AI Customer Support: What to Automate and What to Keep Human

AI Customer Support: What to Automate and What to Keep Human

AI customer support works best when a company decides by type of request, not by channel or by tool. If you are asking what to automate, the answer is that AI can resolve low-risk requests with few exceptions on its own, prepare the work when judgment is involved, and step aside when an error is costly or the customer is in a fragile state. The boundary comes from crossing two questions: what happens if the answer is wrong, and how much context and exception the request carries.

Key takeaways

• Classify each type of request by risk (the impact of an error) and by complexity (exceptions and context), not by channel or topic.
• Status checks, copies of existing documents and questions about documented policy are the best candidates for full automation.
• When risk is high but the flow is simple, AI collects and prepares, and a rule or a person approves the action.
• Legal threats, suspected fraud, health, safety and emotionally distressed customers go straight to a person.
• The boundary is not fixed. It moves based on conversation logs, sample reviews and a success metric agreed before the pilot.

Why the right question is not AI or people

Many companies start the project by asking whether AI will replace the agent. That question leads to two bad decisions: automating everything the tool is technically able to answer, or banning AI from any topic that feels sensitive. In both cases the criterion is the tool, not the customer.

A more useful question is: for this type of request, what is the worst reasonable outcome of a wrong answer, and who bears it? If the worst outcome is the customer having to repeat a question, AI can work with plenty of room. If the worst outcome is a wrongful charge, exposed personal data or a customer filing a complaint with a consumer protection agency (in Brazil, Procon), the conversation changes.

Agent designs tend to be more reliable when they start with simple workflows and only gain autonomy where there is evidence it works. Anthropic makes this recommendation in its engineering guide, Building effective agents, and it applies directly to customer support: autonomy is granted per type of task, not out of enthusiasm.

The two axes: risk and complexity

To classify requests, use two independent axes. A topic can be simple and risky, or complex and harmless.

Risk is the cost of being wrong. Consider four components:

• Financial: the answer or action could lead to a refund, an improper discount, a charge or a lost sale.
• Legal and regulatory: the answer could be treated as a promise, an admission or a failure to meet an obligation to the consumer.
• Personal: the conversation involves personal data, sensitive data or access to someone's account (see the data protection section below).
• Emotional: the customer is angry, vulnerable or in a delicate situation, and an automated reply makes things worse.

Complexity is the amount of judgment required. It goes up when:

• the answer depends on several policy exceptions;
• information must be combined from more than one system;
• the customer's request is ambiguous or mixes several topics;
• the earlier history changes the decision;
• no document says what the correct answer is.

A practical rule: if even your best agents disagree on the answer, the AI has no reference to get it right yet.

The four zones and what to do in each

Crossing the two axes gives four zones. Each one calls for a different role for AI.

• Low risk and low complexity: automate end to end. Examples: order or ticket status, opening hours, a copy of a document already issued, tracking, a branch address, a documented step-by-step procedure.
• Low risk and high complexity: AI prepares, a person decides. Examples: a long technical question, a request that mixes several topics, a product recommendation with many variables. The AI summarizes, looks up information and suggests a reply, and the agent reviews and sends it.
• High risk and low complexity: AI runs the collection, and the action goes through approval. Examples: cancellation, a change to account details, a refund within a clear rule. The AI confirms identity, gathers what is missing and logs the request. A deterministic rule or a person authorizes the execution.
• High risk and high complexity: a person from the start. Examples: a complaint that threatens a lawsuit, suspected fraud, a product accident, a disputed charge with a long history. At most, the AI says an agent will take over and organizes a summary of the case.

The most valuable part of this design is the third zone. Many companies treat cancellations and refunds as all or nothing, when a middle path works: the AI does the tedious part of collecting information and the decision stays with whoever has the authority. A template to formalize this is in the AI agent approval policy.

Which requests should go straight to a person?

Some situations are not worth an automation attempt, even if the model can answer fluently. Recognizing them early keeps the customer from going through several messages before being heard.

• Any mention of a lawyer, a consumer protection agency, a lawsuit or the press.
• Suspected fraud, a scam, a cloned account or misuse of data.
• A report of harm to health, an accident or a safety risk.
• A customer showing despair, grief, fear or intense anger.
• A request for a policy exception, a discount outside the rules or a debt negotiation.
• A request for someone else's personal data, or from someone who cannot identify themselves.
• A customer who explicitly asks to speak to a person.

The last item deserves emphasis. If the customer asks for an agent, the correct response is to hand over, not to try to persuade them to stay with the AI. In regulated sectors, access to a human agent is often required by customer service regulation, so confirm with your legal team what applies to your case before designing the flow.

The handoff itself also needs a method: the agent should receive the summary, what the AI already tried and the reason for the transfer, without making the customer repeat everything. The WhatsApp AI agent human handoff playbook details this design.

Illustrative scenario: a furniture store handling support on WhatsApp

Imagine, as an illustrative scenario, a furniture store that handles support on WhatsApp. Requests of different natures arrive throughout the day. Here is how the classification organizes each one.

One customer asks where their delivery is. This is low risk and low complexity: the AI checks the status in the system and replies with the expected date and the carrier's contact channel.

Another customer says the sofa arrived with a tear and wants to exchange it. The financial risk is moderate and the complexity depends on the damage rules. The AI asks for photos, the order number and a description, confirms identity and opens the ticket. The decision on exchange, refund or a technician visit stays with a person or with a rule approved in advance.

A third customer writes that a chair broke and their daughter was hurt. This involves a safety risk, strong emotion and possible legal implications. The AI does not try to solve it. It recognizes the signal, tells the customer that an agent will take over the conversation as a priority and hands the summarized case to the team.

What changes from one case to another is not the channel or the general topic (delivery, defective product), but the type of impact and the need for judgment. For how this applies to orders, returns and tracking, the article on AI customer service for ecommerce takes a more operational look.

How to draw the boundary in your company: map, limit, pilot, hand off

The method we use in support projects has four steps. None of them requires sophisticated technology. They require process discipline.

• Map: list the real request types from past conversations, not from what the team remembers. Group them by type and note who decides each one, which systems are consulted and what the expected outcome is. Record the baseline: response time, rework and cases that come back.
• Limit: for each type, define what the AI can read, what it can write and which actions it can take. Access to personal data follows the minimum necessary. Actions with financial or account effects go through approval. The limits are written down, not implied in the prompt.
• Pilot: choose one flow from the low-risk zone, run it with every decision logged and agree in advance on the metric that defines success. Review conversation samples often, including the ones that ended well. The guide on how to measure an AI automation pilot helps you set decision gates.
• Hand off: design the escalation triggers (keywords, sentiment, attempts without resolution, customer request) and the content of the handoff. At the end of the pilot, decide whether to expand, adjust or stop based on the logs.

A note on order: only move the boundary into the next zone once the previous one is stable. It is tempting to jump straight to refunds and cancellations because the gain looks larger, but that is where errors cost the most.

What controls hold the boundary in place day to day?

Classifying request types defines the intent. For the intent to hold, you need technical and operational controls.

First, validation on input and output. The AI should not state deadlines, amounts or policies that are not in the knowledge base it consults, and actions should be verified before they run. The article on guardrails for AI agents shows how to organize these layers.

Second, protection against manipulation. In customer support, anyone can write anything to the agent, including instructions disguised as a question. The OWASP Top 10 for LLM Applications describes risks such as prompt injection and excessive agency, which matter when the agent has access to customer data and to actions in your systems.

Third, risk governance. The NIST AI Risk Management Framework offers a structure for identifying, measuring and treating risks in AI systems, and it can serve as the basis for your matrix, even in a lean version for a small company.

Fourth, privacy. Support handles names, addresses, ID numbers, purchase history and sometimes sensitive data. Define the legal basis, what is stored and for how long, and who can see the records. The ANPD is the official data protection authority in Brazil, and the article on LGPD compliance for AI on WhatsApp covers consent, legal basis and records.

How do you know the boundary is in the right place?

The right boundary is not the one that impresses in a demo, but the one that holds over time. A few signs tell you whether to pull back or move forward.

Signs to pull back:

• More reopened cases that the AI marked as resolved.
• Customers asking for an agent in the first or second message.
• Agents correcting the same answers again and again.
• Complaints that cite the conversation with the bot as part of the problem.

Signs to move forward:

• Reviewed samples show consistently correct answers for a given request type.
• Agents almost always approve the draft without meaningful edits.
• Transferred cases arrive well summarized and with the data needed.

A single metric, such as the AI resolution rate, can mislead. If the AI resolves a lot because customers gave up, the number looks good while the service is worse. Combine resolution with satisfaction, reopen rate and sample review. To evaluate responses at scale, a second model can help with triage, but it does not replace human review of a sample.

Checklist before putting a request type on AI

Use this list as the final decision, one type at a time.

• Is there a document or rule that defines the correct answer?
• Is the worst outcome of an error acceptable and reversible?
• Does the AI need personal data beyond the minimum to answer?
• Is there a clear handoff trigger if something goes off script?
• Does someone have named responsibility for reviewing conversations of this type?
• Can the customer reach a person without effort?
• Is there an agreed criterion to expand, adjust or stop?

If any answer is no, the type is not ready for full automation. That does not rule out using AI to support the agent on that same type.

Frequently asked questions

Does AI customer support replace agents? No, and the safest design assumes it does not. AI takes over simple, low-risk requests, prepares more complex cases and frees the team for the ones that need judgment, empathy or authority. The human team also has to stay in place to review, correct and decide exceptions.

Where should you start when automating customer support? With the most frequent, lowest-risk request type, such as order status, a copy of a document or a documented policy question. Start with one flow, log the conversations and define the success metric before going live. Only once it is stable is it worth moving to types that involve some action in your systems.

Can AI handle cancellations and refunds on its own? Generally not at the start. These flows have financial effects and can lead to disputes, so the safer path is for the AI to collect the data and confirm identity while a clear rule or a person approves the execution. If volume justifies it, automatic approval can be enabled only for cases within well-defined, auditable limits.

What does data protection law (LGPD in Brazil) change for AI in customer support? The LGPD requires a company to have a legal basis for processing personal data, collect only what is necessary and be able to explain what it does with it. In customer support, this affects what the AI can look up, what gets recorded and who can access the conversations. Consult your legal team and the ANPD guidance for your case, and document the decisions before going into production.

Sources

• Anthropic: Building effective agents
• OWASP Top 10 for LLM Applications
• NIST AI Risk Management Framework
• ANPD: Autoridade Nacional de Proteção de Dados