Customer Support KPIs with AI: How to Adjust AHT, Resolution, CSAT

Customer Support KPIs with AI: How to Adjust AHT, Resolution, CSAT

To measure customer support KPIs with AI, keep using average handling time (AHT), resolution rate and CSAT, but change how you calculate each one. Measure the conversation end to end, separate what the AI resolved on its own from what passed through a person, and compare only groups of equivalent contacts. Without that, your dashboard mixes a change in the type of demand with real quality, and it leads to the wrong decisions.

Key takeaways

• AHT stops being a single number: separate AI time, active human time and total time to resolution.
• Resolution must mean the customer's need was met, not that the conversation was closed.
• Read CSAT by journey (AI only, AI with handoff, human only), because each group responds differently.
• Comparing with the previous period only works after you recalculate the past with the same definition and the same set of contacts.
• A pilot with one real flow, logs and a target agreed beforehand keeps you from arguing about the metric after the result.

Why classic metrics mislead once AI is involved

AHT, resolution and CSAT were built for a world where a person handled every conversation. The time measured was the agent's time, the ticket was closed by the agent, and the satisfaction survey was in practice a rating of that agent. When an AI takes over part of the contacts, all three assumptions stop holding at once.

The most common effect is a mix shift. The AI tends to absorb the simple, repetitive questions: duplicate invoices, order status, opening hours, required documents. What remains for the human team is what requires judgment, exceptions or negotiation. The result: human AHT can go up and human CSAT can swing even when the whole operation is performing better. Anyone looking only at the overall average concludes that the AI made support worse, or the opposite, depending on which number appears first in the report.

There is also a definition problem. A WhatsApp conversation that the AI closes for inactivity may be counted as resolved. A customer who gave up and came back through another channel becomes two contacts. So before talking about targets, decide what each metric now means. If your focus is customer service, the article on what to automate and what to keep human in AI customer support helps you define which types of contact go into each group.

How to adapt AHT without fooling yourself

Traditional AHT adds up handling time and divides it by the number of contacts. With AI, that blends seconds of automated replies with minutes of human work. In asynchronous channels such as WhatsApp the problem gets worse: a conversation can stay open for hours with nobody working on it.

The most useful adaptation is to break the time into parts that answer different questions:

• Time to first response: shows whether the customer stopped waiting. This is usually where the AI's gain is most obvious.
• AI-led conversation time: tells you whether the flow is direct or whether the customer goes in circles to reach the point.
• Active human time after handoff: the team's real effort, excluding the customer's waiting time.
• Total time to resolution: what the customer feels, adding everything up, including pauses.

Handoff quality deserves special attention. If the AI passes the case with a clear summary of what was already asked, human time tends to drop, because the person does not start from zero. If the handoff arrives without context, the customer repeats everything and human time goes up. That is a valuable signal, and it only shows up when human AHT is measured separately and tagged with which cases came from the AI.

Also look at the distribution, not just the average. A few very long cases distort the number and hide the fact that most conversations got faster. The median plus a list of the longest cases usually tells a more accurate story.

How to measure resolution when AI answers

Containment and resolution are different things. Containment says the AI did not transfer the conversation to a person. Resolution says the customer got what they needed. An AI can contain a lot and resolve little, simply by leaving the customer with no way out until they give up. If the goal is only to reduce handoffs, the system learns not to hand off, and that is a bad incentive.

A practical definition of resolution combines signals instead of relying on just one:

• Customer confirmation, when they say it is solved or say thanks after a direct answer.
• A completed action in your systems, such as an invoice reissued, an appointment created or a ticket opened with the correct data.
• No follow-up on the same subject within a window the company defines for each type of contact.
• A sample reviewed by a person, to check that what the dashboard called resolved really was.

With that, classify each conversation into clear categories: resolved by AI, resolved after handoff, abandoned, closed without confirmation and reopened. The closed-without-confirmation category is the most honest of all, because it admits that not every silent conversation was a success. To organize this reading in a controlled test, the guide on how to measure an AI automation pilot details metrics and decision gates.

One more point: record why the AI decided what it decided. A decision log with the identified intent, the action taken and the reason for any handoff lets you audit resolution later. The OpenTelemetry observability documentation describes the general idea of using logs, metrics and traces to understand how a system behaves, and the same reasoning applies to a support agent.

CSAT: who answers and what the score measures

CSAT with AI carries two risks. The first is response bias: the people who answer a survey are usually those who were very satisfied or very irritated. If the AI changes who receives the survey (for example, by sending it only after formally closed conversations), the score changes without support itself having changed.

The second is attribution. A customer who went through a confusing AI flow and was then well served by a person may give a low score, and the dashboard charges it to the human. To avoid this, record the path taken and analyze the score by journey:

• AI only, no handoff.
• AI with handoff to a human.
• Human only, no AI involvement.

Add a direct question about the outcome to the survey, in addition to the overall score. Something like: was your problem solved? That separates satisfaction with tone from satisfaction with the result. If the survey is sent proactively on WhatsApp, keep in mind the rules on templates and consent, which the article on WhatsApp outbound messages, templates and opt-in explains.

Because many customers never answer any survey, complement it with indirect signals. Sentiment analysis in customer support can flag conversations with frustration even without a score, as long as you treat it as a clue for review and not a verdict. And remember that the data in these conversations is personal: limit what is stored and who can see it, in line with LGPD.

Traps when comparing with the previous period

Comparing this month with last month looks simple, but once AI is in place almost nothing stays the same. The most frequent traps:

• Mix shift: simple contacts moved to the AI, so the human numbers now reflect a harder set of cases.
• New demand: after-hours service and fast answers bring contacts that did not happen before, which changes volume and profile.
• Changed definitions: if resolution and closing are now counted differently, the historical series lost comparability.
• A different CSAT sample: the timing, channel or audience receiving the survey changed.
• Seasonality and campaigns: peak periods have their own profile, and comparing a peak month with a calm one distorts everything.
• Novelty effect: at first customers test the AI out of curiosity, and behavior shifts after a few weeks.
• Simultaneous changes: team, deadline policy and AI changing together make it impossible to attribute the effect to any one of them.

The safe rule is to recalculate the previous period with the new definition whenever the data allows. When it does not, state and record that the comparison is approximate. Another good practice is to compare by contact type (for example, order status against order status) rather than by total. The task-based evaluation approach described by Anthropic in its material on agent evals follows this logic: measure behavior on defined cases instead of trusting a broad average.

Illustrative scenario: the average got worse, but support got better

Imagine, as an illustrative scenario, a services company that handles customers on WhatsApp. Before the AI, every contact went to the team. Afterward, the AI started answering questions about opening hours, documents and status, and handing off change requests and complaints.

In the following month's report, human AHT went up, overall CSAT looked similar and the resolution rate increased. Leadership asks whether the AI helped or hurt. Looking only at the old dashboard, the answer is confusing.

The correct analysis breaks the numbers apart. Human AHT went up because the team now receives only change requests and complaints, which are naturally longer. Time to first response dropped for customers asking something simple. CSAT for customers served only by the AI is high on objective questions, but low in cases where the customer could not find the option to talk to a person. Resolution went up, but a sample review shows that some of the conversations closed without confirmation were not actually resolved.

Conclusion of the scenario: the AI helped with simple contacts and needs adjustment in the path to a human. The right decision is to fix the handoff trigger and change the resolution rule, not to shut the project down or celebrate the average.

A plan to adapt your KPIs without overcomplicating

The method we apply in support projects follows the same sequence: map, limit, pilot and hand over. For metrics, it looks like this:

• Map: list the contact types, how they are handled today and what each metric was before the AI, noting how each one was calculated.
• Define: write on one page what resolved, abandoned, reopened and transferred mean, and which time windows apply to each contact type.
• Limit: choose one flow and restrict what the AI can do in it, with clear rules for handing off to a person.
• Pilot: run that real flow with decision logs and a target agreed before starting, including what will make the company expand, adjust or stop.
• Review by sample: every week, a person reads a set of conversations from each category and compares them with what the dashboard recorded.
• Hand over the operation: document the definitions, give the dashboard to the team and decide the next flow based on the evidence.

This design is consistent with what the NIST AI Risk Management Framework proposes in general terms: map the context, then measure and manage risks continuously. It also reflects the concern of measuring productivity without turning the team into a target of surveillance: the metrics exist to improve the process, not to rank people. The article on how to measure AI productivity without surveillance goes deeper into that care.

Frequently asked questions

Does AHT still make sense when AI handles part of the customers? Yes, as long as it stops being a single number. Measure time to first response, AI time, active human time and total time to resolution. That way you see where the gain shows up and where there is rework.

What is the difference between containment and resolution rate? Containment measures how many conversations the AI did not transfer to a person. Resolution measures whether the customer's need was met. An AI can have high containment and low resolution if customers give up, so the two need to be tracked separately.

How do I keep CSAT from being distorted by the AI? Analyze the score by journey (AI only, AI with handoff, human only) and add a question about whether the problem was solved. Also watch who receives the survey, so that a change in audience does not look like a change in quality. Use indirect signals for customers who do not respond.

Can I compare today's numbers with the period before the AI? Yes, with care. Recalculate the previous period with the new definition whenever possible, compare by contact type and account for seasonality, new demand and simultaneous changes. When that is not possible, record that the comparison is approximate and use the pilot as your main reference.

Sources

• OpenTelemetry: observability concepts
• Anthropic: Demystifying evals for AI agents
• NIST AI Risk Management Framework