Your AI resolves tickets, but customers are still unhappy: The measurement gap in B2B SaaS support

Start with the denominator

Resolution rates are difficult to compare when teams count different populations. One dashboard may include only conversations the AI attempted. Another may include every conversation across every support channel.

Consider an illustrative week with 1,000 support conversations. The AI handles 800 and resolves 560 without a human. Its resolution rate on AI-handled conversations is 70%. Its autonomous resolution share across all support conversations is 56%. Both numbers are useful, but they answer different questions.

Report the counts beside the rates. Define which channels, ticket types, and time period are included, how duplicates are handled, and what qualifies as a resolution. Track AI coverage separately so that changing the mix of attempted tickets does not look like a quality improvement.

Choose a repeat-contact observation window that fits the workflow. If a seven-day window is appropriate, wait until it has elapsed before treating that cohort's outcome as final. Mark more recent results as provisional.

Separate an answer, a deflection, and a resolution

An answer means the system responded. Deflection describes an interaction that did not reach a human under the reporting definition. Neither, on its own, establishes that the customer's problem was solved.

A vendor example shows why the labels matter. In its Papaya Pay customer story, Maven AGI reports 90% of inquiries answered autonomously via chat and a 70% first-contact resolution rate. These are vendor-reported results from a consumer bill-payment service, rather than a B2B SaaS benchmark. The story does not establish that the two metrics use identical populations, so the difference should not be treated as a measured failure rate.

For your own product, agree the resolution definition with the customer. A password-reset question may be resolved when access is restored. A billing dispute may need an approved adjustment. Closing a chat after describing the steps is not equivalent to completing them.

Show CSAT response counts, especially at account level

Customer satisfaction surveys capture the experience of respondents. Low participation and small account-level samples make the score difficult to interpret on its own.

Survicate's 2025 CSAT report reports a SaaS median score of 77.8% across 275 surveys from 87 companies. Its 7.5% median response rate comes from a different, broader subset of 560 surveys across 145 companies. These are platform-specific samples, not universal B2B SaaS standards.

The response-rate denominator is survey views. Forty tickets do not automatically mean forty survey views. As an illustration, forty survey views at a 7.5% response rate would yield three responses on average. With only three responses, moving from three satisfied respondents to two changes CSAT from 100% to about 66.7%.

Always show the response count, invitation or view count as applicable, and reporting period. Break results down by channel, issue type, customer tier, and resolution path when the sample supports it. Avoid presenting tiny segments as reliable rankings.

Track who owns escalations

A handoff to a queue is an intermediate step. Track whether an accountable person or team accepts the case and whether it progresses within the customer's agreed service commitment.

Capture the escalation trigger, request time, assignment time, acceptance time, and eventual outcome. Decide how to treat cases that are still open at the reporting cutoff. Otherwise, slow escalations can disappear from the denominator.

Measure the handoff's quality too. Review whether the receiving agent got a useful summary, the attempted actions, the relevant account details, and a clear statement of what remains unresolved. Ask customers whether they had to repeat information.

Set targets from the support arrangement and the risk of the issue. An arbitrary assignment percentage cannot replace a contractual commitment or make an urgent case safe to ignore.

Identify tickets that need account context

Some requests can be answered from public documentation. Others depend on the customer's plan, permissions, configuration, usage, or contract. Track those separately.

For the account-context group, record whether the required information was available, current, and accessible to the AI under the user's permissions. Compare resolution, handoff, and repeat-contact outcomes within similar issue types. A raw comparison with human-only tickets may be misleading if people receive the hardest cases.

Missing context is often an integration or authorization problem. Adding more help-center text will not tell an agent whether a particular customer has a contractual entitlement or permission to make an account change.

Measure priority service commitments directly

Map ticket severity and customer tier to the applicable agreement. Record whether the commitment concerns acknowledgement, a substantive first response, updates, restoration, or resolution. These are different events.

For each tier, report cases that met the commitment divided by cases whose deadline has passed in the period. Include still-open cases that have already missed their deadline. Keep cases that are not yet due visible rather than quietly treating them as successes.

Review the exceptions individually. A single overdue access outage for a large customer can deserve more attention than a high aggregate automation rate suggests.

Build an account-level review from the ticket data

Start with a small set of measures you can explain and reproduce. Keep the same definitions across product, support, customer success, and sales.

• AI coverage and autonomous resolution share, with counts and included channels.
• Repeat contact about the same issue within the chosen observation window.
• Escalation assignment, acceptance, and handoff quality.
• Outcomes for tickets that require account context.
• Compliance with each priority service commitment.
• CSAT and customer effort feedback, with sample sizes and reviewed comments.

Roll these up by account without hiding the underlying tickets. Review unresolved high-impact cases with the people responsible for the relationship, and record the corrective action and its owner.

These measures help locate problems; they do not prove that a change caused a renewal or prevented churn. Use account reviews and customer conversations to understand the commercial implications.

Use the findings to change the product

If repeat contacts rise, review what the system counted as resolved. If escalations stall, fix ownership and routing. If account-context requests fail, inspect the integration, permissions, and information freshness. If customers report high effort, watch the handoff and ask where they had to start again.

An automation dashboard becomes useful when it helps a team decide what to fix. Keep the customer outcome close enough to the metric that a good-looking percentage cannot conceal an unresolved problem.

If you need help making your support evidence clear to buyers, Book a discovery call.