What your customer service AI should do when it doesn't know the answer

When customer service AI doesn't know the answer, it should say so, hand the customer to someone who can help (with the conversation record attached), and turn the missed question into a knowledge update. It should never guess. Many hallucinations in customer service start right here: a system built to reply to anything produces a reply when it has nothing approved to say.

Why does customer service AI answer questions it can't answer?

Most customer service AI is built to generate. A generative assistant receives a question, retrieves whatever text looks relevant, and writes a reply. Writing a reply is what the system does, so it does it whether the retrieved text settles the question or not. When the source material is thin, out of date or silent on the point, the model fills the gap with text that sounds right.

The business may not have a policy on what the customer asked. The problem is that the system has no dependable way to notice, and no design reason to stop. A fluent wrong answer looks the same in the logs as a fluent right one, which is why these errors tend to surface as complaints by customers after the fact rather than alerts before it.

Why can't the model just tell you when it doesn't know?

It can try. The result is unreliable, because asking a language model to rate its own confidence asks the component that produced the error to detect it.

Prompt rules such as "answer only from the provided context" and "say you don't know if unsure" lower the rate of invented answers, and they are worth running. But they don't change who makes the final decision: the same model that writes the answer also decides whether it had grounds to write it. Retrieval-augmented generation has the same limit, as Is RAG enough? Why retrieval still hallucinates in customer service explains.

Confidence thresholds on retrieval scores help a little more. But a high similarity score only measures how closely a passage matches the words of a question, not whether it actually answers it. A refund policy for a different product can score well and still be completely wrong.

The most dependable signal comes from how the system is built. When the answers are written and approved before any customer asks, checking whether an approved answer exists is a simple lookup with a yes-or-no result. The system either finds a matching answer or it doesn’t. That’s far more reliable than asking a model to judge its own confidence while it’s still writing the reply.

How does a knowledge-first architecture know when there's no answer?

In a knowledge-first architecture, answers are built and approved at ingestion. When Inbenta Encore ingests a business's content (policy documents, help center pages, PDFs, call transcripts, SharePoint libraries and systems such as Salesforce and ServiceNow), it works out the ways a customer might ask each question and links each one to a resolution. A second, independent model validates each answer before it goes live. It all happens before a customer sees it, and that approval is the audit record.

At runtime, Encore matches the customer's request against those governed answers. Either one fits or it doesn't. When one fits, a language model checks that the question and the answer align and shapes the wording for the conversation. The model shapes the wording, not the facts. When nothing fits, Encore does not force a weak match, and the model is not asked to fill the gap.

The matching step itself is covered in How does contact center AI determine caller intent before the question is asked?. For why it’s safer to build and approve answers ahead of time than to generate them while the customer waits, see How to prevent AI hallucinations in customer service: fix the architecture, not the prompt.

What should happen when the AI has no approved answer?

A well-designed customer service AI does four things when it reaches the edge of its knowledge.

It recognizes the gap. The system registers that it has no governed answer, rather than letting a generation step cover the hole with plausible text.

It asks, when the question is ambiguous. Some misses are gaps in the question, not in the knowledge. "What's the fee?" can mean four different fees. A short clarifying question settles that without a handoff.

It escalates with context. When the knowledge doesn't cover the question, Encore escalates to a content expert or a live agent instead of guessing, and passes along the conversation thread so the customer doesn't repeat themselves. With Live Agent Assist, the agent sees the case history at escalation and works from governed reply drafts, which the agent reviews before anything reaches the customer.

It logs the gap. A question the system couldn't answer is one of the most useful data points it produces. It should go somewhere that leads to a fix, not into a transcript nobody reads.

Is escalation a failure for customer service AI?

Not when it is designed in. Escalation reads as failure under a containment metric, where the goal is to keep the customer inside the automated channel. Under that metric a customer who gives up counts the same as one who got an answer, and an assistant that guesses scores better than one that hands off.

Resolution is the better measure. A correct handoff that ends in a resolved contact beats a confident wrong answer that ends in a complaint, a repeat contact or a compliance review. Gartner puts full self-service resolution at roughly 22% of interactions, which means the handoff is a large part of the customer journey, not an edge case. An assistant that hides its gaps behind generated text makes that journey harder to see and harder to fix.

The quality of the handoff decides what escalation costs. When context is lost, the customer re-explains and the agent starts cold. In Live Agent Assist, which puts governed knowledge and conversation history in front of the agent, Inbenta measures +80% first-contact resolution and 65% lower escalated call volume. Both figures are specific to the assisted channel.

How does a missed question become a better answer next week?

This is where the gap pays for itself. Encore Elevate learns from real interactions and surfaces the questions the knowledge doesn't cover: new intents customers are asking for, missing sources, undefined acronyms and escalation fixes. It drafts the update, and the operator publishes it with one-click approval. It also shows where customers struggle and escalate, so the CX team knows what to fix first.

The loop matters because knowledge decays after launch. Products change, fees change, policies get rewritten, and customers find new ways to ask old questions. A system that guesses through those gaps hides the decay until it turns into a complaint. A system that escalates and logs them turns each miss into a candidate update, approved by the team that owns the knowledge. Against GenAI-reliant stacks, Encore runs at up to around 50% lower overhead.

Doesn't declining to answer frustrate customers?

Less than a wrong answer does. A customer told "I'll connect you with someone who can confirm that" loses a minute. A customer given the wrong refund window, the wrong fee or the wrong eligibility rule loses money, comes back angrier, and in a regulated industry may have grounds for a complaint the business has to answer.

The trade-off shrinks as coverage grows. AI Assistants in production answer at 98 to 99% accuracy, and Encore matches compound questions to several governed answers and returns one response, so a two-part question doesn't fail because one half is unusual. Each gap Elevate closes is one fewer handoff. The goal is not an assistant that escalates often. It is one that escalates for the right reasons and answers correctly the rest of the time.

What should you ask a vendor about no-answer behavior?

Five questions show how an assistant behaves at the edge of its knowledge.

What does the assistant do when no approved answer exists: escalate, ask a clarifying question, or generate something anyway? Who decides that no answer exists: a model rating its own confidence, or a match against an approved set? What does the human agent receive at handoff: the conversation and its context, or a transfer and a fresh start? Where do unanswered questions go after the conversation ends? How long does it take a missed question to become an approved answer, and who approves it?

Then ask for a demonstration with a question that isn't in the knowledge base. How a system handles that one question tells you more than a demo of fifty it can answer.