Few things frustrate customers more than having to repeat themselves.
I’ve spent over two decades designing and implementing contact centre technology. If there’s one question that decides whether an AI project earns trust or eventually gets switched off, it’s this: does the AI actually understand the customer?
Contact centres have made real progress here, with better knowledge management, smarter routing, and more capable agents. But as AI agents handle a growing share of conversations, there’s a new version of the same problem. When the AI can’t understand your customers, that progress unravels fast.
This blog is about why language is what makes or breaks conversational AI, and what it takes to get right.
How conversational AI actually works now
It’s worth being clear on this: the technology has moved on considerably from what most people picture.
The traditional model, where a customer types or says something, a natural language processing (NLP) engine maps it to a predefined intent, and a scripted response fires back, still exists, and it isn’t simply a leftover from an earlier era. In regulated environments especially, some platforms deliberately hold onto this structure, because a defined, testable set of intents is predictable, deterministic and auditable, in a way that’s harder to guarantee once a model is generating novel responses on the fly. It’s capable within that defined scope, but it can be brittle outside it, and every new way a customer might phrase something has to be built in rather than picked up automatically.
Alongside this, AI agents built on large language models (LLMs) take a different approach: understanding language contextually rather than matching inputs to trained intents, so they handle variation, ambiguity, corrections and follow-ups without every permutation needing to be manually trained in.
For voice, the picture is more nuanced. There are three architectures worth understanding:
STT/TTS-based (the current mainstream). Speech-to-text converts the customer’s voice to a transcript, an LLM processes it and generates a response, and text-to-speech delivers it back. Accurate and widely deployed, but each step adds latency, and nuance like tone and hesitation can get lost in translation if not designed correctly.
Hybrid architectures. This is where the more advanced platforms are heading: blending STT/TTS and speech-to-speech processing within the same conversation flow, then layering genuinely agentic capability on top, so the AI isn’t just replying, it’s reasoning about what’s needed and triggering the right action or system behind the scenes. A fast speech-to-speech layer keeps the exchange natural, while the agentic layer handles account lookups, verification and case updates in parallel.
Speech-to-speech (the emerging frontier). The latest voice AI models process audio end-to-end, without converting to text first, removing latency and preserving everything in the voice: tone, pace, emotional register. Still maturing, but clearly the direction of travel.
Understanding which architecture your platform uses matters because: it shapes what the AI can do and where its limits are.
Where language design still makes or breaks the experience
Better underlying models don’t mean the design problem goes away, they just raise the floor. Here’s where it still gets difficult.
Different words for the same need
“I need to pay my bill”, “can I clear my balance?” and “I want to make a payment” all mean the same thing. LLM-based agents handle this better than intent-based systems, but still need good examples and clear knowledge to route and respond correctly.
Accents, dialects and speech patterns
For voice AI, this remains a genuine inclusivity issue. An STT model trained on a narrow sample, often skewed towards certain accents, will degrade in accuracy for everyone outside that sample. Choose platforms with broad training data and test across the actual diversity of your customer base.
Sector-specific language
Policy names, product codes, local place names, industry terminology and: anything specific to your business needs explicit handling. An AI agent with no knowledge of your products will confidently misunderstand customers explaining their issue perfectly clearly.
Pauses, corrections and real speech
Nobody talks in clean, complete sentences. Customers pause, backtrack, correct themselves mid-sentence. A good AI agent needs to track the full intent of an utterance, not latch onto the first thing it detected. “I need to change my address, actually sorry, my delivery address, not my billing one” should land cleanly.
Emotion and what it signals
“I want to cancel” isn’t always a cancellation request. It might be dissatisfaction, financial pressure, or a vulnerable customer in difficulty. LLMs read emotional context significantly better than intent classifiers, but this still needs designing for, including what the AI does when it detects a signal like that.
Context and reference
“I need to change it” only makes sense if the AI knows what “it” is. Good AI agents maintain conversation context and use it, so customers aren’t asked to repeat information they’ve already given.
Designing AI conversations that actually work
The technology is better than it’s ever been. But the design still does the heavy lifting.
Map your integration landscape first.
This must be examined as a priority before any AI engagement begins, not discovered once building is already underway. Look at exactly what the AI needs to plug into: CRM, billing, telephony, case management, identity verification. This is where AI projects genuinely succeed or fail, more often than the model or the script ever will.
An AI agent that can’t see or update the systems your human agents use is limited to conversation, not resolution, so integration has to be scoped and proven out before anything else starts.
Start with the right use cases
AI handles a growing range of conversations well, but not everything, and automating what it shouldn’t is where most AI projects lose customer trust. Complaints, vulnerability, complex account issues, anything emotionally charged: these typically need a human. Be clear about what the AI is for, and design that boundary deliberately.
Set clear guardrails
What can the AI access? What can it do? What should it say when it’s uncertain? When does it hand off? These are risk decisions, not just technical ones. In UK financial services and insurance especially, that means designing explicitly for Consumer Duty and vulnerable customer identification from the outset, not retrofitting it once the AI is already live.
The platforms that hold up best under scrutiny are the ones where governance and audit trail are built into the core architecture, not bolted on afterwards, and where these decisions are made consciously and documented.
Build around real customer language
Don’t design from how your business describes itself. Pull actual contact reasons, call recordings, chat logs, and agent insight, that’s the vocabulary your AI needs, not the version in your FAQ. The strongest platforms keep learning from that real language after go-live too, refining understanding from actual conversations rather than needing a full manual retrain every time a customer phrases something a new way.
Connect it to trusted knowledge
An AI agent is only as good as the information it can draw on. Accurate, approved, maintained knowledge, including policies, product details and process guidance, is the foundation. Without it, the AI fills gaps with guesses.
Design for when it goes wrong
Every AI agent misunderstands customers sometimes. The question is what happens next. “I didn’t quite catch that, could you tell me more about what you need?” is a recoverable moment. “I’m sorry, I don’t understand” repeated three times is not. Design the repair path as carefully as the happy path.
Make escalation feel seamless
The handover from AI to human is one of the most important moments in the journey, and one of the most frequently botched. When a customer moves to an agent, the agent needs context: what was discussed, what was tried, what the customer actually needs.
Without that, you’ve just made them repeat everything they’ve already said, the exact frustration you were trying to avoid. This is also where genuinely agentic AI, not just conversational AI, earns its keep: updating the account, logging the case, kicking off the right workflow in the background, so the agent picks up mid-journey instead of starting cold.
Keep improving
Where are customers dropping out? Which queries aren’t handled? Where are misunderstandings clustering? What’s being escalated that shouldn’t need to be? The data is there, use it. Conversational AI design is never finished.
Getting this right
Well-designed AI agents can meaningfully improve customer experience: faster resolution, more consistent support, less pressure on your teams. But the gains are in the design and execution, not just the technology selection. That’s the standard I hold every project to, and the standard I’d encourage you to hold your AI partners to as well.
At SVL, we help contact centres plan and implement conversational AI that delivers. Through our partnerships with boost.ai and NiCE Cognigy, we work with businesses to build AI-powered experiences grounded in how their customers communicate, not just what looks good in a demo.
If you want to explore what that looks like in practice, get in touch.
