Learn

What is backchanneling? The “mm-hmm” in voice AI

Backchannels are the “mm-hmm”s a listener drops in while someone else talks. People do it without thinking. A voice agent has to tell a caller’s backchannel from an interruption, and decide when to give one of its own.

6 min read

  • Zach KamranCo-Founder & CTO
In this guide

Backchanneling is how a listener shows they’re following without taking over: “mm-hmm,” “uh-huh,” “right,” “oh no,” “got it.” The speaker keeps the floor, and the listener signals in passing that the message is landing.

The linguist Victor Yngve named it. In a 1970 paper he described conversation as running on two channels at once. The person with the turn speaks on the main channel, and the listener sends short messages back on the “back channel,” which the speaker hears without stopping. Listen to any phone call and you’ll hear long stretches of one person talking, dotted with the other’s small sounds.

Kinds of backchannel

KindExamplesWhat it says
Continuer“mm-hmm,” “uh-huh,” “yeah”Keep going, I’m with you
Acknowledgement“I see,” “got it,” “okay”I’ve taken that in
Sympathy“oh no,” “oh dear”That sounds bad
Assessment“nice,” “that’s great,” “oh good”That sounds good
NonverbalNods, smiles, raised eyebrowsAny of the above, silently

Timing matters as much as wording. Listeners seldom backchannel mid-phrase. They do it in small pauses, at the end of a clause, or when the speaker’s pitch drops. The kind has to fit, too: “that’s great” after someone describes a broken furnace is worse than silence.

Why they matter on the phone

Face to face, a nod does the work of “mm-hmm.” On the phone the voice carries all of it. A listener who stays silent while a caller explains a problem for thirty seconds sounds absent, and the caller fills the gap with “hello? are you still there?”

Contact center calls are full of these stretches. Callers read out addresses, describe what went wrong with an order, or explain what they’re shopping for, and they expect the small signals a person would give. Too few and the line feels dead. Too many, or mistimed, and the listener sounds impatient or mechanical.

The two backchannel problems for voice agents

Hearing the caller’s

When the agent is talking and the caller says “mm-hmm,” the right response is to carry on, because the caller is agreeing. To a simple voice-activity detector, though, “mm-hmm” looks like the start of an interruption. An agent that stops whenever it hears sound will stutter through every explanation, stopping, restarting and losing its place.

The fix is a turn-taking model that understands what the sound means, and decides quickly, since a real interruption deserves an immediate stop. That’s the barge-in problem, where the agent should stop for “wait, can I change that to Thursday?” and keep going through “uh-huh.”

Giving its own

The other direction is harder. While the caller talks, the agent has to keep deciding whether this pause wants a sound at all, since most don’t and a cue every few seconds sounds like someone hurrying you along. If it does, the agent has to pick the right kind, such as a continuer mid-thought, “oh no” after bad news, “I see” after an explanation. The cue has to land inside the pause, neither mid-word nor so late that the caller is already talking again.

Then there’s delivery. The cue should be in the agent’s own voice and quieter than its speech, the way a listener sounds. And the agent’s turn-taking must not mistake its own “mm-hmm” for the agent taking the floor, or it will cut off the caller it meant to encourage.

How Simple handles backchannels

On the listening side, our Tango model tells a real interruption from a quick “mm-hmm,” so the agent keeps talking when a customer is only agreeing. It recognizes a real interruption about 100 ms after the caller starts talking, and it cut interruptions by 64% against our previous agent in production. The Tango post and the turn-taking page explain how.

On the speaking side, Simple’s agent gives short listening cues of its own. When the caller pauses for about half a second mid-turn, a small model reads what they’ve said so far and scores whether a listener would respond and with which kind of cue (a continuer, an acknowledgement, sympathy or a short assessment). Mid-sentence, only a continuer or sympathy fits, so those are the only cues it uses there.

It rations them to a couple per turn, spaced several seconds apart. When the caller finishes and the agent takes its turn, it can say a brief “Okay.” straight away, so the caller hears a reaction while the full answer is generated; the answer then drops its own opening acknowledgement, so nobody hears “Okay. Okay, so…”

Each cue is rendered in the agent’s own voice, played more quietly than its replies, on a separate audio track that turn-taking ignores, so a cue never counts as the agent taking the floor. Cues are recorded on their own channel, so call reviews can hear them.

Backchannels and latency

A reply takes a moment to generate, and on the phone even a short gap sounds like hesitation. An “okay” at the right moment tells the caller they were heard while the answer is on its way, which is what a person does while they think. It doesn’t replace low latency or good end-of-turn detection, but it makes the remaining gap sound like part of the conversation.

How to test an agent’s backchanneling

Call the agent and read out a long address slowly, with pauses. It should wait for the whole thing, maybe with the odd “mm-hmm,” rather than jumping in at the first gap. Then tell it a delivery arrived broken, and listen for whether any cue fits the mood.

While it’s talking, say “mm-hmm” and check that it keeps going, then interrupt properly and check that it stops at once. Over a two-minute call, count the cues; more than a few a minute starts to sound mechanical. Last, play the recording back and listen for cues at full speech volume, clipped, or on top of your words.

Frequently asked questions

What is backchanneling?

The short signals a listener gives while someone else is speaking, such as “mm-hmm,” “uh-huh,” “I see” or “oh no,” to show they’re following without taking the turn.

What are examples of backchannel responses?

Continuers like “mm-hmm” and “uh-huh,” acknowledgements like “I see” and “got it,” sympathy like “oh no,” and assessments like “nice.” Nodding and smiling are nonverbal backchannels.

Who coined the term backchannel?

Victor Yngve, in his 1970 paper “On getting a word in edgewise,” which called the listener’s short responses a “back channel” running alongside the speaker’s main channel.

Is “mm-hmm” an interruption?

No. The listener is encouraging the speaker to go on. A voice agent should keep talking through it and stop only for a real interruption.

Do AI voice agents backchannel?

Some do. Simple’s agent tells a caller’s “mm-hmm” from a real interruption, and gives short cues of its own in the caller’s pauses, chosen to fit what was said and spoken in its own voice.

Hear a voice agent handle a real call