What is a good response latency for an AI voice agent?

Under about a second of silence between the caller finishing and hearing a reply. Human conversation runs on gaps of roughly 200ms, so anything approaching a second reads as a fault on the line and callers start repeating themselves, which causes both parties to talk at once. The percentile matters as much as the median — a fast average with a slow tail fails on the hardest calls.

What causes latency in a voice AI phone call?

Five stages in series: deciding the caller has finished speaking, finalising the transcript, the language model producing its first token, speech synthesis producing its first audio, and the audio travelling back through the carrier. Three of those are network round trips to model providers, which is why where the models are hosted matters more than how fast they are.

Why is voice AI slower in India?

Because most model providers are hosted in the United States, and each trans-Pacific round trip costs a couple of hundred milliseconds before any work happens. Three such hops consume most of a one-second budget on geography alone. The fix is running every hop in-region, which is available but usually more expensive and offers a shorter list of models — particularly for Indian languages.

How do I test an AI voice agent’s latency myself?

Call it and count the silence after you stop speaking. Then stop mid-sentence, the way real callers do, and see whether it waits or interrupts. Then talk over its reply and see whether it stops. Five minutes of that is more informative than any published figure, which will have been measured somewhere else at a percentile nobody stated.

Why latency decides whether a voice AI call works

30 August 2026

Most voice AI demos sound good. The ones that fail in production usually fail on timing rather than on understanding: the caller stops talking, nothing happens for a second and a half, and they start talking again over the top of the reply. From there the call degrades. Latency is the property that decides whether these systems are usable, it is the hardest one to fix, and it is the one number vendors are vaguest about.

What the silence is made of

The gap between a caller finishing their sentence and hearing a reply is not one delay. It is five, in series, and each one can be optimised separately.

Deciding they have finished. The system has to be confident the caller has stopped rather than paused, which is a judgement call with no perfect answer.
Finalising the transcript. Streaming transcription produces a provisional result while the caller talks and a final one afterwards; the gap between them is dead time.
The model’s first token. How long the language model takes to begin its reply, which is mostly network round trip plus queueing rather than computation.
The first byte of audio. Synthesis has to start producing sound; a streaming voice starts before the sentence is finished, a non-streaming one waits for all of it.
Getting it there. The audio still has to travel back through the carrier to a phone.

Why a second is the threshold

Human conversation runs on gaps of around two hundred milliseconds. People do not consciously measure this, but they react to it: a pause noticeably longer than that reads as hesitation, and one approaching a second reads as a problem with the line.

What callers then do is the actual damage. They repeat themselves, or say "hello?", and now the system is processing new speech that arrived while it was answering the previous sentence. Both parties are talking. The call does not recover gracefully from this, and it is why a demo that felt fine at one and a half seconds falls apart with a real caller who is not being patient.

This is also why perceived latency is worth engineering separately from real latency. A short acknowledgement played quickly — the equivalent of "mm" — buys most of the budget back, because the caller has evidence they were heard and stops waiting.

The problem with running from India

Three of the five stages above are network calls to a model provider, and most of the good ones are hosted in the United States. Each trans-Pacific round trip costs a couple of hundred milliseconds before any work is done.

Stack three of those and the geography alone has spent most of a second, which is the entire budget. This is why latency figures quoted by global vendors do not transfer: a number measured between a US datacentre and a US caller is not the number an Indian business will see.

The fix is unglamorous and expensive. Every hop has to be in-region — inference, telephony and the application — and the in-region options are usually a shorter list than the global ones. A vendor optimising purely for cost per minute has already made this trade-off, and made it against you.

What to ask, and what to ignore

Ignore any latency figure quoted without a location and a percentile. "Under 500ms" measured where, and at which percentile? A median of 500ms with a long tail is a system that feels fast most of the time and infuriating on the calls that matter.

Ask where the models run, and whether that answer changes for Indian languages — it frequently does, because the Indic-capable options and the fast options are not always the same list.

Then stop asking and test it. Call the demo line, stop talking mid-sentence the way real callers do, and count. Say something while it is replying and see whether it stops. That five-minute test tells you more than any number on a website, including this one.

One clarification about our own copy, since it uses a similar word: when this site says calls are answered in under a second, that is the phone being picked up, not the length of a conversational turn. They are different measurements and it is worth keeping them apart when comparing vendors, because some of them do not.