The latency budget of a voice AI call
Callers notice silence long before they notice bad grammar. Where the milliseconds go in a voice AI turn — and how to win them back.
A natural conversation leaves a few hundred milliseconds between turns. Go much past that and callers start talking over your agent — or hang up. Every voice AI call is a race against that budget.
Where the time goes
The caller stops speaking. Speech recognition has to decide they are done, the model has to start answering, text to speech has to produce the first audio, and the audio has to reach the caller. Each step adds delay; the network adds it twice.
Stream everything
Streaming recognition, streamed model output, and sentence-by-sentence speech mean the caller hears the first words while the rest is still being generated.
Handle barge-in
When the caller interrupts, clear the queued audio immediately. In the dialpress SDK that is one call: media.clear().
Shorten the path
Keep the gateway, the speech engine, and the agent close together. Infrastructure that owns the whole media path can remove hops a model-only product cannot.
Field notes
Keep reading
Build an AI voice bot like an Express app
The dialpress Node SDK turns SIP into routes and handlers. Here is a PBX and an AI voice bot that share one gateway — and pass calls between them.
SIP vs WebRTC for AI voice agents
SIP reaches every phone number on earth; WebRTC reaches every browser. A production voice agent needs both — here is how they fit together.
Giving voice agents tools with MCP
Your booking system, your CRM, your order lookup — any API can become a voice agent’s tool without writing a line of agent code.
