OpenAI details GPT-Live realtime voice system: full-duplex model, six months of latency engineering

OpenAI published a technical deep-dive explaining how it built GPT-Live, its third-generation voice system, in six months. Unlike earlier turn-based voice assistants, GPT-Live puts a full-duplex voice model at the center of the conversation, letting it listen and speak at the same time for a more immediate, natural rhythm.

Previous systems relied on small “turn detector” models to decide when the assistant should speak: guess too early and the user gets cut off, guess too late and the response feels sluggish. GPT-Live removes the detector from the audio path entirely, and when deeper reasoning or tool use is required, it can asynchronously consult frontier models such as GPT-5.5 without interrupting the flow of conversation.

Delivering this at scale required reworking model inference, context management, and media transport. The architecture cleanly separates the media path from application logic: audio flows between client and voice model on a dedicated fast path, while delegation, tool use, and other work happen behind an asynchronous RPC boundary, so a slow tool call cannot stall the voice stream.

OpenAI rewrote the media frontend and inference logic in Go, replacing a Python asyncio implementation; the new system’s p95 frame delivery matches the previous system’s p50. Transport is built on WebRTC, and a new “Instant Connect” mechanism moves SDP signaling off the critical path, letting a client start a session with a single UDP packet.

For long-running calls, the team built stateful inference with seamless handoffs: a replacement model instance can be warmed, prefilled with current context, and cut over in parallel without media interruption. Context compaction is treated as another managed transition, so long conversations never miss a beat.

The work extends to the protocol layer. OpenAI designed WARP, a set of open specifications being advanced through the IETF’s TSVWG working group, with support already added to libwebrtc and Pion. The architecture powers ChatGPT Voice, including the newly launched ability to control your computer and coordinate agents in the ChatGPT desktop app.

OpenAI says the same foundation will underpin the upcoming GPT-Live API and will let voice experiences span more devices, apps, and modalities. For the industry, the shift from turn-based to continuous conversation sets a new baseline for what responsive voice AI should feel like.