TL;DR for operators A realtime agent has at least two different jobs: participate in a conversation at the right moment, and complete whatever task the conversation initiates. Gander separates those jobs. A 9B multimodal front layer continuously handles audio, video, turn-taking, and interruption, while a separate asynchronous back brain can take over longer reasoning or tool workflows. ...
TL;DR for operators A speech model can perform well when given valid speech and still behave badly on audio that should never have reached generation. Mengzhe Geng’s SURE-Voice study1 isolates that earlier decision. On the 480-example held-out SURE-Extended test, six raw speech/audio LLMs achieved only 0.000 to 0.133 accuracy on unsupported inputs. Placing the same fixed audio gate in front of each model raised unsupported accuracy to 0.919, preserved each backbone’s stored supported accuracy of 0.919 to 0.970, and reduced downstream model calls from 480 to 287. ...
Voice is supposed to be the easy interface. No menus. No forms. No training session. A user speaks, the agent understands, and some neat piece of software magic happens in the background. That is the sales pitch. It is also mostly true in a demo room, which is a place where microphones behave, users speak politely, and nobody’s child interrupts from the back seat. ...
A voice assistant can transcribe your question correctly and still answer like it heard something else. That is the awkward part of modern audio-language models. The obvious diagnosis is usually “better speech recognition.” The less obvious diagnosis is nastier: the model may receive an audio input that is semantically equivalent to the text prompt, but once generation begins, its audio-conditioned reasoning trajectory drifts away from the reasoning trajectory it would have followed if the same question had been typed. ...