Let the candidate control when agent takes the turn - #192
Merged
Merged
Conversation
jserv
force-pushed
the
turn-taking
branch
3 times, most recently
from
September 30, 2026 21:58
629cb84 to
5727b3d
Compare
This was referenced Oct 1, 2026
ColtenOuO
reviewed
Oct 1, 2026
jserv
force-pushed
the
turn-taking
branch
2 times, most recently
from
October 1, 2026 20:39
0c6c00d to
80b9c52
Compare
Jim answered the pause a candidate took to find the next word: Gemini ended the turn after one second of silence, and a second-language speaker pauses longer than that mid-sentence. The default window is now three seconds, and the live prompt says a pause within a sentence is not a finished answer. A longer window alone would slow every reply, so the page adds "Your turn is done" (Alt+Enter, or a click on the ring), which ends the audio stream so Gemini answers at once: in a measured session the first audio came about 0.8s after the last word instead of 3.4s. A ring beside the status line fills through the window the agent publishes as a participant attribute while the microphone is quiet, and resets on speech, so the moment Jim takes the turn is visible before it comes. The prompt golden changes on purpose: contract bundle 25, live prompt 17. Close #140
Jim kept talking after a candidate asked for time to think: "let me think" was just another turn to answer, and silence alerts, hints and editor reviews filled the gap within seconds. A Thinking button, or an explicit spoken request in English, now holds the floor. Replies the model generates meanwhile are dropped, nudges and editor reviews wait, and hints and endings are refused, while editor, microphone and test evidence stay live and the deadline does not move. Speaking again, Continue or "Your turn is done" ends the hold, and so do the five-minute warning, the round change and one check-in after two silent minutes. A spoken request stays provisional until its utterance ends, so reasoning that merely begins with "let me think" is not a hold, and the transcript of speech from before a click cannot end the hold it was clicked into. The model is told when something it said during a hold went unheard, test runs reach it as context it does not answer, a reconnection during a hold is briefed without being asked to speak, and a Continue whose reply never comes is asked for after four seconds. The prompt golden changes on purpose within bundle 25. Close #172
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jim answered the pause a candidate took to find the next word, and kept talking after a candidate asked for time to think. The first commit raises the default silence window from one second to three, tells the interviewer a pause within a sentence is not a finished answer, and adds "Your turn is done" (a button, Alt+Enter, or a click on a countdown ring that fills through the window while the microphone is quiet), which ends the audio stream so Gemini answers without waiting it out. The second adds a thinking hold: a Thinking button or an explicit spoken request in English ("let me think", "can you give me a minute") keeps the floor, generated replies are dropped, nudges wait, hints and endings are refused, and evidence and the deadline carry on. Speaking again, Continue or yielding ends it, and so do the five-minute warning, the round change and one check-in after two silent minutes.
The trade worth a reviewer's eye is the longer window: three seconds would slow every reply, which is why the yield exists, and it only helps if ending the audio stream really ends the turn. A probe against the live API (9 sessions,
gemini-3.1-flash-live-preview, 3s window) measured the first reply audio at a median 0.76s after the last word withaudioStreamEndagainst 3.36s without; the next utterance after it was detected normally, and 1 turn in 12 ignored the end and fell back to the full window. The same probe showedinputTranscription.finishedis never sent and the transcript arrives about 0.3s after the stream ends, which is what the pre-click transcript window in the hold is sized from. Spoken requests are matched by phrase, so recognition is English only and deliberately strict: short phrases such as "one second" or "hold on" count only as a whole sentence.Verified with
scripts/test.shat each commit (all gates pass; at the tip 1061 Rust tests and 647 browser tests, with the Playwright lane skipped because Chromium is unsupported on this Ubuntu 20.04 host). Before the rebase onto current main: with Chromium 130 linked in by hand the browser suite passed 753 of 757, the 4 failures being PDF tests that fail identically on main, and credentialedscripts/browser-check.shwith the rust agent passed the voice, barge and report flows, and a scripted run against a real room clicked Thinking, saw Jim stay listening for the whole hold, released it by speaking and by Continue, and yielded with Alt+Enter.scripts/gemini-check.shgetssetupCompletewith the new window.cargo mutants --in-diffover the change found 10 missed and 4 timeouts; after new tests, and with the three functions that only write to the room added to the exclusion list with the reason, a re-run over the affected functions caught all 63 viable mutants. It was not re-run after the final cleanup. The prompt golden changes on purpose: contract bundle 22, live prompt 14.Closes #140
Closes #172
Summary by cubic
Lets candidates control when Jim takes or gives up the turn. The default silence window increases from one to three seconds so a pause mid-sentence is not treated as a finished answer, while Your turn is done (a button, Alt+Enter, or the countdown ring) ends the audio turn immediately.
Written for commit 271e939. Summary will update on new commits.