Skip to content

feat(deepgram): use Configure control message for mid-stream keyterm updates#5809

Open
Panmax wants to merge 5 commits into
livekit:mainfrom
Panmax:feat/deepgram-configure-keyterm-mid-stream
Open

feat(deepgram): use Configure control message for mid-stream keyterm updates#5809
Panmax wants to merge 5 commits into
livekit:mainfrom
Panmax:feat/deepgram-configure-keyterm-mid-stream

Conversation

@Panmax
Copy link
Copy Markdown
Contributor

@Panmax Panmax commented May 22, 2026

Summary

  • When only keyterm, EOT thresholds, or language_hint are updated via update_options(), send a Deepgram Configure control message instead of reconnecting the WebSocket
  • This avoids dropping in-progress speech recognition when options change mid-stream (e.g. updating keyterms while the user is speaking)
  • Options that cannot be updated mid-stream (model, sample_rate, mip_opt_out, endpoint_url, tags) still trigger a full reconnect as before

Motivation

Currently, any call to SpeechStreamv2.update_options() triggers _reconnect_event.set(), which tears down the active WebSocket and reconnects. If this happens while the user is speaking, any in-progress INTERIM_TRANSCRIPT is lost and the final transcript may be incomplete.

Deepgram's Flux models support a Configure control message that allows updating keyterms, EOT thresholds, and language hints mid-stream without reconnecting:

{
  "type": "Configure",
  "keyterms": ["term1", "term2"],
  "thresholds": {
    "eot_threshold": 0.8,
    "eot_timeout_ms": 5000
  }
}

Ref: https://developers.deepgram.com/docs/flux/configure#configure-message

Changes

  • Added self._ws instance variable to SpeechStreamv2 to hold the current WebSocket reference
  • Added _send_configure() method that constructs and sends the Configure control message
  • Modified update_options() to check whether the update requires a reconnect or can be applied mid-stream via Configure message
  • Handle ConfigureSuccess / ConfigureFailure responses from Deepgram — on failure, fall back to reconnect so options still take effect
  • Use asyncio.create_task with tracked reference instead of fire-and-forget ensure_future; cancel pending task on WebSocket cleanup
  • Fall back to reconnect if Configure message send fails (e.g. WebSocket closing concurrently)

Test plan

  • Verify keyterm updates via Configure message don't interrupt ongoing speech recognition
  • Verify EOT threshold updates via Configure message work correctly
  • Verify model/sample_rate/endpoint changes still trigger reconnect
  • Verify behavior when WebSocket is not yet connected (falls back to reconnect)
  • Verify ConfigureFailure triggers reconnect and logs warning
  • Verify send failure (e.g. ws closed mid-send) falls back to reconnect

🤖 Generated with Claude Code

…pdates

When only keyterm, eot thresholds, or language_hint are updated, send a
Deepgram Configure control message instead of reconnecting the WebSocket.
This avoids dropping in-progress speech recognition when options are
updated mid-stream (e.g. updating keyterms while the user is speaking).

Options that cannot be updated mid-stream (model, sample_rate, mip_opt_out,
endpoint_url, tags) still trigger a full reconnect.

Ref: https://developers.deepgram.com/docs/keyterm#configure-control-message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Panmax and others added 4 commits May 22, 2026 17:19
Wrap the WebSocket send in an async helper that re-checks connection
state and catches exceptions gracefully, avoiding unhandled task errors
if the WebSocket is closing when the coroutine executes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace asyncio.ensure_future with tracked asyncio.create_task
- Fall back to reconnect when Configure send fails instead of silently dropping
- Handle ConfigureSuccess/ConfigureFailure responses from Deepgram
- Cancel pending configure task on WebSocket cleanup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant