feat(openai, google): Plugins for OpenAI & Gemini realtime speech-to-speech translation

[claralp]

feat(openai, google): realtime speech-to-speech translation models

Adds RealtimeTranslationModel for both OpenAI (gpt-realtime-translate) and Google Gemini (gemini-3.5-live-translate-preview). Both perform live, simultaneous speech-to-speech translation — audio in one language comes out as audio + transcript in the target language while the speaker is still talking, with no STT/TTS/VAD plugins required.

New additions

livekit-plugins-openai — realtime/translation_model.py

Wraps the dedicated /v1/realtime/translations WebSocket endpoint. Since this API has no response/turn lifecycle (only continuous session.output_transcript.delta / session.output_audio.delta deltas), turn boundaries are synthesized from idle gaps in the output stream: the first output delta opens a segment (GenerationCreatedEvent) and a configurable idle timeout closes it. Features:

• Input (source-language) and output (target-language) transcript streaming
• Audio resampling to 24 kHz mono PCM16
• update_options(target_language=...) pushes a live session.update to in-flight sessions
• max_session_duration for proactive WebSocket recycling
• Full retry/backoff via conn_options
• RealtimeModelMetrics per segment (TTFT, duration)
• Exposes raw openai_server_event_received / openai_client_event_queued events for debugging

livekit-plugins-google — realtime/translation_model.py

Thin subclass of the existing RealtimeModel/RealtimeSession. The entire connection, receive, generation, transcription, and metrics machinery is inherited unchanged — only _build_connect_config is overridden to send a TranslationConfig instead of a conversational system prompt. Capabilities mutable_chat_context, mutable_instructions, and auto_tool_reply_generation are disabled to match the model's constraints.

livekit-plugins-google README — documents RealtimeTranslationModel, usage as an AgentSession LLM, live language switching, and the multi-user example.

examples/other/translation/realtime-multi-user-translator.py

End-to-end multi-user translator example, supporting both providers via TRANSLATION_PROVIDER=openai|google. Demonstrates running one translation session per participant/direction simultaneously.

Tests

• tests/test_openai_realtime_translation.py — hermetic tests (no network, monkeypatched WebSocket) covering: session.update payload, output segment streaming and metrics, back-to-back segments, input transcription (interim → final), late-delta segment reopening, and generate_reply failure contract.
• tests/test_google_realtime_translation.py — hermetic tests covering model capabilities/properties, minimal LiveConnectConfig (no system instruction, tools, speech config, or session resumption), and update_options language switching.

Usage (both providers are drop-in compatible)

# OpenAI
from livekit.plugins.openai.realtime import RealtimeTranslationModel
session = AgentSession(llm=RealtimeTranslationModel(target_language="es"))

# Google Gemini
from livekit.plugins.google.realtime import RealtimeTranslationModel
session = AgentSession(llm=RealtimeTranslationModel(target_language="es"))

await session.start(room=ctx.room, agent=Agent(instructions=""))

[CLAassistant]

All committers have signed the CLA.