Skip to content

feat(vad): set default vad in agent session#5812

Open
chenghao-mou wants to merge 1 commit into
chenghao/feat/inline-silero-vadfrom
chenghao/feat/default-vad
Open

feat(vad): set default vad in agent session#5812
chenghao-mou wants to merge 1 commit into
chenghao/feat/inline-silero-vadfrom
chenghao/feat/default-vad

Conversation

@chenghao-mou
Copy link
Copy Markdown
Member

@chenghao-mou chenghao-mou commented May 22, 2026

AgentSession now provisions a bundled silero VAD automatically. Existing call-site behaviour is preserved via an is_default marker on the VAD instance — the framework-supplied default is transparent to every code path that previously cared about "user configured a VAD."

Before

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT("deepgram/nova-3"),
    llm=inference.LLM("openai/gpt-4.1-mini"),
    tts=inference.TTS("cartesia/sonic-3"),
    vad=inference.VAD(model="silero"),  # required boilerplate
)

…or, in examples using the prewarm pattern:

def prewarm(proc: JobProcess) -> None:
    proc.userdata["vad"] = inference.VAD(model="silero")

server.setup_fnc = prewarm

@server.rtc_session()
async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        # ...
        vad=ctx.proc.userdata["vad"],
    )

After

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT("deepgram/nova-3"),
    llm=inference.LLM("openai/gpt-4.1-mini"),
    tts=inference.TTS("cartesia/sonic-3"),
    # vad= omitted — bundled silero is provided automatically
)

# Opt out explicitly if needed:
# AgentSession(..., vad=None)

# Customise by passing your own:
# AgentSession(..., vad=inference.VAD(model="silero", min_silence_duration=0.3))

What changes (and what doesn't)

Concern Behaviour
AgentSession() with no vad= Now gets inference.VAD(model="silero") with is_default=True
AgentSession(vad=None) Unchanged — no VAD
AgentSession(vad=my_vad) Unchanged — user-supplied VAD, is_default=False
RealtimeModel with server-side turn detection Default VAD is constructed but not wired into the audio pipeline (saves CPU); server-side VAD remains canonical
turn_detection="vad" / AudioTurnDetector opt-ins Now satisfied by the default — no more "VAD model not provided" warnings
Realtime fallback mode = "vad" when server-side turn detection is off No change — only flips when the user explicitly supplied a VAD
Adaptive interruption detection eligibility No change — opt-in stays opt-in; default VAD doesn't auto-enable it
STT hook speaking= payload type and _last_speaking_time source No change for sessions that didn't configure a VAD — default VAD is treated as absent at these sites

The marker is exposed as a read-only vad.VAD.is_default property on the base class, so plugin-supplied VADs inherit False automatically.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional findings.

Open in Devin Review


def _build_default_vad() -> vad.VAD:
instance = inference.VAD(model="silero")
instance._is_default = True
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we create a separate field? Like one inside the session itself? self._using_default_vad = True

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that, but then we need to pass extra flag to activity for resolution or audio recognition like in update_vad.

@chenghao-mou chenghao-mou force-pushed the chenghao/feat/default-vad branch from 15d36dd to 5644401 Compare May 23, 2026 00:40
@chenghao-mou chenghao-mou force-pushed the chenghao/feat/inline-silero-vad branch from 0526efb to 122cd9c Compare May 23, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants