feat: add voice latency and interruptions scorers

[kev (kevchoi)]

AI-created or modified and not human-reviewed in its current form; treat this artifact as provisional. Updated 2026-10-07.

Summary

Adds two deterministic voice scorers, VoiceLatency and VoiceInterruptions, in TypeScript and Python. They score the timing of a voice call from a list of utterances and make no model call.

• Both take a named utterances argument: { speaker: "user" | "agent", start_unix_ms, end_unix_ms, interrupted? }[]. These are the timing fields of Utterance in Braintrust's voice trace reader (@braintrust/local-voice). Customers build the list from their own traces.
• Both group the user's speech into turns the same way and find the agent's reply to each turn.
• VoiceLatency: the share of agent replies that start within maxGapMs (default 1500) of the user finishing a turn. Metadata has p50_ms, p95_ms, gaps_ms, replies, unanswered_turns and max_gap_ms.
• VoiceInterruptions: the share of user turns the agent didn't talk over. Talking over means the agent starts during the turn and overlaps it by more than minOverlapMs (default 300). Barge-ins, where the user cuts the agent off, are reported in metadata and don't affect the score.
• Invalid utterances are skipped and counted in skipped. Without timed utterances the score is null.
• Lists both under a "Voice" group in js/manifest.ts, so they appear as built-in scorers. The server passes them utterances from the trace in a separate braintrust PR; until that lands, the built-ins score null.
• SCORERS.md documents the turn rules, timestamp expectations and limits; README.md lists both.

Testing

• Voice tests pass: the same 8 in JS (vitest) and Python (pytest).
• JS build and pre-commit linters passed on the changed files. The one tsc --noEmit error is the existing one in js/render-messages.test.ts.
• Not run on real voice calls. The default thresholds follow Hamming's published bands and haven't been tuned.

🤖 Generated with Claude Code

[github-actions]

Braintrust eval report

Autoevals (HEAD-1791387780)

Score	Average	Improvements	Regressions
NumericDiff	78.5% (+1pp)	7 🟢	5 🔴
Time_to_first_token	6.6tok (-1.84tok)	182 🟢	37 🔴
Llm_calls	1.55 (+0)	-	-
Tool_calls	0 (+0)	-	-
Errors	0 (+0)	-	-
Llm_errors	0 (+0)	-	-
Tool_errors	0 (+0)	-	-
Prompt_tokens	516.5tok (+2.61tok)	18 🟢	25 🔴
Prompt_cached_tokens	0tok (+0tok)	-	-
Prompt_cache_creation_tokens	0tok (+0tok)	-	-
Prompt_cache_creation_5m_tokens	0tok (+0tok)	-	-
Prompt_cache_creation_1h_tokens	0tok (+0tok)	-	-
Completion_tokens	473.1tok (+6.38tok)	95 🟢	117 🔴
Completion_reasoning_tokens	359.27tok (+11.64tok)	75 🟢	99 🔴
Completion_accepted_prediction_tokens	0tok (+0tok)	-	-
Completion_rejected_prediction_tokens	0tok (+0tok)	-	-
Completion_audio_tokens	0tok (+0tok)	-	-
Total_tokens	989.6tok (+8.99tok)	96 🟢	117 🔴
Estimated_cost	0$ (+0$)	59 🟢	73 🔴
Duration	6.6s (-1.84s)	182 🟢	37 🔴
Llm_duration	7.11s (-1.98s)	182 🟢	37 🔴