Skip to content

fix(proto): stringify map values in encode_by_name - #7962

Merged
u9g merged 1 commit into
mainfrom
jason/proto-function-call-extra
Oct 19, 2026
Merged

u9g merged 1 commit into
mainfrom
jason/proto-function-call-extra

Conversation

@u9g

@u9g u9g commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

livekit/protocol#1853 adds map<string, string> extra = 6 to livekit.agent.FunctionCall. Once livekit-agents runs with a livekit-protocol that has the field, encoding a function call whose extra holds anything other than a string raises:

TypeError: bad argument type for built-in operation

From SessionHost this comes back as session request run_input failed: bad argument type for built-in operation. Agent-simulation text runs on livekit-agents 1.7.1 with a livekit-protocol built from protocol main failed every simulation on turn 1 with this error.

Who hits it, and when

No released livekit-protocol on PyPI has the field yet. The latest is 1.1.27, and the staged v1.1.28 (livekit/python-sdks#823) still pins a protocol commit from before #1853. The first PyPI release built from a protocol that includes livekit/protocol@1e406a8 will have it. livekit-agents accepts that release (livekit-protocol>=1.1.21,<2), so upgrading is enough to trigger the bug. These calls carry non-string extra:

  • Every tool call through inference.LLM, for every model. The LiveKit Inference gateway puts extra_content.livekit (deployment and tier metadata) on every streamed tool call, and the client stores it as FunctionCall.extra = {"livekit": {...}} (inference/llm.py, generation.py). Gemini 3 calls also carry {"google": {"thought_signature": ...}}.
  • openai.LLM and the plugins built on its stream, when the endpoint returns extra_content. It reuses the same stream class (openai/llm.py).
  • Any tool that calls ctx.update(), with any LLM. The first update sets extra["__livekit_agents_tool_non_blocking"] = True (events.py).

The google.LLM plugin is not affected. It keeps thought signatures on the LLM instance, not in FunctionCall.extra (google/llm.py).

Everything that serializes a ChatItem fails on such a call:

  • SessionHost: run_input (remote_session.py), get_chat_history, get_agent_info and conversation_item_added events.
  • The LiveKit Cloud session report. _upload_session_report raises at the first such item in the transcript (traces.py). The rest of the transcript, the evaluations, the outcome and the audio recording are never uploaded. The only trace is the log line failed to upload the session report to LiveKit Cloud (job.py).

Root cause

#6756 changed encode_by_name to copy every field the proto declares off the source object by name. That way a new schema field is carried without editing a call site (_proto.py), and the FunctionCall branch depends on it entirely (L100-L101). llm.FunctionCall.extra is dict[str, Any] (chat_context.py), but the wire field is map<string, string> (proto), and encode_by_name passes the dict through unconverted.

ChatMessage.extra has the same types and never failed, because its call site converts each value with str(v) (L97). Both serializers that existed before #6756 did the same for that map.

Fix

The str(v) conversion moves into encode_by_name, next to the Timestamp conversion, so it applies to every map field. Every map these protos declare is map<string, string>. The ChatMessage call site no longer needs its own copy, so it is removed.

  • ChatMessage output is unchanged. It is the same str(v).
  • On released protocols, which don't declare FunctionCall.extra, nothing else changes.
  • With the field declared, a call carrying gateway metadata encodes as {"livekit": "{'inference_deployment': ...}"}. That is the same form ChatMessage.extra already uses for dict values.

Tests

  • test_map_values_are_stringified encodes a dict value and a bool value through encode_by_name. On main it raises the TypeError with any protocol version. With the fix it passes.
  • The coverage guard's saturated FunctionCall now has an extra with a dict value. With a protocol that declares the field, the guard exercises it. Run against main's _proto.py with a protocol-main build, function_call and test_saturated_items_cover_every_item_type fail with the TypeError. With the fix they pass.
  • An end-to-end check runs RemoteSession → SessionHost → AgentSession, with a fake LLM that returns a tool call carrying extra={"livekit": {...}}. On main: RuntimeError: session request run_input failed: bad argument type for built-in operation. With the fix, the turn returns function_call, function_call_output and message.

Results:

  • Unit suite (pytest --unit, locked livekit-protocol 1.1.21): 5359 passed, 13 failed. The same 13 fail on a pristine origin/main in this environment: 12 in test_vad_buffer_resize.py (numpy resize refcheck on Python 3.14) and 1 in test_google_credentials.py (it picks up local ADC).
  • Against a protocol-main build of livekit-protocol: tests/test_chat_item_proto_coverage.py and tests/test_session_host.py pass.
  • Lint and types: ruff check, ruff format --check and scripts/check_types.py are clean.

agents-js

agents-js is not affected. Its encoder lists FunctionCall fields explicitly and never sets extra (proto.ts), and its lockfile pins @livekit/protocol 1.50.4. @livekit/protocol 1.53.0 already declares the field. When agents-js moves to it, its coverage guard (test) will report function_call drops: extra. The fix there is the String(v) that its ChatMessage path already uses (proto.ts).

livekit/protocol#1853 declares FunctionCall.extra as map<string, string>.
encode_by_name copied llm.FunctionCall.extra (dict[str, Any]) into it
unconverted, so with that protocol any call whose extra holds a non-str
value raised "TypeError: bad argument type for built-in operation" on
every ChatItem serialization path. Every inference.LLM tool call carries
{"livekit": {...}}, and ctx.update() sets a bool flag.

ChatMessage.extra already got str() at its call site; the coercion now
applies to every map field in encode_by_name.
@u9g
u9g marked this pull request as ready for review October 19, 2026 21:53
@u9g
u9g requested a review from a team as a code owner October 19, 2026 21:53

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@theomonnom

Copy link
Copy Markdown
Member

Right I guess that's why I wasn't a fan of auto-forwarding args

@u9g
u9g merged commit 0338d0e into main Oct 19, 2026
26 checks passed
@u9g
u9g deleted the jason/proto-function-call-extra branch October 19, 2026 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants