Skip to content

fix(llm): tolerate unparseable stored tool-call arguments in Anthropic/Google/AWS formatters#6321

Open
PranavMishra28 wants to merge 1 commit into
livekit:mainfrom
PranavMishra28:fix/provider-format-unparseable-tool-args
Open

fix(llm): tolerate unparseable stored tool-call arguments in Anthropic/Google/AWS formatters#6321
PranavMishra28 wants to merge 1 commit into
livekit:mainfrom
PranavMishra28:fix/provider-format-unparseable-tool-args

Conversation

@PranavMishra28

Copy link
Copy Markdown

This PR contains:

  • New features
  • Changes to dev-tools e.g. CI config / github tooling
  • Docs
  • Bug fixes
  • Code refactor

What is the current behavior? (You can also link to an open issue here)

Fixes #6308.

FunctionCall.arguments holds the model's raw tool-call output and is only canonicalized to valid JSON when it parses successfully. When it can't be parsed — an open-weight model's output that json_repair can't recover (the call fails as a ToolError and the raw string persists), or history built outside the execution path (ChatContext.from_dict, a custom llm_node, restored session state) — the raw string stays in chat history verbatim.

The Anthropic, Google, and AWS chat-context formatters call json.loads(msg.arguments) directly in to_chat_ctx. A later, healthy turn that formats that history then raises json.JSONDecodeError and the whole request aborts. OpenAI and Mistral pass arguments through as an opaque string, so they don't hit this.

from livekit.agents.llm import ChatContext, FunctionCall, FunctionCallOutput

ctx = ChatContext.empty()
ctx.insert(FunctionCall(call_id="c1", name="lookup", arguments="not-a-json-object"))
ctx.insert(FunctionCallOutput(call_id="c1", name="lookup", output="tool error", is_error=True))
ctx.to_provider_format(format="anthropic")   # raises json.JSONDecodeError (same for "google" / "aws")

What is the new behavior?

Adds a shared parse_tool_call_arguments() helper in _provider_format/utils.py that returns the parsed dict when the stored arguments are a JSON object, and otherwise logs a warning and falls back to an empty object {}. The three JSON-object formatters call it instead of json.loads(...).

{} is preferred over a best-effort repair: the historical tool call already produced its output, so fabricating arguments the model never sent risks feeding wrong data back to the provider. A tool call's arguments are always a named-parameter object, so JSON that parses to a non-object (array, scalar, null) degrades the same way.

Tests added in tests/test_chat_ctx.py cover all three providers against unparseable arguments, a valid-args passthrough regression, and empty/non-object arguments. The unparseable/non-object cases fail on main (raise JSONDecodeError) and pass with this change.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No. Valid arguments format exactly as before; only previously-crashing inputs change behavior — from a raised JSONDecodeError to an empty-args object plus a warning log.

Other information:

Developed with Claude Code; reviewed and tested by Pranav before marking ready for review.

@CLAassistant

CLAassistant commented Jul 5, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@PranavMishra28 PranavMishra28 marked this pull request as ready for review July 5, 2026 01:32
@PranavMishra28 PranavMishra28 requested a review from a team as a code owner July 5, 2026 01:32
devin-ai-integration[bot]

This comment was marked as resolved.

… formats

The Anthropic, Google, and AWS chat-context formatters call
json.loads(msg.arguments) directly. FunctionCall.arguments is only
canonicalized to valid JSON when the model output parses; unparseable output
(unrecoverable open-weight calls, or history restored via ChatContext.from_dict)
persists verbatim, so formatting that history on a later turn raised
json.JSONDecodeError and aborted the whole request.

Add a shared parse_tool_call_arguments() helper that falls back to an empty
object (with a warning) when the arguments aren't a JSON object, and use it in
the three formatters. OpenAI/Mistral already pass arguments through as an opaque
string and are unaffected.

Fixes livekit#6308

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PranavMishra28 PranavMishra28 force-pushed the fix/provider-format-unparseable-tool-args branch from eaa86ee to 3e886e8 Compare July 7, 2026 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

to_provider_format crashes with JSONDecodeError when replaying a tool call whose arguments never parsed (Anthropic / Google / AWS)

2 participants