Skip to content

fix(query): recover from OpenAI-compat length truncations; forward non-interactive to headless - #830

Merged
ericleepi314 merged 1 commit into
mainfrom
fix/openai-compat-length-recovery
Aug 12, 2026
Merged

fix(query): recover from OpenAI-compat length truncations; forward non-interactive to headless#830
ericleepi314 merged 1 commit into
mainfrom
fix/openai-compat-length-recovery

Conversation

@ericleepi314

Copy link
Copy Markdown
Collaborator

Two model-agnostic fixes extracted from the DeepSeek-flash harness work (feat/deepseek-flash-harness) — the parts that benefit every provider, with all DeepSeek-gated and eval-specific changes left behind.

1. Recover from OpenAI-compat finish_reason="length" truncations

Every stop_reason consumer in the query loop keys on the internal (Anthropic) word "max_tokens" — the withheld-content escalation lane (→ 64K retry), the "resume, break into smaller pieces" recovery nudge, and _is_withheld_max_output_tokens. None of them matched the OpenAI wire's "length", so a truncated response from any OpenAI-compatible provider (openai, openrouter, zai, deepseek, …) fell through as a normal end-of-turn: no escalation, no resume nudge, and a headless run ended with a silently-clipped answer.

_call_model_sync now maps finish_reason == "length"stop_reason = "max_tokens" (Anthropic already emits "max_tokens", so this only normalizes the other wire).

The _api_error = "max_output_tokens" tag is now applied only to tool-free truncations. A truncation that still carries a surviving tool call must be yielded — headless persistence (on_message) is the only route into the saved session — not withheld while its tools execute anyway. This also closes the same latent leak on the Anthropic wire, where max_tokens + tool_use has always been possible.

2. Forward non-interactive mode to the headless prompt

build_effective_system_prompt (the headless/TUI cutover builder) never forwarded options.is_non_interactive_session into build_full_system_prompt_blocks, so the headless (-p) model never received the # Non-Interactive Mode section and behaved as if a human would reply. Now forwarded; interactive callers (TUI bridge, agent-server) leave the flag unset and are unchanged. Model-agnostic.

Tests

  • tests/test_openai_length_stop_reason.py — the 64K escalation fires on finish_reason="length"; a "stop" turn is untouched; a truncation carrying a surviving tool call is yielded (not withheld).
  • tests/test_headless_non_interactive_section.py — section present when the flag is set, absent otherwise, null-safe when tool_context has no .options.

Verification

  • Focused + wider sweep: 2345 passed, 9 skipped, 0 failures.
  • Existing max_tokens escalation tests stay green (they use finish_reason="max_tokens" + tool_uses=None, so the tool-free guard still fires).
  • Only "length" is remapped — stop / tool_calls / content_filter pass through unchanged.

🤖 Generated with Claude Code

…n-interactive to headless

Two model-agnostic fixes ported from the DeepSeek-flash harness work
(feat/deepseek-flash-harness), benefiting every provider:

1. finish_reason="length" -> stop_reason "max_tokens" normalization.
   Every stop_reason consumer in the query loop keys on the internal
   (Anthropic) "max_tokens" vocabulary — the withheld-content escalation
   lane, the resume-recovery nudge, _is_withheld_max_output_tokens — and
   none matched the OpenAI wire's "length". So a truncated response from
   ANY OpenAI-compatible provider (openai, openrouter, zai, deepseek, …)
   fell through as a normal end-of-turn: no escalation to 64K, no resume
   nudge, and a headless run ended with a silently-clipped answer. Anthropic
   already emits "max_tokens"; this maps the other wire onto the same word.
   The tag is now applied ONLY to tool-free truncations — a truncation that
   still carries a surviving tool call must be yielded (headless persistence
   is the only route into the saved session) rather than withheld while its
   tools execute; this also closes the same latent leak on the Anthropic
   wire, where max_tokens + tool_use has always been possible.

2. build_effective_system_prompt now forwards is_non_interactive_session
   into build_full_system_prompt_blocks, so the headless (-p) path actually
   receives the "# Non-Interactive Mode" section (it was silently omitted).
   Interactive callers leave the flag unset and are unchanged.

Tests: test_openai_length_stop_reason.py (escalation fires on "length";
tool-carrying truncation is yielded not withheld) and
test_headless_non_interactive_section.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ericleepi314
ericleepi314 merged commit 5441d5d into main Aug 12, 2026
3 of 6 checks passed
@ericleepi314
ericleepi314 deleted the fix/openai-compat-length-recovery branch August 12, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant