Repository navigation
feat(provider): exact signed-thinking capture, persistence and replay (incl. Bedrock) #4653
Copy link
Copy link
Closed
Copy link
Labels
area/providers/anthropicFor features/issues/fixes related to the usage of Anthropic modelsFor features/issues/fixes related to the usage of Anthropic modelsarea/providers/bedrockAWS Bedrock provider supportAWS Bedrock provider supportarea/sessionsFor features/issues/fixes related to session lifecycle (resume, persistence, export)For features/issues/fixes related to session lifecycle (resume, persistence, export)
Description
Goal
Capture, persist and replay signed thinking exactly, on all three transports. If the stored state can't be trusted, refuse to continue instead of replaying a rebuilt version.
Why
- Bedrock flattens reasoning into text + signature fields (adapter.go) and rebuilds the blocks later. It treats a signature without text as redacted content (convert.go), but a signed thinking block can legitimately have empty text. Order between thinking, text and tool calls is also rebuilt rather than preserved.
thinkingBindingBehaviorpicksdrop_blockautomatically when the primary or any fallback checks thinking prefixes (thinking_binding.go). The preserved thinking docs makeerrorthe default and describedrop_blockas a stopgap that can cost extra output tokens. Switching the default is a deliberate client policy change.- The Anthropic client already stores raw response blocks in
ProviderStateand replays them when they still describe the message (response_content.go). Extend that mechanism; don't create a second one.
Scope
- Keep blocks in order with their index: thinking (including signed-empty), redacted thinking bytes, text, tool use IDs/inputs and tool results. No "last signature wins".
- Extend
ProviderStateadditively with a version and the serving model. Unknown versions, incomplete or interrupted streams are never replayed as if they were verified. - Bedrock: store and replay native reasoning blocks instead of rebuilding them from flattened fields.
Haiku 5.5 exceptions: dropped (see Decisions).Make: dropped (see Decisions).thinking_prefix_mismatch: errorthe defaultdrop_blockstays the default for prefix-checking 5.x models;errorremains available as an explicit opt-in.drop_blocknever deletes the stored original.
Delivery plan and decisions (2026-10-20)
A code survey found more than this issue first described, so it ships as three stacked PRs:
- (a) Bedrock native reasoning blocks — merged in fix(bedrock): capture and replay native Claude reasoning blocks #4666 (follow-up perf fix: perf(bedrock): accumulate native block text in a strings.Builder #4667). Capture, persist and replay Converse content blocks exactly, in
ProviderState{provider: "amazon-bedrock"}. This fixes redacted bytes being corrupted on save, multiple thinking blocks collapsing to the last signature, signed-empty thinking being replayed as redacted, and lost ordering between thinking and tool calls. - (b) Raw-state contract — in review as feat(provider): versioned raw state and explicit policy for unreplayable thinking #4668. Add a version and the serving model to
ProviderState, seal it after generated media is attached, and apply the not-replayable policy below. (c)— dropped (see Decisions).errordefault
Decisions:
- Raw state that can no longer be replayed (edited, redacted by a hook, media stripped): omit that earlier thinking from the outgoing request with a warning. Inside the active tool-use loop, where the API requires the thinking, refuse with a clear error instead. Today the Anthropic fallback drops it silently.
errordefault: dropped (2026-10-20). A survey of the runtime found that default features change the request prefix mid-session: automatic session compaction (on by default, fires at 90% of the context window, often mid tool loop, and keeps a tail that contains signed thinking) and agent handoffs (swap system prompt and tools mid-loop); plus, depending on config, MCP servers starting late or reconnecting, tools appearing,num_history_itemstrimming andadd_date/ prompt-file changes. Undererrorthe first mismatch is permanent: every later request replays the same thinking and gets the same 400, including after a new user turn. Local stripping cannot help inside the active tool loop, where the API requires the thinking.drop_blocktherefore stays the default. A possible future improvement is provider-side prefix tracking (fingerprint the prefix each turn was produced under, strip stale thinking and switch only affected requests todrop_block); measure first how often "Anthropic dropped invalidated thinking" is logged in practice.- Haiku 5.5 exceptions: dropped (2026-10-20). The omission code from fix: make Haiku 5.5 requests compatible across Claude providers #4553 was never merged, and the Haiku 5.5 migration guide documents no such requirement: earlier thinking blocks "stay in context and count as input tokens". The guide also notes that Haiku 5.5 thinking blocks only work in the account that produced them; from another account the API drops them and the request still succeeds.
Acceptance criteria
- Round-trip tests per transport: stream →
ProviderState→ SQLite and JSON → new runtime → continue an unchanged tool conversation; the replayed request matches the original blocks byte for byte. - Signed-empty thinking, redacted thinking and interleaved thinking/text/tool calls survive the round trip.
- Edited or trimmed history, an imported session and an unknown state version lead to a clear refusal or a documented fallback, never to a silently rebuilt replay.
-
Default: dropped, see Decisions.erroron prefix-checking 5.x modelsdrop_blockremains the documented default, with dropped blocks logged. - Older binaries reading the new state: the limitation is documented (no promise of transparent downgrade).
-
task build,task test,task lint, andgo test -raceon the touched packages pass.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/providers/anthropicFor features/issues/fixes related to the usage of Anthropic modelsFor features/issues/fixes related to the usage of Anthropic modelsarea/providers/bedrockAWS Bedrock provider supportAWS Bedrock provider supportarea/sessionsFor features/issues/fixes related to session lifecycle (resume, persistence, export)For features/issues/fixes related to session lifecycle (resume, persistence, export)
Activity