Skip to content

feat(provider): exact signed-thinking capture, persistence and replay (incl. Bedrock) #4653

Closed

Description

@aheritier

Goal

Capture, persist and replay signed thinking exactly, on all three transports. If the stored state can't be trusted, refuse to continue instead of replaying a rebuilt version.

Why

  • Bedrock flattens reasoning into text + signature fields (adapter.go) and rebuilds the blocks later. It treats a signature without text as redacted content (convert.go), but a signed thinking block can legitimately have empty text. Order between thinking, text and tool calls is also rebuilt rather than preserved.
  • thinkingBindingBehavior picks drop_block automatically when the primary or any fallback checks thinking prefixes (thinking_binding.go). The preserved thinking docs make error the default and describe drop_block as a stopgap that can cost extra output tokens. Switching the default is a deliberate client policy change.
  • The Anthropic client already stores raw response blocks in ProviderState and replays them when they still describe the message (response_content.go). Extend that mechanism; don't create a second one.

Scope

  • Keep blocks in order with their index: thinking (including signed-empty), redacted thinking bytes, text, tool use IDs/inputs and tool results. No "last signature wins".
  • Extend ProviderState additively with a version and the serving model. Unknown versions, incomplete or interrupted streams are never replayed as if they were verified.
  • Bedrock: store and replay native reasoning blocks instead of rebuilding them from flattened fields.
  • Haiku 5.5 exceptions: dropped (see Decisions).
  • Make thinking_prefix_mismatch: error the default: dropped (see Decisions). drop_block stays the default for prefix-checking 5.x models; error remains available as an explicit opt-in.
  • drop_block never deletes the stored original.

Delivery plan and decisions (2026-10-20)

A code survey found more than this issue first described, so it ships as three stacked PRs:

  1. (a) Bedrock native reasoning blocks — merged in fix(bedrock): capture and replay native Claude reasoning blocks #4666 (follow-up perf fix: perf(bedrock): accumulate native block text in a strings.Builder #4667). Capture, persist and replay Converse content blocks exactly, in ProviderState{provider: "amazon-bedrock"}. This fixes redacted bytes being corrupted on save, multiple thinking blocks collapsing to the last signature, signed-empty thinking being replayed as redacted, and lost ordering between thinking and tool calls.
  2. (b) Raw-state contract — in review as feat(provider): versioned raw state and explicit policy for unreplayable thinking #4668. Add a version and the serving model to ProviderState, seal it after generated media is attached, and apply the not-replayable policy below.
  3. (c) error default — dropped (see Decisions).

Decisions:

  • Raw state that can no longer be replayed (edited, redacted by a hook, media stripped): omit that earlier thinking from the outgoing request with a warning. Inside the active tool-use loop, where the API requires the thinking, refuse with a clear error instead. Today the Anthropic fallback drops it silently.
  • error default: dropped (2026-10-20). A survey of the runtime found that default features change the request prefix mid-session: automatic session compaction (on by default, fires at 90% of the context window, often mid tool loop, and keeps a tail that contains signed thinking) and agent handoffs (swap system prompt and tools mid-loop); plus, depending on config, MCP servers starting late or reconnecting, tools appearing, num_history_items trimming and add_date / prompt-file changes. Under error the first mismatch is permanent: every later request replays the same thinking and gets the same 400, including after a new user turn. Local stripping cannot help inside the active tool loop, where the API requires the thinking. drop_block therefore stays the default. A possible future improvement is provider-side prefix tracking (fingerprint the prefix each turn was produced under, strip stale thinking and switch only affected requests to drop_block); measure first how often "Anthropic dropped invalidated thinking" is logged in practice.
  • Haiku 5.5 exceptions: dropped (2026-10-20). The omission code from fix: make Haiku 5.5 requests compatible across Claude providers #4553 was never merged, and the Haiku 5.5 migration guide documents no such requirement: earlier thinking blocks "stay in context and count as input tokens". The guide also notes that Haiku 5.5 thinking blocks only work in the account that produced them; from another account the API drops them and the request still succeeds.

Acceptance criteria

  • Round-trip tests per transport: stream → ProviderState → SQLite and JSON → new runtime → continue an unchanged tool conversation; the replayed request matches the original blocks byte for byte.
  • Signed-empty thinking, redacted thinking and interleaved thinking/text/tool calls survive the round trip.
  • Edited or trimmed history, an imported session and an unknown state version lead to a clear refusal or a documented fallback, never to a silently rebuilt replay.
  • Default error on prefix-checking 5.x models: dropped, see Decisions. drop_block remains the documented default, with dropped blocks logged.
  • Older binaries reading the new state: the limitation is documented (no promise of transparent downgrade).
  • task build, task test, task lint, and go test -race on the touched packages pass.

Parent: #4583 · Depends on #4651

Metadata

Metadata

Assignees

Labels

area/providers/anthropicFor features/issues/fixes related to the usage of Anthropic modelsarea/providers/bedrockAWS Bedrock provider supportarea/sessionsFor features/issues/fixes related to session lifecycle (resume, persistence, export)

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions