Resumed terminal tool output cannot recover after a Session append failure

[FU-max-boop]

Please read this first

• I read the Session, RunState, approval, lifecycle, and output-guardrail docs.
• I searched open and closed issues/PRs for Session append,
  NextStepFinalOutput, and terminal tool-output persistence. The related work
  listed under Scope boundaries does not cover this terminal-finalization
  failure mode.

Describe the bug

After an approval-gated local tool completes, tool_use_behavior="stop_on_first_tool"
can make that tool output the terminal agent output. If the final client-managed
Session.add_items() append then fails, the exception is correctly propagated,
but the resulting RunState cannot recover or explicitly reject that accepted
terminal result.

At the failure boundary:

• the live runtime step is NextStepFinalOutput;
• JSON serialization records current_step = null;
• there is no pending_session_write;
• the passing output guardrail has already evaluated the original output, but
  its result is not retained in the failed state.

Retrying the same live or JSON-restored state therefore enters the model again,
runs the output guardrail again, and returns a new final output. The completed
tool effect is not repeated in this reproducer, but terminal output identity,
guardrail evidence, and durable Session history can diverge.

Debug information

• Agents SDK: latest release v0.22.0, and exact current
  main@a40ae9803e6b7a79faa246293f56adb100d5868b
• Exact src tree: 56eeb964e8924a6d0034b3e2289699c719f5450b
• Python: 3.13.15
• Operating system: macOS 26.5, arm64
• Model/provider: deterministic public ScriptedModel; no provider, API key,
  network request, or timing race
• Latest release: yes
• Reproducibility: consistent; focused repro and all 16 frozen matrix rows

The injected exception is:

RuntimeError: injected Session.add_items failure before commit

Repro steps

This focused fail-before-commit case is self-contained and also reproduces
against the installed openai-agents==0.22.0 package:

Reproducer

import asyncio
import copy
import json
from typing import Any

from openai.types.responses import (
    ResponseFunctionToolCall,
    ResponseOutputMessage,
    ResponseOutputText,
)
from agents import (
    Agent,
    GuardrailFunctionOutput,
    RunContextWrapper,
    Runner,
    function_tool,
    output_guardrail,
)
from agents.run import RunConfig
from agents.testing import ScriptedModel


class FailingSession:
    def __init__(self):
        self.session_id = "terminal-write-repro"
        self.items = []
        self.fail_next = False

    async def get_items(self, limit=None):
        items = copy.deepcopy(self.items)
        return items[-limit:] if limit is not None else items

    async def add_items(self, items):
        if self.fail_next:
            self.fail_next = False
            raise RuntimeError("injected Session.add_items failure before commit")
        self.items.extend(copy.deepcopy(items))

    async def pop_item(self):
        return self.items.pop() if self.items else None

    async def clear_session(self):
        self.items.clear()


async def main():
    effects = []
    guardrail_outputs = []

    @function_tool(needs_approval=True)
    async def charge(amount: int) -> str:
        effects.append(amount)
        return f"receipt-{amount}"

    @output_guardrail
    async def record_output(
        context: RunContextWrapper[Any], agent: Agent[Any], output: Any
    ) -> GuardrailFunctionOutput:
        guardrail_outputs.append(str(output))
        return GuardrailFunctionOutput(output_info=output, tripwire_triggered=False)

    model = ScriptedModel(
        steps=[
            [
                ResponseFunctionToolCall(
                    id="fc-1",
                    call_id="charge-1",
                    type="function_call",
                    name="charge",
                    arguments=json.dumps({"amount": 7}),
                )
            ],
            [
                ResponseOutputMessage(
                    id="msg-2",
                    type="message",
                    role="assistant",
                    status="completed",
                    content=[
                        ResponseOutputText(
                            type="output_text",
                            text="retry-final",
                            annotations=[],
                            logprobs=[],
                        )
                    ],
                )
            ],
        ]
    )
    agent = Agent(
        name="terminal-write-repro",
        model=model,
        tools=[charge],
        tool_use_behavior="stop_on_first_tool",
        output_guardrails=[record_output],
    )
    session = FailingSession()
    config = RunConfig(tracing_disabled=True)

    paused = await Runner.run(agent, "charge 7", session=session, run_config=config)
    state = paused.to_state()
    state.approve(paused.interruptions[0])

    session.fail_next = True
    try:
        await Runner.run(agent, state, session=session, run_config=config)
    except RuntimeError as error:
        print(type(error).__name__, str(error))

    serialized = state.to_json()
    print(
        {
            "runtime_step": type(state._current_step).__name__,
            "serialized_step": serialized.get("current_step"),
            "pending_session_write": serialized.get("pending_session_write"),
            "retained_output_guardrails": len(state._output_guardrail_results),
            "model_calls": len(model.calls),
            "effects": effects,
            "guardrail_outputs": guardrail_outputs,
        }
    )

    retried = await Runner.run(agent, state, session=session, run_config=config)
    print(
        {
            "retry_output": retried.final_output,
            "model_calls": len(model.calls),
            "effects": effects,
            "guardrail_outputs": guardrail_outputs,
        }
    )


asyncio.run(main())

Observed output:

RuntimeError injected Session.add_items failure before commit
{'runtime_step': 'NextStepFinalOutput', 'serialized_step': None,
 'pending_session_write': None, 'retained_output_guardrails': 0,
 'model_calls': 1, 'effects': [7], 'guardrail_outputs': ['receipt-7']}
{'retry_output': 'retry-final', 'model_calls': 2, 'effects': [7],
 'guardrail_outputs': ['receipt-7', 'retry-final']}

Frozen behavior matrix

I also froze a 16-row Cartesian product before inspecting results:

Dimension	Values
failing resume runner	sync, stream
retry runner	sync, stream
retry state	same live object, JSON round trip
append failure	fail before commit, commit then raise (lost acknowledgement)

The tool is charge(amount=7), its exact output is receipt-7, and a second
scripted response retry-final exists only to expose a new model call.

Observation	Rows
injected append failure propagated	16 / 16
retry completed	16 / 16
tool effect and tool start/end hooks remained exactly once	16 / 16
first output guardrail evaluated receipt-7 before failure	16 / 16
pending terminal Session checkpoint retained after failure	0 / 16
first passing output-guardrail result retained after failure	0 / 16
original receipt-7 recovered without another model call	0 / 16
retry made a second model call and returned retry-final	16 / 16
durable Session contained one exact call/output pair after retry	8 / 16
public replay and retry model input contained one exact pair	16 / 16

The eight durable-Session successes are only the append-then-raise rows: the
pair had already committed before the simulated lost acknowledgement. Those
rows still make a second model call because no terminal checkpoint recognizes
the committed batch as the already accepted result. In all eight
fail-before-commit rows, the Session lacks the tool call/output even after
retry, while replay/model input contain the pair and the Session contains the
later assistant message.

Relevant control flow

At the pinned commit:

• the resumed state is updated, but the pre-guardrail save is skipped for
  NextStepFinalOutput in
  run.py#L1132-L1152;
• output guardrails run before the final append and result construction in
  run.py#L1242-L1403;
• resumed pending-write ownership is limited to NextStepRunAgain and
  NextStepInterruption in
  session_persistence.py#L717-L747;
• RunState declares only those two resumable steps and serializes a terminal
  runtime step as null in
  run_state.py#L846-L874 and
  run_state.py#L1977-L2011;
• the streamed terminal path has the same guardrail-then-save boundary in
  run_loop.py#L491-L649.

Expected behavior / design question

I do not want to force terminal finalization into NextStepRunAgain: the
matrix shows that doing so re-enters the model and changes an already accepted
terminal result.

Would maintainers prefer one of these contracts?

1. Recoverable terminal continuation. Persist enough state to own the
   pending terminal batch, accepted final output, and completed guardrail
   results; reconcile the same Session before any model call. An unchanged
   fail-before tail can append and finalize, and a provable lost acknowledgement
   can finalize without repeating model/tool/hooks/guardrails.
2. Explicitly non-resumable terminal failure. Preserve an actionable,
   serializable recovery payload, but reject subsequent Runner.run*() calls
   before model execution instead of silently treating the state as run-again.

For either choice, changed, partial, or concurrently modified Session history
should fail closed rather than guess. Sync/stream and live/JSON recovery should
have the same contract.

I can prepare the focused implementation and regression matrix after the
contract is selected.

Scope boundaries

• This is narrower than distributed exactly-once execution. It does not cover
  process death before state capture, independent concurrent state copies,
  non-atomic partial appends, or external tools without idempotency.
• fix(sessions): recover failed resumed Session writes before model calls #4630 covers NextStepRunAgain; fix(sessions): recover failed resumed Session writes on a renewed interruption #4650 covers renewed interruption; Resumed approval followed by handoff can leave Session history without tool outputs after append failure #4685
  covers handoff. This issue is only terminal NextStepFinalOutput.
• feat: add approval override argument parity for RunState resumes #2695 covers approval argument overrides, not terminal Session ownership.
• Output guardrail trip leaves orphaned tool call on streamed resume #4125 and its fixes cover output-guardrail tripwire history, not recovery
  after a successful guardrail followed by append failure.
• The broader closed fix: retry resumed Session writes before model calls #4621 was superseded by the narrower maintainer change in
  fix(sessions): recover failed resumed Session writes before model calls #4630; this report does not reuse that implementation.

Evidence integrity

The full provider-neutral harness was run twice with byte-identical JSON and
JSONL results. Its standard-library verifier reconstructs the pinned Git tree,
checks all 315 source files and every trace/state/Session/model cross-field
relationship, and rejects re-sealed source/trace/state/package-boundary
mutations in negative tests.

• source snapshot SHA-256:
  5779a3d6d2e2e94b078d85785b0886d470636cd7513b4946d35c46340f448417
• baseline JSON SHA-256:
  5f4102aef87b7875f61562ac6f649437fc711edfb29f92ae5815e1665ff757ae
• baseline JSONL SHA-256:
  49e7bca13ca4d1fd155ab34b3ca38a8734f9c0ba5d06ffd06e11100ae20d27c1
• external manifest root SHA-256:
  f53612af88d0180b55bfb58ea5fe12cf8db251b583c9050e8b655010d28aad46

[samovermind]

Because NextStepFinalOutput skips the pre-guardrail checkpoint and has no pending_session_write retrying forces a new model turn instead of settling the accepted tool output. For regression testing state divergence under persistence faults trace replay helps snapshot and validate recovery idempotency.

[FU-max-boop]

One material addition from the frozen hook trace: this boundary can repeat application lifecycle hooks, not only model/guardrail observations.

Across all 16 rows, the failing attempt had already completed exactly one on_agent_end(..., "receipt-7") before Session.add_items() raised. Retrying then produced a second on_agent_start and a second on_agent_end(..., "retry-final") in 16/16 rows. The tool start/end hooks and tool effect stayed exactly once, but an application can put its own local side effect in an agent lifecycle hook, so the retry is not side-effect neutral.

That adds one requirement to either contract above:

• a recoverable terminal continuation must remember that final-output hooks already completed and must finalize the original result without replaying agent start/end hooks; or
• an explicitly non-resumable terminal state must reject before any new agent/model/tool/guardrail hook runs.

The current frozen result is therefore:

failure: agent_end(receipt-7) = 16/16
retry:   agent_start count 2  = 16/16
retry:   agent_end count 2    = 16/16

I will include hook-count and hook-order assertions in the regression matrix after the persistence contract is selected.

[FU-max-boop]

A second implementation boundary emerged from the RunState codec audit: recoverable terminal continuation should not initially claim JSON-equivalent recovery for arbitrary NextStepFinalOutput.output: Any.

The existing generic codec can fall back to json.dumps(..., default=str) and then str(value) (pinned source). That is useful for diagnostic payloads, but it is not an identity-preserving final-output codec: a custom object can silently change type and value across a JSON round trip.

For contract 1, my proposed initial support boundary is therefore:

• recover only the terminal tool-output case where Agent.output_type is None or str and the runtime already normalizes the accepted output to str (pinned source);
• persist an explicit terminal-unrecoverable marker and fail closed before any new agent/model/tool/guardrail/hook for structured or custom Any output;
• if structured recovery is added later, reconstruct only through the current trusted AgentOutputSchema.validate_json() contract, never pickle, dynamic type import, or a class identity taken from serialized state.

This keeps live/JSON behavior honest and makes the 16-row plain-text reproducer recoverable without turning a narrow Session repair into a new arbitrary-object serialization promise.

[seratch]

Resolved by #4725

[FU-max-boop]

Thanks for the pointer. I reran the issue’s focused reproducer against both #4725’s merge commit 287594c6ce2b723de162c6f77d7e60c2c3e686b8 and current main@f1a806ad35071053f5248b38b58b8c0c40f67350; the terminal NextStepFinalOutput case still reproduces. After the injected fail-before-commit append, current_step serializes as null, pending_session_write is absent, and the retained output-guardrail result count is zero. Retrying makes a second model call and returns retry-final. #4725 correctly fixes the NextStepHandoff path, but its diff does not cover NextStepFinalOutput. Could we keep #4690 open for that remaining terminal path? I can prepare the focused implementation and regression matrix once the intended recovery contract is confirmed.