Skip to content

test(sessions): consolidate persistence fault scenarios behind an internal harness #4677

Description

@FU-max-boop

Describe the feature

Recent Session recovery work in #4630, #4650, and #4654 exercises the same persistence boundary through two private list-backed Session doubles:

  • tests/test_run_impl_resume_paths.py::_FailingResumeSession
  • tests/test_runner_guardrail_resume.py::_ResumeWriteFailureSession

Both need to distinguish an append that raises before mutation from a commit whose acknowledgement is lost. The surrounding regression matrices also repeat the same live/JSON and streaming/non-streaming recovery axes. Keeping those mechanics separate makes it easy for the durability matrix to drift while each individual test still looks reasonable.

I propose a small internal-only fault-injection harness under tests/utils/. It should not be exported from agents.testing.

The harness should provide:

  • a list-backed Session with explicit initial history and session ID;
  • one-shot add_items() outcomes for success, raise-before-commit, and commit-then-raise;
  • detached records of attempted append batches;
  • optional support for the existing partial-write or controlled-blocking cases only where current tests already require them; and
  • a small immutable case description for run mode and state transport, without encoding expected SDK results.

Runner invocation and caller-visible assertions should remain in the owning test modules. The shared helper should model only Session-boundary mechanics, not become the oracle for recovery behavior.

Acceptance criteria

  • Existing recovery coverage remains explicit for:
    • non-streaming and streaming failure/recovery;
    • same-mode and cross-mode recovery;
    • live and JSON-restored RunState;
    • raise-before-commit and commit-then-raise outcomes.
  • Exactly-once tool effects, Session history, model input, guardrail results, hooks, and public replay continue to be asserted at the public Runner boundary.
  • No runtime package, public import, Session protocol, documentation, or released API contract changes.
  • Specialized cancellation, concurrency, compaction, and real-backend tests may keep local doubles when their lifecycle cannot be represented faithfully.
  • Coordinate any change to tests/utils/simple_session.py with feat: add approval override argument parity for RunState resumes #2695.

Non-goals

  • Adding a public agents.testing.ScriptedSession.
  • Claiming atomicity, CAS, distributed exactly-once behavior, or partial-write semantics for real Session backends.
  • Changing Session recovery behavior.
  • Replacing backend-specific integration tests.

I searched open and closed issues and pull requests for ScriptedSession, scripted Session testing, failing Session helpers, and Session scenario harnesses and did not find a duplicate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions