Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions python/packages/ag-ui/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,27 @@ AG-UI protocol integration for building agent UIs with the AG-UI standard.
`state` entry, so conversation continuation is unreachable from client input by construction; keep it that way.
- `confirm_changes` snapshot cleanup resolves the synthetic confirmation back to its original `function_call_id`;
it must never concatenate unrelated tool results or record accepted changes without a matching real result.
- Configured stateless agent Thread Snapshot stores persist after function/MCP tool-result and approval safe points,
then persist the terminal state. Never treat a yielded `finish_reason` or a following metadata update as successful
model-turn finalization: inner finalizers and result hooks may still reject that output. Save only after the complete
result/approval update and lifecycle side effects are applied, and project only completed call/result groups and
current approval controls into the intermediate snapshot. Sibling text, reasoning, and unrelated pending calls remain
terminal-only. Do not write every text delta or schedule unordered background snapshot writes. Service-session and
workflow Thread Snapshots retain terminal-save cadence so replayable messages cannot advance without matching
provider continuation state; workflow checkpoints own incremental workflow runtime state.
- Disconnect-safe execution is endpoint-owned and opt-in through
`add_agent_framework_fastapi_endpoint(detached_runs=True)`. Keep its producer queue bounded, retain and observe
producer/drainer tasks, and let the producer own final snapshot/checkpoint/approval persistence after the SSE reader
disconnects. Bound endpoint-wide producer admission and cancel abandoned producers after the configured timeout.
This mode discards unread events; it is not a resumable event log.
- While a detached mutation is active, the endpoint may serve an empty Snapshot Hydrate Request but must reject another
mutation for the same `(Snapshot Scope, threadId)` with HTTP 409. Classify hydration once and keep it on the direct
response path so it bypasses detached admission and producer wrapping. Endpoint admission, agent hydration, and
workflow hydration must all use `_run_common._is_snapshot_hydration_request`; pass workflow checkpoint capability
explicitly so checkpoint resumes never become hydration. The guard is process-local and does not replace cross-replica
coordination.
- Detached runs may outlive FastAPI request-scoped disposable resources. Resolve authorization and Snapshot Scope before
spawning the producer, and do not rely on request-owned clients or sessions remaining open after disconnect.
- SSE keepalive is endpoint-owned transport behavior configured through
`add_agent_framework_fastapi_endpoint(keepalive_seconds=...)`. It emits SSE comments only; do not add `PING`,
`HEARTBEAT`, or `KEEPALIVE` AG-UI events, and do not add runner-level keepalive settings.
Expand Down
52 changes: 49 additions & 3 deletions python/packages/ag-ui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,6 +389,14 @@ add_agent_framework_fastapi_endpoint(
)
```

Configured stateless agent snapshot stores are updated after function/MCP tool-result batches and approval safe points,
then written once more with the terminal run state. These boundaries capture completed model/tool rounds without
persisting sibling text or reasoning whose stream finalizer may still reject it. Intermediate snapshots project only
completed call/result groups and current approval controls; the terminal snapshot retains the complete finalized
output. Service-session snapshots retain terminal-save cadence so replayable messages cannot advance without their
matching provider continuation state. Workflow Thread Snapshots also keep their terminal-save cadence; workflow
checkpointing remains the mechanism for incremental workflow runtime state.

A frontend can then hydrate the latest stored snapshot for the scoped thread:

```json
Expand All @@ -398,6 +406,42 @@ A frontend can then hydrate the latest stored snapshot for the scoped thread:
}
```

### Disconnect-safe runs

By default, AG-UI execution remains attached to the SSE response: disconnecting the client cancels the response
generator and can stop the run. Set `detached_runs=True` when a finite agent or workflow run must continue through
snapshot, checkpoint, and approval-state finalization after the HTTP reader disconnects:

```python
add_agent_framework_fastapi_endpoint(
app,
agent,
"/",
snapshot_store=snapshot_store,
snapshot_scope_resolver=resolve_snapshot_scope,
detached_runs=True,
max_detached_runs=32,
detached_run_timeout_seconds=3600,
)
```

Detached execution uses a bounded endpoint-owned producer queue. While a detached mutating request is active, another
mutating request for the same `(Snapshot Scope, threadId)` returns HTTP 409; an empty snapshot Hydrate Request remains
allowed, bypasses detached producer capacity, and returns the latest committed safe point. Equal Thread ids in
different Snapshot Scopes remain independent.
Each endpoint registration retains at most `max_detached_runs` producers (32 by default); requests beyond that limit
receive HTTP 503. After a reader disconnects or never starts, `detached_run_timeout_seconds` cancels a stalled producer
and releases its capacity (one hour by default).

This option does not add resumable event replay. Events emitted while no client is attached are consumed and discarded,
so a reconnecting client recovers from Thread Snapshots rather than resuming the original SSE position. Applications
that require replay of every in-flight event must provide their own authenticated event log and resume route with
retention and cross-replica semantics appropriate to their deployment.

FastAPI dependencies and other request-scoped resources may be released after the disconnected response ends. Resolve
authorization, Snapshot Scope, and other durable values before the run starts; detached tools and providers must not
retain request-owned clients or sessions that are expected to close with the HTTP request.

Endpoint configuration requires `snapshot_scope_resolver` whenever a snapshot store is configured, including when
the store is already set on a pre-wrapped `AgentFrameworkAgent` or `AgentFrameworkWorkflow`. The resolver returns
the application-defined Snapshot Scope used with the AG-UI Thread id as the storage key. The endpoint also derives
Expand Down Expand Up @@ -468,9 +512,11 @@ encryption, integrity protection, access control, retention, audit, data residen
custom stores remain source-compatible because `session_state` is optional, but they provide Session State Continuity
only when they round-trip that field unchanged with the rest of the snapshot.

The supported consistency model is one active run per `(Snapshot Scope, threadId)`. Concurrent writes to the same
scoped thread remain last-writer-wins. Applications that require stronger consistency must serialize those runs using
coordination appropriate to their deployment; a process-local lock does not provide distributed consistency.
The supported consistency model is one active run per `(Snapshot Scope, threadId)`. With `detached_runs=True`, one
registered endpoint enforces that rule in-process by rejecting concurrent mutations while allowing hydration.
Coordination is not shared across endpoint registrations, workers, or replicas; applications that require distributed
serialization must provide it using infrastructure appropriate to their deployment. Without detached execution,
concurrent writes to the same scoped thread remain last-writer-wins.

## Architecture

Expand Down
Loading
Loading