Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/brave-otters-recover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/sdk": patch
---

Fixed a chat agent hanging after an interrupted turn: when a run was killed mid-answer (out of memory, crash, or eviction) and only the one message it was answering was still outstanding, the new run never replied to it. That message is now re-answered on the new run.
11 changes: 9 additions & 2 deletions docs/ai-chat/patterns/recovery-boot.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ On a continuation boot, the runtime reads:
- **`session.out` tail past the snapshot cursor** — closed assistant turns plus, optionally, a `partialAssistant` (the trailing message whose stream never received a `finish` chunk). `cleanupAbortedParts` has already stripped streaming-in-progress fragments.
- **`session.in` tail past the last `turn-complete` cursor** — user messages the dead run hadn't acknowledged.

If both `partialAssistant` and `inFlightUsers` are non-empty, the runtime splices `[firstInFlightUser, partialAssistant]` onto the chain. The remaining in-flight users dispatch as fresh turns. The model sees:
If there's a `partialAssistant` and two or more `inFlightUsers`, the runtime splices `[firstInFlightUser, partialAssistant]` onto the chain. The remaining in-flight users dispatch as fresh turns. The model sees:

```
[ ...settledMessages, // chain through the last completed turn
Expand Down Expand Up @@ -138,10 +138,17 @@ type RecoveryBootResult<TUIM extends UIMessage = UIMessage> = {
};
```

- **`chain`** — replaces the seed chain. Defaults to `[...settledMessages, firstInFlightUser, partialAssistant]` when both partial and in-flight users exist, otherwise `settledMessages` alone.
- **`chain`** — replaces the seed chain. Defaults to `[...settledMessages, firstInFlightUser, partialAssistant]` when there's a partial **and two or more** in-flight users, otherwise `settledMessages` alone.
- **`recoveredTurns`** — user messages to dispatch as fresh turns after the chain is restored. Defaults to `inFlightUsers.slice(1)` when the smart default consumed the first user, otherwise `inFlightUsers`.
- **`beforeBoot`** — runs after the writer flushes and before the first recovered turn fires. Use for blocking persistence (write the partial to your DB so a later turn can reference it). Errors bubble — wrap your own try/catch if you want to soft-fail.

<Note>
The splice needs a follow-up user to answer, so it only applies with two or
more in-flight users. With exactly one — the plain OOM or crash-mid-answer
case — the orphan partial is dropped and that single user is re-dispatched as
a fresh turn, so the interrupted question still gets answered.
</Note>

## Examples

### Drop the partial — strict "cancel means discard"
Expand Down
41 changes: 31 additions & 10 deletions packages/trigger-sdk/src/v3/ai.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4239,8 +4239,11 @@ export type RecoveryBootEvent<TUIM extends UIMessage = UIMessage> = {
/**
* User messages that arrived on `session.in` past the cursor — i.e.
* the message(s) the predecessor was processing or had queued when
* it died. The runtime's default is to re-dispatch each as a fresh
* turn after the chain is restored. Return a different list via
* it died. The runtime's default re-dispatches each as a fresh turn
* after the chain is restored, except when a `partialAssistant` is
* present AND there are two or more of them: the first is then
* spliced into the chain (as the question the partial was answering)
* rather than dispatched. Return a different list via
* `recoveredTurns` to skip / reorder / collapse them.
*/
inFlightUsers: TUIM[];
Expand Down Expand Up @@ -4285,8 +4288,12 @@ export type RecoveryBootResult<TUIM extends UIMessage = UIMessage> = {
chain?: TUIM[];
/**
* The user messages to re-dispatch as fresh turns after the chain is
* restored. Default: `inFlightUsers` (re-process every in-flight
* user). Return `[]` to suppress all of them; return a filtered /
* restored. Default: `inFlightUsers.slice(1)` when a
* `partialAssistant` is present and there are two or more in-flight
* users (the first one is spliced into the chain instead), otherwise
* `inFlightUsers` — including the single-user case, where the
* interrupted user is re-dispatched and the orphan partial is
* dropped. Return `[]` to suppress all of them; return a filtered /
* reordered subset to skip specific ones.
*/
recoveredTurns?: TUIM[];
Expand Down Expand Up @@ -4853,8 +4860,13 @@ export type ChatAgentOptions<
* customer's DB.
*
* Defaults (returned when the hook is omitted or returns no field):
* - `chain` = `settledMessages` (drop the orphan partial)
* - `recoveredTurns` = `inFlightUsers` (re-dispatch every user)
* - With two or more in-flight users, the partial and the user it
* was answering are spliced into the chain:
* `chain` = `[...settledMessages, inFlightUsers[0], partialAssistant]`
* and `recoveredTurns` = `inFlightUsers.slice(1)`.
* - Otherwise `chain` = `settledMessages` (drop the orphan partial)
* and `recoveredTurns` = `inFlightUsers` (re-dispatch every user)
* — so a single interrupted user is answered on the new run.
*
* @example
* ```ts
Expand Down Expand Up @@ -5840,20 +5852,29 @@ function chatAgent<
}
}

// Default: splice partial + the user it was answering into
// the chain so follow-ups like "keep going" still have context.
// Default: splice partial + the user it was answering into the chain
// so follow-ups like "keep going" still have context, and re-dispatch
// the users that arrived after it.
//
// The splice needs a follow-up user to answer — it consumes
// `inFlightUsers[0]` into the chain instead of dispatching it. With
// exactly ONE in-flight user (the plain OOM / crash-mid-answer case)
// there is nothing left to dispatch, so splicing would strand that
// user unanswered and idle the run. Require `length > 1` on both
// branches: at n=1 the orphan partial is dropped and the interrupted
// user is re-dispatched as a fresh turn instead.
let seedChain: TUIMessage[];
let recoveredTurns: TUIMessage[];
if (hookChain !== undefined) {
seedChain = hookChain;
} else if (partialAssistant !== undefined && inFlightUsers.length > 0) {
} else if (partialAssistant !== undefined && inFlightUsers.length > 1) {
seedChain = [...settledMessages, inFlightUsers[0]!, partialAssistant];
} else {
seedChain = settledMessages;
}
if (hookRecoveredTurns !== undefined) {
recoveredTurns = hookRecoveredTurns;
} else if (partialAssistant !== undefined && inFlightUsers.length > 0) {
} else if (partialAssistant !== undefined && inFlightUsers.length > 1) {
recoveredTurns = inFlightUsers.slice(1);
} else {
recoveredTurns = inFlightUsers;
Expand Down
48 changes: 48 additions & 0 deletions packages/trigger-sdk/test/recovery-boot.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,54 @@ describe("onRecoveryBoot — chat.agent recovery hook", () => {
}
});

it("smart default: a single in-flight user is re-dispatched, not swallowed by the splice", async () => {
// The plain OOM / crash-mid-answer shape: the run died while answering
// the only outstanding user message. Splicing that user into the chain
// would leave nothing to dispatch, so the run would boot and idle with
// the message unanswered. The default must re-dispatch it instead (and
// drop the orphan partial).
let observedChain: Array<{ role: string; idHead: string }> = [];
let turnCount = 0;
const model = new MockLanguageModelV3({
doStream: async () => {
turnCount++;
return { stream: textStream("ok") };
},
});
const partial = assistantMessage("partial answer in progress", "a-partial");
const u1 = userMessage("the question that OOM'd", "u-1");
const agent = chat.agent({
id: "recovery-boot.single-inflight-user",
// NO onRecoveryBoot — exercise the default path
onTurnStart: async ({ uiMessages }) => {
if (turnCount === 0) {
observedChain = uiMessages.map((m) => ({
role: m.role,
idHead: m.id.slice(0, 10),
}));
}
},
run: async ({ messages, signal }) => streamText({ model, messages, abortSignal: signal }),
});
const harness = mockChatAgent(agent, {
chatId: "single-inflight-user",
continuation: true,
previousRunId: "run_prior",
});
harness.seedSessionOutPartial(partial as never);
harness.seedSessionInTail([u1 as never]);
try {
await new Promise((r) => setTimeout(r, 100));
// One turn fires, for the interrupted user.
expect(turnCount).toBe(1);
// The orphan partial is dropped — the chain is just the re-dispatched user.
expect(observedChain.map((m) => m.role)).toEqual(["user"]);
expect(observedChain[0]!.idHead).toBe("u-1");
} finally {
await harness.close();
}
});

it("hook's recoveredTurns: [] suppresses re-dispatch of in-flight users", async () => {
let turnCount = 0;
const model = new MockLanguageModelV3({
Expand Down
Loading