Skip to content

Commit e9fe30c

Browse files
committed
fix(sdk): re-dispatch a single in-flight user on recovery boot
The recovery-boot smart default spliced the partial assistant plus the user it was answering into the seed chain whenever a partial existed and there was at least one in-flight user. With exactly one in-flight user — the plain OOM / crash-mid-answer shape — that consumed the only message there was to dispatch: `recoveredTurns` came out empty, the boot queue stayed empty, the session.in cursor was advanced past the message anyway, and on a preload or continuation boot (no message on the wire payload) neither dispatch site fired. The interrupted question was never answered and the run idled until its timeout. Require two or more in-flight users for the splice, on both the chain and the recoveredTurns branch. With n >= 2 nothing changes. With n = 1 the orphan partial is dropped and the interrupted user is re-dispatched as a fresh turn, which is what the OOM-resilience docs promise. The submit-message boot is unaffected: the existing dedup skips a queued message identical to the one already on the wire payload. Also corrects the docstrings and the recovery-boot docs, which described the default as "re-dispatch every user" without mentioning the splice. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent cc69ff4 commit e9fe30c

4 files changed

Lines changed: 93 additions & 12 deletions

File tree

.changeset/brave-otters-recover.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@trigger.dev/sdk": patch
3+
---
4+
5+
Fixed a chat agent hanging after an interrupted turn: when a run was killed mid-answer (out of memory, crash, or eviction) and only the one message it was answering was still outstanding, the new run never replied to it. That message is now re-answered on the new run.

docs/ai-chat/patterns/recovery-boot.mdx

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ On a continuation boot, the runtime reads:
3232
- **`session.out` tail past the snapshot cursor** — closed assistant turns plus, optionally, a `partialAssistant` (the trailing message whose stream never received a `finish` chunk). `cleanupAbortedParts` has already stripped streaming-in-progress fragments.
3333
- **`session.in` tail past the last `turn-complete` cursor** — user messages the dead run hadn't acknowledged.
3434

35-
If both `partialAssistant` and `inFlightUsers` are non-empty, the runtime splices `[firstInFlightUser, partialAssistant]` onto the chain. The remaining in-flight users dispatch as fresh turns. The model sees:
35+
If there's a `partialAssistant` and two or more `inFlightUsers`, the runtime splices `[firstInFlightUser, partialAssistant]` onto the chain. The remaining in-flight users dispatch as fresh turns. The model sees:
3636

3737
```
3838
[ ...settledMessages, // chain through the last completed turn
@@ -138,10 +138,17 @@ type RecoveryBootResult<TUIM extends UIMessage = UIMessage> = {
138138
};
139139
```
140140

141-
- **`chain`** — replaces the seed chain. Defaults to `[...settledMessages, firstInFlightUser, partialAssistant]` when both partial and in-flight users exist, otherwise `settledMessages` alone.
141+
- **`chain`** — replaces the seed chain. Defaults to `[...settledMessages, firstInFlightUser, partialAssistant]` when there's a partial **and two or more** in-flight users, otherwise `settledMessages` alone.
142142
- **`recoveredTurns`** — user messages to dispatch as fresh turns after the chain is restored. Defaults to `inFlightUsers.slice(1)` when the smart default consumed the first user, otherwise `inFlightUsers`.
143143
- **`beforeBoot`** — runs after the writer flushes and before the first recovered turn fires. Use for blocking persistence (write the partial to your DB so a later turn can reference it). Errors bubble — wrap your own try/catch if you want to soft-fail.
144144

145+
<Note>
146+
The splice needs a follow-up user to answer, so it only applies with two or
147+
more in-flight users. With exactly one — the plain OOM or crash-mid-answer
148+
case — the orphan partial is dropped and that single user is re-dispatched as
149+
a fresh turn, so the interrupted question still gets answered.
150+
</Note>
151+
145152
## Examples
146153

147154
### Drop the partial — strict "cancel means discard"

packages/trigger-sdk/src/v3/ai.ts

Lines changed: 31 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4239,8 +4239,11 @@ export type RecoveryBootEvent<TUIM extends UIMessage = UIMessage> = {
42394239
/**
42404240
* User messages that arrived on `session.in` past the cursor — i.e.
42414241
* the message(s) the predecessor was processing or had queued when
4242-
* it died. The runtime's default is to re-dispatch each as a fresh
4243-
* turn after the chain is restored. Return a different list via
4242+
* it died. The runtime's default re-dispatches each as a fresh turn
4243+
* after the chain is restored, except when a `partialAssistant` is
4244+
* present AND there are two or more of them: the first is then
4245+
* spliced into the chain (as the question the partial was answering)
4246+
* rather than dispatched. Return a different list via
42444247
* `recoveredTurns` to skip / reorder / collapse them.
42454248
*/
42464249
inFlightUsers: TUIM[];
@@ -4285,8 +4288,12 @@ export type RecoveryBootResult<TUIM extends UIMessage = UIMessage> = {
42854288
chain?: TUIM[];
42864289
/**
42874290
* The user messages to re-dispatch as fresh turns after the chain is
4288-
* restored. Default: `inFlightUsers` (re-process every in-flight
4289-
* user). Return `[]` to suppress all of them; return a filtered /
4291+
* restored. Default: `inFlightUsers.slice(1)` when a
4292+
* `partialAssistant` is present and there are two or more in-flight
4293+
* users (the first one is spliced into the chain instead), otherwise
4294+
* `inFlightUsers` — including the single-user case, where the
4295+
* interrupted user is re-dispatched and the orphan partial is
4296+
* dropped. Return `[]` to suppress all of them; return a filtered /
42904297
* reordered subset to skip specific ones.
42914298
*/
42924299
recoveredTurns?: TUIM[];
@@ -4853,8 +4860,13 @@ export type ChatAgentOptions<
48534860
* customer's DB.
48544861
*
48554862
* Defaults (returned when the hook is omitted or returns no field):
4856-
* - `chain` = `settledMessages` (drop the orphan partial)
4857-
* - `recoveredTurns` = `inFlightUsers` (re-dispatch every user)
4863+
* - With two or more in-flight users, the partial and the user it
4864+
* was answering are spliced into the chain:
4865+
* `chain` = `[...settledMessages, inFlightUsers[0], partialAssistant]`
4866+
* and `recoveredTurns` = `inFlightUsers.slice(1)`.
4867+
* - Otherwise `chain` = `settledMessages` (drop the orphan partial)
4868+
* and `recoveredTurns` = `inFlightUsers` (re-dispatch every user)
4869+
* — so a single interrupted user is answered on the new run.
48584870
*
48594871
* @example
48604872
* ```ts
@@ -5840,20 +5852,29 @@ function chatAgent<
58405852
}
58415853
}
58425854

5843-
// Default: splice partial + the user it was answering into
5844-
// the chain so follow-ups like "keep going" still have context.
5855+
// Default: splice partial + the user it was answering into the chain
5856+
// so follow-ups like "keep going" still have context, and re-dispatch
5857+
// the users that arrived after it.
5858+
//
5859+
// The splice needs a follow-up user to answer — it consumes
5860+
// `inFlightUsers[0]` into the chain instead of dispatching it. With
5861+
// exactly ONE in-flight user (the plain OOM / crash-mid-answer case)
5862+
// there is nothing left to dispatch, so splicing would strand that
5863+
// user unanswered and idle the run. Require `length > 1` on both
5864+
// branches: at n=1 the orphan partial is dropped and the interrupted
5865+
// user is re-dispatched as a fresh turn instead.
58455866
let seedChain: TUIMessage[];
58465867
let recoveredTurns: TUIMessage[];
58475868
if (hookChain !== undefined) {
58485869
seedChain = hookChain;
5849-
} else if (partialAssistant !== undefined && inFlightUsers.length > 0) {
5870+
} else if (partialAssistant !== undefined && inFlightUsers.length > 1) {
58505871
seedChain = [...settledMessages, inFlightUsers[0]!, partialAssistant];
58515872
} else {
58525873
seedChain = settledMessages;
58535874
}
58545875
if (hookRecoveredTurns !== undefined) {
58555876
recoveredTurns = hookRecoveredTurns;
5856-
} else if (partialAssistant !== undefined && inFlightUsers.length > 0) {
5877+
} else if (partialAssistant !== undefined && inFlightUsers.length > 1) {
58575878
recoveredTurns = inFlightUsers.slice(1);
58585879
} else {
58595880
recoveredTurns = inFlightUsers;

packages/trigger-sdk/test/recovery-boot.test.ts

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,6 +295,54 @@ describe("onRecoveryBoot — chat.agent recovery hook", () => {
295295
}
296296
});
297297

298+
it("smart default: a single in-flight user is re-dispatched, not swallowed by the splice", async () => {
299+
// The plain OOM / crash-mid-answer shape: the run died while answering
300+
// the only outstanding user message. Splicing that user into the chain
301+
// would leave nothing to dispatch, so the run would boot and idle with
302+
// the message unanswered. The default must re-dispatch it instead (and
303+
// drop the orphan partial).
304+
let observedChain: Array<{ role: string; idHead: string }> = [];
305+
let turnCount = 0;
306+
const model = new MockLanguageModelV3({
307+
doStream: async () => {
308+
turnCount++;
309+
return { stream: textStream("ok") };
310+
},
311+
});
312+
const partial = assistantMessage("partial answer in progress", "a-partial");
313+
const u1 = userMessage("the question that OOM'd", "u-1");
314+
const agent = chat.agent({
315+
id: "recovery-boot.single-inflight-user",
316+
// NO onRecoveryBoot — exercise the default path
317+
onTurnStart: async ({ uiMessages }) => {
318+
if (turnCount === 0) {
319+
observedChain = uiMessages.map((m) => ({
320+
role: m.role,
321+
idHead: m.id.slice(0, 10),
322+
}));
323+
}
324+
},
325+
run: async ({ messages, signal }) => streamText({ model, messages, abortSignal: signal }),
326+
});
327+
const harness = mockChatAgent(agent, {
328+
chatId: "single-inflight-user",
329+
continuation: true,
330+
previousRunId: "run_prior",
331+
});
332+
harness.seedSessionOutPartial(partial as never);
333+
harness.seedSessionInTail([u1 as never]);
334+
try {
335+
await new Promise((r) => setTimeout(r, 100));
336+
// One turn fires, for the interrupted user.
337+
expect(turnCount).toBe(1);
338+
// The orphan partial is dropped — the chain is just the re-dispatched user.
339+
expect(observedChain.map((m) => m.role)).toEqual(["user"]);
340+
expect(observedChain[0]!.idHead).toBe("u-1");
341+
} finally {
342+
await harness.close();
343+
}
344+
});
345+
298346
it("hook's recoveredTurns: [] suppresses re-dispatch of in-flight users", async () => {
299347
let turnCount = 0;
300348
const model = new MockLanguageModelV3({

0 commit comments

Comments
 (0)