Skip to content

Commit 7655a5c

Browse files
committed
fix(chat,sdk): stop an unconsumed control record wedging the mailbox
`chat.messages.next()` and `hasPending()` only inspect the head of the `.in` buffer. A control record whose kind has no consumer on this boot therefore parked at the head forever, and every message queued behind it became undeliverable with no error: `hasPending()` stayed false and `next()` timed out on every call. Records of a kind nothing on the run consumes are now discarded at dispatch instead. Consuming at dispatch keeps the resume cursor exact, since the record never enters the buffer and so leaves no unconsumed barrier for `lastDispatchedSeqNum()` to clamp behind. `message` is always claimed, and handover kinds are claimed for the window in which a handover-prepare boot is actually waiting for them. The mixed-kinds test now builds its blocked-head state on a handover-prepare boot, where the handover kind is claimed and so stays buffered for the raw read it asserts.
1 parent f4d1826 commit 7655a5c

4 files changed

Lines changed: 128 additions & 11 deletions

File tree

.changeset/tidy-mailboxes-wait.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,5 @@
44
---
55

66
Custom agent loops can now inspect pending chat input without consuming it and consume one mailbox record at a time with `chat.messages.hasPending()` and `chat.messages.next()`. Mailbox records include stable identifiers for tracing and redelivery.
7+
8+
A control record that nothing on the run consumes is now discarded rather than left at the head of the `.in` channel, where it would have made every message queued behind it undeliverable. `chat.messages.next()` returning `undefined` now always means the mailbox is idle.

docs/ai-chat/custom-agents.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -267,6 +267,11 @@ comes before a message, `hasPending()` stays `false` and `next()` leaves the
267267
control record for its own consumer. After that record is handled, the message
268268
becomes pending.
269269

270+
A control record that nothing on the run consumes is discarded rather than left
271+
at the head of the channel. `hasPending()` and `next()` only look at the head, so
272+
a record parked there would make every message behind it undeliverable. This
273+
means `next()` returning `undefined` always means the mailbox is idle.
274+
270275
A complete loop:
271276

272277
```ts trigger/my-chat-raw.ts

packages/trigger-sdk/src/v3/ai.ts

Lines changed: 118 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1885,17 +1885,118 @@ async function waitForHandover(options: {
18851885
spanName?: string;
18861886
}): Promise<HandoverSignal | null> {
18871887
if (options.payload.trigger !== "handover-prepare") return null;
1888-
const result = await handoverInput.waitWithIdleTimeout({
1889-
idleTimeoutInSeconds:
1890-
options.idleTimeoutInSeconds ?? options.payload.idleTimeoutInSeconds ?? 60,
1891-
timeout: options.timeout,
1892-
spanName: options.spanName ?? "waiting for handover signal",
1888+
try {
1889+
const result = await handoverInput.waitWithIdleTimeout({
1890+
idleTimeoutInSeconds:
1891+
options.idleTimeoutInSeconds ?? options.payload.idleTimeoutInSeconds ?? 60,
1892+
timeout: options.timeout,
1893+
spanName: options.spanName ?? "waiting for handover signal",
1894+
});
1895+
// Non-ok = idle timeout or the warm handler crashed without signaling.
1896+
if (!result.ok) return null;
1897+
return result.output;
1898+
} finally {
1899+
// The handover window is over either way. A signal arriving after this
1900+
// point has no consumer, so hand it to the drain rather than letting it
1901+
// park at the head of the channel.
1902+
releaseChatInputKinds(CHAT_HANDOVER_KINDS);
1903+
}
1904+
}
1905+
1906+
/**
1907+
* Record kinds on `session.in` that some consumer on THIS boot is responsible
1908+
* for. `"message"` is always claimed; handover kinds are claimed only for the
1909+
* window in which `waitForHandover` is actually waiting for them.
1910+
*
1911+
* Anything not in this set has no consumer on this boot, so leaving it buffered
1912+
* would park it at the head of the channel forever — `chat.messages.next()` and
1913+
* `hasPending()` only ever inspect the head, so every record queued behind it
1914+
* becomes undeliverable with no error. The drain below discards unclaimed kinds
1915+
* instead.
1916+
* @internal
1917+
*/
1918+
const chatClaimedKindsKey = locals.create<Set<string>>("chat.claimedKinds");
1919+
1920+
/** Kinds carried on `.in` that are not user messages. @internal */
1921+
const CHAT_HANDOVER_KINDS = ["handover", "handover-skip"] as const;
1922+
1923+
/** The run's attached drain subscription, so it can be re-offered the buffer. @internal */
1924+
const chatInputDrainKey = locals.create<{ off: () => void }>("chat.inputDrain");
1925+
1926+
function chatClaimedKinds(): Set<string> {
1927+
let claimed = locals.get(chatClaimedKindsKey);
1928+
if (!claimed) {
1929+
claimed = new Set<string>(["message"]);
1930+
locals.set(chatClaimedKindsKey, claimed);
1931+
}
1932+
return claimed;
1933+
}
1934+
1935+
/**
1936+
* Attach the unclaimed-control drain for this run.
1937+
*
1938+
* Consuming at dispatch (returning `true`) is what makes this safe for the
1939+
* resume cursor: the record is never buffered, so it leaves no unconsumed
1940+
* marker and `lastDispatchedSeqNum()` stays exact rather than being clamped
1941+
* behind a record nobody will ever take.
1942+
*
1943+
* `#dispatch` resolves a matching `once()` waiter BEFORE invoking handlers, so
1944+
* this can never take a record out from under a claimed consumer that is
1945+
* actively waiting for it.
1946+
*
1947+
* MUST be attached after `seedSessionInResumeCursorForCustomLoop`, like every
1948+
* other `.in` listener — attaching first would replay from seq 0.
1949+
* @internal
1950+
*/
1951+
function attachUnclaimedChatInputDrain(): { off: () => void } {
1952+
return getChatSession().in.on<ChatInputChunk>((chunk) => {
1953+
const kind = (chunk as { kind?: unknown } | undefined)?.kind;
1954+
// Malformed record: nothing can consume it, so don't let it wedge the head.
1955+
if (typeof kind !== "string") {
1956+
logger.warn("chat: discarded a malformed session.in record with no usable kind");
1957+
return true;
1958+
}
1959+
if (chatClaimedKinds().has(kind)) return undefined;
1960+
logger.warn("chat: discarded a session.in record that no consumer handled on this boot", {
1961+
kind,
1962+
});
1963+
return true;
18931964
});
1894-
// Non-ok = idle timeout or the warm handler crashed without signaling.
1895-
if (!result.ok) return null;
1896-
return result.output;
18971965
}
18981966

1967+
/**
1968+
* Release claimed kinds and re-offer the buffer to the drain.
1969+
*
1970+
* Re-attaching is the sweep: `on()` re-offers every buffered record to the
1971+
* newly attached handler, so a record that was buffered while its kind was
1972+
* still claimed (the waiter-gap window between `once()` iterations) is
1973+
* discarded now and the cursor advances past it.
1974+
* @internal
1975+
*/
1976+
function releaseChatInputKinds(kinds: readonly string[]): void {
1977+
const claimed = chatClaimedKinds();
1978+
let changed = false;
1979+
for (const kind of kinds) {
1980+
if (claimed.delete(kind)) changed = true;
1981+
}
1982+
if (!changed) return;
1983+
1984+
const drain = locals.get(chatInputDrainKey);
1985+
if (!drain) return;
1986+
drain.off();
1987+
locals.set(chatInputDrainKey, attachUnclaimedChatInputDrain());
1988+
}
1989+
1990+
/**
1991+
* Declare that this run will consume the given `session.in` record kinds
1992+
* itself (via raw `session.in` reads). Claimed kinds are never discarded by
1993+
* the unclaimed-control drain: they stay buffered, block
1994+
* `chat.messages.next()` at the head of the channel, and hold the resume
1995+
* cursor behind them until consumed. Call `release()` when the loop stops
1996+
* consuming them — any still-buffered records of those kinds are then
1997+
* discarded and the cursor advances.
1998+
*/
1999+
18992000
/**
19002001
* Per-turn deferred promises. Registered via `chat.defer()`, awaited
19012002
* before `onTurnComplete` fires. Reset each turn.
@@ -5428,6 +5529,14 @@ function chatCustomAgent<
54285529
// listener — otherwise a continuation boot replays already-answered
54295530
// messages into the loop's first wait.
54305531
await seedSessionInResumeCursorForCustomLoop(payload);
5532+
// Claim the kinds this boot actually has a consumer for, then attach the
5533+
// drain for everything else. Handover kinds are only claimed on a
5534+
// handover-prepare boot, which is the only boot that waits for them.
5535+
const claimed = chatClaimedKinds();
5536+
if (payload.trigger === "handover-prepare") {
5537+
for (const kind of CHAT_HANDOVER_KINDS) claimed.add(kind);
5538+
}
5539+
locals.set(chatInputDrainKey, attachUnclaimedChatInputDrain());
54315540
return userRun(payload, runOptions);
54325541
},
54335542
});
@@ -10765,6 +10874,7 @@ export const chat = {
1076510874
response: chatResponse,
1076610875
/** Pre-built input stream for receiving messages from the transport. */
1076710876
messages: messagesInput,
10877+
/** Declare `session.in` record kinds this run consumes itself. See {@link chatClaimInputKinds}. */
1076810878
/** Create a managed stop signal wired to the stop input stream. See {@link createStopSignal}. */
1076910879
createStopSignal,
1077010880
/** Signal the frontend that the current turn is complete. See {@link chatWriteTurnComplete}. */

packages/trigger-sdk/test/chat-messages-mailbox.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ describe("chat.messages mailbox", () => {
6464

6565
await runInMockTaskContext(async (drivers) => {
6666
const runPromise = run(
67-
{ chatId, trigger: "preload" },
67+
{ chatId, trigger: "handover-prepare" },
6868
{ ctx: drivers.ctx, signal: new AbortController().signal }
6969
);
7070
await ready.promise;
@@ -156,7 +156,7 @@ describe("chat.messages mailbox", () => {
156156

157157
await runInMockTaskContext(async (drivers) => {
158158
const runPromise = run(
159-
{ chatId, trigger: "preload" },
159+
{ chatId, trigger: "handover-prepare" },
160160
{ ctx: drivers.ctx, signal: new AbortController().signal }
161161
);
162162
await ready.promise;
@@ -229,7 +229,7 @@ describe("chat.messages mailbox", () => {
229229

230230
await runInMockTaskContext(async (drivers) => {
231231
const runPromise = run(
232-
{ chatId, trigger: "preload" },
232+
{ chatId, trigger: "handover-prepare" },
233233
{ ctx: drivers.ctx, signal: new AbortController().signal }
234234
);
235235
await ready.promise;

0 commit comments

Comments
 (0)