feat(history): replay a window of a task's history, with older messages on demand - #19
feat(history): replay a window of a task's history, with older messages on demand#19Antisophy wants to merge 4 commits into
Conversation
bafaa73 to
dc622ec
Compare
|
Reworked and extended since opening, with a force-push (
Each commit's tree passed the full check suite. |
dc622ec to
bcbea05
Compare
|
Force-pushed again (
Each commit's tree passed the full check suite again. |
I think this is not enough. Agents can run a program in the background, with the output continuing to stream in even as the session is otherwise idle and the user can submit a non-steering message. So, I think any kind of partial loading in reverse order requires a new mechanism to allow the reducer to somehow bank events that it cannot apply yet, and then apply them once the base event arrives. The complexity required by this is the main reason I haven't pursued this myself. One observation I have is that images are usually the most significant contributor to a task's history load time. Have you observed this as well? If so, I'm wondering if the lowest hanging fruit here is to make image payloads out-of-band, and requested by the frontend lazily. |
Opening a task replays its entire history, one WebSocket frame per event, which on an old task means tens of seconds of streaming, a DOM of tens of thousands of nodes, and on a phone a stalled client that drops its WebSocket. Replay a window instead. A new optional top-level history_window config carries desktop and mobile sizes in messages; absent or zero replays everything, so current behaviour is the default. The client sends only its device class and the server picks the number, because the server is the only side that always knows it: tasks_list precedes server_status and is what triggers the first history request, so a client deciding for itself asks for everything whenever those two messages arrive in separate ticks. The window holds exactly the requested number of rendered message bubbles (user or assistant), counted from the end, regardless of turn structure; every event from that point rides along, and seqs stay true history indices. A window may start mid-turn: events whose base item started before the window are dropped by the client's reducers and appear once older history is loaded. A window must not sever session identity: the newest session/init before the window, and the newest session/metadata after that init, replay ahead of the range with their true seqs. A codex transcript carries metadata mid-stream, and metadata without its init is a hard error in the client's reducers. Claude transcripts carry neither event. content-visibility on messages now applies only to unwindowed replays, where the whole transcript is in the DOM and the guessed placeholder heights pay for themselves.
A windowed replay needs a way to reach what it held back. A row atop the list offers one step, five steps, or everything; the server replays only the requested slice, ending where the loaded window began (the new request_history_before message), so nothing the client already holds is fetched twice. The client keeps every frame that built the current timeline in arrival order. A load-more captures the fetched slice between the prepend markers and re-reduces the slice plus the kept frames through the same reset and reduction an initial replay uses, so the end state is exactly what an initial load of the larger window would have produced; a test pins that invariant. Reducing frames through one shared path also means cross-references between old and new messages (a queued send's pending placeholder healed by its later delivery, consumption upgrades) behave identically however the history arrived. Session context replayed from before the window stays ahead of the slice in that merge, and context the slice itself covers is dropped, since the slice carries the authoritative copy at those seqs. The reader stays where they were: scrollTop is captured at the click and restored when the older messages land above.
Clicking a "Load [N] more" button gave no feedback until the slice landed, which on a long fetch reads as the click not registering. The buttons swap for the same status band that plays while awaiting a response, recolored grey through a scoped --status-color override and sized to the buttons' height so the row does not shift. The window start moving brings the buttons back, or removes the row entirely once everything is loaded. The band mounts fresh on every click, and the stock 22s ease-in-out sweep begins at its slowest phase, so a short fetch would end before any visible motion; a scoped negative delay drops the fresh mount into the fast phase and a shorter cycle keeps the motion obvious within a one-second load, with the same keyframes and layer.
Extra history loaded with "Load more" stayed in the task state forever, so every task ever expanded kept its full timeline in memory across task switches. Track which tasks were grown by route id (the tid-to- uuid map may not be populated when the tracking effect first runs); when the active task changes away from a grown one, drop its frames and reset its timeline, and let the activation effect replay the plain window fresh on refocus. Tasks never grown keep the existing behavior, staying cached across switches.
bcbea05 to
2318bd3
Compare
|
Force-pushed once more ( On the boundary cut not being enough: agreed, and the design changed accordingly rather than defending it. Background output streaming into an otherwise idle window, and generally any event whose base item started before the window, is exactly the case a turn-boundary cut cannot make airtight. What the PR now does:
A correction to my previous comment here: the claim that moving On images: I think you're right in general, but they weren't what dominated here. The stalls behind this PR reproduced on transcripts with essentially no images; the measured task was ~26,000 events building ~21,000 DOM nodes, and event count and DOM size tracked the stall. Image payloads certainly dominate transfer bytes on photo-heavy tasks, and out-of-band lazily-fetched images would help windowed and unwindowed replays alike, so it reads as complementary rather than an alternative, and worth doing as its own change. |
|
Following up on the flakiness investigation promised above, with a correction and, this time, a mechanism verified end to end rather than a correlation. The correction: the "daily wall-clock window after UTC midnight" claim in my previous comment was wrong too, in an instructive way. The failures clustered in evening hours because that's when the retries ran serially on an otherwise idle machine; the passes clustered when the full check suite happened to be running in parallel. Time of day was a proxy for machine load. Controlled experiments (same tree, same clock, libfaketime's LD_PRELOAD shim active with a zero offset) flipped the outcome from reliably-failing to reliably-passing, proving the clock value irrelevant: only the added per-syscall overhead mattered. The mechanism, traced with server-side logging and browser-console instrumentation on the reliably-failing configuration (fast idle machine):
This presumably covers the claude and copilot variants of the same spec family, and likely other focus-asserting specs in the flaky set. Two possible fixes, either of which I'm happy to send as a separate PR: have the mock delay the child's answer in focus-asserting specs so the focused state has a deterministic observable duration; or record route transitions in the page (e.g. via a test hook on navigation) and assert the transition happened rather than polling for a transient state. |
That's a good solution. However, then my concern would be that this turns the cost of loading the entire history quadratic: re-applying all events received so far through the reducer now causes the reducer's work to grow quadratically with the size of the conversation. This may or may not be a problem in practice; @Antisophy what are your experiences with this implementation, does "scrolling all the way up" still stay reasonably responsive? If not, one potential improvement to keep the total cost reasonable is to make the total number of chunks constant (e.g. load no less than 10% at a time), or make the chunk size grow in a geometric or exponential tempo. |
|
I won't be able to check on the scrolling performance after loading all messages in one of my biggest tasks for at least a few days. I do like your idea of keeping bounded chunks only. I'll have to test the current implementation and then some bounded-chunk implementations when I can. Until then, I'll have to stick with loading these smaller message windows. Please hold on this PR until I get back to you. |
2 participants
Opening a task replays its entire history, one WebSocket frame per event. That is fine for a young task and increasingly not fine for an old one: a task here reached ~26,000 events, which takes tens of seconds to stream, builds a DOM of roughly 21,000 nodes, and on a phone can stall the client long enough to drop its WebSocket (which disables the composer, since sending needs a live connection).
Four commits, separable and reviewable on their own:
1. Replay only a window (
feat(history): replay only a window of a task's history)history_windowwithdesktopandmobilekeys, in messages. Absent or zero replays everything, so current behaviour is the default and nothing changes for anyone who does not opt in.tasks_listis sent beforeserver_statusand is what triggers the first history request, so a client deciding for itself asks for everything whenever those two messages arrive in separate ticks. That failure is invisible on a fast local connection and reliable on a slow one.item/delta,item/result, oritem/completedwhose base item started before the window is dropped by the reducers (each already no-ops on an unknown item), so a background program's late output is simply not visible in the initial window rather than wrong, and loading older history renders it fully.session/initbefore the window (and the newestsession/metadataafter that init) replay ahead of the range with their true seqs. A codex transcript carries metadata mid-stream, and metadata without its init is a hard error in the client's reducers; claude transcripts carry neither event and are unaffected.content-visibilityon messages now applies only to unwindowed replays. It reserves a guessed 100px per offscreen message, so heights change as they scroll into view and the scroll region grows under a dragged scrollbar handle; that trade only pays when the whole transcript is in the DOM.2. Load older history on demand (
feat(web): load older history on demand)request_history_before: the client names the seq it currently starts at, and the server replays only the messages before it, framed bytask_history_prepend_start/_end.column-reverse, soscrollTopmeasures distance from the bottom, which is exactly the quantity held constant across the batch: the reader stays put and older messages arrive above them.3. Show the awaiting-response band while the slice loads (
feat(web): show the awaiting-response band while older history loads)--status-coloroverride and sized to the buttons' height so the row does not shift. The buttons return when the window start moves, or the row disappears once everything is loaded.4. Release grown history when a task loses focus (
feat(web): release grown history when a task loses focus)Measured end to end on that task, the initial replay carries only the last thirty messages' events instead of all ~26,000, and a load-more click fetches only its slice instead of the whole history.
One of five independent changes for using cydo on a phone: #15 (composer spacing), #16 (the page dragging sideways), #17 (hiding the composer while reading history), #18 (attaching images with a file picker). They touch different code and can merge in any order.