Repository navigation
fix(transcript): keep streamed reply text when a repair read's page is older - #1538
Conversation
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan I reviewed 69f97018. Two P2s remain:
-
A shorter prefix can be authoritative (
opencode-data-read-merge.ts:81–83). OpenCode 2.3.0 explicitly supports final replacements"Draft"→"D"or""(packages/ai/test/provider/open-responses-finals.test.ts:19–47). I reproduced this with the official 2.3.0 executable and a synthetic loopback provider: native HTTP returns finalized"D", but this merge retains"Draft"while acceptingfinish: "stop". Disabling the defense returns"D". Please distinguish open-fragment persistence from authoritative finals; prefix comparison alone cannot decide which is newer. -
Neither new test file runs in CI. Add
opencode-data-delivery-repair.test.tsandopencode-rotation-replay.test.tsto the browser-conditioned UI unit list. Removing the defense still passes the entire current 683-test CI command, while the new delivery regressions fail.
Typecheck, 144 focused units twice and 78 browser checks pass. The native final-prefix reproduction exposes the first issue despite those green suites.
|
@pascalandr Both P2s addressed in 7fb0f8e:
140 store tests and |
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan Thanks, "Draft" → "D"/"" on completed rows and the CI registration are fixed. One P2 remains in 7fb0f8ee:
A committed empty final can precede step completion (opencode-data-read-merge.ts:66–71). I reproduced this on OpenCode 2.3.0 with a synthetic Responses provider: it streams "Draft", finalizes the fragment as "", then starts a tool input in the same open step. The page returns [{type:"text",text:""}, {type:"tool",state:{status:"streaming"}}] and the row has no time.completed. This merge keeps "Draft". A row's completion therefore cannot tell a committed empty final from an uncommitted one. One possible signal: an uncommitted fragment is the row's last fetched part, so any later part means its ending has committed. Please add this open-step case as a regression and fix the comment.
The 687 CI-listed tests, 145 focused tests and typecheck pass on the merged tree.
|
@pascalandr Addressed in 49b9c39: an uncommitted fragment is now required to be the row's last fetched part — any later part proves its ending committed (parts append in stream order, and a following part's append commits after the fragment's ending), so an empty fetched text beside a later part, or beside the row's completion, keeps full page authority. The comment now states this ordering rule instead of row completion alone.
141 store tests and typecheck pass locally. |
…m page Sending a prompt could truncate the previous reply's tail. The inbox delivery repair read — like every reducer page read — lets the fetched page replace any row that did not change during the fetch, so a page whose stored copy of a completed assistant row still held mid-stream text (a strict prefix of what its full-text ending already finalized) overwrote the complete reply the moment the user sent the next message. Keep the live text and reasoning parts of a completed row when the fetched copy is a strict prefix of them; the page stays authoritative for every other field and for part shapes that do not pair up. A prefix-shorter copy can only be the stream state an ending already superseded, never a newer value, so page authority for genuinely newer pages is unchanged. Covered by opencode-data-delivery-repair.test.ts through the real event admission, delivery repair and read pipeline.
…pages The native persists a text part empty until its full-text ending commits, so a page fetched while a fragment's deltas are paused holds the fragment with no text even though the stream already delivered it. When sending a prompt mid-stream triggered the delivery repair read, that page rewound the assembled deltas of the still-streaming reply the same way completed replies were truncated. Drop the completion-stamp condition: within a fragment a stream only appends, so a fetched text strictly shorter than and prefixed by the live text is always the superseded stream state, completed or not. A page holding it is now logged with the message id, so the next real occurrence names the read that carried a stale page instead of leaving no trace. Covered by both delivery-repair scenarios in opencode-data-delivery-repair.test.ts.
Documents the audited rotation path beside the delivery-repair guard: a completion queued behind a full-transcript rotation replays onto the bounded snapshot without rewinding streamed text.
…committed finals A completed item's final text can legitimately replace the streamed draft with a shorter, prefixed or even empty value (Open Responses completed items), and that replacement commits before its ending event is delivered, so a fetched page can be strictly newer while its text is a prefix of the live draft. The prefix comparison could not decide which side was newer and kept streamed drafts over authoritative finals. Guard only the shape that cannot be authoritative: a fetched part holding no text on a row whose step is open on both sides is a fragment whose full-text ending has not committed, older than the deltas the live row already displays. Committed finals carry a value or the row's completion and keep full page authority, verified for both a prefixed and an empty replacement. Adds the delivery-repair and rotation-replay suites to the CI unit list so the regressions they pin actually run there.
…ommitted A committed empty final can precede the step's completion: a fragment finalizes as empty and a tool input starts in the same open step, so the page holds an empty text part beside a later tool part on a row with no completion stamp. Row completion alone cannot tell that final from an uncommitted fragment. Parts append in stream order and a following part's append commits after the fragment's ending, so an uncommitted fragment is always the row's last fetched part: keep live text only there. An empty fetched part with any later part, or beside the row's completion, keeps full page authority. Covered by the open-step regression from the review.
49b9c39 to
ac9bfd2
Compare
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan Thanks, the empty final followed by a tool part is fixed in ac9bfd27. One P2 remains, in the gap just before that next part:
Being the last part does not prove the fragment is uncommitted (opencode-data-read-merge.ts:68–74). On OpenCode 2.3.0, the fragment's ending commits before the next part is appended, in a separate step. A read between the two returns the committed text: "" as the row's last part, and this merge keeps "Draft". It heals once the queued ending arrives, but the base shows "" correctly.
In my captures, the committed empty part carries the provider's native metadata ({ itemId, phase }), while the uncommitted one has none. Treating a fetched part with native metadata as committed closes this gap. I have not seen a final replacement without that metadata. Please add this last-part case to the regression.
The merge-base diff contains only the four intended files, and the 712 CI-listed tests and typecheck pass.
… as committed A fragment's ending commits before the next part is appended, in a separate durable step, so a read between the two returns the committed empty final as the row's last part. Being last therefore does not prove a fragment uncommitted. A committed ending stamps the provider's native metadata onto the stored part while an uncommitted fragment carries none, so a fetched part with native metadata is committed and keeps full page authority. Covered by the last-part regression from the review; the later-part case stays.
|
@pascalandr Addressed in efe6a73: a fetched part carrying The regression suite now pins all four shapes: the uncommitted last-part case keeps the streamed text, and the committed empty final wins beside a later part, as the row's last part with native metadata, and on a completed row ( 142 store tests and typecheck pass locally. |
Reasoning persists its provider native metadata when it starts, so treating any native-bearing part as committed rewound an open reasoning fragment to the page's empty copy during a delivery repair read. Text keeps the native marker as its commit proof; reasoning keeps its own time.completed ending. A new delivery-repair regression keeps streamed reasoning over an open fragment carrying its started native metadata. Dropping the text-only condition fails it; the CI-listed restore integration command (714 tests) and the UI typecheck pass.
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan The native metadata check fixed the text gap, but reasoning persists native when it starts, so an open reasoning fragment was rewound to the page's empty copy. I pushed 8a210aa3: the native commit marker now applies to text only, and reasoning keeps its time.completed ending, with a regression for it.
No P0-P2 findings. All native 2.3.0 witnesses (text finals, empty final before and after the next part, open and ended reasoning) behave correctly, and the 714 CI-listed tests and typecheck pass.
Problem
Sending a prompt could truncate the tail of the previous reply: a reply that showed ~1000 characters could drop to ~800 the moment the next message was sent. The loss was intermittent and self-healed when the fragment's full-text ending later arrived — unless that ending never reached the reducer.
Root cause
Three behaviors line up:
Sending triggers an authoritative read. When the native has already admitted the next turn, the delivered prompt's row is not the reducer's last row, so
session.inbox.deliveredowes the delivery repair read (reconcileDeliveredPromptPage) — amessage.listfetch whose result the SDK applies by replacing rows wholesale.Unchanged rows take the page.
mergePageWithSnapshotgives the fetched page unconditional authority over any row that did not change during the read. A row whose deltas merely paused across the fetch is "unchanged", so the page's copy replaces the text the stream had already assembled and displayed.Open fragments persist empty by design.
text.deltais ephemeral ("Stream fragments are live-only; Text.Ended is the replayable full-value boundary"), and the ending's projector write commits before the event is delivered, so while a fragment is open the stored copy of the message holds it empty — while the UI already shows its streamed deltas. A repair read landing in that gap (steering a busy session with paused deltas) returns a page that is "front complete, tail missing".Together: send while a fragment is open → repair read → page holds the open fragment empty → wholesale replace → the displayed tail vanishes until the ending arrives. The same merge also applied any page older than the stream from non-fresh sources (snapshot-seeded pages, older runtimes), so a completed reply could be truncated the same way there.
Fix
In
mergePageWithSnapshot, an unchanged row now keeps its livetextandreasoningparts when the fetched copy is strictly shorter than and prefixed by the live text.The prefix shape is a sufficient discriminator, completed row or not: within a fragment a stream only ever appends, so the live text grows monotonically; the native persists either the empty part or the ended full value, never a partial one; therefore a fetched text that is a strict prefix of the live text can only be stream state the live row already superseded — it can never be a newer authoritative value, whatever route the page took (a genuinely newer page carries the longer ended value or a divergent one, and still wins).
Everything else keeps full page authority: all other fields, part shapes that do not pair up, identified parts of changed rows, and pages that are longer or divergent. Destructive operations (revert, compaction, deletion) still invalidate fetched pages before the merge runs.
A page holding a superseded prefix is now logged with the message id (
sessionlogger), so a real-world occurrence identifies the read that carried a stale page instead of leaving no trace.Testing
opencode-data-delivery-repair.test.ts(new) drives the real event admission, delivery repair and read pipeline for both shapes: a still-streaming reply whose deltas paused, with the page holding the fragment's empty persistence (the deterministic current-native window), and a completed reply whose page holds the mid-stream prefix of its ended text (the non-fresh-page defense). Both keep the streamed text; both failed before the fix.opencode-rotation-replay.test.ts(new) pins the audited rotation path: a completion queued behind a full-transcript rotation replays without rewinding streamed text.tsc --noEmitpasses.No help-page changes: the fix changes no labels, controls or documented steps.