Skip to content

feat(conversations): unify native session continuation - #939

Open
Neonforge98 wants to merge 5 commits into
feat/pm-expansion-batchfrom
codex/audience-routing
Open

feat(conversations): unify native session continuation#939
Neonforge98 wants to merge 5 commits into
feat/pm-expansion-batchfrom
codex/audience-routing

Conversation

@Neonforge98

@Neonforge98 Neonforge98 commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

Imported provider histories (Codex app, Claude Code) could be read in ORG2 but not continued: there was no canonical owner of message content and delivery state, the window-scoped queue and a separate executions store disagreed after restarts, and nothing could materialize a conversation into a provider's own catalog so the native CLI could resume it. Interrupted tool results, Codex is_error, and compaction were handled inconsistently across the two existing paths.

Solution

Provider-native session continuation stacked on #844 as three reviewable commits:

Commit Scope
feat(conversations): add canonical event plane and durable continuation Canonical EventStore as the single owner of message content and pending/sent/failed; single persisted deliveries document replacing the window-scoped queue + separate executions store
feat(native): materialize and resume provider-native conversations Provider codecs and portable role/tool IR; Codex app-server adapter; Claude transcript materialization into the provider's own catalog so the native App can open and resume
refactor(conversations): unify native continuation and delivery owners One canonical timeline for UI and execution, including local (non-Cloud) roots whose execution children are merged into the root view; explicit Stop/footer wiring to the runner; removal of the second tail outbox, second runner registry, fixed post-terminal waits and hidden-child tab jumps

Potential risks

  • The canonical EventStore and the single persisted deliveries document replace two older owners; any code still reading the window-scoped queue or the separate executions store is now dead by construction and was removed in the third commit.
  • Materialization writes into the provider's real catalog (for Claude: ~/.claude/projects/<repo>/), so continued conversations appear in the native app's own history.
  • Fail-closed convergence is now scoped to proven divergence; absence-of-evidence cases warn and proceed. This is deliberate but changes behaviour for sessions that previously failed early.
  • Recovery treats a missing execution episode as transient (see the robustness gap under Acceptance status); a deleted child session wedges that conversation root until addressed.
  • Root timelines of locally continued sessions now include execution-child tails; a child whose canonical prefix cannot be matched falls back to its creation time, and one whose creation time is unavailable is skipped rather than guessed.
  • Opening a root with many completed episodes hydrates each child's CLI history once (measured: 12 children → one-time RSS burst to ~890 MB, settling at ~245 MB, idle CPU 0%, no repeating RPC). Each later episode is materialized from the pristine root only, so earlier episodes' turns are not carried into a new episode's provider context; that is existing continuation behaviour, surfaced here for the author's awareness.

Defects fixed during verification

  • Interrupted tool results did not round-trip: the read-back projection invented a phantom ToolCall/ToolResult pair for an unresolved provider call (the provider's own representation of an interrupt), so every later synchronize_native_conversation failed its semantic-prefix check. The projection now mirrors the canonical owner; interrupted (not model-visible, no provider can carry it) is no longer compared, is_error still is.
  • Codex materialization dropped is_error: a failed or killed tool was replayed to Codex as a successful function_call_output. Failures now use the exec envelope Codex's own reader restores byte-exact (exit_code 1; interrupted → 130).
  • Fail-closed convergence was scoped too broadly: healthy sessions (Codex without a local account, rollout not yet on disk) got their turn marked Failed. Absence-of-evidence cases warn and return Ok(None); proven divergence still fails closed and blocks the next turn.
  • Cloud plane read failed the whole conversation on one bad historical row; poisoned rows are now quarantined and lastSeq keeps advancing. Inbound writes stay strictly validated.
  • context_compacted truncated the native projection to summary + suffix; the documented skip policy is restored.
  • Retry on a failed Team Chat row could fall through to onEditSubmit, submitting the message to the agent as a prompt instead of retrying the comment. Discussion rows now only retry.
  • Admission-repair failure lost the retained-row identity (would have repopulated the composer — the exact behaviour the delivery contract forbids); restored with its regression test. editComment no longer RPCs a local-only optimistic id.
  • CI Rust gate: MSRV raised to 1.89 for File::lock/unlock; two question_mark lints.
  • E2E: the @-mention row activates on mousedown, so clickRendered was a silent no-op reported as success; the C3 scenario now uses the driver's visibleClick. Its pending → failed DOM assertion is scoped to what an instantly-refused endpoint can render; the state machine stays covered at the store layer.

Acceptance status — stated plainly

The native-continuation acceptance (E2E_NATIVE_CONTINUATION_LIVE=1) had never been executed before this pass: its anchors read callId / displayStatus / displayVariant, none of which exist on the rawEvents contract. Anchors were re-expressed against the declared contract, assertHistory fixed (JSON-escaped haystack could never match multi-line anchors), and the ≥40-event bar kept as a hard floor. With real accounts and a real 649-turn Codex session the run opens the session, passes the size bar, and stopped at the runtime-switch step. Root cause: harness, not product. The Claude source option did render (the driver's own waitUntil for it passed), but isSessionPatchedTo asserts that the imported session row was mutated — which this PR deliberately never does for external_history sessions: the pick is written to conversationTargetOverridesAtom keyed by the conversation root, and the execution episode created on send is the durable authority. The reported count:0 was the last retry strategy after the palette had closed; model: gpt-5.5 is the imported rollout's own metadata (the configured gpt-5.3-codex is the return-leg target). The driver's predicate now reads the committed target from the runtime pill's projection for conversation-bound sessions and keeps the row check for ordinary sessions. Two further harness defects surfaced once that passed: the acceptance input must be a session whose last turn contains a completed tool call (imported sessions load only the final turn's body on open, so the tool-result anchor cannot exist otherwise — the spec now preflights this before spending a live turn), and the shared driver clicked the last visible variant of a model group (claude-opus-5-high) while its expectation was clamped to the account's enabled_models (claude-opus-5), rejecting the model it had itself picked; it now prefers the exact requested row and always accepts the clicked model id. In every case the pill projection showed the pick had applied — the product behaved correctly. Local imported roots now show their continuation. The acceptance first proved the execution path with a valid Claude account (episode created, full history materialized into Claude's own catalog, claude --resume, exact reply in 22 s, durable turn intent completed, tail recovered) and then exposed that the root never showed it: ConversationStreamProvider composed the canonical timeline only from a Cloud family and gated the live runner overlay on an org2-cloud registry key, so for an imported-history/local-session root the reply lived only in the hidden execution child and the root kept the queue-synthesized user row as pending. Fixed in this PR: for a root without a Cloud target the provider resolves the root locator from the imported-history source (or the local session), loads the root's execution children through the same backend query the dispatcher uses (es_get_child_sessions, hidden children are not in the roster atom), subscribes to each child's chat events, and merges each child's new turn(s) into the root timeline (runlanded-* ids bound to the root). The boundary is the same semantic-prefix rule the dispatcher uses: the root's canonical events project to native items that must be a prefix of the child's projection, and the items beyond that prefix are the child's own turns; the child's created_at is only a fallback when the prefix cannot be established. Children that were never opened in this app instance are hydrated from their authoritative CLI history into provider state (not through the event store, whose loadFromCache on tap mount would race and wipe them); the live tap still wins whenever the store has events, so the active runner streams as before. The queue-synthesized pending row is suppressed once the same user turn has landed, and the footer/Stop runner binding follows deliveries matched on the local root key. The E2E inspector now exposes the provider-published stream as streamEvents, and the acceptance asserts on it, so the check reflects what the user sees. Acceptance result: green in both directions. With a valid Claude account and Codex quota, E2E_NATIVE_CONTINUATION_LIVE=1 passes end to end (round-trips large Codex/Claude histories in both directions, 1m59s): Codex→Claude→Codex on the real 47-turn Codex session and Claude→Codex→Claude on the real Claude session. Four live turns completed as durable turn intents with their provider-native tails recovered, three fresh execution episodes were created and one compatible episode reused, and every reply appeared on the imported root's timeline where the harness asserts it. Earlier failing runs of the same acceptance were all environment, not code: an expired auto-detected Claude token (refreshed through the CLI, then re-imported with the app's own detect→save hooks), gpt-5.3-codex being rejected for ChatGPT-login Codex accounts, and that account's usage limit (also hit by the pre-sampling compact that gpt-5.3-codex-spark needs for a 512-line thread). Picking spark at all exposed a picker bug (distinct Codex models folded into the effort-variant family and therefore unselectable); that fix is general model-picker code and ships as #1266.

Robustness gap found while testing (not fixed here — needs a design call): if the execution episode behind an accepted delivery row disappears (the child session is deleted), recoverLocalConversationTurn returns null and conversationTurnRunner treats that as recovery-pending forever, so the conversation root stays wedged with a retry every minute and every later message queues behind it. A confirmed-missing runner is a permanent state and should turn the row into a failed, retryable delivery. Related pre-existing note: the durable queue store (chat-message-queue.json) lives in the app-support directory, not under ORGII_HOME, so E2E runs share it with the developer's own instance.

The dual-instance suite's C3 (Team Chat pending/failed/retry) reaches the retry step; retry correctness is proven in-process end to end (SessionCommentsRetryProjection.test.ts, new coverage for a seam that had none).

Verification

All five gates run on the final tree of this PR (stacked on #844, both on develop c1d0e2976):

Gate Result
pnpm typecheck 0 errors
pnpm lint (oxlint + eslint --max-warnings 0) clean
vitest --changed <#844 head> 600 files / 4723 tests passed
cargo clippy --workspace --all-targets -- -D warnings 0 errors
cargo test --workspace 91 suites, 0 failed (run concurrently with #844's full suite — the load that previously exposed the WorktreeLockGuard release flake fixed in #844)

Also run against a fresh build: the rendered dual-instance suite (A/B pass; remaining failures triaged to harness staleness on develop, one pre-existing product gap, and shared-session cascades — none attributable to this stack) and the stack's own new specs (work-item-features, routine-wizard, session-account-switch), whose failures were root-caused to a spec reading E2E_ORGII_HOME instead of the published ORGII_HOME, a 1680px viewport collapsing the detail pane below its 300px placeholder threshold, and the two Routine regressions fixed in #844.

@Harry19081 Harry19081 added bug Something isn't working cloud-collaboration Cloud sync, organizations, channels, or collaboration project-management Projects, work items, routines, GitHub work, or team inbox labels Aug 25, 2026
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 98d0821 to f1420a5 Compare August 26, 2026 01:01
@Neonforge98 Neonforge98 changed the title fix(collaboration): unify human audience routing feat(conversations): unify audience routing and local continuation Aug 26, 2026
@Neonforge98
Neonforge98 changed the base branch from develop to feat/pm-expansion-batch August 26, 2026 01:02
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch 2 times, most recently from 60fb115 to 2ad6f12 Compare August 26, 2026 03:58
@Neonforge98
Neonforge98 marked this pull request as draft August 26, 2026 08:37
@Neonforge98 Neonforge98 changed the title feat(conversations): unify audience routing and local continuation feat(conversations): unify native session continuation Aug 26, 2026
@Neonforge98
Neonforge98 marked this pull request as ready for review August 26, 2026 10:08
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 157e735 to 8e9b3b4 Compare August 26, 2026 23:50
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 5e9bded to a91320a Compare August 26, 2026 23:53
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 8e9b3b4 to 61ef78b Compare August 27, 2026 16:48
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from ec54e67 to fdc296b Compare August 27, 2026 16:48
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 61ef78b to 223ebfa Compare August 27, 2026 23:25
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch 2 times, most recently from e801165 to 1c4a772 Compare August 27, 2026 23:42
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 7361aff to d601dcf Compare August 30, 2026 20:34
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 05f80a6 to bf692a2 Compare August 31, 2026 10:58
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from d601dcf to 320dbf1 Compare August 31, 2026 10:58
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from bf692a2 to 1112c74 Compare August 31, 2026 11:50
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 320dbf1 to 87e4b35 Compare August 31, 2026 11:50
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 1112c74 to 2d86124 Compare August 31, 2026 14:12
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 87e4b35 to 758b2a9 Compare August 31, 2026 14:13
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from 2d86124 to aa7c9df Compare August 31, 2026 15:12
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 758b2a9 to 36e7649 Compare August 31, 2026 15:21
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from aa7c9df to c0b0f5a Compare September 1, 2026 16:52
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from 36e7649 to 5c2d481 Compare September 1, 2026 16:52
@Neonforge98
Neonforge98 force-pushed the feat/pm-expansion-batch branch from c0b0f5a to 2b8c2f2 Compare September 2, 2026 00:11
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch 2 times, most recently from 2d06a5a to c941f95 Compare September 2, 2026 00:20
@Neonforge98
Neonforge98 changed the base branch from feat/pm-expansion-batch to develop September 2, 2026 00:44
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch from c941f95 to ce232a9 Compare September 2, 2026 00:45
@Neonforge98
Neonforge98 changed the base branch from develop to feat/pm-expansion-batch September 2, 2026 00:46
@Neonforge98
Neonforge98 changed the base branch from feat/pm-expansion-batch to develop September 2, 2026 00:51
@Neonforge98
Neonforge98 changed the base branch from develop to feat/pm-expansion-batch September 2, 2026 00:52
@Neonforge98
Neonforge98 changed the base branch from feat/pm-expansion-batch to develop September 2, 2026 01:08
@Neonforge98
Neonforge98 force-pushed the codex/audience-routing branch 2 times, most recently from 6538c8e to 4e8928a Compare September 2, 2026 01:09
@Neonforge98
Neonforge98 changed the base branch from develop to feat/pm-expansion-batch September 2, 2026 01:10
@Neonforge98
Neonforge98 changed the base branch from feat/pm-expansion-batch to develop September 2, 2026 01:42
@Neonforge98
Neonforge98 changed the base branch from develop to feat/pm-expansion-batch September 2, 2026 01:42
Define canonical conversation roots, typed events, turn intents, local continuation bindings, Cloud-plane synchronization, incremental timelines, durable publication outbox, and provider-independent sender metadata.
Convert canonical role/tool events into Codex and Claude Code native histories, bind provider session identities, preserve interrupted tool activity, resume through the selected CLI transport, and recover from provider context exhaustion.
Scope native identities by source session, reconcile durable queue projections, preserve interrupted and failed turns, keep runtime selection aligned with provider capabilities, and harden Cloud/Team conversation replay plus dual-instance coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cloud-collaboration Cloud sync, organizations, channels, or collaboration project-management Projects, work items, routines, GitHub work, or team inbox

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants