diff --git a/docs/architecture/rfcs/controller-executor-unified-contract-v0.md b/docs/architecture/rfcs/controller-executor-unified-contract-v0.md index 2bb8eab64b..03e658b045 100644 --- a/docs/architecture/rfcs/controller-executor-unified-contract-v0.md +++ b/docs/architecture/rfcs/controller-executor-unified-contract-v0.md @@ -881,6 +881,15 @@ are stable design references, not claims that a canonical Todo is complete. | **C8 — Product configuration and operational readback; C3/C6** | Existing App/settings/Chat projections: one effective configuration owner, role-aware activity, failure/recovery, Stop and artifact return | Packaged managed and supported attached heartbeat journeys, including unavailable host and late results; installed qualification separate; no new packet-format toggle | | **C9 — Retire old paths and qualify adoption; C1–C8** | Contract/runtime maintainers: remove superseded producers, format switches, duplicated prompts and tests; update protocols, guides and RFC delivery facts | Caller/field/decoder retirement inventory, historical replay and rollback evidence; App + benchmark acceptance and matched benefit before any default promotion | +The existing native context entry now supports a bounded C2 precursor: complete +captures remain on disk, the normal response exposes file navigation, and exact +file deltas do not require model retention. After compaction the next work Turn +establishes a new full saved baseline, without an immediate full reread. +Required facts still precede dependent decisions; admission and settlement are +unchanged. This does not qualify C1's unified producer/consumer cutover or the +matched effectiveness/full-cost result in §9. Rollback uses the existing +context-delivery off path; staged material composition remains separately owned. + C1's inventory is part of its usable replacement, not a standalone architecture framework. C3/C4/C5 can proceed independently once C1's contract is stable; C7 can prepare measurements before C6 but cannot claim two-session outcomes. diff --git a/docs/architecture/rfcs/controller-executor-unified-contract-v0.zh-CN.md b/docs/architecture/rfcs/controller-executor-unified-contract-v0.zh-CN.md index 21435b9949..df361d5832 100644 --- a/docs/architecture/rfcs/controller-executor-unified-contract-v0.zh-CN.md +++ b/docs/architecture/rfcs/controller-executor-unified-contract-v0.zh-CN.md @@ -663,6 +663,13 @@ backoff 继续归当前 scheduler owner。本文不引入模型空轮询或新 | **C8 — 产品配置与运行 readback；C3/C6** | 既有 App/settings/Chat 投影：同一 effective configuration owner，按角色展示 activity、失败/恢复、Stop 和产物返回 | 打包 managed 及受支持 attached heartbeat 流程，包括 host 不可用和结果迟到；安装资格另验；不新增 packet 格式开关 | | **C9 — 旧路径退役与采用资格；C1–C8** | Contract/runtime 维护者：删除被替代 producer、格式开关、重复提示/测试，更新 protocol、guide 和 RFC 交付事实 | 调用方/字段/decoder 退役清单、历史 replay 与回退证据；App + benchmark 验收及可比收益成立后再决定默认提升 | +现有原生上下文入口增加一个有界 C2 前置改进：完整捕获保存在磁盘，常规响应提供 +文件导航，精确文件差分不依赖模型保留；压缩后下一个新 Turn 建立新的完整文件基线，不要求立即 +全量重读。必要事实仍须在 +依赖性决策前理解；准入、结算不变。这不代表 C1 统一 producer/consumer 切换或 +第 9 节可比效果/完整成本验收完成。沿既有 context-delivery off 路径回滚；材料分阶段 +组合仍由原 owner 独立收敛。 + C1 盘点是可用替换的一部分，不单独建设架构框架。C1 契约稳定后，C3/C4/C5 可以独立 推进；C7 可以提前准备测量，但不能提前宣称双会话成果。实现 PR 列明替换的调用方、剩余 缺口、实际测试与回退。完整语言迁移或无关 manager 重写不是前置条件；新抽象能编译 diff --git a/docs/development/README.md b/docs/development/README.md index 109dcda400..8d5aaba2f0 100644 --- a/docs/development/README.md +++ b/docs/development/README.md @@ -246,6 +246,13 @@ cross-work dependencies and contrary or unresolved evidence require expansion before a dependent decision. Reuse an existing typed scope owner instead of adding keyword classifiers or another relevance engine. +Keep file availability separate from model retention. Saving a complete capture +permits exact file comparison, not a claim that the model read it. In native +context delivery, a new compaction epoch starts with a full saved baseline on +the next work Turn; ordinary resume and selective reads preserve the epoch. +Recovery continues the original Turn and reads missing necessary facts, without +replaying admission or draining an entire packet to enable later deltas. + Separate **acceptance criteria** from **acceptance evidence**. Criteria and current unmet/unknown status guide execution; historical reviewer notes, full provenance, logs and detailed evidence support a specific verification, dispute, replan or diff --git a/docs/reference/required-work-context.md b/docs/reference/required-work-context.md index 80e8ca59d9..ad3f4ef8a1 100644 --- a/docs/reference/required-work-context.md +++ b/docs/reference/required-work-context.md @@ -15,8 +15,8 @@ The RFC's [candidate context by decision stage](../architecture/rfcs/controller- proposes complete captured alternatives with discoverable decision facts and same-capture batch detail reads. It does not change current read obligations. The current capability gate leaves candidate selection to the Agent; a selected -Todo display is not proof that it can ignore alternatives. Existing full-context -captures must still be read completely. A future staged caller must replace its +Todo display is not proof that it can ignore alternatives. Default-off and signed-envelope captures retain their existing full-read obligations. +Enabled native file delivery instead follows the progressive-read contract below. A future staged caller must replace its delivery, required reads and recovery together, including unknown-scope full fallback; an index read cannot establish delivery of hidden bodies. @@ -62,7 +62,7 @@ change receipt replay validation or declare a model-input/token saving. Unmarked or mixed ownership keeps the original full progressive read in `agent_channel.required_reads`. Malformed markers, non-generated content inside a marked region, unavailable files and invalid UTF-8 fail closed. Unknown -narrative is never classified by headings or clipped to a size budget. Read all +narrative is never classified by headings or clipped to a size budget. For default-off and signed-envelope callers, read all delivered context and remaining reads before dependent work; changed sources need a fresh guard. Enabled scoped acceptance does not replace original Goal intent. @@ -95,8 +95,9 @@ is opened. Agent review still does not establish independent acceptance. The archive remains `projection.json`; the required model view is `required-context.json`. Required reads, full/delta preparation and recovery use -that same model view. A selected detail page cannot acknowledge the required -view, establish a full baseline or prove that other hidden details were read. +that same model view. A selected detail page cannot acknowledge full model retention of the required +view or prove that other hidden details were read. Native file baselines are +established independently from complete saved captures. The adapter edits paths selected by the typed owner without round-tripping the original JSON values through JavaScript. Unknown outcome fields, unknown source read obligations, unsupported schemas, signed envelopes and unavailable planning @@ -125,12 +126,13 @@ existing behavior. The existing size comparison retains a full view when the reference guidance would cost more than the duplicate rows. A context epoch is not a work Turn. Compaction or a failed native hook invalidates -delta reuse; it does not require another admission or unconditional replay of the +the previous epoch’s delta reuse; it does not require another admission or unconditional replay of the old quota packet. Within the existing Turn, continue from the working brief with the Goal, applicable requirements/stops, task and original capture/settlement identity. Before an action that depends on missing or uncertain facts, recover -those facts from the same immutable capture. An unfinished initial required read -still requires its missing content; summarizing it does not fulfill that read. +those facts from the same immutable capture. An unfinished selected section needs its missing facts before dependent work; +summarizing it does not prove those facts were read. Legacy captures retain +their original read contract until their current work Turn ends. Do not repeat completed work or settle the original Turn twice. If the original reference is lost, repair it before dependent work; independently authorized work can continue. A later work Turn obtains its own current guard and required @@ -279,6 +281,10 @@ transport flag does not restore the old exact Todo schema. ## Opt-in native reuse +The retained-input codec described first in this section serves explicit +confirmed-input clients. Native Codex uses the separate saved-file basis +described under Machine default; it does not require full model retention. + The existing [native context delivery option](../heartbeat-automation-prompt.md#opt-in-native-context-delivery) uses one TypeScript codec for ordinary quota and TurnEnvelope. Within a usable baseline, both representations repeat `planning_horizon.core_goal`'s original @@ -361,12 +367,11 @@ patch or an incompatible reader automatically recovers the same original full capture. Recovery never reruns the guard, asks the owner, spends quota or grants execution. Wire reduction alone does not establish actual model costs or outcomes. -After reading a native delta completely, the Agent uses its changes with the -retained model context. A resumed tool process may have no earlier Code Mode -variables; that alone does not mean the model lost the baseline. Materializing a -JavaScript object does not require another full read. If required baseline content -is unavailable or continuity is uncertain, recover the same capture in full before -dependent work. Compaction still invalidates the baseline and requires recovery. +The retained-input rules above apply to explicit confirmed-input clients. Native +Codex delivery instead compares verified saved files (see below); the model may +read changes and necessary facts progressively, without importing the full base. +Compaction starts a new baseline epoch for the next work Turn, without invalidating +the existing Turn or its recovery files. ### Machine default @@ -428,51 +433,48 @@ projection path and does not acquire a second native journal. Unrelated quota commands and digest-bound recovery pages do not activate this setting. Ordinary machine validation still applies to business operations. -Native `quota should-run` and the saved `context-project` entry share the same -paged presentation. Each returns page zero, a digest-pinned continuation and full -recovery without another admission. Inline and continuation pages use the same -64000-byte output budget, including recovery, instructions, escaping, commands -and newline. Content shrinks at UTF-8 boundaries to fit that complete stdout; -captures that fit finish inline. This default does not depend on a shell output -declaration. An explicit `--max-page-output-bytes` also bounds the initial enabled -`should-run` or `context-project` frame, its continuations and full-recovery -command. For example, append `--max-page-output-bytes 7900` to the existing guard -command for a smaller tool-output budget. This does not change host history -limits; a batch must fit their aggregate budget too. Explicit smaller reader -budgets remain available to recover a truncated range of the same capture. Consume the inline -page before following next_command; when the retained-history limit is unknown, -read one bounded output at a time. This is not a guarantee of model retention. -Forward exec stdout directly in code mode. Repeating the packet in -a hashing tool call does not establish that its original bytes were retained. -Keep these short local reader commands in the foreground; for `exec_command`, -use its normal 10000 ms wait rather than an aggressive early yield. A native -host can deliver a page before the process exits but give `PostToolUse` only the -remaining output when polling finishes. Complete model-visible text then does -not imply complete hook coverage. If a reader yields, consume all its output -chunks; missing hook coverage keeps the next decision on full delivery. Do not -fabricate a read receipt or rerun admission to make it differential. Continuing -that captured decision after context loss still uses its exact full recovery. -The reader verifies the immutable capture digest. Native lifecycle delivery uses -the same byte-range validator as explicit input observations: before merging -reader ranges, the post-tool adapter checks actual UTF-8 content against the -trusted journal's digest-pinned physical capture. A changed body, missing or -replaced file, symlink or digest failure cannot advance the baseline. It keeps -the original recovery reference; it does not collect another decision. This -checks output observed at the hook boundary, not downstream model retention. -Older metadata-only candidates require a new complete verified read before -resume or the codec can reuse them; page metadata alone cannot mark them verified. -Native lifecycle continuity still uses -an explicit engineering assumption: a normally completed full read was read and -remains available until invalidated. This is not a retained-input observation or -a model ACK. The shared codec still validates the exact baseline digest, scope, -current controls and reconstruction. Explicit final-input Host clients retain -their separate confirmed-input contract. +Native `quota should-run` and saved `context-project` persist the complete +capture and return a small file reference, not an eager first page. The response +names the delta/full `path`, digest and reader `command`; `recovery` names the +current complete file. A file delta additionally carries `baseline_reference`. +`index_command` lists current fields and `select_command` batches exact pointers. +The reference stays below 7900 serialized bytes (or the explicit smaller output +ceiling); if metadata cannot fit, recovery retains the current capture. Reader +pages remain independently bounded. A complete raw tool log is not proof that +the Host retained the same bytes in model history. + +The native codec's `captured_file` basis verifies the previous full file and +compares exact versions, including unchanged control fields. It does not depend +on a full read, page EOF, a model ACK or a retained-input claim. Goal, Agent, +session, surface, projection version and configuration generation remain scoped; +the context epoch is also bound. After compaction the next work Turn starts a new +full saved baseline; later Turns may use file deltas without a full model read. +Missing, changed, unsafe or incompatible baselines fall back to the current +full-file reference. Integers outside JavaScript's exact range and changes to +integral-float source values (including type or signed-zero changes) use that +same fallback: the bridge cannot preserve all their source distinctions. +Unchanged numeric values stay inherited in place; the codec avoids replacing +their parents or relocating normalized numeric subtrees. The original values remain +available through the same progressive reader, without a full-read obligation. +Each base is a full saved version, +so selective reading never creates a requirement to replay a chain of deltas. + +The model chooses the changes and details its current decision needs. It must +understand applicable Goal intent, acceptance criteria, constraints and stops +before dependent work, and the relevant operation/settlement contract before +effects. Supporting evidence is read when verification needs it. There is no +prerequisite to drain the entire packet, baseline or delta. After compaction, +recover only missing necessary facts from the same current capture; preserve +Turn, task and settlement identity and do not re-admit or repeat effects. A file +reference grants neither authority nor freshness. Selected/index reads never +establish full model retention; complete-page observations retain that separate +meaning for explicit retained-input consumers. For native setup, `quota context-config --codex-app --goal-id GOAL --agent-id AGENT` under machine on prepares only that private transport leaf and renders the existing hooks, recovery instructions and `config_overrides`. Applying those overrides selects a 20000-token native history limit and declares the existing 64000-byte -output profile. Both the initial guard and continuations use that page budget, +output profile. The initial guard returns the small file manifest; requested reader pages use that budget, including when the shell lacks the declaration. The worker applies this configuration automatically inside its isolated opt-in trial. The persistent instruction requests 22000 tokens for each context-read tool result and 24000 for its code-mode emission, @@ -502,11 +504,11 @@ endpoints, provider selection and connection mode, including implicit history vi `previous_response_id`. It neither enables input diagnostics nor scans model-input history. Benchmark preparation stages those same hooks in its isolated trial home. -`PostToolUse` records complete digest-bound reader ranges in the existing session -journal; EOF without preceding pages, a known malformed/truncated page or a reader -error cannot establish a baseline. Concurrent and repeated pages coalesce. +`PostToolUse` can record complete digest-bound reader ranges in the existing session +journal for the separate model-retention observation. EOF without preceding pages, +a known malformed/truncated page or a reader error cannot establish that observation. Concurrent and repeated pages coalesce. `PreCompact`, `PostCompact`, clear, fork and a changed session/workspace invalidate -it. A resume may supply different history under the same session ID; the +that observation and rotate the file epoch for the next work Turn. A resume may supply different history under the same session ID; the [native event contract](https://learn.chatgpt.com/docs/hooks#sessionstart) alone does not attest retention. The shared Codex adapter can qualify a resume against the complete read's native replay checkpoint: the native-selected physical @@ -529,19 +531,20 @@ it neither scans model requests nor asks the model for an ACK. Opted-in benchmark native children create owner-only files automatically; the parent process and off-mode file creation policy stay unchanged. Group-writable existing replay sources cannot establish cross-process continuity. -A new independent decision uses its normal guarded -context command and reads the resulting current full capture; it need not first -read an older capture to seed a delta. Continuing a captured decision after context -loss still requires complete recovery of that exact original, never a new guard -as a substitute. Both paths establish a new chain only after a complete verified -full read. Ordinary tools and turn endings do not invalidate it. An interrupted -pending delivery requires full recovery. Repeated reset events are idempotent. -Recovery reads the same immutable capture without re-admission or spend. +A new independent decision uses its normal guarded context command. Actual +compaction selects a new full saved baseline; ordinary resume preserves the file +epoch without claiming model retention. A previously qualified replay checkpoint +that now fails may conceal compaction and conservatively rotates the epoch. +No prior complete-read checkpoint is required for file comparison. Continuing a +captured decision after loss recovers only the necessary missing facts from that +same original, with no new guard or spend. Ordinary tools and Turn endings do +not invalidate file comparison. Repeated reset events are idempotent. The adapter commits the current complete capture reference and pending delivery under the existing session lock before optional material planning or encoding. A process exit during planning therefore recovers the current source rather -than its predecessor. The planner reuses that archive; an unread current source -still cannot establish a full native baseline. +than its predecessor. The planner reuses that archive. A saved file can support +file comparison without implying that its contents were read by the model. + This deliberately replaces per-request complete-input proof with lifecycle assumptions. Downstream output loss or truncation not exposed to the hook, a @@ -680,10 +683,9 @@ quota 能力候选沿用同一精确 codec 复用已登记 Todo 的正文、标 或读者不兼容时，自动恢复同一原始 full capture，不重跑 guard、不问询、不扣额， 也不授予执行权限。传输减少本身不证明实际模型费用或结果改善。 -完整读完原生差分后，Agent 将变化与模型仍保留的上下文一起使用。恢复后的工具进程可能 -没有此前的 Code Mode 变量；仅此并不表示模型丢失了基线。不必为了组装 JavaScript -对象再读一次全量。若当前工作所需的基线内容不可用或连续性不确定，仍须先完整恢复 -同一捕获，再做依赖它的工作。压缩仍会使基线失效，并要求恢复。 +上面的保留输入规则适用于显式 confirmed-input 客户端。原生 Codex 改用校验后的 +文件版本比较：模型可渐进式读取变化和必要事实，无须把完整基线装入上下文。 +压缩后下一个新 Turn 建立新的完整文件基线，不使已有 Turn 或恢复文件失效。 ### 机器级默认 @@ -718,38 +720,26 @@ Codex App SSH Goal（`--runtime-profile codex_app_ssh_goal`），以及现有 Co 私有传输末级目录并生成已有 hook 和恢复指令，不安装或信任 hook，也不更改宿主配置。 SSH Goal 使用 `--runtime-profile codex_app_ssh_goal` 生成 context-config 和 thin/bootstrap 提示， 沿用可见 Goal 的执行与结算身份；仅启用差分时接入捕获目录，不转换成自动心跳。 -原生 `quota should-run` 与已保存捕获的 `context-project` 共用分页呈现，直接返回第零页、 -固定摘要的后续读取与完整恢复，不另行准入。内联页与续页共用 64000 字节输出预算， -包含恢复字段、完整指引、转义、后续命令与换行；正文按 UTF-8 边界缩小以适配完整 stdout， -能放下的捕获一次返回。默认值不依赖 shell 输出声明；显式更小的读取预算仍可用于恢复 -同一捕获中被截断的范围。先消费内联页，再跟随 next_command；保留历史 -上限未知时，每次只读一个有界输出。该上限不证明模型保留。Code Mode 直接转发 -exec stdout；把整包重打进哈希工具不构成原始字节已被保留的证明。 -这些短时本地读取命令应前台等到结束；`exec_command` 使用正常的 10000 ms 等待， -避免激进提前返回。原生宿主可能先把页面交给模型，却只把轮询结束时剩余的输出交给 -`PostToolUse`；模型可见正文完整不等于 hook 观察范围完整。读取器若提前返回，仍须 -消费全部输出片段；hook 覆盖缺失时，下次决定继续全量投递。不伪造读取回执或重跑 -准入来制造差分；丢失上下文后继续该已捕获决定，仍沿它的精确全量恢复引用读取。 -原生配置同时返回 `config_overrides`：应用后，原生历史上限为 20000 token，并声明已有 -64000 字节输出配置。首次 guard 和续页均使用此分页预算，shell 缺少声明时也一样。 -开启的 benchmark worker 在其 -隔离会话内自动应用；持久指引要求每次续读直接输出预算 22000 token、Code Mode -输出预算 24000 token，每次直接转发一份 stdout。 -压缩/恢复 hook 同样携带这段调用预算指引，避免恢复依赖已经丢失的配置正文。 -内层读取与外层 Code Mode 输出可分别截断；单次预算请求不证明历史保留，也不修改宿主配置。 -任一层截断时，减小正文和完整输出字节预算，重读同一 capture 的受影响范围。 -这仅修复传输重试故障，不代表超大的完整上下文已能容纳，也不证明范围裁剪合格。 -回滚时同时移除生成的输出覆盖项；仅保存机器偏好并不会应用这些宿主配置。 -该设置不证明输入保留、总费用降低或任务效果改善。 -读取器校验不可变 capture 的摘要。Post-tool 适配层复用显式输入观察的字节范围校验， -先将实际 UTF-8 正文与可信 journal 指向的物理捕获及固定摘要核对，再合并读取范围。 -正文改变、文件丢失或替换、符号链接及摘要失败均不得晋升基线；保留原始恢复引用， -不重新收集决策。这只验证 hook 边界观察到的输出，不证明下游模型输入保留。 -旧版仅凭页元数据建立的候选须先完整读取并通过字节校验，才能在 resume 或 codec 中复用； -页元数据本身不能写入已校验标记。 -原生连续性仍明确采用工程假设：正常完整交付的首包已读， -在失效事件发生前仍可用；这不是实际输入保留证明或模型 ACK。共享 codec 仍校验精确基线 -摘要、作用域、本轮控制与完整重建。显式 final-input Host 客户端保留独立的确认输入契约。 +原生 `quota should-run` 与 `context-project` 先保存完整捕获，默认返回小型文件入口， +不内联第一页。`path`、摘要与 `command` 指向当前差分或全量；`recovery` 指向当前 +完整文件；文件差分另带 `baseline_reference`。`index_command` 展示当前字段目录， +`select_command` 批量选读精确路径。引用输出不超过 7900 序列化字节或调用者显式更小 +的上限；元数据放不下时保留当前捕获恢复入口。读取页另受输出预算约束。 +原始工具日志完整，不证明宿主保留给模型的字节也完整。 + +原生 `captured_file` 基线来自校验后的前一完整文件，可差分复用所有未变字段，包括 +控制字段；不依赖通读、EOF、模型 ACK 或保留输入声明。Goal、Agent、会话、surface、 +投影版本、配置 generation 和上下文 epoch 仍隔离；压缩后下一个新 Turn 建立完整 +文件基线，同周期后续 Turn 再差分，无须模型先通读这份基线。 +基线缺失、损坏、不安全或不兼容时回退当前完整文件入口；超出 JavaScript 精确范围的 +整数也回退，以保留原值。每个基线都是完整保存版本，无须因选读而重放差分链。 + +模型自主选择本次决策需要的变化和详情：依赖性工作前理解适用的 Goal 意图、验收标准、 +约束及停止条件，执行副作用前理解相关操作/结算契约，验证时按需查证材料。 +不要求把整包、基线或差分读完才能工作。压缩后仅从同一当前捕获恢复缺失的必要事实， +保留 Turn、任务和结算身份，不重新准入或重复副作用。引用不授予权限或新鲜度。 +目录/选读不构成全量模型保留证明；完整页面观察仍为显式 retained-input 消费者 +保留独立含义。现有原生配置的读取预算只约束被选择的页面，不强制通读。 为下次原生会话配置并信任精确 hooks，不重启当前 App。已加载线程可能继续沿用旧 hook 配置；`hooks/list` 的 enabled/trusted 读回不等于向该线程交付 `SessionStart` 或初始化 @@ -759,10 +749,11 @@ context journal。当前工作到达正常安全边界后，沿宿主正常启 和连接方式，包括带 `previous_response_id` 的隐式历史；不启用输入诊断，也不逐轮扫描 模型完整输入。Benchmark 在各自隔离的 trial home 内自动准备同一组 hooks。 -`PostToolUse` 在现有 session journal 中合并摘要固定的完整读取范围；缺少前页的 EOF、 -已知的格式损坏／截断或读取失败不能建立基线。并发、重复页幂等合并。`PreCompact`、 -`PostCompact`、清空、分叉、会话／工作区切换使基线失效；同一会话 ID 的 resume -也可能使用不同历史，[原生事件契约](https://learn.chatgpt.com/docs/hooks#sessionstart)本身不证明保留。 +`PostToolUse` 可在现有 session journal 中合并摘要固定的完整读取范围，用于独立的 +模型保留观察；缺少前页的 EOF、已知格式损坏／截断或读取失败不能建立该观察。 +并发、重复页幂等合并。`PreCompact`、`PostCompact`、清空、分叉、会话／工作区切换 +使该观察失效，并为下一个新工作 Turn 轮换文件 epoch；同一会话 ID 的 resume 也可能 +使用不同历史，[原生事件契约](https://learn.chatgpt.com/docs/hooks#sessionstart)本身不证明保留。 共享 Codex adapter 可核对完整读取时的原生重放检查点：原生实际选择的物理历史文件， 须保留从当前基线首次 reader 开始的摘要固定区间，以及每份精确 reader 输出，完整覆盖固定捕获的全部字节； 会话、工作区、模型和权限模式相同， @@ -776,14 +767,14 @@ code-mode 父调用的保留输出须包含嵌套 reader hook 观察到的全部 工程假设，不声称证明未来模型输入；不扫描模型请求，也不要求模型 ACK。 开启后的 benchmark 原生子进程自动以仅所有者可访问的权限创建文件，父进程与关闭态 保持原权限策略；既有可组写的重放来源不能建立跨进程连续性。 -新的独立决定走正常准入与上下文命令，完整读取其返回的当前全量捕获；不必为了建立差分 -基线而先读旧捕获。若丢失上下文后继续某个已捕获决定的依赖工作，仍须完整恢复那份精确 -原捕获，不能用新 guard 替代。两条路径都只在完整且已校验的全量读取后建立新差分链。 -普通工具和正常轮次结束不无条件失效。尚未完成的交付被中断 -后须完整恢复。重复重置事件幂等；恢复读取同一不可变捕获，不重新准入或扣额。 +新的独立决定走正常准入与上下文命令。实际压缩后，下一个新工作 Turn 建立新的完整 +文件基线；普通 resume 保留文件 epoch，不声称模型保留。曾合格的重放检查点若核验 +失败，可能隐藏压缩，保守地轮换 epoch；文件比较本身不要求先有完整读取检查点。 +上下文丢失后继续原决定，只从同一原始捕获恢复缺失的必要事实，不重新准入或扣额。 +普通工具和 Turn 结束不会使文件比较失效；重复重置事件幂等。 Adapter 在可选材料规划或编码前，先持既有 session 锁提交当前完整 capture 引用及 pending delivery。规划过程中进程退出后，恢复指向当前来源而非上一份；planner -复用同一 archive。尚未读取的当前来源仍不能建立完整 native baseline。 +复用同一 archive。完整落盘可支持文件比较，不表示模型已经读过其内容。 这是以生命周期假设替换逐轮完整输入证明。Hook 未暴露的下游丢失／截断、漏掉失效事件， 或在声明开关之外静默禁用 hooks，仍可能漏检；不把这些假设称为实测保留，也不新增审批门。 @@ -828,12 +819,15 @@ original JSON values (including large integers), and performs no admission, settlement or live Goal lookup. Selection requires the existing typed runtime; ordinary full reads retain their file-only path. -Selected EOF, including a root selection, cannot establish a full-context -baseline in the native hook. This shared reader enables exact detail expansion; -it does not change which facts a caller must read, relax required reads, or -switch existing native input to a necessary-only view. Those consumers must -integrate their scope, instructions and recovery together before claiming a -progressive-delivery or total-cost improvement. +`quota context-read --list` discovers immediate child pointers, JSON types, +serialized value sizes and object/array counts without returning their bodies. +Combine `--list` with `--select` to explore nested sections; omit the selectors +for the root. Its `context_capture_index_v1` view shares the digest-bound +pagination contract, but an index cursor cannot be used for a value selection. + +Selected/index EOF, including root selection, never proves full model retention. +Native file comparison is independent of that claim. The model can inspect the +current file directly, or consult its baseline and delta, without a full drain. `quota context-read --select /path --select /another/path` 用 RFC 6901 JSON pointer 从原始摘要绑定的 capture 批量读取详情。提供已有 `--capture-file` 与 @@ -851,9 +845,11 @@ pointer 从原始摘要绑定的 capture 批量读取详情。提供已有 `--ca 沿 `next_command` 继续，它保留两个摘要及精确路径；非零偏移必须携带 `--selection-sha256`。读取器校验两者，保留包括大整数在内的原 JSON 值， 不准入、不结算、不查询实时 Goal。选择模式依赖已有 typed runtime；普通全量 -读取仍只走文件路径。包括根对象选择在内的详情 EOF 均不能被原生 hook 记为 -完整上下文基线。此共享读取器提供精确展开能力，不改变调用方必读事实或自动切换 -必要视图；调用方须共同接入范围、指引与恢复后，再验收渐进投递及完整成本收益。 +读取仍只走文件路径。`--list` 列出直接子字段的精确 pointer、类型、序列化字节数 +及对象/数组成员数，不返回正文。与 `--select` 组合可探索嵌套章节；省略路径从根开始。 +`context_capture_index_v1` 沿用双摘要分页，目录 cursor 不能用作正文选择 cursor。 +包括根选择在内的目录/详情 EOF 不证明全量模型保留；原生文件比较与此独立。 +模型可直接查当前完整文件，也可查基线和差分，无须先通读。 ### Saved context cache / 已保存上下文缓存 @@ -868,34 +864,14 @@ Read-only recovery never creates a cache. With context delivery off, projection remains inert and reads the original capture. Failures retain that capture's exact digest and Turn receipt; they never rerun admission or spend quota. -With delivery on, a successful saved projection also returns page zero using -the existing `context_capture_page_v1` fields. It uses the same 64000-byte -complete-stdout budget as continuations, including recovery, activation and -instructions; captures that fit finish inline. JSON `context-read` with no page -arguments also defaults to 64000 content bytes under a 64000-byte output cap. -Markdown retains its 4096-byte content default. Explicit page/output arguments -retain their prior semantics; a page above 8192 bytes still requires a sufficient -output cap, within the existing capture limit. The JSON-only -`--max-page-output-bytes` also bounds the initial enabled native guard/projection -frame; it shrinks content at UTF-8 boundaries and persists in -`next_command`, including for explicitly smaller same-capture recovery. An -insufficient cap returns an error without a zero-progress page; increase it for -the same capture. Error messages are not subject to an impossible page limit. -The reader changes no host or retained-history setting. Historical Codex 0.162.1 -qualification used a 10,000-token history setting and a 12,000-token code-mode -result budget with one directly forwarded stdout per call; that observation -does not establish another host's output capacity. The shared native setup above -supplies matching history/execution budgets; only the opted-in worker applies -them automatically. Larger defaults never prove history retention, and missing -output still requires recovery from this same capture. -Consume `content`, then follow `next_command` through `eof` for full byte coverage. -The reader verifies the original digest; do not retype or hash the content. The -existing `command` still restarts at zero for older readers or incomplete inline -output. If the optional first-page read fails, that same command remains usable. -Off responses do not add an inline page or activate native reuse; their recovery -references use the same JSON reader default. No admission -or retention receipt is introduced; an inline page alone cannot prove complete -input retention. +With delivery on, the saved projection uses the file-first native entry above. +An explicit full read remains available: follow `next_command` through EOF only +when that complete content is needed. JSON `context-read` defaults to 64000 +content bytes under a 64000-byte serialized-output ceiling; Markdown retains its +4096-byte content default. Explicit limits and UTF-8 byte cursors are unchanged. +Oversized or missing tool output is recovered from the affected range of this +same capture, never by another admission. Off responses retain their original +contract. Reader/output sizing alone proves neither model retention nor savings. `quota context-project` 开启上下文投递后，不再要求另外手工准备缓存目录。 省略 `--decision-output-root` 时复用原生会话的私有上下文目录；显式指定的新目录 @@ -904,24 +880,8 @@ input retention. 只读恢复不创建缓存；关闭上下文投递时保持无副作用，读取原始 capture。失败仍保留 原始摘要和 Turn 回执，不重新准入或扣减额度。 -开启时，成功的已保存投影同时用现有 `context_capture_page_v1` 字段返回第一页。 -该页与续页共用 64000 字节完整 stdout 预算，包含恢复、启用配置与完整指引；能放下的 -捕获一次返回。JSON `context-read` 省略分页参数时，也默认在 64000 字节输出上限下 -请求 64000 字节正文；Markdown 保留 4096 字节正文默认值。显式分页/输出参数保留 -原有语义：超过 8192 字节的页仍须足够的输出上限，且不超过既有 capture 限制。 -JSON 专用参数 `--max-page-output-bytes` 按 UTF-8 边界缩减正文，并在 `next_command` -中保留，也约束开启原生投递的 `should-run`／`context-project` 首次输出及完整恢复命令。 -例如在已有 guard 命令后追加 `--max-page-output-bytes 7900`；省略时仍为 64000 字节。 -该上限不修改宿主历史容量，批量输出还须满足总预算。显式更小的同捕获恢复也一样。 -上限不足时返回错误，不返回无进展分页； -增大上限后重读同一捕获。错误消息不受无法满足的分页上限限制。 -读取器不修改宿主或保留历史设置。历史 Codex 0.162.1 定向验证采用 10,000 token 历史 -设置和 12,000 token code-mode 输出预算，每次直接转发一份 stdout；该观察不证明 -其他宿主的输出容量。上述共享原生配置提供配套历史和执行预算，仅开启的 worker -自动应用。较大默认值不证明历史保留，缺失输出仍恢复同一 capture。 -先读取 `content`，再沿 `next_command` 读至 `eof`，确保完整字节覆盖。 -读取器校验原始摘要；无需重抄正文或计算哈希。 -原有 `command` 仍从零开始，供旧读取方或内联输出不完整时重读；可选的首屏读取 -失败时也保留该命令。关闭态不增加内联分页或启用原生复用；其恢复引用同样采用 JSON -读取器默认值。不新增准入或保留回执； -单页内联内容不证明完整输入仍被保留。 +开启时采用上面的文件入口。需要全文时仍可显式读取并沿 `next_command` 到 EOF； +不将此操作设为通用前置条件。JSON `context-read` 默认请求 64000 字节正文，完整输出 +上限 64000 字节；Markdown 保留 4096 字节正文默认值。显式上限和 UTF-8 字节 cursor +语义不变。工具输出过大或缺失时，只恢复同一捕获的受影响范围，不重新准入。 +关闭态保持原有契约。页大小与输出预算本身不证明模型保留或成本收益。 diff --git a/loopx/cli_commands/quota_capture.py b/loopx/cli_commands/quota_capture.py index 5f836e5ff5..5693e6a568 100644 --- a/loopx/cli_commands/quota_capture.py +++ b/loopx/cli_commands/quota_capture.py @@ -18,7 +18,7 @@ from ..control_plane.quota.error_codes import QuotaCommandValidationError, QuotaDecisionTransportError from ..control_plane.turn_driver.decision_capture import ( validate_handoff_socket, CONTEXT_CAPTURE_READ_GUIDANCE, CONTEXT_CAPTURE_PAGE_GUIDANCE, context_capture_read_command, - NATIVE_CONTEXT_PAGE_BYTES, NATIVE_CONTEXT_PAGE_OUTPUT_BYTES, + NATIVE_CONTEXT_PAGE_BYTES, NATIVE_CONTEXT_PAGE_OUTPUT_BYTES, CONTEXT_CAPTURE_NAVIGATION_GUIDANCE, native_context_page_budget, read_context_capture_file as _read_capture, ) @@ -39,7 +39,8 @@ def _capture_page(path: Path, raw: bytes, digest: str, *, offset: int, size: int next_page_bytes: int | None = None, next_page_output_bytes: int | None = None, select: list[str] | None = None, - selection_sha256: str | None = None) -> dict[str, Any]: + selection_sha256: str | None = None, + list_fields: bool = False) -> dict[str, Any]: end = min(offset + size, len(raw)) while end < len(raw) and raw[end] & 0xC0 == 0x80: end -= 1 @@ -52,6 +53,8 @@ def page(text: str) -> dict[str, Any]: "total_bytes": len(raw), "eof": cursor == len(raw), "content": text} if select is not None: result.update(selection_sha256=selection_sha256, selected_pointers=select) + if list_fields: + result["list_fields"] = True result["read_instruction"] = ("This is selected detail from the original capture, not a complete context read. " "Offsets and EOF describe only this selected view. Follow next_command to finish it; " "keep both digests. Detail delivery does not establish a full-context baseline or grant authority.") @@ -60,7 +63,7 @@ def page(text: str) -> dict[str, Any]: if cursor < len(raw): result["next_command"] = context_capture_read_command(path, digest, offset=cursor, page_bytes=next_page_bytes or size, max_page_output_bytes=next_page_output_bytes or output_limit, - select=select, selection_sha256=selection_sha256) + select=select, selection_sha256=selection_sha256, list_fields=list_fields) return result def fits(result: dict[str, Any]) -> bool: @@ -134,7 +137,10 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An raise ValueError("context-read requires a nonnegative byte offset and page-bytes between 256 and 8192; " "larger pages require a sufficient max-page-output-bytes within the capture limit") path, raw, payload = _read_capture(value, digest) + list_fields = getattr(args, "list_fields", False) selectors = getattr(args, "select", None) + if list_fields and selectors is None: + selectors = [""] selection_digest = getattr(args, "selection_sha256", None) if selectors is None and selection_digest is not None: raise ValueError("selection-sha256 requires --select") @@ -145,14 +151,30 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An raise ValueError("selected continuation requires selection-sha256; follow next_command") from ..control_plane.quota.turn_envelope import _turn_envelope_result - plan = _turn_envelope_result("context_select", context=payload, select=selectors) + plan = _turn_envelope_result("context_select", context=payload, select=selectors, list_fields=list_fields) views: list[dict[str, Any]] = [] for item in plan["selections"]: selected: Any = payload for key in item["path"]: selected = selected[key] - views.append({"pointer": item["pointer"], "value": selected}) - raw = json.dumps({"schema_version": "context_capture_selection_v1", "selections": views}, + if list_fields: + fields = [] + for child in item["children"]: + value = payload + for key in child["path"]: + value = value[key] + field = {"pointer": child["pointer"], "type": + "object" if isinstance(value, dict) else "array" if isinstance(value, list) + else "null" if value is None else "boolean" if isinstance(value, bool) + else "string" if isinstance(value, str) else "number", + "bytes": len(json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode("utf-8"))} + if isinstance(value, (dict, list)): + field["count"] = len(value) + fields.append(field) + views.append({"pointer": item["pointer"], "fields": fields}) + else: + views.append({"pointer": item["pointer"], "value": selected}) + raw = json.dumps({"schema_version": "context_capture_index_v1" if list_fields else "context_capture_selection_v1", "selections": views}, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n" expected = hashlib.sha256(raw).hexdigest() if selection_digest is not None and selection_digest != expected: @@ -164,7 +186,7 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An return _capture_page(path, raw, digest, offset=offset, size=size, output_limit=output_limit, instruction=bool(context_dir), presentation=presentation, next_page_bytes=next_page_bytes, - next_page_output_bytes=next_page_output_bytes, select=selectors, selection_sha256=selection_digest) + next_page_output_bytes=next_page_output_bytes, select=selectors, selection_sha256=selection_digest, list_fields=list_fields) except (OSError, ValueError, RecursionError, EffectRuntimeRemoteError, EffectRuntimeStartupError) as exc: # Recovery is presentation-only: keep failures separate from quota's # collection error path and retain the existing receipt/capture. @@ -178,10 +200,10 @@ def native_context_capture_view(projected: dict[str, Any], directory: Path, *, context_directory: str | None = None, metadata: dict[str, Any] | None = None, max_page_output_bytes: int | None = None) -> dict[str, Any]: - """One paged presentation for ordinary and already-captured native calls. + """Publish file references for an already-decided native capture. - The guard/projector already ran. Only serialize its wire and prefetch the - existing digest-checked reader; never collect or admit another decision. + Persist the complete wire and expose navigation without eagerly importing + content into model history. Reading never collects or admits a decision. """ from ..control_plane.turn_driver.decision_capture import write_decision_capture @@ -208,34 +230,51 @@ def native_context_capture_view(projected: dict[str, Any], directory: Path, *, result["recovery"] = {**recovery, "command": context_capture_read_command( Path(recovery["path"]), recovery["sha256"], page_bytes=page_bytes, max_page_output_bytes=output_bytes)} - # The codec puts native continuity semantics before its change tree. Keep - # this shared transport frame neutral instead of copying that instruction - # or replacing an assumption with a claim of confirmed final input. - presentation = ("This is the full current capture; after complete reading, continue from it." - if mode == "full" else "Read the current continuity basis and instruction before its change tree.") - instruction = ( - "Presentation of the original decision only; authority and freshness are unchanged. " - + presentation + " ") - result["instruction"] = (instruction + "This result includes page zero; consume it before next_command. " - "command restarts at zero if the inline page is incomplete or unsupported. " + CONTEXT_CAPTURE_PAGE_GUIDANCE) - # Bound the complete stdout, including caller-owned recovery and activation - # fields. Capping only content leaves the guard itself vulnerable to native - # truncation before the consumer can discover its continuation command. - page = read_context_capture(argparse.Namespace(capture_file=reference["path"], - capture_sha256=reference["sha256"], offset=0, page_bytes=page_bytes, - format="json", max_page_output_bytes=output_bytes), - presentation=result, next_page_bytes=page_bytes, next_page_output_bytes=output_bytes) - if page["ok"]: - result = page - from ..control_plane.turn_driver.codex_context import register_native_context_view - - register_native_context_view(context_directory, - {**result, "ancestry": projected.get("ancestry", 0)}) - else: - result["instruction"] = instruction + CONTEXT_CAPTURE_PAGE_GUIDANCE - if len(json.dumps(result, ensure_ascii=False, separators=(",", ":")).encode()) + 1 > output_bytes: - raise QuotaDecisionTransportError("Native context metadata exceeds the output limit; " - "read the saved original decision and receipt without rerunning admission.") + # Persist first. A normal guard returns a small reference, never an eager + # page that can be cut by an unrelated Host/history budget. The optional + # delta remains readable in full; navigation always addresses current facts. + current = projected.get("recovery") if projected.get("schema_version") == "context_delivery_v1" else reference + current = current or reference + current_path = Path(current["path"]) + result.setdefault("recovery", {"path": str(current_path), "sha256": current["sha256"], + "command": context_capture_read_command(current_path, current["sha256"], + page_bytes=page_bytes, max_page_output_bytes=output_bytes)}) + current_raw = current_path.read_bytes() + current_document = json.loads(current_raw) + if projected.get("continuity_basis") == "captured_file": + result["continuity_basis"] = "captured_file" + result["baseline_reference"] = dict(projected["baseline_reference"]) + result["total_bytes"] = Path(reference["path"]).stat().st_size + result["index_command"] = context_capture_read_command(current_path, current["sha256"], + page_bytes=min(page_bytes, 24000), max_page_output_bytes=min(output_bytes, 24000), list_fields=True) + result["select_command"] = context_capture_read_command(current_path, current["sha256"], + page_bytes=page_bytes, max_page_output_bytes=output_bytes) + " --select ''" + # A complete scalar preview helps orientation without inventing a second + # decision owner. Oversized or structured values remain discoverable in the + # index; no prose is truncated or treated as an authorization summary. + result["overview"] = {key: value for key, value in current_document.items() + if value is None or isinstance(value, (str, bool, int, float)) + and len(json.dumps(value, ensure_ascii=False).encode("utf-8")) <= 240} + result["instruction"] = CONTEXT_CAPTURE_NAVIGATION_GUIDANCE + if mode != "full": + result["instruction"] += (" command reads the delta; baseline_reference identifies its exact saved base when file-backed. " + "index_command/select_command read current facts directly without that baseline.") + # The reference must fit even under an explicit small caller ceiling. + def fits(): + return len(json.dumps(result, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= min(output_bytes, 7900) + if not fits(): + result.pop("overview") + if not fits(): + result.pop("select_command") # --select also works on the current recovery command. + if not fits() and "baseline_reference" in result: + result["baseline_reference"].pop("command", None) # Exact path/hash remain usable. + if not fits(): + raise QuotaDecisionTransportError("Native context reference exceeds the output limit; " + "read the saved original decision and receipt without rerunning admission.") + from ..control_plane.turn_driver.codex_context import register_native_context_view + + register_native_context_view(context_directory, + {**result, "ancestry": projected.get("ancestry", 0)}) return result @@ -354,7 +393,7 @@ def recovery_reference(value: dict[str, Any]) -> dict[str, str]: "capture_sha256": args.capture_sha256, **({"context_delivery": activation} if activation["enabled"] else {}), "instruction": "Presentation of the original decision only; authority and freshness are unchanged. " - + CONTEXT_CAPTURE_READ_GUIDANCE} + + (CONTEXT_CAPTURE_NAVIGATION_GUIDANCE if activation["enabled"] else CONTEXT_CAPTURE_READ_GUIDANCE)} if native_view is not None: result.update(native_view) return result @@ -557,6 +596,9 @@ def handoff_decision(directory: Path | None, socket_path: str | None, def handle_native_context_command(args: argparse.Namespace, *, print_payload: Callable[..., None], registry_path: Path | None = None, runtime_root_arg: str | None = None) -> int | None: """Dispatch native presentation/lifecycle IO before any quota authority read.""" + if getattr(args, "list_fields", False) and args.quota_command != "context-read": + print_payload({"ok": False, "reason": "--list is only supported by context-read"}, args.format, str) + return 1 has_output_limit = getattr(args, "max_page_output_bytes", None) is not None budget_scope_error = ("max-page-output-bytes is only supported by JSON context-read " "or enabled native should-run/context-project") @@ -619,7 +661,7 @@ def handle_native_context_command(args: argparse.Namespace, *, print_payload: Ca "Omit per-command activation and location flags to retain that inheritance. " if machine_runtime else "Pass --context-delivery on with the same declared host mode, context location and a decision-output-root to heartbeat-prompt/should-run. ") + - "Normal completed full reads establish an explicit continuity assumption; no input diagnostics or provider change is required. " + "Verified saved files support progressive reads and exact deltas without a full model read; compaction starts a new full baseline on the next work Turn. " "The generated output profile uses 64000-byte continuations and a 20000-token native history limit; " "the same page profile applies without the shell override. Explicit smaller reader budgets remain available for truncated-range recovery. " "Set --context-delivery off and remove the generated context configuration, including its output overrides, to restore ordinary output. " diff --git a/loopx/cli_commands/quota_registration.py b/loopx/cli_commands/quota_registration.py index 446ce24702..3dbce09fd4 100644 --- a/loopx/cli_commands/quota_registration.py +++ b/loopx/cli_commands/quota_registration.py @@ -231,6 +231,9 @@ def register_quota_command( quota_parser.add_argument("--select", action="append", help="For context-read: repeat an exact RFC 6901 JSON pointer to read a batch from the original capture. " "Selected detail pages never acknowledge full-context retention; omit for unchanged full capture reads.") + quota_parser.add_argument("--list", dest="list_fields", action="store_true", + help="For context-read: list immediate fields at the selected pointers (root by default), " + "with exact pointers, types and sizes. Does not read bodies or establish a delta baseline.") quota_parser.add_argument("--selection-sha256", help="For selected context-read continuations: exact derived-view digest from the preceding page.") quota_parser.add_argument("--offset", type=int, default=0, help="For context-read: byte offset returned by the previous page.") diff --git a/loopx/control_plane/quota/context_delivery.ts b/loopx/control_plane/quota/context_delivery.ts index 9924f532fc..bed18be2e9 100644 --- a/loopx/control_plane/quota/context_delivery.ts +++ b/loopx/control_plane/quota/context_delivery.ts @@ -343,7 +343,7 @@ function groupCurrentValues(delta: Exclude(); @@ -366,6 +366,26 @@ function changeTree(before: unknown, after: unknown, forced: Path[]): Delta | nu }; const exact = new Set(forced.map(pathKey)), ancestors = new Set(); for (const path of forced) for (let i = 0; i < path.length; i++) ancestors.add(pathKey(path.slice(0, i))); + // Saved files distinguish 1 from 1.0 and positive from negative zero even + // when this bridge cannot. The adapter rejects changed source tokens. + // Preserve unchanged numeric leaves without rewriting their parents or + // relocating a normalized numeric subtree from another file location. + const numericSubtrees = new Set(), inheritedNumbers = new Set(); + const numericPaths = (a: unknown, b: unknown, path: Path): void => { + if (typeof a === "number" && Number.isInteger(a)) { + for (let i = 0; i <= path.length; i++) { + const key = pathKey(path.slice(0, i)); + numericSubtrees.add(key); + if (Object.is(a, b)) inheritedNumbers.add(key); + } + } else if (Array.isArray(a)) { + a.forEach((value, i) => numericPaths(value, Array.isArray(b) ? b[i] : undefined, [...path, i])); + } else if (jsonObject(a)) { + for (const [key, value] of Object.entries(a as JsonObject)) + numericPaths(value, jsonObject(b)?.[key], [...path, key]); + } + }; + if (preserveFileNumbers) numericPaths(before, after, []); // Binding or source reordering can relocate already-read authoring. Index // exact strings and structured subtrees from the immutable base. A target // with current descendants needs literal overrides, never a bare reference. @@ -379,7 +399,7 @@ function changeTree(before: unknown, after: unknown, forced: Path[]): Delta | nu strings.set(value, reference); } else if (Array.isArray(value) || jsonObject(value)) { const reference = {from: path}, valueSize = size(value); - if (bytes(reference) < size({value})) { + if (!numericSubtrees.has(pathKey(path)) && bytes(reference) < size({value})) { const bucket = subtrees.get(valueSize) ?? {candidates: []}; bucket.candidates.push({reference, value}); subtrees.set(valueSize, bucket); @@ -444,7 +464,7 @@ function changeTree(before: unknown, after: unknown, forced: Path[]): Delta | nu } else delta = {value: clone(b)}; // Sizing does not publish a value. Copy only a replacement that is chosen, // rather than every unchanged subtree and every discarded ancestor value. - let result = size(delta) < size({value: b}) ? delta : {value: clone(b)}; + let result = inheritedNumbers.has(key) || size(delta) < size({value: b}) ? delta : {value: clone(b)}; if (reference && size(reference) < size(result)) result = clone(reference); return overlay && size(overlay) < size(result) ? overlay : result; }; @@ -536,17 +556,28 @@ export function prepareContextDelivery(request: JsonObject, legacyQuotaFields: r const assumed = !retained && continuity?.basis === "native_hook_assumption" && continuity.capture_bytes_verified === true && equal(continuity.target, binding); - if (!binding || !prior || (!confirmed && !assumed) || !compatible(binding, target) + // File comparison is exact artifact reconstruction, not a retention claim. + // A Host must verify its immutable base within the current context epoch. + const fileBacked = !retained && continuity?.basis === "captured_file" + && continuity.capture_bytes_verified === true && equal(continuity.target, binding); + if (!binding || !prior || (!confirmed && !assumed && !fileBacked) || !compatible(binding, target) || binding.context_digest !== digest(prior)) return full("baseline_unavailable"); const depth = baseline?.ancestry; if (!Number.isSafeInteger(depth) || Number(depth) < 0 || Number(depth) >= MAX_ANCESTRY) return full("full_refresh_due"); const before = freshPaths(prior, surface, legacyQuotaFields, binding), after = freshPaths(context, surface, legacyQuotaFields, target); - if (!before.valid || !after.valid) return full("source_not_reusable"); - const changes = changeTree(prior, context, after.paths); + if (!fileBacked && (!before.valid || !after.valid)) return full("source_not_reusable"); + const changes = changeTree(prior, context, fileBacked ? [] : after.paths, fileBacked); const delta: JsonObject = {schema_version: VERSION, mode: "delta", target, base: binding, ancestry: Number(depth) + 1, changes, - ...(assumed ? {continuity_basis: "native_hook_assumption"} : {}), - instruction: "Current controls are inline even when unchanged. Other fields inherit the " + ...(fileBacked ? {continuity_basis: "captured_file"} : assumed ? {continuity_basis: "native_hook_assumption"} : {}), + instruction: fileBacked ? "This is an exact difference between saved files, not proof of retained model context. " + + "Use the baseline file for inherited fields or select necessary facts directly from the current full file. " + + "Choose the relevant changes and details; you need not read the entire baseline or delta. " + + "Unchanged fields, including controls, are inherited from the verified baseline file; all current constraints remain binding. " + + "Before dependent work recover any missing necessary facts from the current file. This view grants no authority. " + + "value replaces; from copies from the immutable baseline and edit patches that copy; values assigns fields; " + + "changed/items recurse, removed deletes keys, length sets array length. " + : "Current controls are inline even when unchanged. Other fields inherit the " + (assumed ? "exact baseline under the native-hook continuity assumption" : "confirmed baseline") + "; all requirements remain binding. " + (assumed ? "After reading this delta completely, use it with the retained model context. " @@ -564,7 +595,7 @@ export function prepareContextDelivery(request: JsonObject, legacyQuotaFields: r // Native consumers page the wire before they can use its baseline. Deliver // the existing basis and interpretation before the potentially large tree; // do not duplicate them in a second host instruction or change strict IO. - return assumed ? {schema_version: VERSION, mode: "delta", + return assumed || fileBacked ? {schema_version: VERSION, mode: "delta", continuity_basis: delta.continuity_basis, instruction: delta.instruction, ...delta} : delta; } @@ -583,13 +614,14 @@ export function restoreContextDelivery(request: JsonObject, legacyQuotaFields: r const surface = surfaceOf(target.surface); const delivery = requireJsonObject(request.delivery, "delivery"), baseline = requireJsonObject(request.baseline, "baseline"); const base = requireJsonObject(baseline.target, "baseline target"), context = requireJsonObject(baseline.context, "baseline context"); + const fileBacked = delivery.continuity_basis === "captured_file"; if (delivery.schema_version !== VERSION || delivery.mode !== "delta" || !equal(delivery.target, target) || !equal(delivery.base, base) || !compatible(base, target) || base.context_digest !== digest(context) || !Number.isSafeInteger(baseline.ancestry) || Number(baseline.ancestry) < 0 || delivery.ancestry !== Number(baseline.ancestry) + 1 || Number(delivery.ancestry) > MAX_ANCESTRY) return fallback(); const before = freshPaths(context, surface, legacyQuotaFields, base), after = freshPaths(currentContext, surface, legacyQuotaFields, target); - if (!before.valid || !after.valid) return fallback(); - const expected = changeTree(context, currentContext, after.paths); + if (!fileBacked && (!before.valid || !after.valid)) return fallback(); + const expected = changeTree(context, currentContext, fileBacked ? [] : after.paths, fileBacked); // Also rejects omitted unchanged controls, redundant/conflicting edits, // malformed array indices/lengths and unsupported operations before applying. if (!equal(delivery.changes, expected)) return fallback(); diff --git a/loopx/control_plane/quota/context_selection.ts b/loopx/control_plane/quota/context_selection.ts index 3538deb75f..4e1e8055e9 100644 --- a/loopx/control_plane/quota/context_selection.ts +++ b/loopx/control_plane/quota/context_selection.ts @@ -28,6 +28,13 @@ export function capturedContextSelection(request: JsonObject): JsonObject { } // Return the verified address, not a JS-number-coerced copy of the value. // The file adapter retains original JSON values and owns byte serialization. + if (request.list_fields === true) { + const children = value !== null && typeof value === "object" ? Object.keys(value).map(key => ({ + pointer: pointer + "/" + key.replaceAll("~", "~0").replaceAll("/", "~1"), + path: [...path, Array.isArray(value) ? Number(key) : key], + })) : []; + return {pointer, path, children}; + } return {pointer, path}; }); return {selections}; diff --git a/loopx/control_plane/quota/native_context_lifecycle.ts b/loopx/control_plane/quota/native_context_lifecycle.ts index d8d3d50cd8..4ba2e50773 100644 --- a/loopx/control_plane/quota/native_context_lifecycle.ts +++ b/loopx/control_plane/quota/native_context_lifecycle.ts @@ -36,10 +36,13 @@ export function updateNativeContextLifecycle(request: JsonObject): JsonObject { if (name !== "SessionStart") return state; state = {session_id: session, cwd}; } + const recoveryFile = jsonObject(state.recovery); + const fileEpochCurrent = recoveryFile?.continuity_basis === "captured_file" + && jsonObject(recoveryFile.target)?.context_epoch === state.context_epoch; const invalidate = () => { // Reset is idempotent while already invalid. Pre/PostCompact and the // subsequent SessionStart(compact) therefore do not rotate three epochs. - if (!state.context_epoch || state.candidate || state.pending) + if (!state.context_epoch || state.candidate || state.pending || fileEpochCurrent) state.context_epoch = requireNonEmptyString(request.new_epoch, "new native context epoch"); delete state.candidate; delete state.pending; @@ -53,6 +56,17 @@ export function updateNativeContextLifecycle(request: JsonObject): JsonObject { if (same && event.source === "resume" && request.resume_verified === true && !state.pending && candidate?.basis === NATIVE_CONTEXT_BASIS && candidate.capture_bytes_verified === true) return state; + if (same && event.source === "resume" && fileEpochCurrent + && !(candidate?.native_resume || (state.candidate && !candidate))) { + // Resuming a session is not compaction. File comparison does not depend + // on model retention; retire that separate assumption when unverified. + // A previously qualified history checkpoint that now fails can include + // an unreported compaction/rollback, so conservatively rotate below. + delete state.candidate; + delete state.pending; + delete state.active_call; + return state; + } invalidate(); return state; } @@ -78,6 +92,7 @@ export function updateNativeContextLifecycle(request: JsonObject): JsonObject { for (const value of pages) { const page = jsonObject(value); if (!page || page.schema_version !== "context_capture_page_v1") continue; + if ("selection_sha256" in page || "selected_pointers" in page || "list_fields" in page) continue; const candidate = jsonObject(state.candidate); if (!state.pending && candidate?.capture_sha256 === recovery.sha256 && candidate?.capture_bytes_verified === true diff --git a/loopx/control_plane/turn_driver/codex_context.py b/loopx/control_plane/turn_driver/codex_context.py index c64baa4199..120ad73263 100644 --- a/loopx/control_plane/turn_driver/codex_context.py +++ b/loopx/control_plane/turn_driver/codex_context.py @@ -18,7 +18,7 @@ from .decision_capture import ( write_decision_capture, capture_context_projection, project_captured_context, CONTEXT_CAPTURE_PAGE_GUIDANCE, - capture_page_range, read_context_capture_file, + capture_page_range, read_context_capture_file, CONTEXT_CAPTURE_NAVIGATION_GUIDANCE, context_capture_read_command, NATIVE_CONTEXT_PAGE_BYTES, NATIVE_CONTEXT_PAGE_OUTPUT_BYTES, NATIVE_CONTEXT_LARGE_OUTPUT_BYTES, NATIVE_CONTEXT_HISTORY_TOKENS, NATIVE_CONTEXT_OUTPUT_ENV, native_context_page_budget, @@ -213,12 +213,13 @@ def _context_continuation_instructions(command: str) -> str: preserved. Do not turn every compaction into another full-input loop. """ return ( - "A new work Turn needs its current guard and required reads; never prefetch an old packet to seed a delta. " - "Compaction or a failed hook invalidates delta reuse, not the existing Turn. " + "A new work Turn needs its current guard and the facts its work depends on; never prefetch an old packet to seed a delta. " + "Compaction or a failed hook starts a new baseline epoch for the next work Turn; it does not invalidate the existing Turn or its saved recovery files. " "Continue from its working brief, preserving Goal, requirements/stops and original task/capture/settlement identity. " "Do not replay read context or rerun admission solely because the baseline was invalidated. " - "Before dependent action, recover missing or uncertain necessary facts from the same capture with: " + command + ". " - "An unfinished required read still needs its missing content; a summary cannot declare it read. " + "To find missing necessary facts, discover this same capture with: " + command + " --list. " + "Then use --select for the relevant sections; full-file traversal is not a prerequisite. " + "Finish an interrupted selected section only if the next decision needs it; do not claim unread content was read. " "Never repeat completed effects or settlement. If the original reference is lost, repair it first; " "independently authorized work can continue. A brief or recovery grants no new authority/freshness, " "delta baseline or retention proof. " @@ -244,7 +245,7 @@ def codex_context_recovery_instructions(directory: str, *, goal_id: str, agent_i "--max-page-output-bytes", str(page_output_bytes)]) return ( _context_continuation_instructions(command) + - "When recovering a required full read, follow next_command through eof and verify complete bytes/digest. " + + CONTEXT_CAPTURE_NAVIGATION_GUIDANCE + ("The generated native configuration supports 64000-byte context-read outputs and a 20000-token history limit. " + _NATIVE_CONTEXT_OUTPUT_GUIDANCE + "Read one continuation per tool call; in code-mode, forward successful stdout directly with text(result.output). " @@ -334,7 +335,12 @@ def codex_context_hook(directory: str | None, event: dict[str, Any]) -> dict[str checkpoint_native_read(read_state, updated, event, pages) if updated != state: _save(root, updated) - recovery_needed = same and not updated.get("candidate") + saved = updated.get("recovery") + current_file_epoch = (isinstance(saved, dict) + and saved.get("continuity_basis") == "captured_file" + and isinstance(saved.get("target"), dict) + and saved["target"].get("context_epoch") == updated.get("context_epoch")) + recovery_needed = same and not updated.get("candidate") and not current_file_epoch except (OSError, ValueError, TypeError, RecursionError, EffectRuntimeRemoteError, EffectRuntimeStartupError): # A transient read/reduction failure can miss invalidation just # as a failed write can. While holding the same lock, retire @@ -347,10 +353,10 @@ def codex_context_hook(directory: str | None, event: dict[str, Any]) -> dict[str if (name == "SessionStart" and recovery_needed and isinstance(recovery, dict) and recovery.get("command")): return {"hookSpecificOutput": {"hookEventName": name, "additionalContext": - "LoopX context baseline invalidated. " + "LoopX model-context baseline invalidated. Saved files remain available. " + _context_continuation_instructions(recovery["command"]) + _NATIVE_CONTEXT_OUTPUT_GUIDANCE - + "When completing a required full-capture read: " + CONTEXT_CAPTURE_PAGE_GUIDANCE}} + + "If you choose to read an entire file: " + CONTEXT_CAPTURE_PAGE_GUIDANCE}} return {} @@ -405,6 +411,8 @@ def _verified_context_pages(state: dict[str, Any], pages: list[dict[str, Any]]) files: dict[str, bytes] = {} result = [] for page in pages: + if any(key in page for key in ("selection_sha256", "selected_pointers", "list_fields")): + continue # A directory/detail read makes no full-delivery claim. reference = next((value for value in references if value.get("sha256") == page.get("sha256")), None) if reference is None or page.get("ok") is not True: result.append(page) # the typed owner handles failures and unrelated reads @@ -434,24 +442,49 @@ def _discard_recovery(root: Path) -> None: path.unlink() +def _file_codec_numbers_supported(value: Any) -> bool: + """Keep exact file values when the JSON/JavaScript bridge cannot encode them.""" + if isinstance(value, dict): + return all(_file_codec_numbers_supported(item) for item in value.values()) + if isinstance(value, list): + return all(_file_codec_numbers_supported(item) for item in value) + return not isinstance(value, int) or abs(value) <= 9007199254740991 + + +def _integral_float_sources(value: Any, path: tuple = ()) -> dict[tuple, str]: + """Identify source distinctions that the JSON/JavaScript bridge erases.""" + if isinstance(value, dict): + return {key: token for name, item in value.items() + for key, token in _integral_float_sources(item, (*path, name)).items()} + if isinstance(value, list): + return {key: token for index, item in enumerate(value) + for key, token in _integral_float_sources(item, (*path, index)).items()} + return {path: repr(value)} if isinstance(value, float) and value.is_integer() else {} + + def _native_context_request(state: dict[str, Any], scope: dict[str, Any]) -> dict[str, Any]: - """Use the explicit completed-delivery assumption, never a fabricated ACK.""" + """Compare verified durable versions independently of model retention. + + The same session's last full capture is a file baseline after partial reads + within one context epoch. Compaction starts a fresh full baseline on the + next work Turn. Missing/replaced/unreadable files also fall back to full. + The shared codec verifies scope and content; no ACK is fabricated. + """ request: dict[str, Any] = {"scope": scope} - candidate = state.get("candidate") - if (isinstance(candidate, dict) and candidate.get("basis") == "native_hook_assumption" - and not state.get("pending")): + previous = state.get("recovery") + if isinstance(previous, dict) and isinstance(previous.get("target"), dict): try: - path = Path(candidate["capture_file"]) - _, _, payload = read_context_capture_file(str(path), candidate["capture_sha256"], private=True) - # The shared codec independently validates the complete context - # digest and scope. No second read or full native-history scan. + path, raw, payload = read_context_capture_file(previous["path"], previous["sha256"], private=True) + if not _file_codec_numbers_supported(payload): + return request baseline = dict(schema_version="context_delivery_v1", mode="full", - target=candidate["target"], ancestry=candidate.get("ancestry", 0), context=payload) + target=previous["target"], ancestry=0, context=payload) request.update(baseline=baseline, - continuity={"basis": "native_hook_assumption", "target": candidate["target"], - "capture_bytes_verified": candidate.get("capture_bytes_verified", False)}) + baseline_reference={"path": str(path), "sha256": previous["sha256"], "total_bytes": len(raw), + "command": context_capture_read_command(path, previous["sha256"])}, + continuity={"basis": "captured_file", "target": previous["target"], "capture_bytes_verified": True}) except (OSError, ValueError, KeyError, TypeError, RecursionError): - pass # Missing candidate affects reuse, never the current recovery reference. + pass return request @@ -544,7 +577,7 @@ def project_codex_context(payload: dict[str, Any], *, directory: str | None, cap Serialization prevents invalidation from moving the epoch during preparation. Full fallback must recover this capture or refuse lookup, never an older one. - Completed full or delta pages establish the explicit native-hook assumption. + Saved-file comparison is independent of any complete model-read observation. """ from ..quota.error_codes import QuotaDecisionTransportError @@ -595,11 +628,19 @@ def publish_recovery(reference: dict[str, Any]) -> None: recovery = planned_recovery publish_recovery(recovery) + prior = request.get("baseline", {}).get("context") + if (not _file_codec_numbers_supported(required) + or prior is not None and _integral_float_sources(prior) != _integral_float_sources(required)): + request = {"scope": scope} + result = project_captured_context(required, directory=capture_directory, request=request, captured_recovery=recovery, goal_id=goal_id, agent_id=agent_id, surface=surface, capture_id=capture_id) + if result.get("continuity_basis") == "captured_file" and result.get("mode") == "delta": + result["baseline_reference"] = request["baseline_reference"] if result.get("schema_version") == "context_delivery_v1": state["recovery"]["target"] = result["target"] + state["recovery"]["continuity_basis"] = "captured_file" state["pending"].update(target=result["target"], ancestry=result["ancestry"]) _save(root, state) return result diff --git a/loopx/control_plane/turn_driver/decision_capture.py b/loopx/control_plane/turn_driver/decision_capture.py index 1495ac48c4..a64b001d77 100644 --- a/loopx/control_plane/turn_driver/decision_capture.py +++ b/loopx/control_plane/turn_driver/decision_capture.py @@ -150,6 +150,20 @@ def validate_handoff_socket(value: str) -> Path: "of this same capture with fewer pages or a smaller --page-bytes; do not trust a preview or EOF alone. " "Do not rerun admission or ask the owner. File existence and page EOF are not retention proof." ) +# A durable capture is a source library, not an instruction to import it all. +CONTEXT_CAPTURE_NAVIGATION_GUIDANCE = ( + "The complete decision is saved locally. Choose the sections needed for the current decision; " + "you do not need to read the whole file or drain all pages before working. " + "Use index_command to discover exact pointers, then repeat --select to batch relevant fields. " + "Read applicable goals, acceptance criteria, constraints and stop conditions before dependent work; " + "read the current operation/settlement contract before effects, and supporting evidence when verification needs it. " + "An index is navigation, not evidence of reading or fresh authority. " + "Keep the original Turn/capture/effect identities. After compaction recover only missing necessary facts " + "from this capture; never rerun admission or repeat effects just to recover context. " + "Saved-file deltas may use unread baseline files; this is not a claim of retained model input. " + "Selected/index reads never establish full model-context retention. " +) + # Native paging verifies capture bytes; the existing codec separately declares # its continuity basis. Generic/off readers retain their original obligations # without claiming that a native input observer exists. @@ -187,7 +201,8 @@ def context_capture_read_command(path: Path, digest: str, *, offset: int = 0, page_bytes: int | None = None, max_page_output_bytes: int | None = None, select: list[str] | None = None, - selection_sha256: str | None = None) -> str: + selection_sha256: str | None = None, + list_fields: bool = False) -> str: from ...command_invocation import loopx_cli_argv if page_bytes is None: @@ -197,6 +212,7 @@ def context_capture_read_command(path: Path, digest: str, *, offset: int = 0, return shlex.join([*loopx_cli_argv(), "--format", "json", "quota", "context-read", "--capture-file", str(path), "--capture-sha256", digest, "--offset", str(offset), "--page-bytes", str(page_bytes), + *(["--list"] if list_fields else []), *(argument for pointer in select or [] for argument in ("--select", pointer)), *(["--selection-sha256", selection_sha256] if selection_sha256 is not None else []), *(["--max-page-output-bytes", str(max_page_output_bytes)] if max_page_output_bytes is not None else [])]) diff --git a/tests/control_plane/test_codex_context_lifecycle.py b/tests/control_plane/test_codex_context_lifecycle.py index 37a8bb3fd3..1bdb508503 100644 --- a/tests/control_plane/test_codex_context_lifecycle.py +++ b/tests/control_plane/test_codex_context_lifecycle.py @@ -121,7 +121,7 @@ def test_lifecycle_invalidates_and_recovers_current_capture_without_bodies(tmp_p # The additional work-Turn/read-obligation distinction is intentional; # this synthetic allowance is not a host output or context limit. assert "max_output_tokens=22000" in body and "max_output_tokens=24000" in body - assert payload["requirements"] not in body and len(body) < 2300 + assert payload["requirements"] not in body and len(body) < 2600 else: assert result == {} _, second = project(tmp_path, "two") @@ -174,7 +174,7 @@ def test_complete_reader_delivery_allows_delta_without_diagnostics_or_ack(tmp_pa "tool_response": "ordinary tool output"}) assert context._state(tmp_path) == state _, second = project(tmp_path, "two", payload=payload) - assert second["mode"] == "delta" and second["continuity_basis"] == "native_hook_assumption" + assert second["mode"] == "delta" and second["continuity_basis"] == "captured_file" assert not (tmp_path / "traces").exists() deliver(tmp_path, event, read_pages(tmp_path, second)) _, third = project(tmp_path, "three", payload=payload) @@ -302,7 +302,7 @@ def test_native_workspace_change_retires_previous_recovery(tmp_path, monkeypatch assert second["mode"] == "full" and second["context"] == payload -def test_native_page_zero_exposes_current_continuity_and_preserves_full_recovery(tmp_path, monkeypatch): +def test_native_reference_exposes_file_continuity_and_preserves_full_recovery(tmp_path, monkeypatch): from argparse import Namespace from loopx.cli_commands.quota_capture import native_context_capture_view, read_context_capture @@ -312,12 +312,13 @@ def test_native_page_zero_exposes_current_continuity_and_preserves_full_recovery _, first = project(tmp_path, payload=payload) deliver(tmp_path, event, read_pages(tmp_path, first)) _, second = project(tmp_path, "two", payload=payload) - assert second["mode"] == "delta" and second["continuity_basis"] == "native_hook_assumption" + assert second["mode"] == "delta" and second["continuity_basis"] == "captured_file" view = native_context_capture_view(second, tmp_path / "two", context_directory=str(tmp_path)) - # The consumer must see the codec's actual basis before a long change tree, - # rather than a second transport layer claiming strict final-input proof. - assert json.dumps(second["instruction"]).encode() in view["content"].encode() - assert b'"continuity_basis":"native_hook_assumption"' in view["content"].encode() + # The default names the files without forcing either body into history. + assert "content" not in view + assert view["continuity_basis"] == "captured_file" + assert view["baseline_reference"]["path"] == first["recovery"]["path"] + assert view["recovery"]["path"] == second["recovery"]["path"] for instruction in (view["instruction"], second["recovery"]["instruction"]): assert "Native input observation independently verifies" not in instruction deliver(tmp_path, event, capture_pages(view["path"], view["sha256"])) @@ -349,7 +350,7 @@ def test_native_page_zero_without_metadata_room_retains_original_recovery(tmp_pa @pytest.mark.parametrize("mode", ["full", "delta"]) -def test_resume_requires_complete_original_read_before_reusing_baseline(tmp_path, monkeypatch, mode): +def test_resume_partial_read_does_not_claim_full_model_retention(tmp_path, monkeypatch, mode): event = setup(tmp_path, monkeypatch) payload = {"ok": True, "plan_summary": {"requirements": "Preserve every obligation. " * 1000}} _, wire = project(tmp_path, payload=payload) @@ -363,9 +364,9 @@ def test_resume_requires_complete_original_read_before_reusing_baseline(tmp_path resumed = {**event, "source": "resume"} result = context.codex_context_hook(str(tmp_path), resumed) after = context._state(tmp_path) - assert "candidate" not in after and after["context_epoch"] != before["context_epoch"] + assert "candidate" not in after and after["context_epoch"] == before["context_epoch"] assert after["recovery"] == recovery - assert recovery["command"] in result["hookSpecificOutput"]["additionalContext"] + assert result == {} # Ordinary resume must not induce a recovery/read loop. # Repeated notifications do not rotate the epoch or replace the recovery. context.codex_context_hook(str(tmp_path), resumed) assert context._state(tmp_path) == after @@ -377,7 +378,7 @@ def test_resume_requires_complete_original_read_before_reusing_baseline(tmp_path assert Path(recovery["path"]).read_bytes() == original assert context._state(tmp_path)["candidate"]["target"]["context_epoch"] == after["context_epoch"] _, next_wire = project(tmp_path, "post-resume", payload=payload) - assert next_wire["mode"] == "delta" + assert next_wire["mode"] == "delta" # Ordinary resume keeps this file epoch. @pytest.mark.parametrize("mode", ["full", "delta"]) @@ -392,24 +393,16 @@ def test_fresh_capture_after_resume_does_not_require_reading_old_original(tmp_pa original = wire["recovery"] before = Path(original["path"]).read_bytes() resumed = context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) - guidance = resumed["hookSpecificOutput"]["additionalContext"] - assert guidance.index("new work Turn") < guidance.index(original["command"]) - assert "Do not replay read context or rerun admission solely" in guidance - assert "An unfinished required read still needs its missing content" in guidance + assert resumed == {} assert "candidate" not in context._state(tmp_path) - # A fresh authoritative input must be full. No old reader pages are - # delivered to seed this chain, and the changed requirement remains whole. + # Ordinary resume preserves the file epoch without old-file rereads. + # Actual compaction is tested separately and starts a fresh full baseline. current = {**payload, "current_requirement": "Stop before publishing this new result."} _, fresh = project(tmp_path, "fresh", payload=current) - assert fresh["mode"] == "full" and fresh["context"] == current - pages = read_pages(tmp_path, fresh) - deliver(tmp_path, event, pages[:1]) + assert fresh["mode"] == "delta" and fresh["continuity_basis"] == "captured_file" + assert json.loads(Path(fresh["recovery"]["path"]).read_bytes()) == current assert "candidate" not in context._state(tmp_path) - deliver(tmp_path, event, pages[1:]) - candidate = context._state(tmp_path)["candidate"] - assert candidate["target"] == fresh["target"] - assert candidate["capture_sha256"] == fresh["recovery"]["sha256"] _, next_wire = project(tmp_path, "after-fresh", payload=current) assert next_wire["mode"] == "delta" and next_wire["base"] == fresh["target"] assert Path(original["path"]).read_bytes() == before @@ -439,7 +432,8 @@ def test_known_incomplete_delivery_cannot_establish_baseline(tmp_path, monkeypat deliver(tmp_path, event, [{"ok": False, "schema_version": "context_capture_page_v1"}]) assert "candidate" not in context._state(tmp_path) _, second = project(tmp_path, "two", payload=payload) - assert second["mode"] == "full" + assert second["mode"] == ("delta" if failure in ("tail_only", "gap") else "full") + assert "candidate" not in context._state(tmp_path) assert second["recovery"]["path"] != first["recovery"]["path"] assert json.loads(Path(first["recovery"]["path"]).read_bytes()) == payload @@ -504,7 +498,7 @@ def hook(values): hook(capture_pages(recovery["path"], recovery["sha256"], size=8192)) assert context._state(tmp_path)["candidate"]["capture_sha256"] == recovery["sha256"] _, after = project(tmp_path, "three", payload=payload) - assert after["mode"] == "delta" + assert after["mode"] == "full" # The failed hook retired the old baseline epoch. assert not registry.exists() and not runtime.exists() @@ -543,7 +537,7 @@ def test_unverified_page_facts_cannot_promote_through_the_shared_owner(tmp_path, @pytest.mark.parametrize("route", ["resume", "projection"]) -def test_metadata_only_baselines_require_a_new_verified_read(tmp_path, monkeypatch, route): +def test_metadata_only_model_candidate_does_not_prevent_verified_file_diff(tmp_path, monkeypatch, route): event = setup(tmp_path, monkeypatch) payload = {"ok": True, "plan_summary": {"requirements": "Preserve every obligation. " * 1000}} _, wire = project(tmp_path, payload=payload) @@ -554,7 +548,7 @@ def test_metadata_only_baselines_require_a_new_verified_read(tmp_path, monkeypat context._save(tmp_path, before) if route == "projection": _, next_wire = project(tmp_path, "two", payload=payload) - assert next_wire["mode"] == "full" and next_wire["reason"] == "baseline_unavailable" + assert next_wire["mode"] == "delta" and next_wire["continuity_basis"] == "captured_file" else: context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) assert "candidate" not in context._state(tmp_path) @@ -613,11 +607,11 @@ def test_compaction_reset_recovers_the_same_original_before_new_delta(tmp_path, reset = context._state(tmp_path) assert reset["context_epoch"] != old_epoch and "candidate" not in reset assert first["recovery"]["sha256"] in reply["hookSpecificOutput"]["additionalContext"] - # The unchanged original reader, not a fresh guard, restores the baseline. + # An optional old-file read can restore understanding, not the new epoch baseline. deliver(tmp_path, event, pages) assert context._state(tmp_path)["candidate"]["target"]["context_epoch"] == reset["context_epoch"] _, second = project(tmp_path, "two", payload=payload) - assert second["mode"] == "delta" + assert second["mode"] == "full" def test_configuration_round_trip_requires_a_new_full_baseline(tmp_path, monkeypatch): @@ -703,7 +697,10 @@ def _assert_recovery_after_fallback(root, event, source, capture): digest = hashlib.sha256(raw).hexdigest() assert page["ok"] is True and page["sha256"] == digest assert page["content"].encode() == raw[:len(page["content"].encode())] - assert digest in hook["hookSpecificOutput"]["additionalContext"] + if hook: + assert digest in hook["hookSpecificOutput"]["additionalContext"] + else: + assert source == "resume" # The exact recovery remains available without a reread prompt. @pytest.mark.parametrize("source", ["compact", "resume"]) @@ -1075,3 +1072,131 @@ def test_benchmark_activation_uses_existing_explicit_worker_declaration(monkeypa result = native_context_activation(args) assert result["enabled"] is expected assert result["scenario"] == ("codex_benchmark_heartbeat" if expected else None) + + +@pytest.mark.parametrize("read_mode", ["none", "index", "selected"]) +def test_new_compaction_cycle_gets_full_file_then_unread_file_deltas(tmp_path, monkeypatch, read_mode): + from argparse import Namespace + from loopx.cli_commands.quota_capture import read_context_capture, native_context_capture_view + + event = setup(tmp_path, monkeypatch) + original = {"ok": True, "requirements": {"stop": False, "text": "Keep all requirements. " * 2000}, + "evidence": {"notes": "Optional historical material. " * 2000}} + _, first = project(tmp_path, payload=original) + if read_mode != "none": + page = read_context_capture(Namespace(capture_file=first["recovery"]["path"], + capture_sha256=first["recovery"]["sha256"], format="json", offset=0, + select=None if read_mode == "index" else ["/requirements/stop"], + list_fields=read_mode == "index", page_bytes=8192, max_page_output_bytes=24000)) + assert page["ok"] and page["eof"] + deliver(tmp_path, event, [page]) + assert "candidate" not in context._state(tmp_path) + reply = context.codex_context_hook(str(tmp_path), {**event, "source": "compact"}) + assert "full-file traversal is not a prerequisite" in reply["hookSpecificOutput"]["additionalContext"] + revised = {**original, "requirements": {**original["requirements"], "stop": True}} + _, new_base = project(tmp_path, "after-compact", payload=revised) + assert new_base["mode"] == "full" and new_base["context"] == revised + assert new_base["target"]["context_epoch"] != first["target"]["context_epoch"] + # No full read of this new base is required to compare the next version. + _, next_wire = project(tmp_path, "next-turn", payload=revised) + assert next_wire["mode"] == "delta" and next_wire["continuity_basis"] == "captured_file" + assert next_wire["base"] == new_base["target"] + view = native_context_capture_view(next_wire, tmp_path / "next-turn", context_directory=str(tmp_path)) + assert "content" not in view and "offset" not in view + assert len(json.dumps(view).encode()) < 8192 + assert view["baseline_reference"]["sha256"] == new_base["recovery"]["sha256"] + assert json.loads(Path(view["recovery"]["path"]).read_bytes()) == revised + assert "candidate" not in context._state(tmp_path) + assert hashlib.sha256(Path(view["path"]).read_bytes()).hexdigest() == view["sha256"] + + +@pytest.mark.parametrize("damage", ["deleted", "changed", "symlink", "scope"]) +def test_unread_file_baseline_must_still_be_available_exact_and_scoped(tmp_path, monkeypatch, damage): + setup(tmp_path, monkeypatch) + original = {"requirements": "Keep every constraint. " * 2000} + _, first = project(tmp_path, payload=original) + path = Path(first["recovery"]["path"]) + if damage == "deleted": + path.unlink() + elif damage == "changed": + path.write_bytes(path.read_bytes() + b" ") + elif damage == "symlink": + moved = path.with_suffix(".moved") + path.rename(moved) + path.symlink_to(moved) + else: + state = context._state(tmp_path) + state["recovery"]["target"]["agent_id"] = "foreign-agent" + context._save(tmp_path, state) + _, second = project(tmp_path, "two", payload=original) + assert second["mode"] == "full" and second["context"] == original + assert "baseline_reference" not in second + + +@pytest.mark.parametrize("before,after", [ + (9007199254740992, 9007199254740993), (1, 9007199254740993), (9007199254740993, 1), + (1, 1.0), (1.0, 1), (0.0, -0.0), (-0.0, 0.0), + ({"nested": [1]}, {"nested": [1.0]}), +]) +def test_file_delta_falls_back_before_numeric_normalization_can_hide_changes(tmp_path, monkeypatch, before, after): + from argparse import Namespace + from loopx.cli_commands.quota_capture import native_context_capture_view, read_context_capture + + setup(tmp_path, monkeypatch) + body = "Same unchanged requirements. " * 1000 + project(tmp_path, payload={"body": body, "counter": before}) + _, result = project(tmp_path, "two", payload={"body": body, "counter": after}) + assert result["mode"] == "full" + assert "baseline_reference" not in result + view = native_context_capture_view(result, tmp_path / "two", context_directory=str(tmp_path)) + assert view["path"] == result["recovery"]["path"] + assert hashlib.sha256(Path(view["path"]).read_bytes()).hexdigest() == view["sha256"] + page = read_context_capture(Namespace(capture_file=view["path"], capture_sha256=view["sha256"], + format="json", offset=0, page_bytes=64000, max_page_output_bytes=64000)) + assert page["eof"] + # Numeric equality conflates integer/float types and the sign of zero. + # The public reader must preserve the independently supplied source value. + assert json.dumps(json.loads(page["content"])["counter"]) == json.dumps(after) + + +@pytest.mark.parametrize("before,after", [(1, 2), (True, 1), (0.25, 0.5)]) +def test_exact_bridge_numbers_still_use_file_deltas(tmp_path, monkeypatch, before, after): + setup(tmp_path, monkeypatch) + body = "Same unchanged requirements. " * 1000 + project(tmp_path, payload={"body": body, "counter": before}) + _, result = project(tmp_path, "two", payload={"body": body, "counter": after}) + assert result["mode"] == "delta" and result["continuity_basis"] == "captured_file" + assert "baseline_reference" in result + assert json.dumps(result["changes"]["changed"]["counter"]["value"]) == json.dumps(after) + + +def test_unchanged_integral_floats_inherit_without_rewriting_their_parent(tmp_path, monkeypatch): + setup(tmp_path, monkeypatch) + original = {"body": "Same unchanged requirements. " * 1000, + "state": {"number": 1.0, "a": False, "b": False, "c": False}, "array": [-0.0, "old"]} + current = {**original, "state": {**original["state"], "a": True, "b": True, "c": True}, + "array": [-0.0, "new"]} + _, first = project(tmp_path, payload=original) + _, result = project(tmp_path, "two", payload=current) + assert result["mode"] == "delta" + edits = result["changes"]["changed"] + # A short parent replacement would normalize both original numeric values. + # These literals update only independently specified nonnumeric children. + assert edits["state"] == {"values": {key: True for key in ("a", "b", "c")}} + assert edits["array"] == {"items": {"1": {"value": "new"}}} + saved = json.loads(Path(first["recovery"]["path"]).read_bytes()) + saved["state"].update({key: True for key in ("a", "b", "c")}) + saved["array"][1] = "new" + assert json.dumps(saved, sort_keys=True) == json.dumps(current, sort_keys=True) + + +def test_numeric_subtree_cannot_relocate_a_float_into_an_integer_field(tmp_path, monkeypatch): + setup(tmp_path, monkeypatch) + original = {"source": {"number": 1.0, "description": "Same source text. " * 2000}} + current = {**original, "destination": {**original["source"], "number": 1}} + project(tmp_path, payload=original) + _, result = project(tmp_path, "two", payload=current) + assert result["mode"] == "delta" + destination = result["changes"]["changed"]["destination"] + assert "from" not in destination # Whole-subtree reuse would copy 1.0. + assert json.dumps(destination["changed"]["number"]["value"]) == "1" diff --git a/tests/control_plane/test_codex_context_resume.py b/tests/control_plane/test_codex_context_resume.py index ae2a1c4ea7..326a0e7723 100644 --- a/tests/control_plane/test_codex_context_resume.py +++ b/tests/control_plane/test_codex_context_resume.py @@ -11,6 +11,17 @@ from loopx.control_plane.turn_driver import codex_context as context from tests.control_plane.test_codex_context_lifecycle import setup, project, read_pages +from loopx.control_plane.quota.turn_envelope import restore_context_delivery + + +def assert_file_delta(wire, previous, expected): + baseline = {"schema_version": "context_delivery_v1", "mode": "full", "ancestry": 0, + "target": previous["target"], "context": json.loads(Path(previous["path"]).read_text())} + assert wire["mode"] == "delta" and wire["continuity_basis"] == "captured_file" + restored = restore_context_delivery(wire, baseline=baseline, + current={"schema_version": "context_delivery_v1", "mode": "full", "ancestry": 0, + "target": wire["target"], "context": expected}) + assert restored["mode"] == "delta" and restored["context"] == expected def append(path, kind, payload): @@ -83,14 +94,13 @@ def test_completed_native_resume_keeps_delta_without_old_capture_prefetch(tmp_pa assert result == {} and context._state(tmp_path)["context_epoch"] == before["context_epoch"] current = {**payload, "heartbeat_receipt": {"turn_instance_id": "second"}} _, wire = project(tmp_path, "second", payload=current) - assert wire["mode"] == "delta" - assert wire["changes"]["values"]["heartbeat_receipt"] == current["heartbeat_receipt"] + assert_file_delta(wire, before["recovery"], current) @pytest.mark.parametrize("single_page", [False, True]) @pytest.mark.parametrize("lose_first", [False, True]) @pytest.mark.parametrize("code_mode", [False, True]) -def test_compact_full_recovery_becomes_next_resume_baseline(tmp_path, monkeypatch, single_page, lose_first, code_mode): +def test_compact_recovery_preserves_turn_but_next_turn_starts_new_file_baseline(tmp_path, monkeypatch, single_page, lose_first, code_mode): from argparse import Namespace from loopx.cli_commands.quota_capture import read_context_capture @@ -143,9 +153,9 @@ def test_compact_full_recovery_becomes_next_resume_baseline(tmp_path, monkeypatc context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) current = {**payload, "heartbeat_receipt": {"turn_instance_id": "after-compact"}} _, wire = project(tmp_path, "after-compact", payload=current) - assert wire["mode"] == ("full" if lose_first else "delta") - if not lose_first: - assert wire["changes"]["values"]["heartbeat_receipt"] == current["heartbeat_receipt"] + # Reading the old Turn's recovery cannot seed the new epoch. Even an exact + # read is a recovery of prior work, not the next Turn's current baseline. + assert wire["mode"] == "full" and wire["context"] == current @pytest.mark.parametrize("change", ["compact", "rollback", "retained", "unknown", "truncate", @@ -197,16 +207,19 @@ def test_unqualified_native_version_cannot_retain_resume_baseline(tmp_path, monk before = context._state(tmp_path) assert "native_resume" not in before["candidate"] context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) - assert context._state(tmp_path)["context_epoch"] != before["context_epoch"] + assert "candidate" not in context._state(tmp_path) + assert context._state(tmp_path)["context_epoch"] == before["context_epoch"] _, wire = project(tmp_path, "second", payload=payload) - assert wire["mode"] == "full" and wire["context"] == payload + assert_file_delta(wire, before["recovery"], payload) def test_incomplete_native_read_cannot_gain_resume_continuity(tmp_path, monkeypatch): event, _, payload = observed_read(tmp_path, monkeypatch, complete=False) - context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) + before = context._state(tmp_path) + assert context.codex_context_hook(str(tmp_path), {**event, "source": "resume"}) == {} + assert "candidate" not in context._state(tmp_path) _, wire = project(tmp_path, "second", payload=payload) - assert wire["mode"] == "full" and wire["context"] == payload + assert_file_delta(wire, before["recovery"], payload) @pytest.mark.skipif(not hasattr(os, "mkfifo"), reason="Named pipes require POSIX") @@ -256,16 +269,24 @@ def hook(values): result = hook({"hook_event_name": "SessionStart", "source": "resume"}) after = context._state(tmp_path) assert "candidate" not in after and "pending" not in after - assert after["context_epoch"] != before["context_epoch"] - assert after["recovery"] == reference - guidance = result["hookSpecificOutput"]["additionalContext"] - assert reference["command"] in guidance - # Actual CLI hook recovery must stand alone after persistent context loss. - assert "max_output_tokens=22000" in guidance - assert "max_output_tokens=24000" in guidance - assert "grants no new authority/freshness" in guidance and "retention proof" in guidance - _, fallback = project(tmp_path, "after-fifo", payload=payload) - assert fallback["mode"] == "full" and fallback["context"] == payload + if phase == "post_tool": + # No replay checkpoint was created from the pipe. It cannot prove + # retention, but the regular saved JSON is still a valid file base. + assert after["context_epoch"] == before["context_epoch"] and result == {} + _, wire = project(tmp_path, "after-fifo", payload=payload) + assert_file_delta(wire, reference, payload) + else: + # A previously verified replay became unreadable: this may hide + # compaction, so rotate conservatively without opening the pipe. + assert after["context_epoch"] != before["context_epoch"] + assert after["recovery"] == reference + guidance = result["hookSpecificOutput"]["additionalContext"] + assert reference["command"] in guidance + assert "max_output_tokens=22000" in guidance + assert "max_output_tokens=24000" in guidance + assert "grants no new authority/freshness" in guidance and "retention proof" in guidance + _, fallback = project(tmp_path, "after-fifo", payload=payload) + assert fallback["mode"] == "full" and fallback["context"] == payload assert Path(reference["path"]).read_bytes() == captured assert not registry.exists() and not runtime.exists() diff --git a/tests/control_plane/test_context_capture_selection.py b/tests/control_plane/test_context_capture_selection.py index f1a175b5cb..499452635b 100644 --- a/tests/control_plane/test_context_capture_selection.py +++ b/tests/control_plane/test_context_capture_selection.py @@ -115,3 +115,47 @@ def test_native_hook_does_not_promote_selected_eof(tmp_path, monkeypatch): "tool_use_id": "selected-detail", "tool_response": json.dumps(page)}) after = context._state(tmp_path) assert "candidate" not in after and after["recovery"] == before["recovery"] + assert after["context_epoch"] == before["context_epoch"] and after["pending"] == before["pending"] + + +def test_index_is_a_paginated_directory_not_a_read_receipt(tmp_path): + path, raw, original = fixture_capture(tmp_path) + request = args(path, raw) + request.select = ["", "/rows", "/a~1b", "/large"] + request.list_fields = True + combined = b"" + first = None + while True: + page = read_context_capture(request) + assert page["ok"] and page["offset"] == len(combined), page + assert page["list_fields"] and capture_page_range(page, raw=raw, digest=request.capture_sha256) is None + first = first or page + combined += page["content"].encode() + if page["eof"]: + break + assert "--list" in page["next_command"] + request.offset, request.selection_sha256 = page["next_offset"], page["selection_sha256"] + directory = json.loads(combined) + assert directory["schema_version"] == "context_capture_index_v1" + groups = {row["pointer"]: row["fields"] for row in directory["selections"]} + assert groups["/large"] == [] # listing a scalar never leaks its value + assert groups["/rows"][0]["pointer"] == "/rows/0" + assert groups["/a~1b"] == [{"pointer": "/a~1b/~0", "type": "null", "bytes": 4}] + fields = {row["pointer"]: row for row in groups[""]} + assert fields["/large"] == {"pointer": "/large", "type": "number", "bytes": len(str(original["large"]))} + assert fields["/rows"]["count"] == 1 + assert original["rows"][0]["text"] not in combined.decode() + # A cursor from an index cannot be replayed as a selected-value cursor. + request.list_fields = False + request.offset, request.selection_sha256 = first["next_offset"], first["selection_sha256"] + assert read_context_capture(request)["ok"] is False + + +def test_cli_index_defaults_to_root_without_registry_or_runtime(tmp_path): + path, raw, _ = fixture_capture(tmp_path) + registry, runtime = tmp_path / "missing.json", tmp_path / "missing-runtime" + code, page = _run_cli(registry, runtime, "quota", "context-read", "--capture-file", str(path), + "--capture-sha256", hashlib.sha256(raw).hexdigest(), "--list", cwd=tmp_path) + assert code == 0 and page["eof"], page + assert page["selected_pointers"] == [""] + assert not registry.exists() and not runtime.exists() diff --git a/tests/control_plane/test_context_delivery_cli.py b/tests/control_plane/test_context_delivery_cli.py index f49f85f8b2..4964d0cf3d 100644 --- a/tests/control_plane/test_context_delivery_cli.py +++ b/tests/control_plane/test_context_delivery_cli.py @@ -438,7 +438,7 @@ def unavailable(*args, **kwargs): @pytest.mark.parametrize("provider", ["file", "sqlite"]) @pytest.mark.parametrize("output_limit", [None, 7900]) -def test_native_context_cli_without_retention_stays_full_and_preserves_receipt(tmp_path, monkeypatch, provider, output_limit): +def test_native_context_cli_without_retention_exposes_files_and_preserves_receipt(tmp_path, monkeypatch, provider, output_limit): from loopx.control_plane.turn_driver.codex_context import codex_context_hook from test_quota_settlement_cli import _heartbeat_receipt_count @@ -476,33 +476,32 @@ def consume(page): page = read_context_capture(args) assert page["ok"] - for _ in range(2): + for attempt in range(2): rc, wire = _run_cli(registry, runtime, "quota", "should-run", "--codex-app", "--goal-id", GOAL_ID, "--agent-id", AGENT_ID, "--todo-id", TODO_ID, "--turn-instance-id", "native-full", "--decision-output-root", str(captures), "--context-delivery", "on", "--codex-context-dir", str(root), *([] if output_limit is None else ["--max-page-output-bytes", str(output_limit)]), cwd=project) - assert rc == 0 and wire["mode"] == "full", wire - assert "Native input observation independently verifies" not in wire["instruction"] - assert "Native input observation independently verifies" not in wire["recovery"]["instruction"] - # Consume the advertised first page and continuation, not another guard. - observed = consume(wire) - assert hashlib.sha256(observed).hexdigest() == wire["sha256"] - assert json.loads(observed) == json.loads(Path(wire["recovery"]["path"]).read_text()) + assert rc == 0 and wire["ok"], wire + assert len(json.dumps(wire, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= 7900 + assert "content" not in wire and "eof" not in wire and "context" not in wire + if attempt == 0: + assert wire["mode"] == "full" + # Autonomous readers can choose the current full file directly, whether + # the advertised transport is full or delta. No hook/read ACK is needed. + args = build_parser().parse_args(shlex.split(wire["recovery"]["command"])[1:]) + assert args.max_page_output_bytes == limit + observed = consume(read_context_capture(args)) + assert hashlib.sha256(observed).hexdigest() == wire["recovery"]["sha256"] assert json.loads(observed)["heartbeat_receipt"]["turn_instance_id"] == "native-full" - assert "context" not in wire - if output_limit is None: - assert wire["eof"] and wire["content"].encode() == observed - else: - assert not wire["eof"] - recovery_args = build_parser().parse_args(shlex.split(wire["recovery"]["command"])[1:]) - assert recovery_args.max_page_output_bytes == limit + if output_limit is not None: rc, projected = _run_cli(registry, runtime, "quota", "context-project", "--codex-app", "--goal-id", GOAL_ID, "--agent-id", AGENT_ID, "--turn-instance-id", "native-full", "--capture-file", wire["recovery"]["path"], "--capture-sha256", wire["recovery"]["sha256"], "--decision-output-root", str(projection_root), "--context-delivery", "on", "--codex-context-dir", str(root), "--max-page-output-bytes", str(limit), cwd=project) - assert rc == 0 and projected["mode"] == "full", projected - assert consume(projected) == observed + assert rc == 0 and projected["ok"], projected + args = build_parser().parse_args(shlex.split(projected["recovery"]["command"])[1:]) + assert consume(read_context_capture(args)) == observed assert _heartbeat_receipt_count(runtime, "native-full") == 1 assert _spend_run_count(runtime) == 0 and state.read_bytes() == original @@ -539,7 +538,8 @@ def guard(turn, cwd=project): return view def consume(view): - page, observed = view, b"" + page = read_context_capture(build_parser().parse_args(shlex.split(view["recovery"]["command"])[1:])) + observed = b"" while True: assert page["offset"] == len(observed) observed += page["content"].encode() @@ -548,7 +548,7 @@ def consume(view): if page["eof"]: break page = read_context_capture(build_parser().parse_args(shlex.split(page["next_command"])[1:])) - assert hashlib.sha256(observed).hexdigest() == view["sha256"] + assert hashlib.sha256(observed).hexdigest() == view["recovery"]["sha256"] return json.loads(observed) first = guard("native-work-first") @@ -568,7 +568,8 @@ def consume(view): "--source", "heartbeat", "--execute", cwd=project) assert rc == 0 and spent["ok"], spent original = Path(first["recovery"]["path"]).read_bytes() - context.codex_context_hook(str(root), {**event, "source": "resume"}) + context.codex_context_hook(str(root), {**event, "hook_event_name": "PreCompact"}) + context.codex_context_hook(str(root), {**event, "source": "compact"}) assert "candidate" not in context._state(root) current = guard("native-work-next") @@ -628,8 +629,8 @@ def unavailable_directory(**kwargs): assert printed[0]["heartbeat_receipt"] == original["heartbeat_receipt"] assert printed[0]["captured_decision"] == {"path": str(tmp_path / "decision.json")} else: - assert len(printed) == 1 and printed[0]["eof"] is True - assert json.loads(printed[0]["content"]) == {"native_projection": original} + assert len(printed) == 1 and "content" not in printed[0] + assert json.loads(Path(printed[0]["path"]).read_text()) == {"native_projection": original} assert json.loads((tmp_path / "decision.json").read_text()) == original @@ -742,6 +743,9 @@ def test_native_large_pages_do_not_depend_on_shell_environment(tmp_path, monkeyp page = native_context_capture_view({"requirements": '汉🙂"\\\t\n' * 12000, "last": "Stop before publication."}, tmp_path) original = Path(page["path"]).read_bytes() + assert "content" not in page + assert len(json.dumps(page, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= 7900 + page = read_context_capture(build_parser().parse_args(shlex.split(page["command"])[1:])) limit = 64000 assert len(page["content"].encode()) > 32000 assert len(json.dumps(page, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= limit diff --git a/tests/control_plane/test_progressive_native_context.py b/tests/control_plane/test_progressive_native_context.py index 95dbe5eeb6..a605e10323 100644 --- a/tests/control_plane/test_progressive_native_context.py +++ b/tests/control_plane/test_progressive_native_context.py @@ -76,8 +76,10 @@ def test_cold_warm_and_repeated_compaction_keep_requirements_without_replaying_m assert "Do not replay read context or rerun admission solely" in result["hookSpecificOutput"]["additionalContext"] assert "Previous failure" not in json.dumps(result) _, fresh = project(tmp_path, "fresh", payload={**original, "stop": "Stop all delivery."}) - assert fresh["mode"] == "full" and fresh["context"]["stop"] == "Stop all delivery." - assert fresh["context"]["capability_gate"] == original["capability_gate"] + assert fresh["mode"] == "full" + current = json.loads(Path(fresh["recovery"]["path"]).read_bytes()) + assert current["stop"] == "Stop all delivery." + assert current["capability_gate"] == original["capability_gate"] assert json.loads(Path(archive["path"]).read_bytes()) == original @@ -148,11 +150,14 @@ def check_recovery(view): deliver(tmp_path, event, recovered) assert context._state(tmp_path)["candidate"]["capture_file"] == view["recovery"]["path"] code, warm = _run_cli(registry, runtime, *command, cwd=tmp_path) - assert code == 0 and warm["target"]["context_epoch"] == view["target"]["context_epoch"], warm + assert code == 0 and warm["ok"], warm + if "target" in warm: + assert warm["target"]["context_epoch"] == view["target"]["context_epoch"] check_recovery(warm) context.codex_context_hook(str(tmp_path), {**event, "source": "compact"}) code, compacted = _run_cli(registry, runtime, *command, cwd=tmp_path) - assert code == 0 and compacted["mode"] == "full", compacted + assert code == 0 and compacted["ok"], compacted + assert compacted["mode"] == "full" check_recovery(compacted) assert source.read_bytes() == raw and not registry.exists() and not runtime.exists() diff --git a/tests/control_plane/test_saved_context_project.py b/tests/control_plane/test_saved_context_project.py index fb75bab9e9..190fc7eb24 100644 --- a/tests/control_plane/test_saved_context_project.py +++ b/tests/control_plane/test_saved_context_project.py @@ -115,10 +115,9 @@ def test_saved_projection_manages_private_cache_without_authority_io(tmp_path, m assert code == 0 and result["ok"] and result["mode"] == "full", result assert path.read_bytes() == raw and not registry.exists() and not runtime.exists() if enabled: - assert result["schema_version"] == "context_capture_page_v1" - assert result["offset"] == 0 and result["eof"] - assert result["content"].encode() == Path(result["path"]).read_bytes() - assert json.loads(result["content"]) == original + assert "content" not in result and "offset" not in result + assert "index_command" in result and "select_command" in result + assert json.loads(Path(result["path"]).read_bytes()) == original actual_root = context if cache_location == "default" else root assert actual_root.stat().st_mode & 0o077 == 0 assert list(actual_root.glob("context-*")) @@ -145,7 +144,7 @@ def test_saved_projection_manages_private_cache_without_authority_io(tmp_path, m @pytest.mark.parametrize("body", ["Exact requirement.\n" * 7000, "完整要求与来源🙂\n" * 7000], ids=["ascii", "utf8"]) @pytest.mark.parametrize("deep_path", [False, True], ids=["short", "deep"]) -def test_saved_projection_first_page_continues_exact_bytes_and_keeps_restart(tmp_path, monkeypatch, body, deep_path): +def test_saved_projection_reference_allows_optional_exact_read_and_restart(tmp_path, monkeypatch, body, deep_path): from loopx.control_plane.turn_driver.codex_context import codex_context_hook from loopx.control_plane.turn_driver.codex_input_trace import _capture_page_range @@ -168,28 +167,30 @@ def test_saved_projection_first_page_continues_exact_bytes_and_keeps_restart(tmp "--capture-sha256", digest, "--goal-id", "goal", "--agent-id", "agent", "--turn-instance-id", "turn", "--context-delivery", "on", "--codex-app", "--codex-context-dir", str(context), cwd=tmp_path) assert code == 0 and result["mode"] == "full", result - # Include the final recovery, activation and instruction fields: an inline - # content cap alone does not bound the actual native guard stdout. - assert not result["eof"] and len(result["content"].encode()) > 32000 - assert len(json.dumps(result, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= 64000 + # The default is a reference regardless of file length. An intentional full + # read remains lossless and paginated, without changing the saved decision. + assert "content" not in result and "offset" not in result + assert len(json.dumps(result, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= 8192 delivered = Path(result["path"]).read_bytes() delivered_digest = hashlib.sha256(delivered).hexdigest() assert result["sha256"] == delivered_digest and json.loads(delivered) == json.loads(raw) - assert _capture_page_range(result, raw=delivered, digest=delivered_digest) == (0, result["next_offset"]) - assert _capture_page_range({**result, "content": result["content"][:-1]}, raw=delivered, digest=delivered_digest) is None - combined = result["content"].encode() - page = result - while not page["eof"]: - argv = shlex.split(page["next_command"]) + combined = b"" + command = result["command"] + while True: + argv = shlex.split(command) code, page = _run_cli(registry, runtime, *argv[argv.index("quota"):], cwd=tmp_path) assert code == 0 and page["offset"] == len(combined) + assert _capture_page_range(page, raw=delivered, digest=delivered_digest) == (len(combined), len(combined) + len(page["content"].encode())) + assert _capture_page_range({**page, "content": page["content"][:-1]}, raw=delivered, digest=delivered_digest) is None assert len(json.dumps(page, ensure_ascii=False, separators=(",", ":")).encode()) + 1 <= 64000 combined += page["content"].encode() + if page["eof"]: + break + command = page["next_command"] assert combined == delivered and hashlib.sha256(combined).hexdigest() == delivered_digest argv = shlex.split(result["command"]) code, restarted = _run_cli(registry, runtime, *argv[argv.index("quota"):], cwd=tmp_path) assert code == 0 and restarted["offset"] == 0 - assert restarted["content"].startswith(result["content"]) assert restarted["content"].encode() == delivered[:restarted["next_offset"]] assert source.read_bytes() == raw and not registry.exists() and not runtime.exists() @@ -429,7 +430,9 @@ def test_checkpoint_preflight_selection_inherits_native_transport(tmp_path, monk def consume(wire): data = bytearray() - page = wire + argv = shlex.split(wire["command"]) + rc, page = _run_cli(registry, runtime, *argv[argv.index("quota"):], cwd=project) + assert rc == 0, page while True: assert page["ok"] and page["offset"] == len(data) data.extend(page["content"].encode()) @@ -476,7 +479,7 @@ def consume(wire): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Explicit native Codex; loopback inference only") -@pytest.mark.parametrize("host", ["app_server", "benchmark", "app_machine", "app_code_mode"]) +@pytest.mark.parametrize("host", ["app_server", "benchmark", "app_machine", "app_code_mode", "app_progressive"]) def test_native_saved_preflight_uses_lifecycle_without_guard_replay(tmp_path, monkeypatch, host): from canonical_authority_fixture import initialize_canonical_authority, isolate_sqlite_runtime, read_fixture_todos from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection @@ -576,6 +579,12 @@ def response(request, number, message): queue.append(shlex.join([*cli, "--format", "json", "quota", "context-read", "--capture-file", descriptor["path"], "--capture-sha256", descriptor["sha256"], "--offset", str(offset)])) + if host == "app_progressive": + # Exercise the actual small reference and selective public + # reader in native model input; never consume either file in full. + assert "content" not in descriptor + queue = [descriptor["index_command"], descriptor["recovery"]["command"] + + " --select /goal_id --select /heartbeat_receipt"] phase = name + "_read" if not queue: return response(request, number, message) @@ -583,7 +592,19 @@ def response(request, number, message): else: name = phase.removesuffix("_read") descriptor = observed[name][1] - assert _delivered(request, descriptor["sha256"]) == Path(descriptor["path"]).read_text().rstrip("\n") + if host == "app_progressive": + outputs = [item["output"] for item in request["input"] + if item.get("type") in ("function_call_output", "custom_tool_call_output")] + output = outputs[-1] + if isinstance(output, list): + output = [part["text"] for part in output if part.get("type") == "input_text"][-1] + page = json.loads(output.split("\nOutput:\n", 1)[-1]) + values = {item["pointer"]: item["value"] for item in json.loads(page["content"])["selections"]} + assert values["/goal_id"] == GOAL_ID + assert values["/heartbeat_receipt"]["turn_instance_id"] == "saved-native-turn" + assert "candidate" not in json.loads((context / "session.json").read_bytes()) + else: + assert _delivered(request, descriptor["sha256"]) == Path(descriptor["path"]).read_text().rstrip("\n") if host == "benchmark" and name in ("cold", "warm"): # Consume the command actually delivered to the native # worker, rather than manually supplying missing flags. @@ -686,7 +707,7 @@ def response(request, number, message): assert restored["mode"] == "delta" # A fallback would hide a broken baseline chain. # Match the native journal's exact prior full capture and depth. The # reconstructed mapping need not preserve that capture's field order. - baseline = {**current, "ancestry": wire["ancestry"]} + baseline = {**current, "ancestry": 0 if wire.get("continuity_basis") == "captured_file" else wire["ancestry"]} if host == "benchmark": for name, (_, raw) in zip(("cold", "warm"), sources): presentation = json.loads(Path(observed[name][0]["recovery"]["path"]).read_text()) @@ -694,11 +715,28 @@ def response(request, number, message): selected = presentation["interaction_contract"]["cli_channel"]["selection_command"] original_command = original["interaction_contract"]["cli_channel"]["selection_command"]["command_args_template"] words = shlex.split(selected["command_args_template"]) + original_words = shlex.split(original_command) for option in ("--context-delivery", "--codex-context-dir", "--max-page-output-bytes"): - index = words.index(option) - del words[index:index + 2] - assert words == shlex.split(original_command) + for command_words in (words, original_words): + if option in command_words: + index = command_words.index(option) + del command_words[index:index + 2] + assert words == original_words selected["command_args_template"] = original_command + # Main already deduplicates identical Todo rows within this exact + # capture. Resolve only that registered presentation carrier. + for rows in presentation["agent_todo_summary"].values(): + if not isinstance(rows, list): + continue + for i, row in enumerate(rows): + if not isinstance(row, dict) or "context_ref" not in row: + continue + value = presentation + for part in row["context_ref"].split("/")[1:]: + value = value[int(part)] if isinstance(value, list) else value[part] + assert "context_ref" not in value + rows[i] = value + presentation.pop("context_reference_instruction", None) assert presentation == original # only presentation transport differs else: assert cold["context"] == json.loads(sources[0][1]) and projected == json.loads(sources[1][1]) diff --git a/tests/control_plane_ts/context_delivery.test.ts b/tests/control_plane_ts/context_delivery.test.ts index 1cc931af34..f201e58851 100644 --- a/tests/control_plane_ts/context_delivery.test.ts +++ b/tests/control_plane_ts/context_delivery.test.ts @@ -920,3 +920,60 @@ test("legacy quota fields are bound by the existing adapter, never caller data", // A direct codec call has no legacy binding and conservatively refreshes it. assert.deepEqual(inline(prepareContextDelivery(request), ["heartbeat_recommendation"]), value.heartbeat_recommendation); }); + +test("verified file deltas inherit unchanged controls within an epoch without retained-input claims", () => { + const original = {ok: true, goal_id: "goal", controls: {stop: false, owner: "agent"}, + opaque_control: "Unknown controls also remain available in the exact file. ".repeat(400)}; + const baseline = full(original); + const nextScope = {...scope}; + const current = {...original, controls: {...original.controls, stop: true}}; + const args = {context: current, scope: nextScope, capture_id: "capture-2"}; + const continuity = {basis: "captured_file", target: baseline.target, capture_bytes_verified: true}; + const delta = prepareContextDelivery({...args, baseline, continuity}); + assert.equal(delta.mode, "delta"); + assert.equal(delta.continuity_basis, "captured_file"); + assert.ok(JSON.stringify(delta).length < JSON.stringify(current).length / 3); + const restored = restoreContextDelivery({delivery: delta, baseline, current: prepareContextDelivery(args)}); + assert.deepEqual(restored.context, current); + for (const key of ["goal_id", "agent_id", "session_id", "context_epoch", "continuity_generation"]) { + assert.equal(prepareContextDelivery({...args, scope: {...nextScope, [key]: "other"}, baseline, continuity}).mode, "full"); + } + for (const invalid of [undefined, {...continuity, capture_bytes_verified: false}, {...continuity, target: {}}]) { + assert.equal(prepareContextDelivery({...args, baseline, continuity: invalid}).mode, "full"); + } + const corrupted = {...delta, changes: {value: original}}; + const fallback = restoreContextDelivery({delivery: corrupted, baseline, current: prepareContextDelivery(args)}); + assert.equal(fallback.reason, "automatic_full_recovery"); + assert.deepEqual(fallback.context, current); +}); + +test("file epochs survive ordinary resume but rotate once for an actual compaction", () => { + const state = {session_id: "native", cwd: "/workspace", context_epoch: "original", + recovery: {continuity_basis: "captured_file", target: {context_epoch: "original"}}, pending: {}}; + const event = {hook_event_name: "SessionStart", source: "resume", session_id: "native", cwd: "/workspace"}; + const resumed = updateNativeContextLifecycle({state, event, new_epoch: "unused", pages: []}); + assert.equal(resumed.context_epoch, "original"); + assert.equal(resumed.pending, undefined); + assert.equal(resumed.candidate, undefined); + const compacted = updateNativeContextLifecycle({state: resumed, + event: {...event, hook_event_name: "PostCompact"}, new_epoch: "next", pages: []}); + assert.equal(compacted.context_epoch, "next"); + const repeated = updateNativeContextLifecycle({state: compacted, + event: {...event, source: "compact"}, new_epoch: "do-not-rotate-again", pages: []}); + assert.deepEqual(repeated, compacted); +}); + +test("file delta recovery preserves inherited numeric leaves across sibling edits", () => { + const original = {body: long, state: {number: 1, a: false, b: false, c: false}, array: [0, "old"]}; + const current = {...original, state: {...original.state, a: true, b: true, c: true}, array: [0, "new"]}; + const baseline = full(original), complete = full(current, "capture-2"); + const delta = prepareContextDelivery({context: current, scope, capture_id: "capture-2", baseline, + continuity: {basis: "captured_file", target: baseline.target, capture_bytes_verified: true}}); + assert.equal(delta.mode, "delta"); + const changes = delta.changes as {changed: Record}; + assert.deepEqual(changes.changed.state, {values: {a: true, b: true, c: true}}); + assert.deepEqual(changes.changed.array, {items: {"1": {value: "new"}}}); + const restored = restoreContextDelivery({delivery: delta, baseline, current: complete}); + assert.equal(restored.mode, "delta"); + assert.deepEqual(restored.context, current); +}); diff --git a/tests/control_plane_ts/context_selection.test.ts b/tests/control_plane_ts/context_selection.test.ts index f9a5302768..44472e4da3 100644 --- a/tests/control_plane_ts/context_selection.test.ts +++ b/tests/control_plane_ts/context_selection.test.ts @@ -28,3 +28,14 @@ test("literal prototype-like keys are addressable only as own JSON properties", {selections: [{pointer: "/__proto__/constructor", path: ["__proto__", "constructor"]}]}); assert.throws(() => select({}, ["/__proto__"])); }); + +test("discovery lists exact child addresses without materializing values", () => { + const context = {"a/b": {"~": ["private-body", null]}, n: 3}; + const result = evaluateTurnEnvelope({operation: "context_select", context, + select: ["/a~1b", "/a~1b/~0", "/n"], list_fields: true}); + assert.deepEqual(result, {selections: [ + {pointer: "/a~1b", path: ["a/b"], children: [{pointer: "/a~1b/~0", path: ["a/b", "~"]}]}, + {pointer: "/a~1b/~0", path: ["a/b", "~"], children: [ + {pointer: "/a~1b/~0/0", path: ["a/b", "~", 0]}, {pointer: "/a~1b/~0/1", path: ["a/b", "~", 1]}]}, + {pointer: "/n", path: ["n"], children: []}]}); +});