D-WAL: LanceCycleWriter vs MemWAL measured; capped session cache, version clock, copy-free temporal projections

.claude/board/AGENT_LOG.md

@@ -9,6 +9,11 @@
 - Card `substrate-mapper` (run as a general-purpose agent with the card pasted; the card is not registered in this session), six units: `hyperon-atom`, `hyperon-space`, the minimal-MeTTa interpreter, the grounded stdlib, lance-graph `nal`/`r2il`/`relalg`, `quack`.
 - Ledger: spot-checks `Space::query` L175, `complex_query` L340, `GroundingSpace` L17, interpreter dispatch L417/L453, `query` L604, `FoldDialect` L260 — all correct; 0 wrong. 1 missed: no map recorded that `ModuleSpace` splits conjunctions and requires `ModuleSpace` dependencies (`space/module.rs:33-48`); the orchestrator found it by reading the file in full.
 - Cost: about 515k–646k subagent tokens per run. No conclusion came from a card.
+## 2026-10-14 — WAL shootout harvest: three substrate-mapper runs
+
+- Card `substrate-mapper`, three units: lance 13 `dataset/mem_wal` (5 files, 187 items), lancedb 0.40 MemWAL/LSM (4 files, ~301 items), lance-graph persist sink (`persist_sink.rs`, `batch_writer.rs`, `blw_rows.rs`, `blw_tenant.rs`, 76 items).
+- Ledger: spot-checks 2/2 per map against source (`set_lsm_write_spec` :73, `execute_lsm_merge_insert` :666, `WalSink` :534, `recover_and_apply` :677, and the `LsmWriteSpec` public API); 0 wrong. One stale upstream doc found by the orchestrator, not by a card: `LsmWriteSpec`'s doc calls the `merge_insert` dispatch a follow-up, but `table/merge.rs:320` already dispatches it.
+- Cost: about 587k, 591k and 606k subagent tokens. The maps fed the pre-registration; no conclusion came from a card.
 
 ## 2026-10-14 — D-HOLO H6b harvest: four substrate-mapper runs, one reply each
 

.claude/board/INTEGRATION_PLANS.md

@@ -5,6 +5,13 @@
 - MeTTa is a declaration IR, not an execution IR: the Datalog subset lowers through `lance-graph-relalg` → R2IL → mask-risc; everything else stays in Hyperon's interpreter.
 - Seam: a named `LanceSpace` implementing `hyperon_space::Space`; `ModuleSpace` splits conjunctions, so `&self` is not the seam.
 - Measured PeTTa (SWI-Prolog) near-linear, Hyperon superlinear (anc_800: 0.42 s vs 417 s); new trie panic at ≥ 1,024 children.
+## 2026-10-14 (2) — wal-shootout-v1 — `LanceCycleWriter` vs the lancedb 0.40 MemWAL write path → `.claude/plans/wal-shootout-v1.md`
+
+**Status:** MEASURED (D-WAL-1, D-WAL-2, spec cross-check; D-WAL-3 truncated at 27.5 min by a container restart). Feeds D-LNC-7 without deciding it.
+
+- Same logical cycle on both arms (frame + landing + image rows); M adds a `pk` column for MemWAL.
+- D-WAL-1 validation: SIGKILL atomicity (sweep must bracket the boundary), acknowledged ⇒ durable (read before and after writer replay), idempotent resubmit, read horizon.
+- D-WAL-2 cost per cycle and per row at N = 64 / 1,024 / 8,192; D-WAL-3 60-minute soak per arm with a CSV sample every 30 s.
 
 ## 2026-10-14 — d-holo-address-indexed-algebra-v1 — falsifiers H1, H5, H6 for the address-indexed algebra → `.claude/plans/d-holo-address-indexed-algebra-v1.md`
 

.claude/board/LATEST_STATE.md

@@ -1,3 +1,10 @@
+## 2026-10-15 — main after #1555 (merged); D-WAL measured on branch `ccr-9f3b8075-74u47l-wal-shootout` (unpushed)
+
+- **On `main` (#1555, merged `5366a5ab`):** `crates/lance-graph-metta` (excluded) with the D-MTP-1 join probe and the D-MTP-1b range-hop probe; plan `metta-polyfill-v1.md`. Examples only, no production change.
+- **On branch:** D-WAL-5/6/7 (decisions accepted 2026-10-15): `LanceCycleWriter` opens and writes on a Lance session with an 8 MiB metadata cache (`SESSION_METADATA_CACHE_BYTES`); index forms of `deinterlace` / `local_trajectories` in `lance-graph-planner::temporal`.
+- **Contract (new type):** `temporal_pov::{VersionClock, ClockError}` — rows of an append-only lane per version; `rows_at(&TemporalPov)` resolves a point of view to one half-open row range (v4: 4.2 ns per call; a per-row scan of 400k rows takes 127 µs). Eight bytes per version.
+- **On branch (unpushed):** D-WAL-1/2/3 for `LanceCycleWriter` vs the lancedb 0.40 MemWAL path (probe, harness, example behind `lancedb-sdk`); entry `2026-10-15-d-wal-shootout.md`. No production wiring.
+
 ## 2026-10-14 — the rail walk applies the reader's field mask on branch `ccr-0455e606-wmtsor` (unmerged)
 
 - **Contract:** `selection::walk_rails_authorized(graph, class_view, registry, view_binding, allowed, root, max_depth, visit)`. Same walk as `walk_rails`, with `allowed(class)` added to the AND at every hop: `view.mask ∩ presence ∩ allowed(class)`. A hidden field is not emitted at any depth (each hop uses its own class's mask), and a hidden rail is not followed, so a linked object cannot be read through a link the reader may not see. `allowed` returns a mask, not an `Option`, so there is no permit-all fallback. `walk_rails` is unchanged. Closes H7 for every output built on the walk (OGAR `ogar-doc-ir` resolve uses it); the caller passes the `field_mask` of its `Decision` per class. Tests: four `selection::tests::authorized_walk_*`; three fail with the mask removed, the hidden-rail test fails with the mask applied to emission only.

.claude/board/PR_ARC_INVENTORY.md

@@ -1,3 +1,10 @@
+## 2026-10-14 — lance-graph PR #1555 (merged `5366a5ab`, branch `ccr-9f3b8075-74u47l-metta`, head `457a8c41`, 14 commits) — D-MTP: MeTTa over the masking-algebra polyfill; two-hop join and the hop as a range
+
+- **Added:** plan `metta-polyfill-v1.md` (Hyperon seams, PeTTa vs Hyperon measurements, MeTTa-as-IR verdict, datafrog fork as the exact Rust oracle). `crates/lance-graph-metta` (own workspace, excluded; datafrog from `AdaWorldAPI/datafrog` as a dev-dependency only) with `join_probe` (D-MTP-1, two-hop join through Quack `Semijoin`) and `join_range_probe` (D-MTP-1b, the hop as a range under a monotone `dst`). Entries `2026-10-14-metta-polyfill-map.md`, `-d-mtp-1-join.md`, `-d-mtp-1b-range-hop.md`; STATUS D-MTP rows.
+- **Locked:** pairs equal datafrog at every size; counts equal Hyperon and PeTTa. At 10⁶ (v4): gather count 1.2–1.8 ns/row, emit 4.0–5.7 ns/pair against datafrog's 59–71; the range path counts in 0.0001–0.0004 ns/row with runs recorded at ingest and refuses a non-monotone lane. relalg has no hop yet, so the join lowers through Quack.
+- **Deferred / open:** D-MTP-2..5 (named `LanceSpace`, recursion via the external driver, the trie width fix upstream, PeTTa in the probes); the order witness is probe-local.
+- **Confidence:** high for equality (disable runs red as pre-registered); timings are best of 5 over three runs on one VM.
+
 ## 2026-10-14 — lance-graph PR #1552 (merged `98c03dd1`, branch `ccr-9f3b8075-74u47l-haiku-harvest`) — Haiku harvest tier: six cards and their supervision rules
 
 - **Added:** cards `.claude/agents/{harvest-scout,census-counter,quote-harvester,substrate-mapper,spec-editor,guarded-executor}.md`; `tiered-agent-execution-protocol.md` § Haiku harvest tier (card table, brief sizing, supervision rules, mapping a large surface); CLAUDE.md Model Policy rewritten; BOOT files, worker template, cca2a and v3 BOOT aligned; AGENT_LOG entries for the first runs; #1551 board entries.

.claude/board/STATUS_BOARD.md

@@ -1,3 +1,17 @@
+## D-WAL — `LanceCycleWriter` vs the lancedb 0.40 MemWAL write path (2026-10-14)
+
+Plan: `.claude/plans/wal-shootout-v1.md`. Feeds D-LNC-7; decides nothing.
+
+| D-id | scope | status | gate / falsifier |
+|---|---|---|---|
+| **D-WAL-1** | V1 SIGKILL atomicity, V2 acknowledged ⇒ durable, V3 idempotent resubmit, V4 read horizon; both arms | Measured (§2a): V1–V3 hold on both arms; M's acknowledged writes are invisible to other readers until a flush or a writer replay | no partial cycle, sweep brackets the boundary; no acknowledged cycle lost after the replay read; V3/V4 behaviour recorded |
+| **D-WAL-2** | commit p50/p99 and ns/row at N = 64 / 1,024 / 8,192; disk, files, versions/generations, full-scan latency after 200 cycles | Measured (§3a): M waits for the WAL ticker (~95 ms at the 100 ms default); at 10 ms faster than L from N = 1,024; M holds ~2× the bytes and its scan grows without compaction | numbers with tier; no threshold |
+| **D-WAL-3** | 60-minute soak per arm, CSV every 30 s | Measured, truncated at 27.5 min / 1,649 cycles by a container restart (§4a): M commit p50 flat 2.0–2.8 ms, L 5.8 → ~10.5 ms; M disk 1.57× L (WAL kept after each flush); both scans linear in rows; L RSS 95 → 762 MiB unexplained | drift, growth and RSS reported |
+| **D-WAL-4** | where L's resident memory goes | Measured: every Lance commit caches a full manifest clone per version (lance 13 `record_successful_commit`); append-only adds one fragment per version, so the cache grows as V²/2 to its 1 GiB default. 1,600 versions: default session 628 MiB cache / 570 MiB RSS, 8 MiB-capped session 7.8 MiB / 52 MiB | `examples/wal_rss_probe.rs`, default vs capped arm in separate processes |
+| **D-WAL-5** | cap the writer's Lance session | Done: open and both Create writes use a session with an 8 MiB metadata cache. 400 commits: 42.6 MB uncapped, under 8 MiB capped; reopen path covered (127 MB uncapped at 800) | `session_cache_stays_under_the_cap_over_many_commits`; disables on the Create write and on open each turn it red |
+| **D-WAL-6** | version clock: a point of view as one row range | Done: `contract::temporal_pov::VersionClock`; v4 4.2 ns per `rows_at` vs 127 µs per-row scan at 400k rows | row-version oracle over 200 versions with gaps; disables (gap fill, start offset, base guard) each turn it red. Not yet wired into a writer or reader; seeding from storage at open is open |
+| **D-WAL-7** | `temporal.rs` without row copies | Done: `deinterlace_indices`, `local_trajectories_indices`, `local_trajectory_indices_of`; copying forms keep their signatures as wrappers | clone counter at 0 for the index form; removing the sort turns the two ordering tests red |
+
 ## D-MTP — Hyperon/MeTTa over the masking-algebra polyfill (2026-10-14)
 
 Plan: `.claude/plans/metta-polyfill-v1.md`. Proposed crate `lance-graph-metta` (not created).
@@ -850,7 +864,7 @@ identity) is real and probe-gated.
 | D-GTM-0n / P3 | **Ternlog chaining amortization** (`ndarray/examples/ternlog_amortization_probe.rs`, re-run `--release` 2026-09-06): does chaining pay on THIS mask shape? Pre-registered pass criterion = per-constraint cost flat in K, with a **K=1 control** (nothing to chain ⇒ no win) | **GREEN, bounded.** `T3/T1` bottoms at **0.50** by K=8; T3 total flat in K (57.6 ns @ K=4 → 54.1 ns @ K=32, 16× more constraints). K=1 control reads **1.03** as it must. Two bounds ride with the number and may not be dropped when it is cited: the win is **L2-residency-contingent** (bandwidth `138→15 GB/s` past L2, ratio back to 1.03 at a 512 KiB mask), and mask loses to sparse below **0.1%** active. **Does NOT retire the standing caveat** — 2× on one of three passes still cannot account for 5×. Gates Stage-3 SPEED claims only; the shape/correctness case never depended on it |
 | D-ALPHA-1b | **`AlphaMask::zip` release-mode length guard** (`lance-graph-contract/src/alpha.rs`): the only guard was `debug_assert_eq!`, compiled out in release, so mismatched-length operands produced a mask claiming `self.len` addresses over the SHORTER operand's word count — `count`/`is_empty` under-report silently, `contains`/`materialize_ordinals` panic at a distance | **Fixed** — promoted to `assert_eq!`. Defect reproduced in release BEFORE the fix (the `should_panic` falsifier reported "test did not panic as expected"); two-sided falsifiers both disable-verified. No API change, no external callers. Stage 1b of `.claude/temporal/09-plan.md`, now DONE |
 | D-NGN-AUDIT | **`NodeGuid::new` repurpose audit** (5+3 council, `.claude/plans/nodeguid-new-repurpose-audit-v1.md`): can the V1 mint be repurposed as the V3 cascade constructor? READ-ONLY — operator: *"it's already in production, if you touch it it would change something you half understand."* | **HELD at Phase 2**; SPEC v1 SHIPPED in #1217 (`b518dbf1`). Phase 1 RETURNED (all 5 savants); consolidation is held on a VIOLATES against frozen decision F5 (*"Nobody walks anything"*) — `GET /views/graph/wave.abi` reaches `inherited_part_of` — which is the operator's to rule, not the orchestrator's to resolve. **Do NOT re-cast Phase 1, and do not cast the 3 reviewers until that ruling lands.** Draft v2 also owes F9 (the #1202 dating rule). The spec was orphaned by a branch reset before #1201 merged and recovered on 2026-09-07; #1201 itself merged hygiene-only. Spec carries a §0 retracted-readings table (4 false orchestrator readings, banked so they are not re-derived) |
-| D-LNC-7 | **One-WAL ruling**: `LanceCycleWriter`'s watermark WAL vs lance MemWAL — `applied_through` ≡ index catch-up position; two under one commit = two truths. Plan recommends keep-ours until D-LNC-3, then re-decide | **Operator decision** |
+| D-LNC-7 | **One-WAL ruling**: `LanceCycleWriter`'s watermark WAL vs lance MemWAL — `applied_through` ≡ index catch-up position; two under one commit = two truths. Plan recommends keep-ours until D-LNC-3, then re-decide | Decided 2026-10-15 after D-WAL-1..5: `LanceCycleWriter` stays the default (session metadata cache capped, D-WAL-5); MemWAL stays an option. 60-minute soak deferred |
 | D-TEMPORAL-1 | **Canonical witness identity probe** (`crates/lance-graph/tests/canonical_witness_identity_probe.rs`): does canonical address identity survive the A-store round trip on lance 11? `NodeGuid → NodeRow → NodeRowPacket::as_le_bytes → cycle commit → drop writer → reopen → scan_image`. Minted through `mint_for(classid_read_mode(c).tail_variant, …)`, never a hardcoded constructor | **GREEN, both rails.** V3 (`CLASSID_OSINT_V3`): payload bytes byte-exact across two cycles and a reopen; `payload[0..16] == G.as_bytes()`; value slab survived (so equality is not a zero-match); each cycle minted exactly `base+1`; head recovered at V2 after restart. V1 legacy control (`CLASSID_OSINT`) round-trips too. **Scope:** proves byte and address fidelity of the store; does NOT prove production reachability — `LanceCycleWriter` still has zero non-test callers (see D-MW-P2) |
 | D-TEMPORAL-1a | **Pinned false-green: the V1 accessor on a V3 key.** 256 V3 addresses varying only in `leaf` → 256 distinct 16-byte keys, 256 distinct `to_hex_v2()`, but **1** distinct `identity()`. Byte fidelity does not imply address fidelity | **Pinned as a positive assertion.** Also pins the second half, which is not a defect: `local_key_v2` yields **1** across the same set BY CONSTRUCTION — `leaf` is the 4th HHTL routing tier (bytes 10..12), part of the addressing prefix, deliberately outside the basin-local key (bytes 12..16). So "use the V3 accessor" is two choices, not one, and the wrong V3 one fails like the V1 one. Do not update the numbers if the assertion breaks — re-audit the call sites |
 | D-V3-MINT-1 | **V3 mint reachability** (`crates/lance-graph-contract/tests/v3_mint_reachability_probe.rs`): `mint_for`'s V2/V3 arm is gated on `guid-v2-tail`, which is NOT in `default`; the registry's V3 entries are gated on `guid-v3-tail`, which is. Two different features, so premise and conclusion can come apart — is a V3-registered classid silently minted into the deprecated V1 layout? | **GREEN — no defect.** `guid-v3-tail = ["guid-v2-tail"]` (`Cargo.toml:69`), so the implication holds. Measured, not inferred: `tail_variant=V3 guid-v3-tail=true guid-v2-tail=true leaf_is_live=true`, key bytes show the V2/V3 carving with `leaf` live at byte 10. Discriminator is `leaf` (the V3 arm feeds it to `new_v2`; the V1 fallback discards it), so the probe stays valid if the feature graph is ever flattened |

.claude/board/entries/2026-10-15-d-wal-shootout.md

@@ -0,0 +1,37 @@
+# 2026-10-15 — D-WAL — `LanceCycleWriter` vs the lancedb 0.40 MemWAL write path (measured)
+
+**Status:** MEASURED. Plan `.claude/plans/wal-shootout-v1.md` §2a (D-WAL-1), §3a (D-WAL-2),
+§3b (spec cross-check against lance `v13.0.0` `docs/src/format/table/mem_wal.md`), §4a
+(D-WAL-3, truncated). Probe `crates/lance-graph/tests/wal_shootout_probe.rs`, harness
+`tests/wal_common/`, example `examples/wal_shootout.rs`, all behind `lancedb-sdk`.
+
+**Outcome:**
+- Both paths are crash-atomic per cycle and lose nothing they acknowledged (SIGKILL sweep
+  brackets the boundary on both). MemWAL needs `durable_write = true` for that; with
+  `false` an acknowledged cycle is lost (disable run).
+- MemWAL's acknowledged writes are visible to the writing handle only; another handle or
+  process sees them after a flush or after it replays the WAL by opening a writer, and the
+  base table only after compaction. The spec states this as its consistency model
+  (strong only with live memtable access, otherwise eventual).
+- Cost: with the default 100 ms flush every MemWAL write waits about 95 ms; at 10 ms it is
+  slower than `LanceCycleWriter` at 64 casts per cycle and faster from 1,024. In the soak
+  (256 casts, 10 ms) MemWAL's commit stayed flat at 2.0–2.8 ms while L's grew from 5.8 to
+  about 10.5 ms over 1,649 versions.
+- MemWAL keeps the WAL after flushing: disk grows by a WAL increment per cycle plus a
+  second copy at each memtable flush (every ~195 cycles = the 100k-row default), 1.57× L at
+  1,649 cycles. The spec describes no component that removes WAL files.
+- Neither path compacts; both full scans grow linearly in rows.
+
+- L's RSS growth is Lance's session metadata cache, not our writer (D-WAL-4,
+  `examples/wal_rss_probe.rs`): each commit inserts a clone of the new manifest keyed by
+  version, and in an append-only store manifest v lists v fragments, so the cache holds
+  about V²/2 fragment entries up to its 1 GiB default. At 1,600 versions: default session
+  628 MiB cache, 570 MiB RSS; session with an 8 MiB metadata cache 7.8 MiB, 52 MiB RSS.
+  Capping the metadata cache, or compacting so fragment count stops growing, bounds it.
+
+- Follow-up done the same day (D-WAL-5/6/7): the writer's session cap, a version clock
+  (`contract::temporal_pov::VersionClock`, 4.2 ns per point-of-view read on v4), and index
+  forms of the `temporal.rs` projections. D-LNC-7: L stays default, MemWAL an option.
+
+**Open:** a full-length soak (deferred; this one stopped at 27.5 min when the container restarted); the effect of compaction on both; D-LNC-7
+(which WAL to keep) stays a decision for the user.

.claude/board/entries/README.md

@@ -25,11 +25,12 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
 opposite directions; the stranding this convention prevents shows up in
 exactly one of them, never both.
 
-335 entries, 2026-08-06 .. 2026-10-15.
+336 entries, 2026-08-06 .. 2026-10-15.
 
 | date | entry id | finding | file |
 |---|---|---|---|
 | 2026-10-15 | `ledger-correction-wire-contract` |  | [2026-10-15-ledger-correction-wire-contract.md](2026-10-15-ledger-correction-wire-contract.md) |
+| 2026-10-15 | `D-WAL` | `LanceCycleWriter` vs the lancedb 0.40 MemWAL write path (measured) | [2026-10-15-d-wal-shootout.md](2026-10-15-d-wal-shootout.md) |
 | 2026-10-14 | `revision-layer-invariants` |  | [2026-10-14-revision-layer-invariants.md](2026-10-14-revision-layer-invariants.md) |
 | 2026-10-14 | `D-MTP-0` | where Hyperon/MeTTa attaches to the masking-algebra polyfill | [2026-10-14-metta-polyfill-map.md](2026-10-14-metta-polyfill-map.md) |
 | 2026-10-14 | `ledger-revision-after-1553` |  | [2026-10-14-ledger-revision-after-1553.md](2026-10-14-ledger-revision-after-1553.md) |

.claude/plans/lance-graph-as-the-modelgraph-v1.md

@@ -323,10 +323,9 @@ accordingly to batchwriter."* Followed. The arc, and what it left:
 
 **So `Dataset::write` is real and in-tree** — two call sites, under a module doc
 saying the I/O *"is INTERNAL to this one writer"*:
-`Dataset::write` at `crates/lance-graph/src/graph/cycle_sink.rs:675` (the Create
-inside `LanceCycleWriter::bootstrap`) and `Dataset::write` at
-`crates/lance-graph/src/graph/cycle_sink.rs:710` (the Create-or-Append inside
-`LanceCycleWriter::raw_append`). The plan's G-C above — *"no `Dataset::write`"* —
+`Dataset::write` in `crates/lance-graph/src/graph/cycle_sink.rs`, function
+`LanceCycleWriter::bootstrap` (the Create), and `Dataset::write` in the same file,
+function `LanceCycleWriter::raw_append` (the Create-or-Append). The plan's G-C above — *"no `Dataset::write`"* —
 was **wrong when written.**
 
 > ⊘ The first spelling of this citation was `…rs:675` followed by a bare