diff --git a/.claude/board/AGENT_LOG.md b/.claude/board/AGENT_LOG.md index 5c4087679..31ac79589 100644 --- a/.claude/board/AGENT_LOG.md +++ b/.claude/board/AGENT_LOG.md @@ -1,3 +1,9 @@ +## 2026-10-14 — D-HOLO H6b harvest: four substrate-mapper runs, one reply each + +- Card `substrate-mapper`, four units: `holo_interval_probe.rs` (H6), mask-risc exec, mask-risc IR, ndarray `simd` mask re-exports. +- Ledger: spot-checks 3/3, 2/2, 3/3 and the re-export lines correct; 0 wrong, 0 missed. The ndarray map lists re-exports without signatures, as the card requires; signatures were read from `src/simd_masking_ops.rs` by the orchestrator. +- Cost: about 454k, 518k, 556k and 566k subagent tokens. The maps fed the H6b probe; no conclusion came from a card. + ## 2026-10-14 — first Haiku harvest run (harvest-scout card, policy change) - Card `harvest-scout`, one run: lines still forbidding Haiku across `.claude/{agents,skills,knowledge,v3}`, `.claude/BOOT.md`, `CLAUDE.md`, 8 patterns. diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 8e05023b6..9709f1810 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,8 @@ +## 2026-10-14 — main after #1552 (merged) + +- **On `main` (#1552, merged `98c03dd1`):** the Haiku harvest tier (six cards, supervision rules); policy only. +- **In PR (#1553):** D-HOLO H6b measured (`holo_tile_forms_probe`): structured rule not met, hybrid byte switch refuted, group count tracks P's cost. H6c pre-registered (group-count crossover, group-count switch, declared construction); not run. + ## 2026-10-14 — main after #1551 (merged) - **On `main` (#1551, merged `9337da49`):** D-HOLO H11 and H12 pre-registered, H6b amended with connectedness; plan only, nothing run. diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 49be75a56..f76fc2981 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,10 @@ +## 2026-10-14 — lance-graph PR #1552 (merged `98c03dd1`, branch `ccr-9f3b8075-74u47l-haiku-harvest`) — Haiku harvest tier: six cards and their supervision rules + +- **Added:** cards `.claude/agents/{harvest-scout,census-counter,quote-harvester,substrate-mapper,spec-editor,guarded-executor}.md`; `tiered-agent-execution-protocol.md` § Haiku harvest tier (card table, brief sizing, supervision rules, mapping a large surface); CLAUDE.md Model Policy rewritten; BOOT files, worker template, cca2a and v3 BOOT aligned; AGENT_LOG entries for the first runs; #1551 board entries. +- **Locked:** Haiku is read-only harvest and exact-spec edits; it never authors, reviews or decides. The orchestrator Reads every item relied on, re-runs one count per reply, spot-checks every map, reads every spec-editor diff; two misses on one kind of task drop that kind back to Sonnet. +- **Deferred / open:** per-card miss rates; whether the fixed context cost (~0.45 M tokens per run) makes small searches uneconomic. +- **Confidence:** medium; policy, first runs 9/9, 2/2, 7/7. + ## 2026-10-14 — lance-graph PR #1551 (merged `9337da49`, branch `ccr-9f3b8075-74u47l-holo-prereg`, head `d56f2581`, 1 commit) — D-HOLO: pre-register H11 and H12, connectedness in H6b - **Added:** plan sections H11 (stacked logic: answers no single brick gives, == oracle; cost below separate runs at d = 4; no confidence inflation, gate disable must inflate) and H12 (semantic map of inference functions across OpenNARS, ONA and the NAL polyfill; stability rule: the last system adds zero nodes in both orders). STATUS rows D-HOLO-H11/H12; the #1550 arc and LATEST_STATE entries. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index f4eba615e..af046bf12 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -7,7 +7,8 @@ Plan: `.claude/plans/relalg-v1.md`. Crate: `crates/lance-graph-relalg` (own work | **D-HOLO-H1** | K-branch evaluation of a coordinate-aligned Boolean constraint (plan `d-holo-address-indexed-algebra-v1`) | Measured 2026-10-14: MIXED, C flat in K; beats gate-once only with row-stored coverage and a narrow gate | == row oracle; total cost at K = 64 ≤ 2× K = 1 and ≤ 0.5× best baseline | | **D-HOLO-H5** | Coordinate-native vs coordinate-crossing: prefix count, perspectives, reverse `memberOf` with inverse build/update cost | Measured 2026-10-14, corrected after review: Q1 CONFIRMED at N = 100 (31–37×), FAILS at N = 1 (1.3–1.4×, the occupancy build dominates); Q2 passes only with row words stored; plane inverse refuted on Q3b (break-even > 1,000) | == row oracle; Range ≥ 5× sweep; break-even N reported; permuted-tile run disagrees | | **D-HOLO-H6** | Closure of address restrictions as interval sets vs masks vs value sweep | Measured 2026-10-14: HOLDS; symbolic cheapest in all 24 cells | == row oracle; symbolic cost flat in area, linear in interval count | -| **D-HOLO-H6b** | Product normal form vs Morton intervals vs dense mask vs hybrid on the Context × Operation reading of the `u8:u8` tile; construction, normalization, ∩/∪/¬, emptiness, equality, bytes, reuse at N = 1..1,000 | Queued (pre-registered, not run; amended with a connectedness column and a doughnut fixture) | == row oracle; structured patterns ≤ 0.5× dense at N = 100; hybrid ≤ 1.25× dense on adversarial patterns; component count vs density as predictor reported | +| **D-HOLO-H6b** | Product normal form vs Morton intervals vs dense mask vs hybrid on the Context × Operation reading of the `u8:u8` tile; construction, normalization, ∩/∪/¬, emptiness, equality, bytes, reuse at N = 1..1,000 | Measured 2026-10-14: structured rule NOT MET (build dominates at N = 100; P 0.21–0.73× D per query); hybrid passes checkerboard and random 50 %, fragmented 1.19–1.33; byte switch refuted by sparse/dense (H 1.8–4.8× D); group count, not bytes or density, tracks P's cost | == row oracle; structured patterns ≤ 0.5× dense at N = 100; hybrid ≤ 1.25× dense on adversarial patterns; component count vs density as predictor reported | +| **D-HOLO-H6c** | Group-count crossover (g = 1..64, blocked vs interleaved), a group-count switch for the hybrid, and construction from a declaration with the same declaration for every arm | Queued (pre-registered, not run) | == row oracle; P/D per query crosses 1 between g = 4 and 16 in both layouts; H' ≤ 1.25× D everywhere and never ≥ 2×; declared structured rule P or H ≤ 0.5× D at N = 100 | | **D-HOLO-H8** | One formula in three domains: logical algebra, physical realization and semantic leakage compared separately | Queued (pre-registered, not run) | no semantic leakage; families grow only with representation and value algebra | | **D-HOLO-H9** | Relational emptiness decided symbolically (empty operand, disjoint domains, contradictory constraints) vs by fold | Queued (pre-registered, not run) | zero false empties; fraction decided symbolically reported | | **D-HOLO-H11** | Stacked logic: a chain of NAL polyfill bricks at depth 1–8 composed before one evaluation | Queued (pre-registered, not run; after H6b) | answers no single brick gives, == oracle; cost below d separate runs at d = 4; no confidence inflation, gate disable must inflate | diff --git a/.claude/board/entries/2026-10-14-d-holo-h6b.md b/.claude/board/entries/2026-10-14-d-holo-h6b.md new file mode 100644 index 000000000..1dd182138 --- /dev/null +++ b/.claude/board/entries/2026-10-14-d-holo-h6b.md @@ -0,0 +1,18 @@ +# 2026-10-14 — D-HOLO-H6b — normal forms on the Context × Operation tile (measured) + +**Status:** MEASURED (v4, avx512f, three runs). Plan `.claude/plans/d-holo-address-indexed-algebra-v1.md`, +§ Results: H6b. Probe: `crates/lance-graph-mask-risc/examples/holo_tile_forms_probe.rs`. + +**Outcome:** +- All four arms (product form P, Morton intervals I, dense mask D, hybrid H) match the row oracle on + all 14 patterns, the doughnut included. Three disable runs red. +- Structured rule (P or H ≤ 0.5× D at N = 100): NOT MET, undecided. The 65,536-cell build (~0.3 ms + in every arm) dominates the totals. Query-only, P is 0.21–0.73× D on the structured patterns. +- Adversarial rule (H ≤ 1.25× D): checkerboard and random 50 % pass; fragmented 1.19–1.33. +- Counterexample: the hybrid's byte switch keeps P on sparse and dense (60–61 groups), where H costs + 1.8–4.8× D. P's cost follows its group count (post hoc Spearman +0.81 / +0.88), not its bytes. +- Pre-registered predictor: component count ahead of density (+0.34–0.55 vs −0.07–0.12), weak at + 14 patterns. + +**Open:** the selector stays held. Not measured: a group-count switch, the crossover between 2 and +60 groups (no fixture there), P built from a declaration instead of a cell sweep. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 1e5d891d5..9eb7728c6 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,12 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -329 entries, 2026-08-06 .. 2026-10-14. +330 entries, 2026-08-06 .. 2026-10-14. | date | entry id | finding | file | |---|---|---|---| | 2026-10-14 | `revision-layer-invariants` | | [2026-10-14-revision-layer-invariants.md](2026-10-14-revision-layer-invariants.md) | | 2026-10-14 | `D-NALP-4` | | [2026-10-14-isa-truth-exact-codes.md](2026-10-14-isa-truth-exact-codes.md) | +| 2026-10-14 | `D-HOLO-H6b` | normal forms on the Context × Operation tile (measured) | [2026-10-14-d-holo-h6b.md](2026-10-14-d-holo-h6b.md) | | 2026-10-14 | `D-HOLO` | | [2026-10-14-d-holo-falsifiers.md](2026-10-14-d-holo-falsifiers.md) | | 2026-10-13 | `D-PROD-V4` | | [2026-10-13-production-targets-v4.md](2026-10-13-production-targets-v4.md) | | 2026-10-12 | `D-SAG-1` | | [2026-10-12-software-awareness-graph.md](2026-10-12-software-awareness-graph.md) | diff --git a/.claude/plans/d-holo-address-indexed-algebra-v1.md b/.claude/plans/d-holo-address-indexed-algebra-v1.md index 413f2f410..5262565e6 100644 --- a/.claude/plans/d-holo-address-indexed-algebra-v1.md +++ b/.claude/plans/d-holo-address-indexed-algebra-v1.md @@ -1,6 +1,6 @@ # D-HOLO — address-indexed algebra: falsifiers H1, H5, H6 (v1) -**Status:** MEASURED 2026-10-14 (H1, H5, H6); H6b, H8, H9, H11, H12 +**Status:** MEASURED 2026-10-14 (H1, H5, H6, H6b); H6c, H8, H9, H11, H12 PRE-REGISTERED, not run. The pre-registration below is unchanged; results are in § Results (2026-10-14), the next falsifiers in § Next falsifiers. @@ -34,7 +34,8 @@ is run. - [x] D-HOLO-H5: coordinate-native vs coordinate-crossing probe, same - [x] D-HOLO-H6: closure of address restrictions probe, same - [x] Results section, board entry, status rows -- [ ] D-HOLO-H6b: normal forms on the Context × Operation tile (with the connectedness column) +- [x] D-HOLO-H6b: normal forms on the Context × Operation tile (with the connectedness column) +- [ ] D-HOLO-H6c: group-count switch, the 2–60 group crossover, declared construction - [ ] D-HOLO-H8: semantic diversity does not add instruction families - [ ] D-HOLO-H9: relational emptiness without a fold - [ ] D-HOLO-H11: stacked logic (pre-registered after H6b; runs after it) @@ -669,3 +670,128 @@ reverse order; the nodes our polyfill does not cover; disconnected regions. H13 (attention-limited expansion vs full expansion, "saccades") and an inertness test for every proposed meta-awareness level stay unregistered: H13 needs a named focus signal, and the levels need a stated meaning. + +## Results: H6b (2026-10-14) + +Probe `crates/lance-graph-mask-risc/examples/holo_tile_forms_probe.rs`, v4 +(avx512f), three runs. Every arm matches the row oracle cell by cell on all 14 +patterns, including the doughnut. Disable runs, each red: P without merging +equal operation masks (`product_form_is_canonical_across_routes` and the oracle +test fail); binary ops without the implicit empty group (both fail); the hybrid +ignoring its threshold (`hybrid_switches_both_ways` fails). + +Deviation from the pre-registration: normalization is folded into construction +and into each operation, not timed as its own step. Every arm builds by sweeping +the pattern's membership function over all 65,536 cells. + +**Verdicts against the pre-registered rules.** + +| Rule | Result, three runs | Verdict | +|---|---|---| +| Structured (row, column, rectangle, prefix): P or H total at N = 100 ≤ 0.5× D | P 0.64–1.28, H 0.71–1.01 | NOT MET; undecided (under 2×) | +| Adversarial: H total at N = 100 ≤ 1.25× D | checkerboard 0.60–0.85, random 50 % 1.04–1.17, fragmented 1.19–1.33 | checkerboard and random 50 % pass; fragmented passes in one run, undecided in two | +| Component count predicts the winner better than density (Spearman with ln(best symbolic / D)) | components +0.34 / +0.55 / +0.52, density −0.06 / −0.07 / +0.12 | components ahead, weak at 14 patterns | + +**Why the structured rule fails: construction dominates.** The 65,536-cell +build costs about 0.3 ms in every arm, against a query of 0.6–2.6 µs, so at +N = 100 the totals are mostly build. At N = 1,000 P reaches 0.29–0.77× D on the +structured patterns. Query-only (post hoc, not pre-registered), P is 0.21–0.73× +D on those four patterns, and 0.25–0.62× on checkerboard, complementary, +disjoint and doughnut: about 2–4× per query, at the edge of the 2× noise rule. +A symbolic construction (from a declared rectangle or class, O(groups) instead of +O(cells)) was not measured; the rule cannot be judged without it. + +**Counterexample: the hybrid's byte switch is wrong.** H stays in P while P is +smaller than 8 KiB. On sparse (60 groups, 3.8 KiB) and dense (61 groups) that +keeps P, and H's total is 1.8–4.8× D's (query 6.8–14.6× D's). H as specified is +not safe as a default: it is more than 2× worse than D on two patterns outside +the adversarial set. P's cost follows the group count (a binary op builds the +g_a × g_b refinement), not its bytes. Post hoc, P's group count predicts +ln(P query / D query) with Spearman +0.81 and +0.88; components +0.37 / +0.45, +density −0.17 / −0.12. + +**Thresholds and counterexamples.** +- P is the cheapest per query at 1–2 groups (row, column, rectangle, prefix, + complementary, disjoint, doughnut, checkerboard) and loses at 60 or more + (sparse, dense: 7–11× D; diagonal, random, fragmented: 59–188× D). No fixture + lies between 2 and 60 groups, so the crossover is not located. +- I is cheapest only on the prefix (one interval, 0.04–0.05× D per query), as + in H6; it loses heavily wherever intervals multiply (checkerboard 120×, random + 50 % 400–474× D). +- Density predicts nothing: checkerboard and random 50 % share density 0.5 and + sit at opposite ends (P 0.25–0.30× vs 96–188× D per query). +- Connectedness alone is not enough: the staircase ("fragmented") is one + component with 255 groups and loses as P by 60–77×; the doughnut is one + component with 2 groups and wins. + +**What this changes.** The terminal representation selector stays held. The +evidence points at a switch on group count, not bytes, and at symbolic +construction as the term that decides whether P pays at small N. Neither is +measured. Open: a fixture family between 2 and 60 groups to locate the +crossover; P built from a declaration instead of a cell sweep. + +## Next falsifier: H6c (PRE-REGISTERED 2026-10-14, NOT RUN) + +DECISION (operator, 2026-10-14): pre-register H6c after H6b. SCOPE: examples +only, no new opcode or executor, no production selector. BASIS: H6b's three +open terms. REVISIT WHEN: H6c reports. + +Same universe, arms, oracle, tier, timing and noise rule as H6b (v4 only, tier +printed, median of 11, three runs, min–max reported, differences under 2× +undecided). Every arm still matches the row oracle on every cell first. + +### Q1 — the crossover in group count + +**Fixture family.** For g ∈ {1, 2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64}: g +distinct operation masks (seeded, density 0.5), assigned to contexts in two +layouts, so that group count is separated from contiguity: +- *blocked:* contiguous context ranges of equal size; +- *interleaved:* context c gets mask c mod g. + +A and B are two such patterns with the same g and layout and independent seeds. + +**Measured.** P and D per query (the H6b query), ∩, ∪ and ¬, per g and layout. + +**Prediction (falsifiable).** P's query cost grows with g_A × g_B (the +refinement), D's stays flat, and the per-query ratio P / D crosses 1 between +g = 4 and g = 16 in both layouts. It is falsified if the crossover lies outside +that range, or if the two layouts disagree on it by more than one step of the +g grid (then contiguity, not group count, is the driver). + +### Q2 — a group-count switch for the hybrid + +H' uses P while P's group count is at most G, else D; G ∈ {2, 4, 8, 16, 32}. +The group count is known after construction, so the switch adds no pass. + +**Decision.** H' with the G closest to Q1's crossover is safe as a default if, +on all 14 H6b patterns and the Q1 family, its total at N = 100 and its query +cost are both at most 1.25× D, and it is never 2× or more above D. The byte +switch (H) is run beside it; H6b showed it fails on sparse and dense, and H6c +must reproduce that or say why not. + +### Q3 — construction from a declaration + +Each arm is also built from a declaration instead of a cell sweep: a list of +(context range, operation range) rectangles for row, column, rectangle, +complementary, disjoint and doughnut, and a Morton prefix for prefix. +- P: one group per distinct operation mask, built from the ranges. +- D: rows filled word by word from the ranges. +- I: H6's `box_set` decomposition, or the prefix interval. + +**Decision.** Rerun H6b's structured rule with every arm built from the +declaration: P or H at most 0.5× D's total at N = 100. Report build cost per +arm and the N at which P's total falls below D's. + +**Fairness rule.** D gets the same declaration as P. A rule passed only +because D is swept while P is declared does not count. + +### Tests and disable runs (before any timing) + +- The declared build of every arm equals its swept build (P: equal group list; D + and I: equal words or intervals). Disable: an off-by-one in the declared + range end must fail it. +- H' switches both ways on the Q1 family (P at g = 1, D at g = 64). Disable: + ignoring G must fail it. +- Every Q1 fixture really has g groups (P's group count equals g) and is neither + empty nor full. Disable: assigning every context the first mask must fail it. + diff --git a/crates/lance-graph-mask-risc/Cargo.toml b/crates/lance-graph-mask-risc/Cargo.toml index 2da125c00..703921e15 100644 --- a/crates/lance-graph-mask-risc/Cargo.toml +++ b/crates/lance-graph-mask-risc/Cargo.toml @@ -39,3 +39,7 @@ test = true [[example]] name = "holo_interval_probe" test = true + +[[example]] +name = "holo_tile_forms_probe" +test = true diff --git a/crates/lance-graph-mask-risc/examples/holo_tile_forms_probe.rs b/crates/lance-graph-mask-risc/examples/holo_tile_forms_probe.rs new file mode 100644 index 000000000..c71abd9e4 --- /dev/null +++ b/crates/lance-graph-mask-risc/examples/holo_tile_forms_probe.rs @@ -0,0 +1,1154 @@ +//! D-HOLO H6b: symbolic normal forms on the Context × Operation tile. +//! +//! Pre-registered in `.claude/plans/d-holo-address-indexed-algebra-v1.md` +//! (H6b, with the connectedness amendment). The 256 × 256 tile is read as +//! context (row `c`) × operation (column `o`). Five representations of a set +//! of (c, o) pairs are compared: +//! +//! - P, canonical product form: contexts grouped by identical operation mask, +//! a list of (context mask, operation mask), 2 × 256 bits each. +//! - I, normalized Morton interval unions (as in H6), x = o, y = c. +//! - D, the dense 8 KiB mask, row-major `c * 256 + o`, operated on with +//! `ndarray::simd::{mask_and, mask_or, mask_not, mask_any}`. +//! - H, hybrid: P while it is smaller than D, else D. +//! - O, the row oracle: the pattern's membership function on every cell. +//! +//! Every arm builds from the same membership function by sweeping the 65,536 +//! cells, so construction is the generic case, not a symbolic shortcut. +//! Normalization is part of each build and of each operation: no arm has a +//! separate step that could be skipped. +//! +//! `cargo run --release -p lance-graph-mask-risc --example holo_tile_forms_probe` +//! `cargo test -p lance-graph-mask-risc --example holo_tile_forms_probe` + +use std::hint::black_box; +use std::time::Instant; + +use ndarray::simd::{mask_and, mask_any, mask_not, mask_or}; + +const SIDE: usize = 256; +const CELLS: usize = SIDE * SIDE; +const N: u32 = CELLS as u32; +const DENSE_WORDS: usize = CELLS / 64; +const DENSE_BYTES: usize = DENSE_WORDS * 8; +/// Bytes of one product-form group: a context mask and an operation mask. +const GROUP_BYTES: usize = 64; + +fn tier() -> &'static str { + if cfg!(target_feature = "avx512f") { + "v4 (avx512f)" + } else if cfg!(target_feature = "avx2") { + "v3 (avx2): timings not reportable" + } else { + "below v3: timings not reportable" + } +} + +// ---------------------------------------------------------------- 256-bit masks + +type M256 = [u64; 4]; +const EMPTY: M256 = [0; 4]; + +fn bit(m: &M256, i: usize) -> bool { + m[i / 64] >> (i % 64) & 1 == 1 +} +fn set(m: &mut M256, i: usize) { + m[i / 64] |= 1 << (i % 64); +} +fn and(a: &M256, b: &M256) -> M256 { + [a[0] & b[0], a[1] & b[1], a[2] & b[2], a[3] & b[3]] +} +fn or(a: &M256, b: &M256) -> M256 { + [a[0] | b[0], a[1] | b[1], a[2] | b[2], a[3] | b[3]] +} +fn not(a: &M256) -> M256 { + [!a[0], !a[1], !a[2], !a[3]] +} +fn is_zero(a: &M256) -> bool { + a.iter().all(|&w| w == 0) +} + +// ---------------------------------------------------------------- P: product form + +/// Canonical product form: groups with non-zero, pairwise disjoint context +/// masks and non-zero, distinct operation masks, sorted by operation mask. +#[derive(Clone, PartialEq, Eq, Debug)] +struct Prod { + groups: Vec<(M256, M256)>, +} + +impl Prod { + /// Canonicalize cells with disjoint context masks: drop empty ones, merge + /// equal operation masks, sort. + fn normalize(mut cells: Vec<(M256, M256)>) -> Prod { + cells.retain(|(c, o)| !is_zero(c) && !is_zero(o)); + cells.sort_unstable_by_key(|a| a.1); + let mut groups: Vec<(M256, M256)> = Vec::with_capacity(cells.len()); + for (c, o) in cells { + match groups.last_mut() { + Some(last) if last.1 == o => last.0 = or(&last.0, &c), + _ => groups.push((c, o)), + } + } + Prod { groups } + } + + fn build(f: &dyn Fn(usize, usize) -> bool) -> Prod { + let mut cells = Vec::with_capacity(SIDE); + for c in 0..SIDE { + let mut ops = EMPTY; + for o in 0..SIDE { + if f(c, o) { + set(&mut ops, o); + } + } + let mut ctx = EMPTY; + set(&mut ctx, c); + cells.push((ctx, ops)); + } + Prod::normalize(cells) + } + + /// The groups plus the implicit group of contexts with no operations. + fn with_rest(&self) -> Vec<(M256, M256)> { + let mut covered = EMPTY; + for (c, _) in &self.groups { + covered = or(&covered, c); + } + let mut v = self.groups.clone(); + let rest = not(&covered); + if !is_zero(&rest) { + v.push((rest, EMPTY)); + } + v + } + + /// `f` over the common refinement of both context partitions. + fn binop(&self, b: &Prod, f: fn(&M256, &M256) -> M256) -> Prod { + let (ga, gb) = (self.with_rest(), b.with_rest()); + let mut cells = Vec::with_capacity(ga.len() + gb.len()); + for (ca, oa) in &ga { + for (cb, ob) in &gb { + let c = and(ca, cb); + if !is_zero(&c) { + cells.push((c, f(oa, ob))); + } + } + } + Prod::normalize(cells) + } + + fn intersect(&self, b: &Prod) -> Prod { + self.binop(b, and) + } + fn union(&self, b: &Prod) -> Prod { + self.binop(b, or) + } + fn complement(&self) -> Prod { + let cells = self + .with_rest() + .into_iter() + .map(|(c, o)| (c, not(&o))) + .collect(); + Prod::normalize(cells) + } + fn is_empty(&self) -> bool { + self.groups.is_empty() + } + fn contains(&self, c: usize, o: usize) -> bool { + self.groups.iter().any(|(cm, om)| bit(cm, c) && bit(om, o)) + } + fn bytes(&self) -> usize { + self.groups.len() * GROUP_BYTES + } + fn canonical(&self) -> bool { + let g = &self.groups; + let disjoint = g + .iter() + .enumerate() + .all(|(i, a)| g[i + 1..].iter().all(|b| is_zero(&and(&a.0, &b.0)))); + disjoint + && g.iter().all(|(c, o)| !is_zero(c) && !is_zero(o)) + && g.windows(2).all(|w| w[0].1 < w[1].1) + } + fn to_dense(&self) -> Dense { + let mut d = Dense::zero(); + for (cm, om) in &self.groups { + for c in 0..SIDE { + if bit(cm, c) { + d.w[c * 4..c * 4 + 4].copy_from_slice(om); + } + } + } + d + } +} + +// ---------------------------------------------------------------- I: Morton intervals + +fn spread(v: u32) -> u32 { + let mut x = v & 0xFF; + x = (x | (x << 4)) & 0x0F0F; + x = (x | (x << 2)) & 0x3333; + (x | (x << 1)) & 0x5555 +} +fn gather(v: u32) -> u32 { + let mut x = v & 0x5555; + x = (x | (x >> 1)) & 0x3333; + x = (x | (x >> 2)) & 0x0F0F; + (x | (x >> 4)) & 0x00FF +} +/// Morton code with x = operation, y = context. +fn morton(c: usize, o: usize) -> u32 { + spread(o as u32) | (spread(c as u32) << 1) +} +fn unmorton(code: u32) -> (usize, usize) { + (gather(code >> 1) as usize, gather(code) as usize) +} + +/// A sorted set of disjoint, non-adjacent half-open Morton intervals. +#[derive(Clone, PartialEq, Eq, Debug)] +struct Iv { + s: Vec<(u32, u32)>, +} + +fn push(out: &mut Vec<(u32, u32)>, lo: u32, hi: u32) { + if lo >= hi { + return; + } + if let Some(last) = out.last_mut() { + if last.1 >= lo { + last.1 = last.1.max(hi); + return; + } + } + out.push((lo, hi)); +} + +impl Iv { + fn build(f: &dyn Fn(usize, usize) -> bool) -> Iv { + let mut s = Vec::new(); + for code in 0..N { + let (c, o) = unmorton(code); + if f(c, o) { + push(&mut s, code, code + 1); + } + } + Iv { s } + } + fn union(&self, b: &Iv) -> Iv { + let (a, b) = (&self.s, &b.s); + let mut out = Vec::with_capacity(a.len() + b.len()); + let (mut i, mut j) = (0, 0); + while i < a.len() || j < b.len() { + let take_a = j >= b.len() || (i < a.len() && a[i].0 <= b[j].0); + let (lo, hi) = if take_a { a[i] } else { b[j] }; + if take_a { + i += 1; + } else { + j += 1; + } + push(&mut out, lo, hi); + } + Iv { s: out } + } + fn intersect(&self, b: &Iv) -> Iv { + let (a, b) = (&self.s, &b.s); + let mut out = Vec::new(); + let (mut i, mut j) = (0, 0); + while i < a.len() && j < b.len() { + let lo = a[i].0.max(b[j].0); + let hi = a[i].1.min(b[j].1); + push(&mut out, lo, hi); + if a[i].1 < b[j].1 { + i += 1; + } else { + j += 1; + } + } + Iv { s: out } + } + fn complement(&self) -> Iv { + let mut out = Vec::with_capacity(self.s.len() + 1); + let mut at = 0; + for &(lo, hi) in &self.s { + push(&mut out, at, lo); + at = hi; + } + push(&mut out, at, N); + Iv { s: out } + } + fn is_empty(&self) -> bool { + self.s.is_empty() + } + fn contains(&self, c: usize, o: usize) -> bool { + let code = morton(c, o); + let k = self.s.partition_point(|&(_, hi)| hi <= code); + k < self.s.len() && self.s[k].0 <= code + } + fn bytes(&self) -> usize { + self.s.len() * 8 + } + fn canonical(&self) -> bool { + self.s.iter().all(|&(lo, hi)| lo < hi && hi <= N) + && self.s.windows(2).all(|w| w[0].1 < w[1].0) + } +} + +// ---------------------------------------------------------------- D: dense mask + +#[derive(Clone, PartialEq, Eq, Debug)] +struct Dense { + w: Box<[u64]>, +} + +impl Dense { + fn zero() -> Dense { + Dense { + w: vec![0u64; DENSE_WORDS].into_boxed_slice(), + } + } + fn build(f: &dyn Fn(usize, usize) -> bool) -> Dense { + let mut d = Dense::zero(); + for c in 0..SIDE { + for o in 0..SIDE { + if f(c, o) { + let i = c * SIDE + o; + d.w[i / 64] |= 1 << (i % 64); + } + } + } + d + } + fn intersect(&self, b: &Dense) -> Dense { + let mut d = Dense::zero(); + mask_and(&self.w, &b.w, &mut d.w); + d + } + fn union(&self, b: &Dense) -> Dense { + let mut d = Dense::zero(); + mask_or(&self.w, &b.w, &mut d.w); + d + } + fn complement(&self) -> Dense { + let mut d = Dense::zero(); + mask_not(&self.w, CELLS, &mut d.w); + d + } + fn is_empty(&self) -> bool { + !mask_any(&self.w) + } + fn contains(&self, c: usize, o: usize) -> bool { + let i = c * SIDE + o; + self.w[i / 64] >> (i % 64) & 1 == 1 + } + fn bytes(&self) -> usize { + DENSE_BYTES + } +} + +// ---------------------------------------------------------------- H: hybrid + +#[derive(Clone, Debug)] +enum Hyb { + P(Prod), + D(Dense), +} + +impl Hyb { + fn settle(p: Prod) -> Hyb { + if p.bytes() < DENSE_BYTES { + Hyb::P(p) + } else { + Hyb::D(p.to_dense()) + } + } + fn build(f: &dyn Fn(usize, usize) -> bool) -> Hyb { + Hyb::settle(Prod::build(f)) + } + fn dense(&self) -> Dense { + match self { + Hyb::P(p) => p.to_dense(), + Hyb::D(d) => d.clone(), + } + } + fn binop(&self, b: &Hyb, pf: fn(&Prod, &Prod) -> Prod, df: fn(&Dense, &Dense) -> Dense) -> Hyb { + match (self, b) { + (Hyb::P(x), Hyb::P(y)) => Hyb::settle(pf(x, y)), + (Hyb::D(x), Hyb::D(y)) => Hyb::D(df(x, y)), + _ => Hyb::D(df(&self.dense(), &b.dense())), + } + } + fn intersect(&self, b: &Hyb) -> Hyb { + self.binop(b, Prod::intersect, Dense::intersect) + } + fn union(&self, b: &Hyb) -> Hyb { + self.binop(b, Prod::union, Dense::union) + } + fn complement(&self) -> Hyb { + match self { + Hyb::P(p) => Hyb::settle(p.complement()), + Hyb::D(d) => Hyb::D(d.complement()), + } + } + fn is_empty(&self) -> bool { + match self { + Hyb::P(p) => p.is_empty(), + Hyb::D(d) => d.is_empty(), + } + } + fn equal(&self, b: &Hyb) -> bool { + match (self, b) { + (Hyb::P(x), Hyb::P(y)) => x == y, + _ => self.dense() == b.dense(), + } + } + fn contains(&self, c: usize, o: usize) -> bool { + match self { + Hyb::P(p) => p.contains(c, o), + Hyb::D(d) => d.contains(c, o), + } + } + fn bytes(&self) -> usize { + match self { + Hyb::P(p) => p.bytes(), + Hyb::D(d) => d.bytes(), + } + } + fn is_dense(&self) -> bool { + matches!(self, Hyb::D(_)) + } +} + +// ---------------------------------------------------------------- the query + +/// The per-query formula: X = (A ∩ ¬B) ∪ (B ∩ ¬A), then emptiness of X and +/// equality of X with A. Exercises ∩, ∪ and ¬, plus both tests. +macro_rules! query { + ($a:expr, $b:expr, $eq:expr) => {{ + let (a, b) = ($a, $b); + let x = a + .intersect(&b.complement()) + .union(&b.intersect(&a.complement())); + let e = x.is_empty(); + let q = $eq(&x, a); + (x, e, q) + }}; +} + +// ---------------------------------------------------------------- patterns + +fn mix(mut z: u64) -> u64 { + z = z.wrapping_add(0x9E37_79B9_7F4A_7C15); + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) +} +fn coin(seed: u64, c: usize, o: usize, per_10k: u64) -> bool { + mix(seed ^ ((c * SIDE + o) as u64)) % 10_000 < per_10k +} +fn in_rect(c: usize, o: usize, c0: usize, c1: usize, o0: usize, o1: usize) -> bool { + (c0..c1).contains(&c) && (o0..o1).contains(&o) +} + +type Cell = fn(usize, usize) -> bool; + +struct Pattern { + name: &'static str, + structured: bool, + adversarial: bool, + a: Cell, + b: Cell, +} + +fn patterns() -> Vec { + let p = |name, structured, adversarial, a: Cell, b: Cell| Pattern { + name, + structured, + adversarial, + a, + b, + }; + vec![ + p( + "row", + true, + false, + |c, o| c == 37 && (10..200).contains(&o), + |c, o| c == 37 && (100..250).contains(&o), + ), + p( + "column", + true, + false, + |c, o| o == 91 && (5..230).contains(&c), + |c, o| o == 91 && (40..256).contains(&c), + ), + p( + "rectangle", + true, + false, + |c, o| in_rect(c, o, 20, 120, 30, 170), + |c, o| in_rect(c, o, 60, 200, 100, 240), + ), + p( + "prefix", + true, + false, + |c, o| morton(c, o) >> 12 == 3, + |c, o| morton(c, o) >> 14 == 0, + ), + p("diagonal", false, false, |c, o| c == o, |c, o| c == o + 1), + p( + "checkerboard", + false, + true, + |c, o| (c + o) % 2 == 0, + |c, o| (c ^ o) & 2 == 0, + ), + p( + "random 1%", + false, + false, + |c, o| coin(1, c, o, 100), + |c, o| coin(2, c, o, 100), + ), + p( + "random 50%", + false, + true, + |c, o| coin(3, c, o, 5_000), + |c, o| coin(4, c, o, 5_000), + ), + p( + "sparse", + false, + false, + |c, o| coin(5, c, o, 10), + |c, o| coin(6, c, o, 10), + ), + p( + "dense", + false, + false, + |c, o| !coin(7, c, o, 10), + |c, o| !coin(8, c, o, 10), + ), + p( + "complementary", + true, + false, + |c, o| in_rect(c, o, 20, 120, 30, 170), + |c, o| !in_rect(c, o, 20, 120, 30, 170), + ), + p( + "disjoint", + true, + false, + |c, o| in_rect(c, o, 0, 100, 0, 100), + |c, o| in_rect(c, o, 150, 256, 120, 256), + ), + p( + "fragmented", + false, + true, + |c, o| o < c, + |c, o| o < c * 3 / 4, + ), + p( + "doughnut", + false, + false, + |c, o| in_rect(c, o, 40, 200, 40, 200) && !in_rect(c, o, 90, 150, 90, 150), + |c, o| in_rect(c, o, 100, 240, 0, 256), + ), + ] +} + +/// 4-connected components of a pattern on the 256 × 256 grid. +fn components(f: Cell) -> usize { + let mut seen = vec![false; CELLS]; + let mut stack = Vec::new(); + let mut n = 0; + for start in 0..CELLS { + if seen[start] || !f(start / SIDE, start % SIDE) { + continue; + } + n += 1; + seen[start] = true; + stack.push(start); + while let Some(i) = stack.pop() { + let (c, o) = (i / SIDE, i % SIDE); + let mut try_cell = |c2: usize, o2: usize| { + let j = c2 * SIDE + o2; + if !seen[j] && f(c2, o2) { + seen[j] = true; + stack.push(j); + } + }; + if c > 0 { + try_cell(c - 1, o); + } + if c + 1 < SIDE { + try_cell(c + 1, o); + } + if o > 0 { + try_cell(c, o - 1); + } + if o + 1 < SIDE { + try_cell(c, o + 1); + } + } + } + n +} + +/// The oracle answer for the query: X's membership, emptiness, equality. +fn oracle(p: &Pattern) -> (Vec, bool, bool) { + let x: Vec = (0..CELLS) + .map(|i| { + let (c, o) = (i / SIDE, i % SIDE); + let (a, b) = ((p.a)(c, o), (p.b)(c, o)); + a != b + }) + .collect(); + let empty = x.iter().all(|&v| !v); + let equal = (0..CELLS).all(|i| x[i] == (p.a)(i / SIDE, i % SIDE)); + (x, empty, equal) +} + +// ---------------------------------------------------------------- timing + +fn time(mut f: F) -> f64 { + f(); + let mut v: Vec = (0..11) + .map(|_| { + let t = Instant::now(); + let mut n = 0u32; + while n < 3 || t.elapsed().as_micros() < 2_000 { + f(); + n += 1; + } + t.elapsed().as_nanos() as f64 / f64::from(n) + }) + .collect(); + v.sort_by(|a, b| a.total_cmp(b)); + v[5] +} + +/// Spearman rank correlation (average ranks for ties). +fn spearman(x: &[f64], y: &[f64]) -> f64 { + fn ranks(v: &[f64]) -> Vec { + let mut idx: Vec = (0..v.len()).collect(); + idx.sort_by(|&a, &b| v[a].total_cmp(&v[b])); + let mut r = vec![0.0; v.len()]; + let mut i = 0; + while i < idx.len() { + let mut j = i; + while j + 1 < idx.len() && v[idx[j + 1]] == v[idx[i]] { + j += 1; + } + let avg = (i + j) as f64 / 2.0; + for &k in &idx[i..=j] { + r[k] = avg; + } + i = j + 1; + } + r + } + let (rx, ry) = (ranks(x), ranks(y)); + let n = x.len() as f64; + let (mx, my) = (rx.iter().sum::() / n, ry.iter().sum::() / n); + let cov: f64 = rx.iter().zip(&ry).map(|(a, b)| (a - mx) * (b - my)).sum(); + let vx: f64 = rx.iter().map(|a| (a - mx).powi(2)).sum(); + let vy: f64 = ry.iter().map(|b| (b - my).powi(2)).sum(); + cov / (vx * vy).sqrt() +} + +const ARMS: [&str; 4] = ["P", "I", "D", "H"]; +const NS: [f64; 4] = [1.0, 10.0, 100.0, 1000.0]; + +struct Row { + name: &'static str, + structured: bool, + adversarial: bool, + density: f64, + comps: usize, + intervals: usize, + groups: usize, + hyb_dense: bool, + /// Per arm: build (A + B), query, ∩, ∪, ¬, empty, equal, bytes of A. + build: [f64; 4], + query: [f64; 4], + ops: [[f64; 5]; 4], + bytes: [usize; 4], +} + +fn measure(p: &Pattern) -> Row { + let fa: &dyn Fn(usize, usize) -> bool = &p.a; + let fb: &dyn Fn(usize, usize) -> bool = &p.b; + let (pa, pb) = (Prod::build(fa), Prod::build(fb)); + let (ia, ib) = (Iv::build(fa), Iv::build(fb)); + let (da, db) = (Dense::build(fa), Dense::build(fb)); + let (ha, hb) = (Hyb::build(fa), Hyb::build(fb)); + + let build = [ + time(|| { + black_box((Prod::build(fa), Prod::build(fb))); + }), + time(|| { + black_box((Iv::build(fa), Iv::build(fb))); + }), + time(|| { + black_box((Dense::build(fa), Dense::build(fb))); + }), + time(|| { + black_box((Hyb::build(fa), Hyb::build(fb))); + }), + ]; + let query = [ + time(|| { + black_box(query!( + black_box(&pa), + black_box(&pb), + |x: &Prod, y: &Prod| x == y + )); + }), + time(|| { + black_box(query!(black_box(&ia), black_box(&ib), |x: &Iv, y: &Iv| x == y)); + }), + time(|| { + black_box(query!( + black_box(&da), + black_box(&db), + |x: &Dense, y: &Dense| x == y + )); + }), + time(|| { + black_box(query!( + black_box(&ha), + black_box(&hb), + |x: &Hyb, y: &Hyb| x.equal(y) + )); + }), + ]; + let ops = [ + [ + time(|| { + black_box(black_box(&pa).intersect(black_box(&pb))); + }), + time(|| { + black_box(black_box(&pa).union(black_box(&pb))); + }), + time(|| { + black_box(black_box(&pa).complement()); + }), + time(|| { + black_box(black_box(&pa).is_empty()); + }), + time(|| { + black_box(black_box(&pa) == black_box(&pb)); + }), + ], + [ + time(|| { + black_box(black_box(&ia).intersect(black_box(&ib))); + }), + time(|| { + black_box(black_box(&ia).union(black_box(&ib))); + }), + time(|| { + black_box(black_box(&ia).complement()); + }), + time(|| { + black_box(black_box(&ia).is_empty()); + }), + time(|| { + black_box(black_box(&ia) == black_box(&ib)); + }), + ], + [ + time(|| { + black_box(black_box(&da).intersect(black_box(&db))); + }), + time(|| { + black_box(black_box(&da).union(black_box(&db))); + }), + time(|| { + black_box(black_box(&da).complement()); + }), + time(|| { + black_box(black_box(&da).is_empty()); + }), + time(|| { + black_box(black_box(&da) == black_box(&db)); + }), + ], + [ + time(|| { + black_box(black_box(&ha).intersect(black_box(&hb))); + }), + time(|| { + black_box(black_box(&ha).union(black_box(&hb))); + }), + time(|| { + black_box(black_box(&ha).complement()); + }), + time(|| { + black_box(black_box(&ha).is_empty()); + }), + time(|| { + black_box(black_box(&ha).equal(black_box(&hb))); + }), + ], + ]; + let members = (0..CELLS).filter(|&i| (p.a)(i / SIDE, i % SIDE)).count(); + Row { + name: p.name, + structured: p.structured, + adversarial: p.adversarial, + density: members as f64 / CELLS as f64, + comps: components(p.a), + intervals: ia.s.len(), + groups: pa.groups.len(), + hyb_dense: ha.is_dense(), + build, + query, + ops, + bytes: [pa.bytes(), ia.bytes(), da.bytes(), ha.bytes()], + } +} + +fn total(r: &Row, arm: usize, n: f64) -> f64 { + r.build[arm] + n * r.query[arm] +} + +fn main() { + println!("D-HOLO H6b: normal forms on the 256 × 256 Context × Operation tile"); + println!("tier: {}", tier()); + println!("timings: ns, median of 11; every arm checked against the row oracle first\n"); + + let pats = patterns(); + for p in &pats { + let (x, e, q) = oracle(p); + let fa: &dyn Fn(usize, usize) -> bool = &p.a; + let fb: &dyn Fn(usize, usize) -> bool = &p.b; + let (pa, pb) = (Prod::build(fa), Prod::build(fb)); + let (ia, ib) = (Iv::build(fa), Iv::build(fb)); + let (da, db) = (Dense::build(fa), Dense::build(fb)); + let (ha, hb) = (Hyb::build(fa), Hyb::build(fb)); + let (px, pe, pq) = query!(&pa, &pb, |x: &Prod, y: &Prod| x == y); + let (ix, ie, iq) = query!(&ia, &ib, |x: &Iv, y: &Iv| x == y); + let (dx, de, dq) = query!(&da, &db, |x: &Dense, y: &Dense| x == y); + let (hx, he, hq) = query!(&ha, &hb, |x: &Hyb, y: &Hyb| x.equal(y)); + assert!( + px.canonical() && ix.canonical(), + "{}: not canonical", + p.name + ); + assert_eq!((pe, pq), (e, q), "{}: P", p.name); + assert_eq!((ie, iq), (e, q), "{}: I", p.name); + assert_eq!((de, dq), (e, q), "{}: D", p.name); + assert_eq!((he, hq), (e, q), "{}: H", p.name); + for (i, &want) in x.iter().enumerate() { + let (c, o) = (i / SIDE, i % SIDE); + assert_eq!(px.contains(c, o), want, "{}: P cell {i}", p.name); + assert_eq!(ix.contains(c, o), want, "{}: I cell {i}", p.name); + assert_eq!(dx.contains(c, o), want, "{}: D cell {i}", p.name); + assert_eq!(hx.contains(c, o), want, "{}: H cell {i}", p.name); + } + } + println!( + "oracle: all {} patterns match in all four arms\n", + pats.len() + ); + + let rows: Vec = pats.iter().map(measure).collect(); + + println!("pattern density comps intervals groups H-form | bytes P / I / D / H"); + for r in &rows { + println!( + "{:<13} {:>7.4} {:>5} {:>9} {:>6} {:>6} | {} / {} / {} / {}", + r.name, + r.density, + r.comps, + r.intervals, + r.groups, + if r.hyb_dense { "D" } else { "P" }, + r.bytes[0], + r.bytes[1], + r.bytes[2], + r.bytes[3] + ); + } + + println!("\nper-operation ns (A op B): ∩ / ∪ / ¬A / empty(A) / A==B"); + for r in &rows { + for (k, arm) in ARMS.iter().enumerate() { + let o = r.ops[k]; + println!( + "{:<13} {arm} {:>9.0} {:>9.0} {:>9.0} {:>7.1} {:>7.1}", + r.name, o[0], o[1], o[2], o[3], o[4] + ); + } + } + + println!("\nbuild (A + B) and query ns per arm: P / I / D / H"); + for r in &rows { + println!( + "{:<13} build {:>9.0} {:>9.0} {:>9.0} {:>9.0} | query {:>9.0} {:>9.0} {:>9.0} {:>9.0}", + r.name, + r.build[0], + r.build[1], + r.build[2], + r.build[3], + r.query[0], + r.query[1], + r.query[2], + r.query[3] + ); + } + + println!("\ntotal ns = build + N × query, ratio to D at each N: P / I / H"); + for r in &rows { + let mut line = format!("{:<13}", r.name); + for &n in &NS { + let d = total(r, 2, n); + line += &format!( + " | N={:<4} {:>5.2} {:>5.2} {:>5.2}", + n as u32, + total(r, 0, n) / d, + total(r, 1, n) / d, + total(r, 3, n) / d + ); + } + println!("{line}"); + } + + println!("\ndecision rules at N = 100 (ratio to D; < 2× either way is undecided)"); + for r in &rows { + let d = total(r, 2, 100.0); + let (p, i, h) = ( + total(r, 0, 100.0) / d, + total(r, 1, 100.0) / d, + total(r, 3, 100.0) / d, + ); + let rule = if r.structured { + format!( + "structured: P {} (≤ 0.5), H {}", + verdict_le(p, 0.5), + verdict_le(h, 0.5) + ) + } else if r.adversarial { + format!("adversarial: H {} (≤ 1.25)", verdict_le(h, 1.25)) + } else { + String::from("reported only") + }; + let best = [(p, "P"), (i, "I"), (1.0, "D"), (h, "H")] + .into_iter() + .min_by(|a, b| a.0.total_cmp(&b.0)) + .map_or("?", |t| t.1); + println!( + "{:<13} P {p:>5.2} I {i:>5.2} H {h:>5.2} cheapest {best:<2} {rule}", + r.name + ); + } + + let ratio: Vec = rows + .iter() + .map(|r| (total(r, 0, 100.0).min(total(r, 1, 100.0)) / total(r, 2, 100.0)).ln()) + .collect(); + let comps: Vec = rows.iter().map(|r| r.comps as f64).collect(); + let dens: Vec = rows + .iter() + .map(|r| r.density.min(1.0 - r.density)) + .collect(); + let ivs: Vec = rows.iter().map(|r| r.intervals as f64).collect(); + println!( + "\npredictors of ln(best symbolic / D) at N = 100, Spearman over {} patterns", + rows.len() + ); + println!( + " components {:+.2}", + spearman(&comps, &ratio) + ); + println!( + " density (min(d, 1 - d)) {:+.2}", + spearman(&dens, &ratio) + ); + println!( + " Morton intervals {:+.2}", + spearman(&ivs, &ratio) + ); + + // Not pre-registered: added after the first run showed that the generic + // 65,536-cell build dominates every total at N = 100. + let grp: Vec = rows.iter().map(|r| r.groups as f64).collect(); + let qratio: Vec = rows + .iter() + .map(|r| (r.query[0] / r.query[2]).ln()) + .collect(); + println!("\npost hoc (not pre-registered): query-only ratio to D, P / I / H"); + for r in &rows { + println!( + "{:<13} {:>6.2} {:>6.2} {:>6.2}", + r.name, + r.query[0] / r.query[2], + r.query[1] / r.query[2], + r.query[3] / r.query[2] + ); + } + println!( + " Spearman, P groups vs ln(P query / D query) {:+.2}", + spearman(&grp, &qratio) + ); + println!( + " Spearman, components vs ln(P query / D query) {:+.2}", + spearman(&comps, &qratio) + ); + println!( + " Spearman, density vs ln(P query / D query) {:+.2}", + spearman(&dens, &qratio) + ); +} + +fn verdict_le(x: f64, bound: f64) -> &'static str { + if x <= bound { + "PASS" + } else if x < 2.0 && x > 0.5 { + "undecided" + } else { + "FAIL" + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn oracle_set(f: Cell) -> Vec { + (0..CELLS).map(|i| f(i / SIDE, i % SIDE)).collect() + } + + /// FAILS IF: any arm's ∩, ∪, ¬ or query result differs from the row + /// oracle on any cell of any pattern, or a P / I result is not canonical. + #[test] + fn every_arm_matches_the_oracle_on_every_pattern() { + for p in patterns() { + let fa: &dyn Fn(usize, usize) -> bool = &p.a; + let fb: &dyn Fn(usize, usize) -> bool = &p.b; + let (pa, pb) = (Prod::build(fa), Prod::build(fb)); + let (ia, ib) = (Iv::build(fa), Iv::build(fb)); + let (da, db) = (Dense::build(fa), Dense::build(fb)); + let (ha, hb) = (Hyb::build(fa), Hyb::build(fb)); + let (pi, pu, pn) = (pa.intersect(&pb), pa.union(&pb), pa.complement()); + let (ii, iu, inn) = (ia.intersect(&ib), ia.union(&ib), ia.complement()); + let (di, du, dn) = (da.intersect(&db), da.union(&db), da.complement()); + let (hi, hu, hn) = (ha.intersect(&hb), ha.union(&hb), ha.complement()); + for x in [&pi, &pu, &pn] { + assert!(x.canonical(), "{}: P not canonical", p.name); + } + for x in [&ii, &iu, &inn] { + assert!(x.canonical(), "{}: I not canonical", p.name); + } + for i in 0..CELLS { + let (c, o) = (i / SIDE, i % SIDE); + let (a, b) = ((p.a)(c, o), (p.b)(c, o)); + let want = [a && b, a || b, !a]; + assert_eq!( + [pi.contains(c, o), pu.contains(c, o), pn.contains(c, o)], + want, + "{} P", + p.name + ); + assert_eq!( + [ii.contains(c, o), iu.contains(c, o), inn.contains(c, o)], + want, + "{} I", + p.name + ); + assert_eq!( + [di.contains(c, o), du.contains(c, o), dn.contains(c, o)], + want, + "{} D", + p.name + ); + assert_eq!( + [hi.contains(c, o), hu.contains(c, o), hn.contains(c, o)], + want, + "{} H", + p.name + ); + } + let (x, e, q) = oracle(&p); + let (px, pe, pq) = query!(&pa, &pb, |x: &Prod, y: &Prod| x == y); + let (hx, he, hq) = query!(&ha, &hb, |x: &Hyb, y: &Hyb| x.equal(y)); + assert_eq!((pe, pq, he, hq), (e, q, e, q), "{}: query flags", p.name); + for (i, &want) in x.iter().enumerate() { + assert_eq!(px.contains(i / SIDE, i % SIDE), want); + assert_eq!(hx.contains(i / SIDE, i % SIDE), want); + } + } + } + + /// FAILS IF: P is not canonical, i.e. the same set reached by two routes + /// gives two different group lists, so equality would lie. A ∩ ⊤ and ¬¬A + /// must equal A exactly. + #[test] + fn product_form_is_canonical_across_routes() { + let full = Prod::build(&|_, _| true); + for p in patterns() { + let a = Prod::build(&p.a); + assert_eq!(a.intersect(&full), a, "{}: A ∩ ⊤", p.name); + assert_eq!(a.complement().complement(), a, "{}: ¬¬A", p.name); + let via_union = a + .intersect(&Prod::build(&p.b)) + .union(&a.intersect(&Prod::build(&p.b).complement())); + assert_eq!(via_union, a, "{}: (A∩B) ∪ (A∩¬B)", p.name); + } + } + + /// FAILS IF: the fixture is degenerate: some pattern is empty or full, no + /// query is empty, every query is empty, or the components column carries + /// no spread (so the connectedness predictor could not discriminate). + #[test] + fn fixture_discriminates() { + let pats = patterns(); + let mut empties = 0; + let mut comps = Vec::new(); + for p in &pats { + let a = oracle_set(p.a).iter().filter(|&&v| v).count(); + assert!(a > 0 && a < CELLS, "{}: A is empty or full", p.name); + if oracle(p).1 { + empties += 1; + } + comps.push(components(p.a)); + } + assert!(empties < pats.len(), "every query is empty"); + assert!(comps.iter().min() == Some(&1) && *comps.iter().max().unwrap() > 1_000); + // The doughnut is one component with a hole: connectedness alone does + // not make it an interval set. + let d = pats.iter().find(|p| p.name == "doughnut").unwrap(); + assert_eq!(components(d.a), 1); + } + + /// FAILS IF: the hybrid never switches, or switches everywhere: it must + /// stay P on a rectangle and fall back to D on a random 50% pattern. + #[test] + fn hybrid_switches_both_ways() { + let pats = patterns(); + let get = |n: &str| pats.iter().find(|p| p.name == n).unwrap(); + assert!(!Hyb::build(&get("rectangle").a).is_dense()); + assert!(Hyb::build(&get("random 50%").a).is_dense()); + } + + /// FAILS IF: the Morton decoding is not the inverse of the encoding. + #[test] + fn morton_round_trips() { + for c in 0..SIDE { + for o in 0..SIDE { + assert_eq!(unmorton(morton(c, o)), (c, o)); + } + } + } +}