diff --git a/.claude/board/entries/2026-10-12-class-method-polyfill-ledger.md b/.claude/board/entries/2026-10-12-class-method-polyfill-ledger.md new file mode 100644 index 000000000..123991dc3 --- /dev/null +++ b/.claude/board/entries/2026-10-12-class-method-polyfill-ledger.md @@ -0,0 +1,143 @@ +# 2026-10-12 — Class-qualified methods compile into Quack algebra (D-CMH-1) + +**Status:** MEASURED for the slice below; everything past it is OPEN. +Code: `crates/lance-graph-glove-parity/src/{method,call}.rs`, +`crates/lance-graph-r2il/src/procs.rs`. Tests: `tests/method.rs`, +`tests/polyfill.rs` (glove-parity), `tests/procs.rs` (r2il). Plan context: +OGAR #357 (RFC, unmerged), lance-graph #1501 (plan, unmerged). + +## What is executable + +```text +SELECT ::() FROM
+ → resolve_call: concept via ogar_vocab::canonical_concept_id, + method label → MethodKey, payload = FROM table = bound table + → MethodTable (prepared once per glove): SchemaSeal, ReadingId, Universe, + WideFieldMask required ⊆ available (Exact|Converted claims) + → MethodBody::Query { filter: [Cond], folds: [Fold], law: Law } (data) + → Quack Query per fold → lower → mask-risc (frontend 1) + → R2IL procs::select_u32 + count|sum|group_sum (frontend 2, parity) + → Law finisher → Answer::Values | Answer::Truths +``` + +`src/` names no method. Every method in the tests is declared there as data: +`count_billable`, `count_not_billable`, `billable_hours` (two folds), +`count_by_duration` (refused), `submit_for_approval` (action, never run), +`revision` and `raw_sums` (NARS fixture concept). + +Rust supplies two reusable layers, each written once: the **algebra** +(conditions, `Count`, `Sum`, `GroupSum`, lowered by both frontends) and the +**laws** (certified finishers: `Values`, `NarsRevision`). A method composes +them and cannot supply its own. + +## Measured + +- `billable_hours` via the call text = the same query written by hand in + Quack = R2IL procs called by hand = a row loop, at n ∈ {1, 64, 65, 4097}. +- `revision` on both frontends = a float oracle (|Δ| < 1e-3) and OpenNARS's + three reference revisions to two places; a statement with no evidence + answers `None`. +- Refusals, each with a disable run that turns its test red: concept, method, + payload, table, extra SQL; schema seal (two swapped positions, every name + kept); reading; population; nullable field; basis of 257; evidence not + selected as independent; law shape; negated condition and summed lane on + each frontend; pure call of an action. + +## Capability / insufficiency ledger + +| Method / need | Missing algebra | Counterexample | Proposed extension | Oracle | +|---|---|---|---|---| +| NARS revision: exclude a repeated `(statement, stamp)` | A first-occurrence mask over an ordered composite key (`key[i] != key[i-1]`). Today the payload's producer computes it. | `a_repeated_stamp_inflates_the_raw_sums`: Σw 19 500 instead of 10 500, c 0.951 instead of 0.91. | Domain-neutral `FirstOfRun(keys…)` predicate over a lane ordered by the key, refused at run when out of order (same precondition `CountKeyRunsU32` already enforces). Reusable for any dedup (IAM, SAP, Odoo). | Row loop with a seen-set (`oracle` in polyfill.rs). | +| Grouped folds: a key outside `0..groups` | The terminal drops it silently. | `evidence_outside_the_declared_domain_is_dropped_silently` (pinned as a gap). | A dropped-row count beside the sink, or a refusal flag; executor-level, not per method. | Count of rows with key ≥ groups. | +| NULL / UNKNOWN | Conditions are two-valued; nullable fields are refused. | `a_nullable_field_is_refused`. | Bind through Quack's `sql_where` (three-valued `WHERE` already exists there) and declare a null policy per law. | SQL three-valued truth table. | +| `sql(query)` frontend | No SQL path reaches Quack (lance-graph's SQL path is DataFusion, in its grace period). `resolve_call` recognises one form. | Any other SQL shape → `Unrecognised`. | A real frontend hands its parsed call to the same `MethodCall`; nothing else changes. | — | +| NARS concept | Not minted in OGAR's codebook. | `SELECT nars::revision(..)` → `UnknownConcept`. | Mint through OGAR; the fixture `0xFFF0` then goes. | — | +| Method identity | OGAR has `MethodDecl` (harvested ORM methods, no key) and #357's `LogicMethodSpec` (RFC only). `MethodKey` is a fixture. | — | Key = `ogar_loco::Inventory::key_of` once methods are minted there; this table becomes the prepared sidecar, not a second inventory. | — | +| Reading and population ids | No ClassView reading id and no plane generation id exist. Both are fixtures here. | — | Issue from the ClassView and the plane owner. | — | +| Compute bodies (fields derived from fields) | Not attempted. The compute-DAG check is fail-open past 64 positions (`10→70→10` reads acyclic). | — | Fix the DAG check to 256 before any compute body lands. | — | +| Fused folds | One mask-risc program per fold; the selection is re-emitted per fold. | — | A multi-sink terminal; an optimisation only, no semantic change. | Same answers. | + +No new mask-risc opcode or R2IL op was added. One R2IL change: `POP_COUNT` +now counts a resident plane in place (mask-risc's `Count` already accepted +one). + +## Metrics so far + +- Methods declared: 7. Executable with existing algebra: 5 (one needs the + producer's mark, see row 1). Refused by design: 2. +- New reusable algebra members: grouped `SUM` exposed to methods (the + terminal existed). New laws: 1 (`NarsRevision`). New ISA: 0. Domain Rust + adapters: 0. + +## Extension criteria + +A new algebra member lands only with (1) a method that needs it, (2) a +counterexample showing no composition of existing members gives the answer, +(3) an independent oracle, (4) both frontends or a stated reason one cannot, +and (5) a second domain that can use it. A new law lands only with an +independent oracle and the body shape it requires checked at preparation. + +## Experiments A–E (same day) + +Machine: `-Ctarget-cpu=x86-64-v3` (the repo default), AVX-512 host, shared +and noisy; hot paths interleaved per rep, medians and minima reported. +Command: `cargo run --release --example method_cost -- 301`. + +**A. The seam.** `SELECT billable_work_entry::billable_hours(cats) FROM cats` += the same query written in Quack = R2IL procs = a row loop, at n ∈ {1, 64, +65, 4097}. The prepared method now holds its Quack programs, lowered once; +the tests assert `programs() == [lower(direct query)]`, so a call executes +the same program a direct query does and no other evaluator exists. There +is no SQL frontend into Quack: "direct Quack" is the query built through +Quack's binder, not parsed SQL. + +**B. Frozen executor.** At e984885, `hours_by_billable` (hours grouped by the +billable code) was declared in `tests/method.rs` only; that commit's stat is +one file. Quack: correct. R2IL: a typed refusal (`WrongOperandKind`), because +its SUM, GROUP_SUM and KEEP took a mask only from a computed slot and a body +with no condition hands them the live plane. Closed in the next commit by one +shared mask resolution in R2IL (domain-neutral; no new op), after which both +frontends agree. So the method needed no Rust; the executor had a gap that +the first condition-free method exposed. + +**C. NARS.** Independent positive and conflicting evidence: matches the float +oracle and OpenNARS's three reference points. A repeated stamp, a replayed +delta, and the same stamp in a second hypothesis layer leave the truth +unchanged, given the producer's two marks (replay by `delta_id`, independence +by `(statement, stamp)`). Retraction: not declared; a negative-weight delta is +exact for an asserted stamp and silently wrong for a phantom one (pinned). +The oracle is the NAL weight formula in floating point plus the OpenNARS +points from `lance-graph-planner/tests/nal_revision_oracle.rs`; no external +NARS binary was run. + +**D. Faults.** Each is a typed refusal before execution, or a red test: +swapped lanes (both frontends disagree with the oracle); schema seal; reading; +population; nullable field read as a value (the refusal test goes red when the +guard is removed); duplicate stamps (refused body / counterexample); law shape; +basis of 257. A compute body has no `MethodBody` variant, so it cannot be +written, rather than being refused at the 64-position DAG limit. + +**E. Cost** (1,048,576 rows; 65,536 in brackets): + +| path | median | min | ns/row (min) | allocs/call | +|---|---|---|---|---| +| cold: prepare one method | 0.8 µs | | | 13 (1.1 KB) | +| cold: resolve call text | 0.2 µs | | | 1 | +| cold: lower one query | 0.2 µs | | | 4 | +| warm method call, Quack | 600 µs (56) | 447 µs (33) | 0.43 (0.51) | 2 (2.1 KB) | +| warm method call, R2IL | 605 µs (56) | 448 µs (35) | 0.43 (0.53) | 5 (16.8 KB) | +| direct Quack, lowered once | 599 µs (54) | 442 µs (37) | 0.42 (0.56) | 0 | +| scalar row loop | 366 µs (23) | 300 µs (10) | 0.29 (0.15) | 0 | + +The interface costs nothing measurable over direct Quack once prepared: the +remaining per-call cost is a scratch arena and the answer vector. Both are +1.5–3× slower than a fused scalar loop, because the program writes a mask +and then sums it (two passes over 8 MB of lanes); that is the executor's +cost, the same for a hand-written Quack query. Fusion of filter into fold is +the lever, not the method layer. Not measured: decoded bytes (lanes are read +resident), and fused-vs-unfused lowering (no fused terminal exists). + +**Metrics after the experiments.** Methods declared: 8; executable on both +frontends: 6; refused by design: 2. New algebra members: 0 since the freeze; +executor fixes: 1 (R2IL plane masks, found by B). New laws: 1. New ISA: 0. +Domain Rust: 0. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index da37d6cf1..028a3912a 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -314 entries, 2026-08-06 .. 2026-10-12. +315 entries, 2026-08-06 .. 2026-10-12. | date | entry id | finding | file | |---|---|---|---| @@ -33,6 +33,7 @@ exactly one of them, never both. | 2026-10-12 | `nars-self-evidence-literature` | NARS internal trace versus evidence, Jøsang dependent/independent fusion and fission, and the snapshot-stable fold assumption; source-grounded red tests for merged #1496 | [2026-10-12-nars-self-evidence-literature.md](2026-10-12-nars-self-evidence-literature.md) | | 2026-10-12 | `hyperon-metta-lg-translation` | Hyperon/MeTTa to LG sibling map: atom/hypergraph roundtrip, binding and rewrite parity, typed folds, scoped effects and 15 open questions | [2026-10-12-hyperon-metta-lg-translation.md](2026-10-12-hyperon-metta-lg-translation.md) | | 2026-10-12 | `D-CMH-0` | Class × Schema × Methods activation connects HotPlug, WideFieldMask, Quack and R2IL without a second registry | [2026-10-12-d-cmh-0-class-method-hotplug-activation.md](2026-10-12-d-cmh-0-class-method-hotplug-activation.md) | +| 2026-10-12 | `D-CMH-1` | | [2026-10-12-class-method-polyfill-ledger.md](2026-10-12-class-method-polyfill-ledger.md) | | 2026-10-12 | `D-HL-0` | CE64 hypothesis layers: plateau of the design (opaque LE bytes, 64 sparse-delta layers merged on read, Narsese-parallel validation), 8 design-vs-code conflicts, 9 unification hypotheses | [2026-10-12-ce64-hypothesis-layers-plateau.md](2026-10-12-ce64-hypothesis-layers-plateau.md) | | 2026-10-11 | `D-TELEPORT-0` | | [2026-10-11-teleport-probe-source-restriction.md](2026-10-11-teleport-probe-source-restriction.md) | | 2026-10-11 | `synergy-ledger-classid-hotplug` | | [2026-10-11-synergy-ledger-classid-hotplug.md](2026-10-11-synergy-ledger-classid-hotplug.md) | diff --git a/.github/workflows/glove-parity.yml b/.github/workflows/glove-parity.yml index 33bf786b3..450988465 100644 --- a/.github/workflows/glove-parity.yml +++ b/.github/workflows/glove-parity.yml @@ -7,6 +7,7 @@ on: - 'crates/lance-graph-ontology/**' - 'crates/lance-graph-quack/**' - 'crates/lance-graph-mask-risc/**' + - 'crates/lance-graph-r2il/**' - 'crates/lance-graph-contract/**' - '.github/workflows/glove-parity.yml' workflow_dispatch: @@ -26,11 +27,21 @@ jobs: ref: master path: ndarray persist-credentials: false + # lance-graph-r2il (method calls through R2IL) path-deps ogar-loco and + # ogar-r2il from the OGAR sibling. + - uses: actions/checkout@v4 + with: + repository: AdaWorldAPI/OGAR + ref: main + path: OGAR + persist-credentials: false - uses: dtolnay/rust-toolchain@stable with: components: rustfmt, clippy - name: Record tested substrate - run: git -C ndarray rev-parse HEAD + run: | + git -C ndarray rev-parse HEAD + git -C OGAR rev-parse HEAD - name: Format run: cargo +stable fmt --manifest-path lance-graph/crates/lance-graph-glove-parity/Cargo.toml -- --check - name: Lint diff --git a/crates/lance-graph-glove-parity/Cargo.toml b/crates/lance-graph-glove-parity/Cargo.toml index 183a61cc4..4d78dcac3 100644 --- a/crates/lance-graph-glove-parity/Cargo.toml +++ b/crates/lance-graph-glove-parity/Cargo.toml @@ -17,9 +17,12 @@ lance-graph-quack = { path = "../lance-graph-quack" } # The capability authority (`OgarAuthority`) and OGAR's promoted classes and # `ClassView`, for the hot-plugs and the canonical field basis (`basis`). lance-graph-ogar = { path = "../lance-graph-ogar" } +# Executes a method through both frontends: Quack (above) and R2IL. R2IL +# path-deps the OGAR sibling (`ogar-loco`, `ogar-r2il`), so CI checks it out. +lance-graph-mask-risc = { path = "../lance-graph-mask-risc" } +lance-graph-r2il = { path = "../lance-graph-r2il" } [dev-dependencies] -lance-graph-mask-risc = { path = "../lance-graph-mask-risc" } lance-graph-sap = { path = "../lance-graph-sap" } # mask-risc names ndarray by its canonical git coordinate; point it at the diff --git a/crates/lance-graph-glove-parity/examples/method_cost.rs b/crates/lance-graph-glove-parity/examples/method_cost.rs new file mode 100644 index 000000000..347b97c8f --- /dev/null +++ b/crates/lance-graph-glove-parity/examples/method_cost.rs @@ -0,0 +1,290 @@ +//! Experiment E: what the declarative method interface costs. +//! +//! Measures, on one SAP CATS batch, separately: +//! +//! 1. cold: `MethodTable::prepare` (binding) and `resolve_call` (text → key); +//! 2. warm: `MethodTable::call`, Quack and R2IL frontends; +//! 3. direct Quack: the same query, lowered once, executed; +//! 4. scalar reference: a row loop over the same lanes. +//! +//! Every timed path is asserted to give the same answer before it is timed. +//! Allocations are counted by a wrapping global allocator. +//! +//! Run: `cargo run --release --example method_cost -- [rows] [reps]` +use std::alloc::{GlobalAlloc, Layout, System}; +use std::hint::black_box; +use std::sync::atomic::{AtomicU64, Ordering::Relaxed}; +use std::time::Instant; + +use lance_graph_glove_parity::basis::{canonical_fields, FieldMap, Grade, SAP_FIELDS}; +use lance_graph_glove_parity::call::resolve_call; +use lance_graph_glove_parity::method::{ + Answer, Cond, EffectClass, Fold, FoldValue, Frontend, Glove, Law, MethodBody, MethodKey, + MethodSpec, MethodTable, ReadingId, SchemaSeal, Universe, +}; +use lance_graph_mask_risc::{execute_into, Foreign, LaneRef, Out, Planes, Scratch, Value}; +use lance_graph_ogar::ogar_vocab::class_ids::BILLABLE_WORK_ENTRY; +use lance_graph_quack::bind::{table, Binder}; +use lance_graph_quack::{lower, Agg, Query}; +use lance_graph_sap::bind::{CatsBatch, BILLABLE, BILLING, HOURS}; +use lance_graph_sap::binder::{live_words, CatsBinder, TABLE}; +use lance_graph_sap::schema::{CatsSchema, FIELD_COUNT}; + +struct Counting; +static ALLOCS: AtomicU64 = AtomicU64::new(0); +static BYTES: AtomicU64 = AtomicU64::new(0); + +// SAFETY: forwards every call to `System` unchanged; only counts. +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + ALLOCS.fetch_add(1, Relaxed); + BYTES.fetch_add(l.size() as u64, Relaxed); + // SAFETY: same contract as the caller's. + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: same contract as the caller's. + unsafe { System.dealloc(p, l) } + } +} + +#[global_allocator] +static A: Counting = Counting; + +const INDICATORS: [&str; 4] = ["Billable", "Non-Billable", "Internal Cost", "Unknown"]; +const HOURS_TEXT: [&str; 3] = ["8.50", "0.125", "2"]; + +fn batch(n: usize) -> CatsBatch { + let base: Vec> = include_str!("../../lance-graph-sap/fixtures/cats.txt") + .lines() + .map(|v| if v == "\\N" { None } else { Some(v) }) + .collect(); + let mut input: [Vec>; FIELD_COUNT] = std::array::from_fn(|i| vec![base[i]; n]); + for i in 0..n { + input[BILLING][i] = Some(INDICATORS[(i * 7 + i / 3) % 4]); + input[HOURS][i] = Some(HOURS_TEXT[i % 3]); + } + CatsBatch::bind( + CatsSchema::new(42, 0), + std::array::from_fn(|i| input[i].as_slice()), + ) + .unwrap() +} + +/// Median of `reps` timings of `f`, in nanoseconds, plus allocations per call. +fn time(reps: usize, mut f: impl FnMut() -> T) -> (u128, f64, f64) { + let mut v = Vec::with_capacity(reps); + let (a0, b0) = (ALLOCS.load(Relaxed), BYTES.load(Relaxed)); + for _ in 0..reps { + let t = Instant::now(); + black_box(f()); + v.push(t.elapsed().as_nanos()); + } + let (a1, b1) = (ALLOCS.load(Relaxed), BYTES.load(Relaxed)); + v.sort_unstable(); + ( + v[reps / 2], + (a1 - a0) as f64 / reps as f64, + (b1 - b0) as f64 / reps as f64, + ) +} + +fn main() { + let mut args = std::env::args().skip(1); + let n: usize = args.next().map_or(1 << 20, |s| s.parse().unwrap()); + let reps: usize = args.next().map_or(51, |s| s.parse().unwrap()); + + let batch = batch(n); + let cats = CatsBinder::new(&batch); + let basis = canonical_fields(); + let claims: Vec = SAP_FIELDS + .iter() + .map(|m| FieldMap { + grade: if m.canonical == "duration" { + Grade::Exact + } else { + m.grade + }, + ..*m + }) + .collect(); + let spec = MethodSpec { + key: MethodKey([0xB2; 16]), + concept: BILLABLE_WORK_ENTRY, + schema: SchemaSeal::of(&basis), + reading: ReadingId(1), + effect: EffectClass::Pure, + body: MethodBody::Query { + filter: &[Cond::Eq { + field: "billable", + literal: "true", + }], + folds: &[Fold::Sum("duration")], + law: Law::Values, + }, + }; + let glove = Glove { + concept: BILLABLE_WORK_ENTRY, + basis: &basis, + claims: &claims, + binder: &cats, + table: TABLE, + reading: ReadingId(1), + universe: Universe(42), + }; + let specs = [("billable_hours", spec)]; + let sql = "SELECT billable_work_entry::billable_hours(cats) FROM cats"; + + let live = live_words(n); + let lanes = batch.lanes(); + let planes = Planes { + n_rows: n, + masks: &[live.as_slice()], + lanes: &lanes, + }; + + // The answer every path must give. + let (LaneRef::U32(bill), LaneRef::I32(hours)) = (lanes[BILLABLE], lanes[HOURS]) else { + panic!("lane kinds") + }; + let scalar = || -> i64 { + bill.iter() + .zip(hours) + .filter(|(b, _)| **b == 1) + .map(|(_, h)| i64::from(*h)) + .sum() + }; + let want = scalar(); + + let t = MethodTable::prepare(&glove, &specs); + let key = resolve_call(sql, &t).unwrap().method; + let expect = Answer::Values(vec![FoldValue::Scalar(want)]); + for f in [Frontend::Quack, Frontend::R2il] { + assert_eq!(t.call(key, &planes, Universe(42), f).unwrap(), expect); + } + + let tid = cats.table(TABLE).unwrap(); + let hours_col = cats.field(tid, "hours_logged").unwrap().col; + let direct = Query { + filter: table(TABLE) + .where_eq("billable", "true") + .count() + .bind(&cats) + .unwrap() + .filter, + agg: Agg::SumI32(hours_col), + }; + let program = lower(&direct).unwrap(); + // The method compiles to the very program a direct query does. + let prepared = t.get(key).unwrap().as_ref().unwrap(); + let method_q = prepared.queries(); + // A call executes the program lowered at preparation, and it is the + // very program the direct query lowers to. + assert_eq!( + prepared.programs(), + std::slice::from_ref(&program), + "same lowered program" + ); + let mut scratch = Scratch::for_program(&program, n).unwrap(); + let mut run_direct = || match execute_into( + &program, + &planes, + &Foreign::NONE, + &mut scratch, + Out::None, + ) + .unwrap() + { + Value::SumI64(s) => s, + v => panic!("{v:?}"), + }; + assert_eq!(run_direct(), want); + + println!( + "tier: avx512f={} avx2={} (compiled target features, not the host)", + cfg!(target_feature = "avx512f"), + cfg!(target_feature = "avx2") + ); + println!("rows {n}, reps {reps} (median), answer {want}"); + println!( + "lowered program: {} mask ops + terminal, {} scratch slots", + program.ops.len(), + program.scratch_slots + ); + let row = |name: &str, (ns, a, b): (u128, f64, f64)| { + println!( + "{name:<34} {:>12.3} µs {:>8.2} ns/row {:>7.1} allocs {:>11.0} B alloc", + ns as f64 / 1e3, + ns as f64 / n as f64, + a, + b + ); + }; + row( + "cold: prepare (bind 1 method)", + time(reps, || MethodTable::prepare(&glove, &specs)), + ); + row( + "cold: resolve_call (text → key)", + time(reps, || resolve_call(sql, &t).unwrap()), + ); + row( + "cold: lower (bound query → program)", + time(reps, || lower(&method_q[0]).unwrap()), + ); + // Hot paths interleaved: each rep runs all four in a rotating order, + // so machine drift lands on every path alike. Median and minimum. + let mut samples: [Vec; 4] = Default::default(); + for r in 0..reps { + for k in 0..4 { + let path = (r + k) % 4; + let t0 = Instant::now(); + match path { + 0 => { + black_box(t.call(key, &planes, Universe(42), Frontend::Quack).unwrap()); + } + 1 => { + black_box(t.call(key, &planes, Universe(42), Frontend::R2il).unwrap()); + } + 2 => { + black_box(run_direct()); + } + _ => { + black_box(scalar()); + } + } + samples[path].push(t0.elapsed().as_nanos()); + } + } + let names = [ + "warm: method call, Quack", + "warm: method call, R2IL", + "direct Quack (lowered once)", + "scalar reference (row loop)", + ]; + for (name, v) in names.iter().zip(samples.iter_mut()) { + v.sort_unstable(); + println!( + "{name:<34} median {:>8.1} µs min {:>8.1} µs {:>6.3} ns/row (min)", + v[v.len() / 2] as f64 / 1e3, + v[0] as f64 / 1e3, + v[0] as f64 / n as f64 + ); + } + let (_, a, b) = time(reps, || { + t.call(key, &planes, Universe(42), Frontend::Quack).unwrap() + }); + println!("allocs per warm Quack call: {a:.1} ({b:.0} B)"); + let (_, a, b) = time(reps, || { + t.call(key, &planes, Universe(42), Frontend::R2il).unwrap() + }); + println!("allocs per warm R2IL call: {a:.1} ({b:.0} B)"); + let (_, a, b) = time(reps, &mut run_direct); + println!("allocs per direct call: {a:.1} ({b:.0} B)"); + println!( + "lane bytes read per call: billable {} + hours {} + live {}", + n * 4, + n * 4, + live.len() * 8 + ); +} diff --git a/crates/lance-graph-glove-parity/src/call.rs b/crates/lance-graph-glove-parity/src/call.rs new file mode 100644 index 000000000..cae5665e5 --- /dev/null +++ b/crates/lance-graph-glove-parity/src/call.rs @@ -0,0 +1,94 @@ +//! The class-qualified call, as a caller's `sql(query)` carries it. +//! +//! No SQL frontend reaches Quack today: `lance-graph`'s SQL path is +//! DataFusion, which is in its grace period and gains nothing new. This +//! module recognises exactly ONE form and refuses everything else: +//! +//! ```text +//! SELECT ::(
) FROM
+//! ``` +//! +//! It is the seam a real SQL frontend hands its parsed call to, not a SQL +//! parser. The text is resolved once, cold, into a typed [`MethodCall`]: +//! +//! - `` through OGAR's codebook (`canonical_concept_id`), never a +//! local name table; +//! - `` through the [`MethodTable`]'s labels, into a [`MethodKey`]; +//! - the payload names the row the method reads, which must be the `FROM` +//! table, which must be the table the methods were bound against. +//! +//! Nothing is substituted back into text: the call carries numbers only. +use lance_graph_ogar::ogar_vocab::canonical_concept_id; + +use crate::method::{MethodKey, MethodTable}; + +/// A resolved class-qualified call. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MethodCall { + /// Canonical concept id. + pub concept: u16, + pub method: MethodKey, +} + +/// Why a call text did not resolve. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum CallParseError { + /// Not of the one recognised form. + Unrecognised, + /// OGAR's codebook has no concept of that name. + UnknownConcept(String), + /// The concept is not the one whose methods the table holds. + WrongConcept { called: u16, table: u16 }, + /// The concept has no method of that name. + UnknownMethod(String), + /// The payload is not the `FROM` table. + PayloadMismatch, + /// The `FROM` table is not the one the methods were bound against. + WrongTable(String), +} + +/// Resolve `SELECT ::(
) FROM
` against `t`. +/// +/// # Errors +/// +/// [`CallParseError`] naming the first part that did not resolve. +pub fn resolve_call(sql: &str, t: &MethodTable) -> Result { + let words: Vec<&str> = sql.split_whitespace().collect(); + let [select, call, from, table] = words.as_slice() else { + return Err(CallParseError::Unrecognised); + }; + if !select.eq_ignore_ascii_case("select") || !from.eq_ignore_ascii_case("from") { + return Err(CallParseError::Unrecognised); + } + let (concept, rest) = call.split_once("::").ok_or(CallParseError::Unrecognised)?; + let (method, payload) = rest + .strip_suffix(')') + .and_then(|r| r.split_once('(')) + .ok_or(CallParseError::Unrecognised)?; + let ident = + |s: &str| !s.is_empty() && s.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'_'); + if ![concept, method, payload, table].into_iter().all(ident) { + return Err(CallParseError::Unrecognised); + } + let id = canonical_concept_id(concept) + .ok_or_else(|| CallParseError::UnknownConcept(concept.to_owned()))?; + if id != t.concept() { + return Err(CallParseError::WrongConcept { + called: id, + table: t.concept(), + }); + } + if payload != *table { + return Err(CallParseError::PayloadMismatch); + } + if *table != t.table() { + return Err(CallParseError::WrongTable((*table).to_owned())); + } + let method = t + .resolve(method) + .ok_or_else(|| CallParseError::UnknownMethod(method.to_owned()))?; + Ok(MethodCall { + concept: id, + method, + }) +} diff --git a/crates/lance-graph-glove-parity/src/lib.rs b/crates/lance-graph-glove-parity/src/lib.rs index 89d414303..b36141217 100644 --- a/crates/lance-graph-glove-parity/src/lib.rs +++ b/crates/lance-graph-glove-parity/src/lib.rs @@ -12,6 +12,8 @@ //! glove's lanes. This is a FIELD MAPPING (bind time, fields → lanes), not a //! report `pivot()` (plan §C.6.1). pub mod basis; +pub mod call; +pub mod method; use lance_graph_ontology::namespace::SchemaKind; use lance_graph_ontology::OntologyRegistry; diff --git a/crates/lance-graph-glove-parity/src/method.rs b/crates/lance-graph-glove-parity/src/method.rs new file mode 100644 index 000000000..4611e8c6c --- /dev/null +++ b/crates/lance-graph-glove-parity/src/method.rs @@ -0,0 +1,889 @@ +//! Methods of a concept, bound once against one glove, compiled into Quack +//! algebra (D-CMH-1; `.claude/plans/class-method-schema-hotplug-v1.md`). +//! +//! A method is DATA, not a Rust function. A [`MethodSpec`] belongs to the +//! concept and declares its body as a query in the concept's canonical field +//! names (the authority's `ClassView`): a conjunction of conditions, a list of +//! folds, and a [`Law`] that turns the folds' values into the result. Adding a +//! method adds a spec; it adds no Rust. +//! +//! [`MethodTable::prepare`] binds every spec once, cold, against one glove's +//! claims and `Binder`, and records what it refused. After that a call carries +//! numbers only, and one generic lowering serves every method: +//! +//! ```text +//! label ──resolve (cold)──► MethodKey +//! MethodKey + borrowed planes ──call──► Quack Query per fold ─► lower ─► mask-risc +//! └─► R2IL calls per fold ─► FoldDialect ─► mask-risc +//! └─► Law ─► result +//! ``` +//! +//! The chain this serves: a caller writes `sql(query)`; a class-qualified +//! method inside it (`Class::method(payload)`) expands into Quack algebra; Quack +//! compiles it. R2IL is the second frontend onto the same positions and checks +//! parity. A label is never an address: [`MethodTable::resolve`] turns it into +//! a [`MethodKey`], and only the key reaches a call. +//! +//! Two masks, two algebras: +//! +//! - [`PreparedMethod::required`] is a [`WideFieldMask`] over the canonical +//! basis: WHICH FIELDS the method reads. A method prepares only when +//! `required ⊆ available`, where `available` is the basis positions this +//! glove has a proven (`Exact` or `Converted`) claim for. +//! - The row mask the method runs on lives in mask-risc and is never stored +//! here. +//! +//! An action body prepares, so the table can name it, and is refused by +//! every pure call ([`CallError::NotPure`]). +//! +//! A spec is bound only to the glove it was written for: its +//! [`SchemaSeal`] (the canonical basis, in order) and its [`ReadingId`] must +//! equal the glove's, and a call must name the [`Universe`] (source +//! generation and population) the method was prepared against. +//! +//! Scope: conditions are `=`/`!=` on non-nullable `Code` fields; folds are +//! `COUNT`, `SUM` and grouped `SUM` over non-nullable `I32` fields; laws are +//! [`Law::Values`] and [`Law::NarsRevision`]. Everything else is refused by +//! name. +//! +//! A [`Law`] is the one place a domain's semantics are written in Rust: it is +//! a certified finisher, checked against an independent oracle, and every +//! method of every concept that names it reuses it. A method never supplies +//! its own. +use lance_graph_contract::WideFieldMask; +use lance_graph_mask_risc::{ + execute_into, ExecError, Foreign, Out, Planes, Program, Scratch, Value, +}; +use lance_graph_quack::bind::{Binder, BoundField, FieldKind, TableId}; +use lance_graph_quack::{lower, Agg, Cmp, Col, Filter, LowerError, Mask, Query}; +use lance_graph_r2il::{procs, FoldDialect, FoldError, Val}; + +use crate::basis::{FieldMap, Grade}; + +/// A method's stable identity. Issued by the method's owner; never derived +/// from its label. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct MethodKey(pub [u8; 16]); + +/// The canonical basis a spec was written against: its field names, in +/// `ClassView` order. Reordering, renaming, adding or removing a field +/// changes the seal, so a spec written for one basis never binds another. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct SchemaSeal(pub u64); + +impl SchemaSeal { + /// FNV-1a over the names, each terminated by `0xFF` (not a UTF-8 byte), + /// so `["ab","c"]` and `["a","bc"]` differ. + #[must_use] + pub fn of(basis: &[String]) -> Self { + let mut h: u64 = 0xcbf2_9ce4_8422_2325; + for name in basis { + for b in name.bytes().chain(std::iter::once(0xFF)) { + h ^= u64::from(b); + h = h.wrapping_mul(0x0000_0100_0000_01b3); + } + } + Self(h) + } +} + +/// Which interpretation of the glove's bytes a spec assumes. Issued with the +/// reading (the ClassView's); a different reading of the same lanes is a +/// different id. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct ReadingId(pub u16); + +/// The population a method was prepared against: source generation and row +/// universe, issued by whoever owns the planes. Rows from another universe +/// are not rows of this method's table, whatever their shape. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct Universe(pub u64); + +/// What a method may do (OGAR #357 §8). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum EffectClass { + /// Reads borrowed planes, returns a value. + Pure, + /// Reads a pinned dataset version. + ReadVersion, + /// Builds an inert proposal. + Propose, + /// Changes the world. Only OGAR's action gate runs it. + ExternalMutate, +} + +/// One condition of a body's `WHERE`, on a canonical field. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Cond { + /// `field = literal`. + Eq { + field: &'static str, + literal: &'static str, + }, + /// `field != literal`. + Ne { + field: &'static str, + literal: &'static str, + }, +} + +impl Cond { + fn field(self) -> &'static str { + match self { + Cond::Eq { field, .. } | Cond::Ne { field, .. } => field, + } + } +} + +/// One aggregate of a body. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Fold { + /// `COUNT(*)` over the selected rows. + Count, + /// `SUM(field)` over the selected rows. + Sum(&'static str), + /// `SUM(val) GROUP BY key` over the selected rows, for the key domain + /// `0..groups`. Rows whose key is at or past `groups` are dropped by the + /// terminal, not refused. + GroupSum { + key: &'static str, + val: &'static str, + groups: u32, + }, +} + +/// How the folds' values become the method's result. A fixed set the +/// compiler knows; a method names one, it never supplies one. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Law { + /// The fold values, in order. + Values, + /// NAL revision of independent evidence, per statement. + /// + /// The body's folds must be exactly `[GroupSum { key, val: w }, + /// GroupSum { key, val: wf }]` over the same key and domain, where each + /// row carries one piece of evidence as a fixed-point evidence weight + /// `w = scale · c / (1 - c)` and `wf = round(w · f)`. Revision of + /// independent evidence adds weights, so for each statement + /// `f = Σwf / Σw` and `c = (Σw/scale) / (Σw/scale + horizon)`, computed + /// once from the sums (never by chaining pairwise revisions). + /// + /// Adding weights is only revision when the evidence is independent: a + /// repeated stamp would count twice. The body's filter must therefore + /// contain `independent = "true"`, the payload's own mark of the first + /// occurrence of each `(statement, stamp)`. The algebra cannot compute + /// that mark yet (see the capability ledger); a body without it is + /// refused. + NarsRevision { + /// The field marking the first occurrence of each evidential stamp. + independent: &'static str, + /// Fixed-point scale of `w`. + scale: u32, + /// The evidential horizon `k`. + horizon: u32, + }, +} + +/// A NAL truth value. +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct Truth { + pub f: f64, + pub c: f64, +} + +/// One fold's value. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum FoldValue { + Scalar(i64), + /// One value per group, `0..groups`. + Groups(Vec), +} + +/// A method's typed result. +#[derive(Debug, Clone, PartialEq)] +pub enum Answer { + /// [`Law::Values`]: the fold values, in order. + Values(Vec), + /// [`Law::NarsRevision`]: one truth per statement; `None` where no + /// independent evidence was selected. + Truths(Vec>), +} + +/// A method's body, in the concept's canonical field names. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum MethodBody { + /// `SELECT folds FROM concept WHERE filter`, finished by `law`. + Query { + filter: &'static [Cond], + folds: &'static [Fold], + law: Law, + }, + /// An OGAR action. Named here; never executed here. + Action { action: &'static str }, +} + +/// A method as its concept declares it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MethodSpec { + pub key: MethodKey, + /// Canonical concept id (`ogar_vocab::class_ids`). + pub concept: u16, + /// The basis the body's field names refer to. + pub schema: SchemaSeal, + /// The reading of the glove's bytes the body assumes. + pub reading: ReadingId, + pub effect: EffectClass, + pub body: MethodBody, +} + +impl MethodSpec { + /// The canonical fields the body reads, each once, in order of use. + #[must_use] + pub fn reads(&self) -> Vec<&'static str> { + let mut out = Vec::new(); + if let MethodBody::Query { filter, folds, .. } = self.body { + let fields = filter + .iter() + .map(|c| c.field()) + .chain(folds.iter().flat_map(|f| match *f { + Fold::Count => vec![], + Fold::Sum(field) => vec![field], + Fold::GroupSum { key, val, .. } => vec![key, val], + })); + for f in fields { + if !out.contains(&f) { + out.push(f); + } + } + } + out + } +} + +/// Why a method did not prepare. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PrepareError { + /// The spec belongs to another concept than the table's. + WrongConcept { spec: u16, table: u16 }, + /// The body names a field the canonical basis does not have. + UnknownField(&'static str), + /// The basis is wider than a field position can address. + BasisTooWide(usize), + /// A required field has no proven claim in this glove + /// (`required ⊄ available`). + UnmappedRequired(&'static str), + /// The glove's binder has no table of that name. + UnknownTable(&'static str), + /// The glove's binder does not serve the claimed field. + Unbound(&'static str), + /// The field is nullable. Three-valued comparison is not in scope. + Nullable(&'static str), + /// The body uses the field as a different kind. + KindMismatch(&'static str), + /// The literal is not in the field's domain. + UnknownValue { + field: &'static str, + literal: &'static str, + }, + /// A query body with no fold computes nothing. + NoFold, + /// The declared effect does not match the body. + EffectMismatch, + /// The spec was written for another canonical basis. + SchemaMismatch { spec: SchemaSeal, glove: SchemaSeal }, + /// The spec assumes another reading of the glove's bytes. + ReadingMismatch { spec: ReadingId, glove: ReadingId }, + /// The body's folds do not have the shape its law requires. + LawShape, + /// The law adds evidence, and the body does not select independent + /// evidence (`Law::NarsRevision::independent`). + EvidenceNotIndependent(&'static str), + /// Quack could not lower the bound body. + Lower(LowerError), + /// Another spec in the same table carries this key. + DuplicateKey(MethodKey), + /// Another spec in the same table carries this label. + DuplicateLabel(&'static str), + /// A law's numeric parameter is out of its domain (a zero + /// `NarsRevision` scale or horizon). + LawParameter, + /// Two proven claims map different native fields onto this canonical + /// field; binding either would depend on claim order. + AmbiguousClaim(&'static str), +} + +/// Why a call did not run. +#[derive(Debug)] +pub enum CallError { + /// No method with that key. + UnknownMethod, + /// The method was refused at preparation. + Refused(PrepareError), + /// The method is an action; a pure call never runs it. + NotPure, + /// The planes belong to another population than the method was + /// prepared against. + UniverseMismatch { + method: Universe, + planes: Universe, + }, + Lower(LowerError), + Exec(ExecError), + Fold(FoldError), +} + +/// Which frontend lowers the call. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Frontend { + Quack, + R2il, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum BoundFold { + Count, + Sum(Col), + GroupSum { key: Col, val: Col, groups: u32 }, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +enum Bound { + Query { + live: Mask, + /// `(lane, code, negate)`. + conds: Vec<(Col, u32, bool)>, + folds: Vec, + law: Law, + /// The Quack programs, lowered once at preparation, one per fold. + programs: Vec, + }, + Action, +} + +/// A method bound against one glove: numbers only. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PreparedMethod { + pub key: MethodKey, + pub effect: EffectClass, + /// The canonical fields the method reads, as basis positions. + pub required: WideFieldMask, + /// The population the method was bound against. + pub universe: Universe, + bound: Bound, +} + +impl PreparedMethod { + /// The bound Quack queries of a query body, one per fold (a mask-risc + /// program has one terminal); empty for an action. + #[must_use] + pub fn queries(&self) -> Vec { + let Bound::Query { + live, conds, folds, .. + } = &self.bound + else { + return Vec::new(); + }; + bound_queries(*live, conds, folds) + } + + /// The lowered Quack programs a call executes, one per fold. Lowered once, + /// when the method was prepared; a call never lowers. + #[must_use] + pub fn programs(&self) -> &[Program] { + match &self.bound { + Bound::Query { programs, .. } => programs, + Bound::Action => &[], + } + } +} + +fn bound_queries(live: Mask, conds: &[(Col, u32, bool)], folds: &[BoundFold]) -> Vec { + let filter = || { + Filter::and(std::iter::once(Filter::plane(live)).chain(conds.iter().map( + |&(col, code, negate)| { + Filter::cmp( + col, + if negate { + Cmp::NeU32(code) + } else { + Cmp::EqU32(code) + }, + ) + }, + ))) + }; + folds + .iter() + .map(|f| Query { + filter: filter(), + agg: match *f { + BoundFold::Count => Agg::Count, + BoundFold::Sum(col) => Agg::SumI32(col), + BoundFold::GroupSum { key, val, .. } => Agg::GroupSumI32 { key, val }, + }, + }) + .collect() +} + +impl PreparedMethod { + /// Run the method over `planes` through `frontend`. + /// + /// # Errors + /// + /// [`CallError::NotPure`] for an action, [`CallError::UniverseMismatch`] + /// for planes of another population, otherwise the frontend's own + /// refusal. + pub fn call( + &self, + planes: &Planes<'_>, + universe: Universe, + frontend: Frontend, + ) -> Result { + let Bound::Query { + live, + conds, + folds, + law, + programs, + } = &self.bound + else { + return Err(CallError::NotPure); + }; + if universe != self.universe { + return Err(CallError::UniverseMismatch { + method: self.universe, + planes: universe, + }); + } + let values = match frontend { + Frontend::Quack => programs + .iter() + .zip(folds) + .map(|(p, f)| run_quack(p, *f, planes)) + .collect::, _>>()?, + Frontend::R2il => { + let conds: Vec<(u16, u32, bool)> = + conds.iter().map(|&(c, v, n)| (c.0, v, n)).collect(); + folds + .iter() + .map(|f| run_r2il(*live, &conds, *f, planes)) + .collect::, _>>()? + } + }; + Ok(match *law { + Law::Values => Answer::Values(values), + Law::NarsRevision { scale, horizon, .. } => { + let [FoldValue::Groups(w), FoldValue::Groups(wf)] = values.as_slice() else { + unreachable!("preparation checked the law's shape") + }; + Answer::Truths(nars_revision(w, wf, scale, horizon)) + } + }) + } +} + +/// [`Law::NarsRevision`]'s finisher: per group, from the evidence sums. +fn nars_revision(w: &[i64], wf: &[i64], scale: u32, horizon: u32) -> Vec> { + w.iter() + .zip(wf) + .map(|(&w, &wf)| { + (w > 0).then(|| { + let weight = w as f64 / f64::from(scale); + Truth { + f: wf as f64 / w as f64, + c: weight / (weight + f64::from(horizon)), + } + }) + }) + .collect() +} + +fn run_quack( + program: &Program, + fold: BoundFold, + planes: &Planes<'_>, +) -> Result { + let mut scratch = Scratch::for_program(program, planes.n_rows).map_err(CallError::Exec)?; + if let BoundFold::GroupSum { groups, .. } = fold { + let mut sink = vec![0i64; groups as usize]; + execute_into( + program, + planes, + &Foreign::NONE, + &mut scratch, + Out::I64(&mut sink), + ) + .map_err(CallError::Exec)?; + return Ok(FoldValue::Groups(sink)); + } + match execute_into(program, planes, &Foreign::NONE, &mut scratch, Out::None) + .map_err(CallError::Exec)? + { + Value::Count(n) => Ok(FoldValue::Scalar( + i64::try_from(n).expect("a row count fits i64"), + )), + Value::SumI64(s) => Ok(FoldValue::Scalar(s)), + other => panic!("a Count or SumI32 terminal, got {other:?}"), + } +} + +/// The same selection is emitted once per fold: a fold ends the epoch of +/// every slot built before it. +fn run_r2il( + live: Mask, + conds: &[(u16, u32, bool)], + fold: BoundFold, + planes: &Planes<'_>, +) -> Result { + let mut stack = Vec::new(); + if let BoundFold::GroupSum { key, val, groups } = fold { + let mut sink = vec![0i64; groups as usize]; + let mut d = FoldDialect::with_groups(planes, &Foreign::NONE, &mut sink); + procs::select_u32(&mut d, &mut stack, live.0, conds).map_err(CallError::Fold)?; + procs::group_sum(&mut d, &mut stack, key.0, val.0).map_err(CallError::Fold)?; + assert!(stack.is_empty(), "GROUP_SUM pushes nothing, got {stack:?}"); + drop(d); + return Ok(FoldValue::Groups(sink)); + } + let mut d = FoldDialect::new(planes, &Foreign::NONE); + procs::select_u32(&mut d, &mut stack, live.0, conds).map_err(CallError::Fold)?; + match fold { + BoundFold::Count => procs::count(&mut d, &mut stack), + BoundFold::Sum(col) => procs::sum(&mut d, &mut stack, col.0), + BoundFold::GroupSum { .. } => unreachable!("handled above"), + } + .map_err(CallError::Fold)?; + match stack.as_slice() { + [Val::Scalar(n)] => Ok(FoldValue::Scalar(*n)), + other => panic!("a fold leaves one scalar, got {other:?}"), + } +} + +/// One concept's methods, bound against one glove. +#[derive(Debug)] +pub struct MethodTable { + concept: u16, + table: &'static str, + labels: Vec<(&'static str, MethodKey)>, + methods: Vec<(MethodKey, Result)>, +} + +impl MethodTable { + /// Bind every method of the glove's concept once. `specs` are + /// `(label, spec)`; the label is display text only. + /// + /// A refused method stays in the table with its reason. + #[must_use] + pub fn prepare(glove: &Glove<'_>, specs: &[(&'static str, MethodSpec)]) -> Self { + let available = available(glove.basis, glove.claims); + let seal = SchemaSeal::of(glove.basis); + let count = + |f: &dyn Fn(&(&'static str, MethodSpec)) -> bool| specs.iter().filter(|x| f(x)).count(); + Self { + concept: glove.concept, + table: glove.table, + // An ambiguous label names nothing. + labels: specs + .iter() + .filter(|(l, _)| count(&|(m, _)| m == l) == 1) + .map(|(l, s)| (*l, s.key)) + .collect(), + methods: specs + .iter() + .map(|(l, s)| { + let prepared = if count(&|(_, t)| t.key == s.key) > 1 { + Err(PrepareError::DuplicateKey(s.key)) + } else if count(&|(m, _)| m == l) > 1 { + Err(PrepareError::DuplicateLabel(l)) + } else { + glove.prepare(seal, available.as_ref(), s) + }; + (s.key, prepared) + }) + .collect(), + } + } + + /// The concept whose methods this table holds. + #[must_use] + pub fn concept(&self) -> u16 { + self.concept + } + + /// The glove table its methods were bound against. + #[must_use] + pub fn table(&self) -> &'static str { + self.table + } + + /// The key a label names. Cold: done once, never per call. + #[must_use] + pub fn resolve(&self, label: &str) -> Option { + self.labels + .iter() + .find(|(l, _)| *l == label) + .map(|(_, k)| *k) + } + + /// The prepared method, or why it was refused. + #[must_use] + pub fn get(&self, key: MethodKey) -> Option<&Result> { + self.methods.iter().find(|(k, _)| *k == key).map(|(_, m)| m) + } + + /// Call a method by key. + /// + /// # Errors + /// + /// [`CallError::UnknownMethod`], [`CallError::Refused`], or the + /// method's own call error. + pub fn call( + &self, + key: MethodKey, + planes: &Planes<'_>, + universe: Universe, + frontend: Frontend, + ) -> Result { + match self.get(key).ok_or(CallError::UnknownMethod)? { + Ok(m) => m.call(planes, universe, frontend), + Err(e) => Err(CallError::Refused(e.clone())), + } + } +} + +/// Basis positions with a proven claim (`Exact` or `Converted`). +fn available(basis: &[String], claims: &[FieldMap]) -> Result { + let mut mask = WideFieldMask::EMPTY; + for (i, name) in basis.iter().enumerate() { + let proven = claims + .iter() + .any(|c| c.canonical == name && matches!(c.grade, Grade::Exact | Grade::Converted)); + if proven { + mask = mask.with(u8::try_from(i).map_err(|_| PrepareError::BasisTooWide(basis.len()))?); + } + } + Ok(mask) +} + +/// One glove of one concept: everything a spec is bound against. +pub struct Glove<'a> { + /// Canonical concept id (`ogar_vocab::class_ids`). + pub concept: u16, + /// The concept's canonical fields in `ClassView` order. + pub basis: &'a [String], + /// This glove's native → canonical claims. + pub claims: &'a [FieldMap], + /// This glove's resolution membrane. + pub binder: &'a dyn Binder, + /// The glove's table for the concept. + pub table: &'static str, + /// The reading of the glove's bytes it serves. + pub reading: ReadingId, + /// The population its planes hold. + pub universe: Universe, +} + +impl Glove<'_> { + fn prepare( + &self, + seal: SchemaSeal, + available: Result<&WideFieldMask, &PrepareError>, + spec: &MethodSpec, + ) -> Result { + if spec.concept != self.concept { + return Err(PrepareError::WrongConcept { + spec: spec.concept, + table: self.concept, + }); + } + // The basis IS the field mask's universe: one wider than a position + // can address is refused whole, even if no claimed or read field + // sits past the edge. + if self.basis.len() > usize::from(u8::MAX) + 1 { + return Err(PrepareError::BasisTooWide(self.basis.len())); + } + if spec.schema != seal { + return Err(PrepareError::SchemaMismatch { + spec: spec.schema, + glove: seal, + }); + } + if spec.reading != self.reading { + return Err(PrepareError::ReadingMismatch { + spec: spec.reading, + glove: self.reading, + }); + } + let available = available.map_err(Clone::clone)?; + let reads = spec.reads(); + let mut required = WideFieldMask::EMPTY; + let mut missing = None; + for field in &reads { + let pos = self.position(field)?; + required = required.with(pos); + if missing.is_none() && !available.has(pos) { + missing = Some(*field); + } + } + if !required.is_subset_of(available) { + return Err(PrepareError::UnmappedRequired( + missing.expect("a field outside available exists"), + )); + } + let bound = match (spec.body, spec.effect) { + (MethodBody::Query { filter, folds, law }, EffectClass::Pure) => { + if folds.is_empty() { + return Err(PrepareError::NoFold); + } + check_law(law, filter, folds)?; + let t = self + .binder + .table(self.table) + .ok_or(PrepareError::UnknownTable(self.table))?; + let conds = filter + .iter() + .map(|c| self.cond(t, *c)) + .collect::, _>>()?; + let folds = folds + .iter() + .map(|f| match *f { + Fold::Count => Ok(BoundFold::Count), + Fold::Sum(field) => { + Ok(BoundFold::Sum(self.lane(t, field, FieldKind::I32)?)) + } + Fold::GroupSum { key, val, groups } => Ok(BoundFold::GroupSum { + key: self.lane(t, key, FieldKind::Code)?, + val: self.lane(t, val, FieldKind::I32)?, + groups, + }), + }) + .collect::, _>>()?; + let live = self.binder.live(t); + let programs = bound_queries(live, &conds, &folds) + .iter() + .map(lower) + .collect::, _>>() + .map_err(PrepareError::Lower)?; + Bound::Query { + live, + conds, + folds, + law, + programs, + } + } + (MethodBody::Action { .. }, EffectClass::Propose | EffectClass::ExternalMutate) => { + Bound::Action + } + _ => return Err(PrepareError::EffectMismatch), + }; + Ok(PreparedMethod { + key: spec.key, + effect: spec.effect, + required, + universe: self.universe, + bound, + }) + } + + /// A required field's lane, which must be of `kind`. + fn lane(&self, t: TableId, field: &'static str, kind: FieldKind) -> Result { + let b = self.field(t, field)?; + if b.kind != kind { + return Err(PrepareError::KindMismatch(field)); + } + Ok(b.col) + } + + fn position(&self, field: &'static str) -> Result { + let pos = self + .basis + .iter() + .position(|b| b == field) + .ok_or(PrepareError::UnknownField(field))?; + u8::try_from(pos).map_err(|_| PrepareError::BasisTooWide(self.basis.len())) + } + + /// A required field, through its proven claim. A converted claim is + /// served under its canonical name (the derived lens the conversion is + /// proven on); an exact one under the glove's own name. + fn field(&self, t: TableId, field: &'static str) -> Result { + let mut proven = self + .claims + .iter() + .filter(|c| c.canonical == field && matches!(c.grade, Grade::Exact | Grade::Converted)); + let claim = proven.next().expect("available implies a proven claim"); + if proven.any(|c| c.native != claim.native || c.grade != claim.grade) { + return Err(PrepareError::AmbiguousClaim(field)); + } + let name = match claim.grade { + Grade::Converted => claim.canonical, + _ => claim.native, + }; + let b = self + .binder + .field(t, name) + .ok_or(PrepareError::Unbound(field))?; + if b.validity.is_some() { + return Err(PrepareError::Nullable(field)); + } + Ok(b) + } + + fn cond(&self, t: TableId, c: Cond) -> Result<(Col, u32, bool), PrepareError> { + let (field, literal, negate) = match c { + Cond::Eq { field, literal } => (field, literal, false), + Cond::Ne { field, literal } => (field, literal, true), + }; + let b = self.field(t, field)?; + if b.kind != FieldKind::Code { + return Err(PrepareError::KindMismatch(field)); + } + let code = self + .binder + .code(t, b.col, literal) + .ok_or(PrepareError::UnknownValue { field, literal })?; + Ok((b.col, code, negate)) + } +} + +/// The body shape a law needs, checked before anything binds. +fn check_law(law: Law, filter: &[Cond], folds: &[Fold]) -> Result<(), PrepareError> { + match law { + Law::Values => Ok(()), + Law::NarsRevision { + independent, + scale, + horizon, + } => { + if scale == 0 || horizon == 0 { + return Err(PrepareError::LawParameter); + } + let [Fold::GroupSum { + key: k1, + val: w, + groups: g1, + }, Fold::GroupSum { + key: k2, + val: wf, + groups: g2, + }] = folds + else { + return Err(PrepareError::LawShape); + }; + if k1 != k2 || g1 != g2 || w == wf { + return Err(PrepareError::LawShape); + } + let selects_independent = filter.iter().any(|c| { + *c == Cond::Eq { + field: independent, + literal: "true", + } + }); + if !selects_independent { + return Err(PrepareError::EvidenceNotIndependent(independent)); + } + Ok(()) + } + } +} diff --git a/crates/lance-graph-glove-parity/tests/method.rs b/crates/lance-graph-glove-parity/tests/method.rs new file mode 100644 index 000000000..170c65af6 --- /dev/null +++ b/crates/lance-graph-glove-parity/tests/method.rs @@ -0,0 +1,515 @@ +//! D-CMH-1: methods of `BillableWorkEntry` (`0x0103`) bound once against the +//! SAP glove and compiled into Quack algebra, with R2IL as the parity frontend. +//! +//! Every method below is DATA declared in this file. `src/` knows no method +//! by name; adding one here adds no Rust (the falsifier for the claim that a +//! new method is a binding task, not a Rust task). +//! +//! The method keys and labels below are FIXTURES. Method identity is minted by +//! the method's owner (OGAR); nothing here mints one. +use lance_graph_glove_parity::basis::{canonical_fields, FieldMap, Grade, SAP_FIELDS}; +use lance_graph_glove_parity::method::{ + Answer, CallError, Cond, EffectClass, Fold, FoldValue, Frontend, Glove, Law, MethodBody, + MethodKey, MethodSpec, MethodTable, PrepareError, ReadingId, SchemaSeal, Universe, +}; +use lance_graph_mask_risc::{LaneRef, Planes}; +use lance_graph_ogar::ogar_vocab::class_ids::BILLABLE_WORK_ENTRY; +use lance_graph_quack::bind::{table, Binder, BoundField, TableId}; +use lance_graph_quack::{lower, Col, Mask}; +use lance_graph_sap::bind::{billable_code, CatsBatch, ACTIVITY, BILLABLE, BILLING, HOURS}; +use lance_graph_sap::binder::{live_words, CatsBinder, TABLE}; +use lance_graph_sap::schema::{CatsSchema, FIELD_COUNT}; + +const COUNT_BILLABLE: MethodKey = MethodKey([0xB1; 16]); +const COUNT_BY_DURATION: MethodKey = MethodKey([0xD0; 16]); +const SUBMIT: MethodKey = MethodKey([0xA0; 16]); +const NOT_BILLABLE: MethodKey = MethodKey([0xB0; 16]); +const BILLABLE_HOURS: MethodKey = MethodKey([0xB2; 16]); +const HOURS_BY_BILLABLE: MethodKey = MethodKey([0xB3; 16]); + +const fn query(filter: &'static [Cond], folds: &'static [Fold]) -> MethodBody { + MethodBody::Query { + filter, + folds, + law: Law::Values, + } +} + +/// FIXTURES: the reading id and population id a real glove is issued. +const READING: ReadingId = ReadingId(1); +const UNIVERSE: Universe = Universe(42); + +fn pure(key: MethodKey, body: MethodBody) -> MethodSpec { + MethodSpec { + key, + concept: BILLABLE_WORK_ENTRY, + schema: SchemaSeal::of(&canonical_fields()), + reading: READING, + effect: EffectClass::Pure, + body, + } +} + +fn scalars(v: &[i64]) -> Answer { + Answer::Values(v.iter().map(|&n| FoldValue::Scalar(n)).collect()) +} + +const BILLABLE_TRUE: Cond = Cond::Eq { + field: "billable", + literal: "true", +}; + +fn specs() -> Vec<(&'static str, MethodSpec)> { + vec![ + ( + "count_billable", + pure(COUNT_BILLABLE, query(&[BILLABLE_TRUE], &[Fold::Count])), + ), + ( + "count_by_duration", + pure( + COUNT_BY_DURATION, + query( + &[Cond::Eq { + field: "duration", + literal: "1", + }], + &[Fold::Count], + ), + ), + ), + ( + "submit_for_approval", + MethodSpec { + effect: EffectClass::ExternalMutate, + ..pure( + SUBMIT, + MethodBody::Action { + action: "submit_for_approval", + }, + ) + }, + ), + // Two more shapes, still data: a negated condition, and a body with + // two folds over one selection. + ( + "count_not_billable", + pure( + NOT_BILLABLE, + query( + &[Cond::Ne { + field: "billable", + literal: "true", + }], + &[Fold::Count], + ), + ), + ), + ( + "billable_hours", + pure( + BILLABLE_HOURS, + query(&[BILLABLE_TRUE], &[Fold::Count, Fold::Sum("duration")]), + ), + ), + // Experiment B: declared after the executor was frozen (e984885), + // using only an existing block (grouped SUM). + ( + "hours_by_billable", + pure( + HOURS_BY_BILLABLE, + query( + &[], + &[Fold::GroupSum { + key: "billable", + val: "duration", + groups: 2, + }], + ), + ), + ), + ] +} + +/// Billing indicators cycle through the three documented values and one the +/// DTO does not document, which is neither billable nor not. +const INDICATORS: [&str; 4] = ["Billable", "Non-Billable", "Internal Cost", "Unknown"]; +const HOURS_TEXT: [&str; 3] = ["8.50", "0.125", "2"]; + +fn indicator(i: usize) -> &'static str { + INDICATORS[(i * 7 + i / 3) % 4] +} + +fn batch(n: usize) -> CatsBatch { + let base: Vec> = include_str!("../../lance-graph-sap/fixtures/cats.txt") + .lines() + .map(|v| if v == "\\N" { None } else { Some(v) }) + .collect(); + let mut input: [Vec>; FIELD_COUNT] = std::array::from_fn(|i| vec![base[i]; n]); + for i in 0..n { + input[BILLING][i] = Some(indicator(i)); + input[HOURS][i] = Some(HOURS_TEXT[i % 3]); + } + CatsBatch::bind( + CatsSchema::new(42, 0), + std::array::from_fn(|i| input[i].as_slice()), + ) + .unwrap() +} + +/// The oracle reads the indicator text and the DTO's documented values, never +/// the lane a method binds. +fn billable_rows(n: usize) -> impl Iterator { + (0..n).filter(|&i| billable_code(indicator(i)) == 1) +} + +/// A test-local claim table that also maps `duration` onto SAP's decimal +/// hours (at the batch's own scale). FIXTURE: SAP's real claim on `duration` +/// is `Hypothesized`; a `Converted` grade needs the unit proof. +fn with_duration() -> Vec { + SAP_FIELDS + .iter() + .map(|m| FieldMap { + grade: if m.canonical == "duration" { + Grade::Exact + } else { + m.grade + }, + ..*m + }) + .collect() +} + +fn glove<'a>(basis: &'a [String], binder: &'a dyn Binder, claims: &'a [FieldMap]) -> Glove<'a> { + Glove { + concept: BILLABLE_WORK_ENTRY, + basis, + claims, + binder, + table: TABLE, + reading: READING, + universe: UNIVERSE, + } +} + +fn table_for(binder: &dyn Binder, claims: &[FieldMap]) -> MethodTable { + MethodTable::prepare(&glove(&canonical_fields(), binder, claims), &specs()) +} + +fn call( + t: &MethodTable, + batch: &CatsBatch, + key: MethodKey, + f: Frontend, +) -> Result { + let n = batch.len(); + let live = live_words(n); + let lanes = batch.lanes(); + let planes = Planes { + n_rows: n, + masks: &[live.as_slice()], + lanes: &lanes, + }; + t.call(key, &planes, UNIVERSE, f) +} + +/// FAILS IF: either frontend stops counting exactly the rows the DTO documents +/// as billable, or the two frontends disagree. +#[test] +fn count_billable_agrees_across_quack_r2il_and_the_oracle() { + for n in [1, 64, 65, 4097] { + let batch = batch(n); + let t = table_for(&CatsBinder::new(&batch), SAP_FIELDS); + let key = t.resolve("count_billable").expect("label"); + let want = scalars(&[billable_rows(n).count() as i64]); + assert_eq!( + call(&t, &batch, key, Frontend::Quack).unwrap(), + want, + "n={n}" + ); + assert_eq!( + call(&t, &batch, key, Frontend::R2il).unwrap(), + want, + "n={n}" + ); + } + // Non-vacuity: some rows are billable, some are not, some are unknown. + let o = billable_rows(4097).count(); + assert!(o > 0 && o < 4097); +} + +/// FAILS IF: a new method declared only as data does not run, or its two +/// frontends disagree with the oracle. `count_not_billable` is a negated +/// condition; `billable_hours` is two folds over one selection. +#[test] +fn methods_declared_as_data_run_without_new_rust() { + for n in [1, 65, 4097] { + let batch = batch(n); + let t = table_for(&CatsBinder::new(&batch), &with_duration()); + // `!= true` keeps the non-billable rows AND the undocumented ones. + let not_billable = (0..n).filter(|&i| billable_code(indicator(i)) != 1).count() as i64; + let LaneRef::I32(hours) = batch.lanes()[HOURS] else { + panic!("hours is an I32 lane"); + }; + let billable_sum: i64 = billable_rows(n).map(|i| i64::from(hours[i])).sum(); + let cases = [ + ("count_not_billable", scalars(&[not_billable])), + ( + "billable_hours", + scalars(&[billable_rows(n).count() as i64, billable_sum]), + ), + ]; + for (label, want) in cases { + let key = t.resolve(label).expect(label); + for f in [Frontend::Quack, Frontend::R2il] { + assert_eq!( + call(&t, &batch, key, f).unwrap(), + want, + "{label} {f:?} n={n}" + ); + } + } + } + // Non-vacuity: hours vary across billable rows, so a sum of the wrong + // lane or of every row would differ. + let batch = batch(65); + let LaneRef::I32(hours) = batch.lanes()[HOURS] else { + panic!() + }; + let all: i64 = hours.iter().map(|&h| i64::from(h)).sum(); + let billable: i64 = billable_rows(65).map(|i| i64::from(hours[i])).sum(); + assert!(billable > 0 && billable < all); +} + +/// The method's Quack query is the query a developer writes against the glove +/// directly, so both lower to one program. +#[test] +fn the_bound_query_is_the_one_quack_binds_from_names() { + let batch = batch(65); + let cats = CatsBinder::new(&batch); + let t = table_for(&cats, SAP_FIELDS); + let m = t.get(COUNT_BILLABLE).unwrap().as_ref().unwrap(); + let by_name = table(TABLE) + .where_eq("billable", "true") + .count() + .bind(&cats) + .unwrap(); + assert_eq!(m.queries(), vec![by_name.clone()]); + assert_eq!(lower(&m.queries()[0]).unwrap(), lower(&by_name).unwrap()); + // What a call executes is that lowered program, prepared once: no + // separate row evaluator sits behind the method. + assert_eq!(m.programs(), [lower(&by_name).unwrap()]); +} + +/// The field mask is over the canonical basis, not the glove's lanes. +#[test] +fn required_names_the_canonical_positions() { + let batch = batch(2); + let t = table_for(&CatsBinder::new(&batch), &with_duration()); + let pos = |f: &str| canonical_fields().iter().position(|c| c == f).unwrap() as u8; + let m = t.get(COUNT_BILLABLE).unwrap().as_ref().unwrap(); + assert!(m.required.has(pos("billable"))); + assert_eq!(m.required.count(), 1); + let m = t.get(BILLABLE_HOURS).unwrap().as_ref().unwrap(); + assert!(m.required.has(pos("billable")) && m.required.has(pos("duration"))); + assert_eq!(m.required.count(), 2); +} + +/// FAILS IF: a method whose field has only a hypothesized claim binds anyway. +/// `duration` is claimed by SAP `hours_logged`, graded `Hypothesized`. +#[test] +fn a_hypothesized_field_is_refused_at_preparation() { + let batch = batch(2); + let t = table_for(&CatsBinder::new(&batch), SAP_FIELDS); + for key in [COUNT_BY_DURATION, BILLABLE_HOURS] { + assert_eq!( + t.get(key), + Some(&Err(PrepareError::UnmappedRequired("duration"))) + ); + } + assert!(matches!( + call(&t, &batch, COUNT_BY_DURATION, Frontend::Quack), + Err(CallError::Refused(PrepareError::UnmappedRequired( + "duration" + ))) + )); + // Twin: with the claim proven, the refusal moves on to binding (a decimal + // lane is not a code), so the availability check was what refused. + let t = table_for(&CatsBinder::new(&batch), &with_duration()); + assert_eq!( + t.get(COUNT_BY_DURATION), + Some(&Err(PrepareError::KindMismatch("duration"))) + ); +} + +/// An action prepares, so the table can name it, and never runs as a pure call. +#[test] +fn an_action_is_named_and_never_run() { + let batch = batch(2); + let t = table_for(&CatsBinder::new(&batch), SAP_FIELDS); + let key = t.resolve("submit_for_approval").unwrap(); + assert!(t.get(key).unwrap().is_ok()); + for f in [Frontend::Quack, Frontend::R2il] { + assert!(matches!(call(&t, &batch, key, f), Err(CallError::NotPure))); + } +} + +#[test] +fn labels_resolve_once_and_unknown_names_resolve_to_nothing() { + let batch = batch(2); + let t = table_for(&CatsBinder::new(&batch), SAP_FIELDS); + assert_eq!(t.resolve("count_billable"), Some(COUNT_BILLABLE)); + assert_eq!(t.resolve("billable"), None, "a field name is not a method"); + assert!(matches!( + call(&t, &batch, MethodKey([0; 16]), Frontend::Quack), + Err(CallError::UnknownMethod) + )); +} + +#[test] +fn malformed_specs_are_refused_by_name() { + let batch = batch(2); + let wrong_concept = MethodSpec { + concept: BILLABLE_WORK_ENTRY + 1, + ..pure(MethodKey([1; 16]), query(&[BILLABLE_TRUE], &[Fold::Count])) + }; + let no_fold = pure(MethodKey([2; 16]), query(&[BILLABLE_TRUE], &[])); + let unknown_field = pure( + MethodKey([3; 16]), + query( + &[Cond::Eq { + field: "colour", + literal: "red", + }], + &[Fold::Count], + ), + ); + let unknown_value = pure( + MethodKey([4; 16]), + query( + &[Cond::Eq { + field: "billable", + literal: "maybe", + }], + &[Fold::Count], + ), + ); + let pure_action = MethodSpec { + effect: EffectClass::Pure, + ..pure( + MethodKey([5; 16]), + MethodBody::Action { + action: "submit_for_approval", + }, + ) + }; + let specs = [ + ("wrong_concept", wrong_concept), + ("no_fold", no_fold), + ("unknown_field", unknown_field), + ("unknown_value", unknown_value), + ("pure_action", pure_action), + ]; + let cats = CatsBinder::new(&batch); + let t = MethodTable::prepare(&glove(&canonical_fields(), &cats, SAP_FIELDS), &specs); + let refused = |k: u8| t.get(MethodKey([k; 16])).unwrap().clone().unwrap_err(); + assert!(matches!(refused(1), PrepareError::WrongConcept { .. })); + assert_eq!(refused(2), PrepareError::NoFold); + assert_eq!(refused(3), PrepareError::UnknownField("colour")); + assert_eq!( + refused(4), + PrepareError::UnknownValue { + field: "billable", + literal: "maybe" + } + ); + assert_eq!(refused(5), PrepareError::EffectMismatch); +} + +/// A binder that serves `billable` from the activity lane (also `U32`). +struct Swapped<'a>(CatsBinder<'a>); + +impl Binder for Swapped<'_> { + fn table(&self, name: &str) -> Option { + self.0.table(name) + } + fn live(&self, t: TableId) -> Mask { + self.0.live(t) + } + fn field(&self, t: TableId, name: &str) -> Option { + let mut f = self.0.field(t, name)?; + if name == "billable" { + f.col = Col(ACTIVITY as u16); + } + Some(f) + } + fn code(&self, t: TableId, col: Col, literal: &str) -> Option { + // Keep the literal's billable code so only the lane differs. + self.0 + .code(t, Col(BILLABLE as u16), literal) + .filter(|_| col.0 == ACTIVITY as u16) + } +} + +/// FAILS IF: the answer does not depend on which lane the method bound. The +/// same method bound to the wrong lane must disagree with the oracle on both +/// frontends. +#[test] +fn a_swapped_lane_breaks_parity() { + let n = 4097; + let batch = batch(n); + let t = table_for(&Swapped(CatsBinder::new(&batch)), SAP_FIELDS); + let want = scalars(&[billable_rows(n).count() as i64]); + for f in [Frontend::Quack, Frontend::R2il] { + assert_ne!(call(&t, &batch, COUNT_BILLABLE, f).unwrap(), want, "{f:?}"); + } +} + +/// Experiment B: a method declared after the executor was frozen, in this +/// file only. Its key is the billable code (0 = not billable, 1 = billable); +/// rows with an undocumented indicator carry neither code and fall outside +/// the two groups. +/// +/// Under the frozen executor (e984885) R2IL refused it: its folds took a mask +/// only from a computed slot, and a body with no condition hands them the +/// live plane. The next commit let every R2IL fold read a resident plane. +/// +/// FAILS IF: the grouped answer stops matching a row loop per group, on +/// either frontend. +#[test] +fn a_method_declared_after_the_freeze_runs_on_both_frontends() { + for n in [1, 65, 4097] { + let batch = batch(n); + let t = table_for(&CatsBinder::new(&batch), &with_duration()); + let key = t.resolve("hours_by_billable").expect("label"); + let LaneRef::I32(hours) = batch.lanes()[HOURS] else { + panic!("hours is an I32 lane"); + }; + let mut want = vec![0i64; 2]; + for (i, &h) in hours.iter().enumerate() { + let code = billable_code(indicator(i)); + if code < 2 { + want[code as usize] += i64::from(h); + } + } + for f in [Frontend::Quack, Frontend::R2il] { + assert_eq!( + call(&t, &batch, key, f).unwrap(), + Answer::Values(vec![FoldValue::Groups(want.clone())]), + "{f:?} n={n}" + ); + } + } + // Non-vacuity: both groups are populated, and the undocumented + // indicator is present, so a drop of it is observable. + let batch = batch(65); + assert!((0..65).any(|i| billable_code(indicator(i)) > 1)); + let t = table_for(&CatsBinder::new(&batch), &with_duration()); + let Answer::Values(v) = call(&t, &batch, HOURS_BY_BILLABLE, Frontend::Quack).unwrap() else { + panic!() + }; + let [FoldValue::Groups(g)] = v.as_slice() else { + panic!() + }; + assert!(g[0] > 0 && g[1] > 0); +} diff --git a/crates/lance-graph-glove-parity/tests/polyfill.rs b/crates/lance-graph-glove-parity/tests/polyfill.rs new file mode 100644 index 000000000..5493a25eb --- /dev/null +++ b/crates/lance-graph-glove-parity/tests/polyfill.rs @@ -0,0 +1,998 @@ +//! D-CMH-1: the class-qualified method as a compiler seam. +//! +//! `SELECT ::() FROM
` resolves through OGAR's +//! codebook to a concept id and a method key, binds once against the glove, +//! and runs as Quack algebra. Its answer is checked against the same query +//! written directly in Quack, against R2IL's procedures, and against a +//! scalar oracle. Then every way the binding could lie is refused by name, +//! and NAL revision is compiled from the same algebra plus one certified +//! law. +//! +//! Every method is declared in this file. The method keys, the reading id, +//! the population id and the NARS concept are FIXTURES. +use lance_graph_glove_parity::basis::{canonical_fields, FieldMap, Grade, SAP_FIELDS}; +use lance_graph_glove_parity::call::{resolve_call, CallParseError, MethodCall}; +use lance_graph_glove_parity::method::{ + Answer, CallError, Cond, EffectClass, Fold, FoldValue, Frontend, Glove, Law, MethodBody, + MethodKey, MethodSpec, MethodTable, PrepareError, ReadingId, SchemaSeal, Truth, Universe, +}; +use lance_graph_mask_risc::{execute_into, Foreign, LaneRef, Out, Planes, Scratch, Value}; +use lance_graph_ogar::ogar_vocab::class_ids::BILLABLE_WORK_ENTRY; +use lance_graph_quack::bind::{table, Binder, BoundField, FieldKind, TableId}; +use lance_graph_quack::{lower, Agg, Col, Mask, Query}; +use lance_graph_r2il::{procs, FoldDialect, Val}; +use lance_graph_sap::bind::{billable_code, CatsBatch, BILLING, HOURS}; +use lance_graph_sap::binder::{live_words, CatsBinder, LIVE, TABLE}; +use lance_graph_sap::schema::{CatsSchema, FIELD_COUNT}; + +const READING: ReadingId = ReadingId(1); +const UNIVERSE: Universe = Universe(42); +const BILLABLE_HOURS: MethodKey = MethodKey([0xB2; 16]); + +const BILLABLE_TRUE: Cond = Cond::Eq { + field: "billable", + literal: "true", +}; + +fn seal() -> SchemaSeal { + SchemaSeal::of(&canonical_fields()) +} + +fn billable_hours() -> MethodSpec { + MethodSpec { + key: BILLABLE_HOURS, + concept: BILLABLE_WORK_ENTRY, + schema: seal(), + reading: READING, + effect: EffectClass::Pure, + body: MethodBody::Query { + filter: &[BILLABLE_TRUE], + folds: &[Fold::Sum("duration")], + law: Law::Values, + }, + } +} + +// --- the SAP fixture (as in tests/method.rs) --------------------------------- + +const INDICATORS: [&str; 4] = ["Billable", "Non-Billable", "Internal Cost", "Unknown"]; +const HOURS_TEXT: [&str; 3] = ["8.50", "0.125", "2"]; + +fn indicator(i: usize) -> &'static str { + INDICATORS[(i * 7 + i / 3) % 4] +} + +fn batch(n: usize) -> CatsBatch { + let base: Vec> = include_str!("../../lance-graph-sap/fixtures/cats.txt") + .lines() + .map(|v| if v == "\\N" { None } else { Some(v) }) + .collect(); + let mut input: [Vec>; FIELD_COUNT] = std::array::from_fn(|i| vec![base[i]; n]); + for i in 0..n { + input[BILLING][i] = Some(indicator(i)); + input[HOURS][i] = Some(HOURS_TEXT[i % 3]); + } + CatsBatch::bind( + CatsSchema::new(42, 0), + std::array::from_fn(|i| input[i].as_slice()), + ) + .unwrap() +} + +/// FIXTURE: SAP's real claim on `duration` is `Hypothesized`; this table +/// promotes it so the method has an `I32` quantity to sum. +fn with_duration() -> Vec { + SAP_FIELDS + .iter() + .map(|m| FieldMap { + grade: if m.canonical == "duration" { + Grade::Exact + } else { + m.grade + }, + ..*m + }) + .collect() +} + +fn glove<'a>(basis: &'a [String], binder: &'a dyn Binder, claims: &'a [FieldMap]) -> Glove<'a> { + Glove { + concept: BILLABLE_WORK_ENTRY, + basis, + claims, + binder, + table: TABLE, + reading: READING, + universe: UNIVERSE, + } +} + +fn hours(batch: &CatsBatch) -> Vec { + let LaneRef::I32(h) = batch.lanes()[HOURS] else { + panic!("hours is an I32 lane") + }; + h.to_vec() +} + +// --- the seam --------------------------------------------------------------- + +/// FAILS IF: the class-qualified call stops agreeing with the same query +/// written directly in Quack, with R2IL's procedures, or with a row loop. +#[test] +fn a_class_qualified_call_agrees_with_direct_quack_r2il_and_the_oracle() { + let sql = "SELECT billable_work_entry::billable_hours(cats) FROM cats"; + for n in [1, 64, 65, 4097] { + let batch = batch(n); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let basis = canonical_fields(); + let t = MethodTable::prepare( + &glove(&basis, &cats, &claims), + &[("billable_hours", billable_hours())], + ); + let call = resolve_call(sql, &t).unwrap(); + assert_eq!( + call, + MethodCall { + concept: BILLABLE_WORK_ENTRY, + method: BILLABLE_HOURS + } + ); + + let live = live_words(n); + let lanes = batch.lanes(); + let planes = Planes { + n_rows: n, + masks: &[live.as_slice()], + lanes: &lanes, + }; + + // 1. The oracle: a row loop over the indicator text and the hours. + let h = hours(&batch); + let oracle: i64 = (0..n) + .filter(|&i| billable_code(indicator(i)) == 1) + .map(|i| i64::from(h[i])) + .sum(); + + // 2. The same query written by hand in Quack, bound from names. + let tid = cats.table(TABLE).unwrap(); + let hours_col = cats.field(tid, "hours_logged").unwrap().col; + let by_name = table(TABLE) + .where_eq("billable", "true") + .count() + .bind(&cats) + .unwrap(); + let direct = Query { + filter: by_name.filter, + agg: Agg::SumI32(hours_col), + }; + let p = lower(&direct).unwrap(); + let mut scratch = Scratch::for_program(&p, n).unwrap(); + let Value::SumI64(direct) = + execute_into(&p, &planes, &Foreign::NONE, &mut scratch, Out::None).unwrap() + else { + panic!() + }; + + // 3. R2IL's procedures, called by hand. + let billable = cats.field(tid, "billable").unwrap().col; + let mut d = FoldDialect::new(&planes, &Foreign::NONE); + let mut stack = Vec::new(); + procs::select_u32(&mut d, &mut stack, LIVE.0, &[(billable.0, 1, false)]).unwrap(); + procs::sum(&mut d, &mut stack, hours_col.0).unwrap(); + let [Val::Scalar(r2il)] = stack.as_slice() else { + panic!() + }; + + let want = Answer::Values(vec![FoldValue::Scalar(oracle)]); + assert_eq!(direct, oracle, "direct quack n={n}"); + assert_eq!(*r2il, oracle, "r2il procs n={n}"); + for f in [Frontend::Quack, Frontend::R2il] { + assert_eq!( + t.call(call.method, &planes, UNIVERSE, f).unwrap(), + want, + "method {f:?} n={n}" + ); + } + } +} + +/// The call text names a concept through OGAR's codebook, a method through +/// the table's labels, and a row; each part that does not resolve is named. +#[test] +fn a_call_that_does_not_resolve_is_refused_by_name() { + let batch = batch(2); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let basis = canonical_fields(); + let t = MethodTable::prepare( + &glove(&basis, &cats, &claims), + &[("billable_hours", billable_hours())], + ); + let cases = [ + ( + "SELECT nars::revision(cats) FROM cats", + CallParseError::UnknownConcept("nars".into()), + ), + ( + "SELECT project::billable_hours(cats) FROM cats", + CallParseError::WrongConcept { + called: lance_graph_ogar::ogar_vocab::class_ids::PROJECT, + table: BILLABLE_WORK_ENTRY, + }, + ), + ( + "SELECT billable_work_entry::frobnicate(cats) FROM cats", + CallParseError::UnknownMethod("frobnicate".into()), + ), + ( + "SELECT billable_work_entry::billable_hours(orders) FROM cats", + CallParseError::PayloadMismatch, + ), + ( + "SELECT billable_work_entry::billable_hours(orders) FROM orders", + CallParseError::WrongTable("orders".into()), + ), + ( + "SELECT billable_work_entry::billable_hours(cats) FROM cats WHERE x = 1", + CallParseError::Unrecognised, + ), + ( + "SELECT billable_work_entry::billable_hours(cats); DROP TABLE cats FROM cats", + CallParseError::Unrecognised, + ), + ]; + for (sql, want) in cases { + assert_eq!(resolve_call(sql, &t), Err(want), "{sql}"); + } +} + +// --- refusals --------------------------------------------------------------- + +/// FAILS IF: a spec written for one basis binds a reordered one. Swapping two +/// canonical positions keeps every name, so only the seal can tell. +#[test] +fn a_spec_for_another_schema_or_reading_is_refused() { + let batch = batch(2); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let basis = canonical_fields(); + let mut swapped = basis.clone(); + swapped.swap(0, 1); + assert_ne!(SchemaSeal::of(&swapped), seal()); + let t = MethodTable::prepare(&glove(&swapped, &cats, &claims), &[("h", billable_hours())]); + assert!(matches!( + t.get(BILLABLE_HOURS), + Some(Err(PrepareError::SchemaMismatch { .. })) + )); + + let other = MethodSpec { + reading: ReadingId(2), + ..billable_hours() + }; + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", other)]); + assert_eq!( + t.get(BILLABLE_HOURS), + Some(&Err(PrepareError::ReadingMismatch { + spec: ReadingId(2), + glove: READING + })) + ); + + // Twin: the unswapped basis and matching reading prepare. + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", billable_hours())]); + assert!(t.get(BILLABLE_HOURS).unwrap().is_ok()); +} + +/// FAILS IF: planes of another population run under a method prepared for +/// this one. +#[test] +fn planes_of_another_population_are_refused() { + let batch = batch(65); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let basis = canonical_fields(); + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", billable_hours())]); + let live = live_words(65); + let lanes = batch.lanes(); + let planes = Planes { + n_rows: 65, + masks: &[live.as_slice()], + lanes: &lanes, + }; + for f in [Frontend::Quack, Frontend::R2il] { + assert!(matches!( + t.call(BILLABLE_HOURS, &planes, Universe(43), f), + Err(CallError::UniverseMismatch { + method: UNIVERSE, + planes: Universe(43) + }) + )); + assert!(t.call(BILLABLE_HOURS, &planes, UNIVERSE, f).is_ok()); + } +} + +/// A binder that declares `billable` nullable. +struct NullableBillable<'a>(CatsBinder<'a>); + +impl Binder for NullableBillable<'_> { + fn table(&self, name: &str) -> Option { + self.0.table(name) + } + fn live(&self, t: TableId) -> Mask { + self.0.live(t) + } + fn field(&self, t: TableId, name: &str) -> Option { + let mut f = self.0.field(t, name)?; + if name == "billable" { + f.validity = Some(Mask(7)); + } + Some(f) + } + fn code(&self, t: TableId, col: Col, literal: &str) -> Option { + self.0.code(t, col, literal) + } +} + +/// FAILS IF: a nullable field binds into a two-valued condition (NULL would +/// read as whatever the lane holds there). +#[test] +fn a_nullable_field_is_refused() { + let batch = batch(2); + let nb = NullableBillable(CatsBinder::new(&batch)); + let claims = with_duration(); + let basis = canonical_fields(); + let t = MethodTable::prepare(&glove(&basis, &nb, &claims), &[("h", billable_hours())]); + assert_eq!( + t.get(BILLABLE_HOURS), + Some(&Err(PrepareError::Nullable("billable"))) + ); +} + +/// FAILS IF: a basis wider than a `WideFieldMask` position can address is +/// truncated instead of refused. +#[test] +fn a_basis_past_256_positions_is_refused() { + let batch = batch(2); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let mut wide = canonical_fields(); + // Pad with names nothing claims, until `billable` sits past position 255 + // only in the second case. + while wide.len() < 257 { + wide.push(format!("pad_{}", wide.len())); + } + let spec = MethodSpec { + schema: SchemaSeal::of(&wide), + ..billable_hours() + }; + let t = MethodTable::prepare(&glove(&wide, &cats, &claims), &[("h", spec)]); + assert_eq!( + t.get(BILLABLE_HOURS), + Some(&Err(PrepareError::BasisTooWide(257))) + ); + // Twin: exactly 256 positions prepare. + wide.pop(); + let spec = MethodSpec { + schema: SchemaSeal::of(&wide), + ..billable_hours() + }; + let t = MethodTable::prepare(&glove(&wide, &cats, &claims), &[("h", spec)]); + assert!(t.get(BILLABLE_HOURS).unwrap().is_ok()); +} + +// --- NAL revision ----------------------------------------------------------- + +/// FIXTURE: no NARS concept is minted in OGAR's codebook yet. +const NARS: u16 = 0xFFF0; +const REVISION: MethodKey = MethodKey([0xEE; 16]); +const RAW_SUMS: MethodKey = MethodKey([0xEF; 16]); +const SCALE: u32 = 1000; +const GROUPS: u32 = 4; + +const EVIDENCE: &str = "evidence"; +const STATEMENT: usize = 0; +const STAMP: usize = 1; +const INDEPENDENT: usize = 2; +const W: usize = 3; +const WF: usize = 4; + +fn nars_basis() -> Vec { + ["statement", "stamp", "independent", "w", "wf"] + .map(String::from) + .to_vec() +} + +const NARS_CLAIMS: [FieldMap; 5] = [ + FieldMap { + native: "statement", + canonical: "statement", + grade: Grade::Exact, + note: "fixture", + }, + FieldMap { + native: "stamp", + canonical: "stamp", + grade: Grade::Exact, + note: "fixture", + }, + FieldMap { + native: "independent", + canonical: "independent", + grade: Grade::Exact, + note: "fixture", + }, + FieldMap { + native: "w", + canonical: "w", + grade: Grade::Exact, + note: "fixture", + }, + FieldMap { + native: "wf", + canonical: "wf", + grade: Grade::Exact, + note: "fixture", + }, +]; + +/// The evidence table's resolution membrane: five non-nullable lanes. +struct EvidenceBinder; + +impl Binder for EvidenceBinder { + fn table(&self, name: &str) -> Option { + (name == EVIDENCE).then_some(TableId(0)) + } + fn live(&self, _: TableId) -> Mask { + Mask(0) + } + fn field(&self, _: TableId, name: &str) -> Option { + let (col, kind) = match name { + "statement" => (STATEMENT, FieldKind::Code), + "stamp" => (STAMP, FieldKind::Code), + "independent" => (INDEPENDENT, FieldKind::Code), + "w" => (W, FieldKind::I32), + "wf" => (WF, FieldKind::I32), + _ => return None, + }; + Some(BoundField { + col: Col(col as u16), + kind, + validity: None, + }) + } + fn code(&self, _: TableId, col: Col, literal: &str) -> Option { + if usize::from(col.0) != INDEPENDENT { + return None; + } + match literal { + "true" => Some(1), + "false" => Some(0), + _ => None, + } + } +} + +const REVISION_FOLDS: &[Fold] = &[ + Fold::GroupSum { + key: "statement", + val: "w", + groups: GROUPS, + }, + Fold::GroupSum { + key: "statement", + val: "wf", + groups: GROUPS, + }, +]; + +const REVISION_LAW: Law = Law::NarsRevision { + independent: "independent", + scale: SCALE, + horizon: 1, +}; + +fn nars_spec( + key: MethodKey, + filter: &'static [Cond], + folds: &'static [Fold], + law: Law, +) -> MethodSpec { + MethodSpec { + key, + concept: NARS, + schema: SchemaSeal::of(&nars_basis()), + reading: READING, + effect: EffectClass::Pure, + body: MethodBody::Query { filter, folds, law }, + } +} + +fn nars_specs() -> Vec<(&'static str, MethodSpec)> { + const INDEP: &[Cond] = &[Cond::Eq { + field: "independent", + literal: "true", + }]; + vec![ + ( + "revision", + nars_spec(REVISION, INDEP, REVISION_FOLDS, REVISION_LAW), + ), + // The same sums without the independence condition, as plain values: + // what revision would read if duplicates were not excluded. + ( + "raw_sums", + nars_spec(RAW_SUMS, &[], REVISION_FOLDS, Law::Values), + ), + ] +} + +/// One piece of evidence: `(statement, stamp, f, c)`. +type Evidence = (u32, u32, f64, f64); + +/// The OpenNARS reference revisions (`lance-graph-planner` +/// `tests/nal_revision_oracle.rs`), as evidence per statement, plus one +/// REPEATED stamp on statement 0. Statement 3 has no evidence. +const ROWS: [Evidence; 7] = [ + (0, 10, 1.0, 0.9), + (0, 11, 0.1, 0.6), + (1, 20, 0.8, 0.5), + (1, 21, 0.3, 0.7), + (2, 30, 0.0, 0.9), + (2, 31, 1.0, 0.9), + (0, 10, 1.0, 0.9), // the same stamp again +]; + +/// OpenNARS's revised truths, to two places. +const REFERENCE: [(f64, f64); 3] = [(0.87, 0.91), (0.45, 0.77), (0.50, 0.95)]; + +struct Lanes { + statement: Vec, + stamp: Vec, + independent: Vec, + w: Vec, + wf: Vec, +} + +/// The payload's producer: fixed-point weights, and the first-occurrence +/// mark of each `(statement, stamp)` (what the algebra cannot compute yet). +fn produce(rows: &[Evidence]) -> Lanes { + let mut seen = Vec::new(); + let mut l = Lanes { + statement: vec![], + stamp: vec![], + independent: vec![], + w: vec![], + wf: vec![], + }; + for &(s, stamp, f, c) in rows { + let w = (f64::from(SCALE) * c / (1.0 - c)).round(); + l.statement.push(s); + l.stamp.push(stamp); + l.independent.push(u32::from(!seen.contains(&(s, stamp)))); + seen.push((s, stamp)); + l.w.push(w as i32); + l.wf.push((w * f).round() as i32); + } + l +} + +/// The independent oracle: revision as the NAL defines it, in floating +/// point, from `(f, c)` directly, over the independent evidence. +fn oracle(rows: &[Evidence]) -> Vec> { + let mut seen = Vec::new(); + let mut acc = vec![(0.0f64, 0.0f64); GROUPS as usize]; + for &(s, stamp, f, c) in rows { + if seen.contains(&(s, stamp)) { + continue; + } + seen.push((s, stamp)); + let w = c / (1.0 - c); + acc[s as usize].0 += w; + acc[s as usize].1 += w * f; + } + acc.iter() + .map(|&(w, wf)| { + (w > 0.0).then(|| Truth { + f: wf / w, + c: w / (w + 1.0), + }) + }) + .collect() +} + +fn nars_table() -> MethodTable { + let basis = nars_basis(); + MethodTable::prepare( + &Glove { + concept: NARS, + basis: &basis, + claims: &NARS_CLAIMS, + binder: &EvidenceBinder, + table: EVIDENCE, + reading: READING, + universe: UNIVERSE, + }, + &nars_specs(), + ) +} + +fn run_nars(t: &MethodTable, key: MethodKey, l: &Lanes, f: Frontend) -> Answer { + let n = l.w.len(); + let live = live_words(n); + let lanes = [ + LaneRef::U32(&l.statement), + LaneRef::U32(&l.stamp), + LaneRef::U32(&l.independent), + LaneRef::I32(&l.w), + LaneRef::I32(&l.wf), + ]; + let planes = Planes { + n_rows: n, + masks: &[live.as_slice()], + lanes: &lanes, + }; + t.call(key, &planes, UNIVERSE, f).unwrap() +} + +/// FAILS IF: revision compiled from two grouped sums and the certified law +/// disagrees with the floating-point oracle or with OpenNARS, on either +/// frontend; or counts the repeated stamp. +#[test] +fn nars_revision_compiles_from_grouped_sums_and_one_law() { + let t = nars_table(); + let lanes = produce(&ROWS); + let want = oracle(&ROWS); + for f in [Frontend::Quack, Frontend::R2il] { + let Answer::Truths(got) = run_nars(&t, REVISION, &lanes, f) else { + panic!("revision answers truths") + }; + assert_eq!(got.len(), GROUPS as usize); + for (s, (g, o)) in got.iter().zip(&want).enumerate() { + match (g, o) { + (Some(g), Some(o)) => { + // Fixed point at scale 1000 moves the third place at most. + assert!((g.f - o.f).abs() < 1e-3, "{f:?} s{s} f {g:?} vs {o:?}"); + assert!((g.c - o.c).abs() < 1e-3, "{f:?} s{s} c {g:?} vs {o:?}"); + } + (None, None) => {} + _ => panic!("{f:?} s{s}: {g:?} vs {o:?}"), + } + } + for (s, &(rf, rc)) in REFERENCE.iter().enumerate() { + let g = got[s].unwrap(); + assert!( + (g.f - rf).abs() < 0.005 && (g.c - rc).abs() < 0.005, + "{f:?} s{s} {g:?}" + ); + } + assert_eq!(got[3], None, "no evidence, no truth"); + } +} + +/// The counterexample for the missing algebra: the same two sums without the +/// independence condition count the repeated stamp, and the confidence that +/// would follow is not NAL revision. +#[test] +fn a_repeated_stamp_inflates_the_raw_sums() { + let t = nars_table(); + let lanes = produce(&ROWS); + let Answer::Values(v) = run_nars(&t, RAW_SUMS, &lanes, Frontend::Quack) else { + panic!() + }; + let [FoldValue::Groups(w), _] = v.as_slice() else { + panic!() + }; + // Statement 0: 9000 + 1500 counted, plus 9000 again. + assert_eq!(w[0], 19_500); + let weight = w[0] as f64 / f64::from(SCALE); + let inflated = weight / (weight + 1.0); + assert!((inflated - 0.951).abs() < 1e-3); + assert!( + (inflated - REFERENCE[0].1).abs() > 0.03, + "the repeat must matter" + ); +} + +/// FAILS IF: a revision body that does not select independent evidence, or +/// whose folds are not the law's shape, prepares. +#[test] +fn a_revision_body_without_independence_or_in_the_wrong_shape_is_refused() { + const MIXED_KEYS: &[Fold] = &[ + Fold::GroupSum { + key: "statement", + val: "w", + groups: GROUPS, + }, + Fold::GroupSum { + key: "stamp", + val: "wf", + groups: GROUPS, + }, + ]; + const INDEP: &[Cond] = &[Cond::Eq { + field: "independent", + literal: "true", + }]; + let basis = nars_basis(); + let specs = [ + ( + "no_indep", + nars_spec(MethodKey([1; 16]), &[], REVISION_FOLDS, REVISION_LAW), + ), + ( + "mixed", + nars_spec(MethodKey([2; 16]), INDEP, MIXED_KEYS, REVISION_LAW), + ), + ( + "one_fold", + nars_spec( + MethodKey([3; 16]), + INDEP, + &REVISION_FOLDS[..1], + REVISION_LAW, + ), + ), + ]; + let t = MethodTable::prepare( + &Glove { + concept: NARS, + basis: &basis, + claims: &NARS_CLAIMS, + binder: &EvidenceBinder, + table: EVIDENCE, + reading: READING, + universe: UNIVERSE, + }, + &specs, + ); + assert_eq!( + t.get(MethodKey([1; 16])), + Some(&Err(PrepareError::EvidenceNotIndependent("independent"))) + ); + assert_eq!( + t.get(MethodKey([2; 16])), + Some(&Err(PrepareError::LawShape)) + ); + assert_eq!( + t.get(MethodKey([3; 16])), + Some(&Err(PrepareError::LawShape)) + ); +} + +/// KNOWN GAP, pinned: a statement at or past the declared domain is dropped +/// by the grouped-sum terminal, not refused. A method cannot see it. +#[test] +fn evidence_outside_the_declared_domain_is_dropped_silently() { + let t = nars_table(); + let mut rows = ROWS.to_vec(); + rows.push((GROUPS + 1, 99, 1.0, 0.9)); + let Answer::Truths(got) = run_nars(&t, REVISION, &produce(&rows), Frontend::Quack) else { + panic!() + }; + let Answer::Truths(base) = run_nars(&t, REVISION, &produce(&ROWS), Frontend::Quack) else { + panic!() + }; + assert_eq!(got, base, "the out-of-domain row left no trace"); +} + +// --- Experiment C: replay, layers, retraction -------------------------------- + +/// One delta as it arrives: `(delta_id, layer, statement, stamp, f, c)`. +/// Replay identity (`delta_id`) and evidential identity (`stamp`) are +/// different things: a replayed delta repeats a delta; two deltas can carry +/// the same stamp (one observation, reported in two hypothesis layers). +type Delta = (u32, u8, u32, u32, f64, f64); + +/// The payload producer: drops a replayed `delta_id` first, then marks the +/// first occurrence of each `(statement, stamp)` across ALL layers as +/// independent. Both marks are the producer's, because the algebra cannot +/// compute a first occurrence yet (ledger row 1). +fn produce_deltas(deltas: &[Delta]) -> Lanes { + let mut seen_delta = Vec::new(); + let rows: Vec = deltas + .iter() + .filter(|d| { + let fresh = !seen_delta.contains(&d.0); + seen_delta.push(d.0); + fresh + }) + .map(|&(_, _, s, stamp, f, c)| (s, stamp, f, c)) + .collect(); + produce(&rows) +} + +/// FAILS IF: a replayed delta, or the same stamp arriving in a second +/// hypothesis layer, changes the revised truth. +#[test] +fn replay_and_cross_layer_repeats_do_not_change_revision() { + let t = nars_table(); + let base: Vec = ROWS[..6] + .iter() + .enumerate() + .map(|(i, &(s, stamp, f, c))| (i as u32, 0, s, stamp, f, c)) + .collect(); + let mut noisy = base.clone(); + noisy.push(base[0]); // the same delta, replayed + noisy.push(base[3]); // replayed again, later + noisy.push((100, 1, 0, 11, 0.1, 0.6)); // stamp 11, seen in layer 1 too + noisy.push((101, 1, 2, 30, 0.0, 0.9)); // stamp 30, layer 1 + for f in [Frontend::Quack, Frontend::R2il] { + let a = run_nars(&t, REVISION, &produce_deltas(&base), f); + let b = run_nars(&t, REVISION, &produce_deltas(&noisy), f); + assert_eq!(a, b, "{f:?}"); + } + // Non-vacuity: without the marks, the noisy payload inflates the sums. + let raw = |d: &[Delta]| { + let rows: Vec = d.iter().map(|&(_, _, s, st, f, c)| (s, st, f, c)).collect(); + let Answer::Values(v) = run_nars(&t, RAW_SUMS, &produce(&rows), Frontend::Quack) else { + panic!() + }; + v + }; + assert_ne!(raw(&base), raw(&noisy)); +} + +/// TYPED INSUFFICIENCY, pinned: retraction is not declared by +/// `Law::NarsRevision`. Writing it as a negative-weight delta is legal +/// arithmetic in the sums, and wrong whenever the retracted stamp was never +/// asserted: the sums cannot see which stamps were asserted. Retraction needs +/// a semijoin of retraction stamps onto asserted stamps (ledger). +#[test] +fn retraction_by_negative_weight_is_not_revision() { + let t = nars_table(); + // Retract stamp 11 from statement 0 (it was asserted): exact. + let mut lanes = produce(&ROWS[..6]); + let retract_11 = (0, 11, 0.1, 0.6); + let w = (f64::from(SCALE) * 0.6 / 0.4).round(); + lanes.statement.push(retract_11.0); + lanes.stamp.push(retract_11.1); + lanes.independent.push(1); + lanes.w.push(-(w as i32)); + lanes.wf.push(-((w * 0.1).round() as i32)); + let Answer::Truths(got) = run_nars(&t, REVISION, &lanes, Frontend::Quack) else { + panic!() + }; + let only_10 = oracle(&[(0, 10, 1.0, 0.9)]); + assert!((got[0].unwrap().c - only_10[0].unwrap().c).abs() < 1e-3); + + // Retract stamp 99 (never asserted): the sums accept it and answer. + let mut lanes = produce(&ROWS[..6]); + lanes.statement.push(0); + lanes.stamp.push(99); + lanes.independent.push(1); + lanes.w.push(-(w as i32)); + lanes.wf.push(-((w * 0.1).round() as i32)); + let Answer::Truths(got) = run_nars(&t, REVISION, &lanes, Frontend::Quack) else { + panic!() + }; + let unchanged = oracle(&ROWS[..6])[0].unwrap(); + assert!( + (got[0].unwrap().c - unchanged.c).abs() > 0.01, + "a phantom retraction changed the truth, and nothing refused it" + ); +} + +// --- review fixes: ambiguity and law parameters are refused ------------------ + +/// FAILS IF: a repeated method key or label is resolved by slice order +/// instead of refused. +#[test] +fn a_repeated_key_or_label_is_refused() { + let batch = batch(2); + let cats = CatsBinder::new(&batch); + let claims = with_duration(); + let basis = canonical_fields(); + let other_key = MethodSpec { + key: MethodKey([0xC1; 16]), + ..billable_hours() + }; + let t = MethodTable::prepare( + &glove(&basis, &cats, &claims), + &[ + ("h", billable_hours()), + ("h2", billable_hours()), // same key, another label + ("dup", other_key), + ( + "dup", + MethodSpec { + key: MethodKey([0xC2; 16]), + ..billable_hours() + }, + ), + ], + ); + assert_eq!( + t.get(BILLABLE_HOURS), + Some(&Err(PrepareError::DuplicateKey(BILLABLE_HOURS))) + ); + assert_eq!(t.resolve("dup"), None, "an ambiguous label names nothing"); + assert_eq!( + t.get(MethodKey([0xC1; 16])), + Some(&Err(PrepareError::DuplicateLabel("dup"))) + ); + // Twin: a unique key and label still prepare. + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", billable_hours())]); + assert!(t.get(BILLABLE_HOURS).unwrap().is_ok()); + assert_eq!(t.resolve("h"), Some(BILLABLE_HOURS)); +} + +/// FAILS IF: a revision law with a zero scale or zero horizon prepares (a +/// zero scale yields NaN, a zero horizon a confidence of exactly 1). +#[test] +fn a_degenerate_revision_law_is_refused() { + const INDEP: &[Cond] = &[Cond::Eq { + field: "independent", + literal: "true", + }]; + let law = |scale, horizon| Law::NarsRevision { + independent: "independent", + scale, + horizon, + }; + let basis = nars_basis(); + let specs = [ + ( + "zero_scale", + nars_spec(MethodKey([4; 16]), INDEP, REVISION_FOLDS, law(0, 1)), + ), + ( + "zero_horizon", + nars_spec(MethodKey([5; 16]), INDEP, REVISION_FOLDS, law(SCALE, 0)), + ), + ( + "ok", + nars_spec(MethodKey([6; 16]), INDEP, REVISION_FOLDS, law(SCALE, 1)), + ), + ]; + let t = MethodTable::prepare( + &Glove { + concept: NARS, + basis: &basis, + claims: &NARS_CLAIMS, + binder: &EvidenceBinder, + table: EVIDENCE, + reading: READING, + universe: UNIVERSE, + }, + &specs, + ); + assert_eq!( + t.get(MethodKey([4; 16])), + Some(&Err(PrepareError::LawParameter)) + ); + assert_eq!( + t.get(MethodKey([5; 16])), + Some(&Err(PrepareError::LawParameter)) + ); + assert!(t.get(MethodKey([6; 16])).unwrap().is_ok()); +} + +/// FAILS IF: two proven claims for one canonical field are resolved by +/// slice order. Reordering the claims must not change which lane binds. +#[test] +fn an_ambiguous_canonical_claim_is_refused() { + let batch = batch(2); + let cats = CatsBinder::new(&batch); + let basis = canonical_fields(); + let mut claims = with_duration(); + // A second proven source for `duration`, from another native field. + claims.push(FieldMap { + native: "work_day", + canonical: "duration", + grade: Grade::Exact, + note: "fixture: a conflicting claim", + }); + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", billable_hours())]); + assert_eq!( + t.get(BILLABLE_HOURS), + Some(&Err(PrepareError::AmbiguousClaim("duration"))) + ); + // Twin: the same claim repeated for the same native field is not + // ambiguous. + let mut claims = with_duration(); + let dup = *claims.iter().find(|c| c.canonical == "duration").unwrap(); + claims.push(dup); + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &[("h", billable_hours())]); + assert!(t.get(BILLABLE_HOURS).unwrap().is_ok()); +} diff --git a/crates/lance-graph-r2il/src/lib.rs b/crates/lance-graph-r2il/src/lib.rs index 2081e881f..944ecc9a3 100644 --- a/crates/lance-graph-r2il/src/lib.rs +++ b/crates/lance-graph-r2il/src/lib.rs @@ -64,8 +64,8 @@ //! //! # Procedures //! -//! [`procs`] holds named call sequences (`filter_lt`, `sum`, `group_sum`, -//! `margin`). They are Rust functions over [`Dialect::call`], not opcodes. +//! [`procs`] holds named call sequences (`filter_lt`, `select_u32`, `count`, +//! `sum`, `group_sum`, `margin`). They are Rust functions over [`Dialect::call`], not opcodes. //! //! # Slot limit //! @@ -438,6 +438,22 @@ impl<'p> FoldDialect<'p> { /// The operand a consumer reads for `slot`. A real slot is itself; a /// virtual one is lowered through mask-risc `fuse` the first time it is /// consumed, its ternlogs appended to this fold's ops. + /// Resolve a fold's mask: a computed slot of the current epoch, or a + /// resident plane read where it stands. Shape is judged by + /// [`fold_mask`] first, so a wrong-kind operand is reported as such + /// before the epoch is. + fn resolve_mask(&mut self, m: FoldMask) -> Result { + match m { + FoldMask::Slot { slot, epoch } => { + if epoch != self.epoch { + return Err(FoldError::StaleSlot); + } + self.operand_of(slot) + } + FoldMask::Plane(p) => Ok(Operand::Plane(p)), + } + } + fn operand_of(&mut self, slot: u16) -> Result { if slot < VIRTUAL { return Ok(Operand::Scratch(slot)); @@ -867,14 +883,9 @@ impl Dialect for FoldDialect<'_> { } } POP_COUNT => { - let mask = stack.pop().ok_or(FoldError::Underflow(f))?; - let Val::Slot { slot: s, epoch } = mask else { - return Err(FoldError::WrongOperandKind(f)); - }; - if epoch != self.epoch { - return Err(FoldError::StaleSlot); - } - let mask = self.operand_of(s)?; + let m = fold_mask(stack.pop().ok_or(FoldError::Underflow(f))?) + .ok_or(FoldError::WrongOperandKind(f))?; + let mask = self.resolve_mask(m)?; let n = self.finalize_and_run_scalar(Terminal::Count { mask })?; stack.push(Val::Scalar(n)); Ok(()) @@ -882,33 +893,19 @@ impl Dialect for FoldDialect<'_> { SUM => { let b = stack.pop().ok_or(FoldError::Underflow(f))?; let a = stack.pop().ok_or(FoldError::Underflow(f))?; - let (mask, lane) = match (a, b) { - ( - Val::Slot { slot: s, epoch }, - Val::Address(Addr::Lane { - idx, - kind: LaneKind::I32, - }), - ) => { - if epoch != self.epoch { - return Err(FoldError::StaleSlot); - } - (self.operand_of(s)?, idx) - } - ( - Val::Address(Addr::Lane { - idx, - kind: LaneKind::I32, - }), - Val::Slot { slot: s, epoch }, - ) => { - if epoch != self.epoch { - return Err(FoldError::StaleSlot); - } - (self.operand_of(s)?, idx) - } + let i32_lane = |v: Val| match v { + Val::Address(Addr::Lane { + idx, + kind: LaneKind::I32, + }) => Some(idx), + _ => None, + }; + // Either order: mask then lane, or lane then mask. + let (m, lane) = match (fold_mask(a), i32_lane(b), i32_lane(a), fold_mask(b)) { + (Some(m), Some(lane), _, _) | (_, _, Some(lane), Some(m)) => (m, lane), _ => return Err(FoldError::WrongOperandKind(f)), }; + let mask = self.resolve_mask(m)?; let n = self.finalize_and_run_scalar(Terminal::MaskedSumI32 { mask, lane })?; stack.push(Val::Scalar(n)); Ok(()) @@ -921,9 +918,7 @@ impl Dialect for FoldDialect<'_> { let val = stack.pop().ok_or(FoldError::Underflow(f))?; let key = stack.pop().ok_or(FoldError::Underflow(f))?; let mask = stack.pop().ok_or(FoldError::Underflow(f))?; - let Val::Slot { slot: s, epoch } = mask else { - return Err(FoldError::WrongOperandKind(f)); - }; + let m = fold_mask(mask).ok_or(FoldError::WrongOperandKind(f))?; let Val::Address(Addr::Lane { idx: val, kind: LaneKind::I32, @@ -943,10 +938,7 @@ impl Dialect for FoldDialect<'_> { }; // Then the epoch, BEFORE a fused expression is lowered and // before the fold, like SUM's arm. - if epoch != self.epoch { - return Err(FoldError::StaleSlot); - } - let mask = self.operand_of(s)?; + let mask = self.resolve_mask(m)?; let terminal = match via { (key, None) => Terminal::GroupSumI32 { mask, key, val }, (key, Some(fk)) => Terminal::GroupSumViaI32 { mask, fk, key, val }, @@ -1004,14 +996,9 @@ impl Dialect for FoldDialect<'_> { // Arity 1, pushes nothing: the mask itself is the demanded result, // written into the caller's `keep_sink` by `Terminal::Keep`. KEEP => { - let mask = stack.pop().ok_or(FoldError::Underflow(f))?; - let Val::Slot { slot, epoch } = mask else { - return Err(FoldError::WrongOperandKind(f)); - }; - if epoch != self.epoch { - return Err(FoldError::StaleSlot); - } - let mask = self.operand_of(slot)?; + let m = fold_mask(stack.pop().ok_or(FoldError::Underflow(f))?) + .ok_or(FoldError::WrongOperandKind(f))?; + let mask = self.resolve_mask(m)?; match self.finalize_and_run(Terminal::Keep { mask }, OutKind::Keep)? { Value::Mask(_) => Ok(()), other => panic!("KEEP terminal shape guarantees Mask, got {other:?}"), @@ -1021,3 +1008,18 @@ impl Dialect for FoldDialect<'_> { } } } + +/// A fold's mask operand, by shape only: a computed slot or a resident plane. +#[derive(Clone, Copy)] +enum FoldMask { + Slot { slot: u16, epoch: u32 }, + Plane(u16), +} + +fn fold_mask(v: Val) -> Option { + match v { + Val::Slot { slot, epoch } => Some(FoldMask::Slot { slot, epoch }), + Val::Address(Addr::Plane(p)) => Some(FoldMask::Plane(p)), + _ => None, + } +} diff --git a/crates/lance-graph-r2il/src/procs.rs b/crates/lance-graph-r2il/src/procs.rs index c9d1963ab..051039f4e 100644 --- a/crates/lance-graph-r2il/src/procs.rs +++ b/crates/lance-graph-r2il/src/procs.rs @@ -15,7 +15,9 @@ use ogar_loco::{Dialect, FnIndex}; use ogar_r2il::{GROUP_SUM, KEEP, SUM}; -use crate::{FoldDialect, FoldError, Val, INT_SUB, INT_S_LESS, LOAD}; +use crate::{ + FoldDialect, FoldError, Val, INT_AND, INT_EQUAL, INT_NOT, INT_SUB, INT_S_LESS, LOAD, POP_COUNT, +}; /// A selection that a procedure re-emits once per fold (see [`margin`]). pub type Select<'s> = dyn Fn(&mut FoldDialect<'_>, &mut Vec) -> Result<(), FoldError> + 's; @@ -24,6 +26,8 @@ pub type Select<'s> = dyn Fn(&mut FoldDialect<'_>, &mut Vec) -> Result<(), const SPACE_U32: u8 = 0; /// `LOAD` space for an `I32` lane. const SPACE_I32: u8 = 1; +/// `LOAD` space for a resident mask plane. +const SPACE_PLANE: u8 = 3; fn call( d: &mut FoldDialect<'_>, @@ -73,6 +77,38 @@ pub fn filter_lt( call(d, KEEP, [0; 3], stack) } +/// The rows of the resident plane `live` that pass every condition: each +/// `(lane, v, negate)` is `U32 lane = v`, or `!=` when `negate`. Pushes the +/// mask. +/// +/// Calls: `LOAD` (plane), then per condition `LOAD`, `IntEqual`, [`IntNot`,] +/// `IntAnd`. +pub fn select_u32( + d: &mut FoldDialect<'_>, + stack: &mut Vec, + live: u16, + conds: &[(u16, u32, bool)], +) -> Result<(), FoldError> { + load(d, stack, SPACE_PLANE, live)?; + for &(lane, v, negate) in conds { + load(d, stack, SPACE_U32, lane)?; + stack.push(Val::Scalar(i64::from(v))); + call(d, INT_EQUAL, [0; 3], stack)?; + if negate { + call(d, INT_NOT, [0; 3], stack)?; + } + call(d, INT_AND, [0; 3], stack)?; + } + Ok(()) +} + +/// How many rows the mask on top of the stack holds. Pushes the count. +/// +/// Calls: `PopCount`. +pub fn count(d: &mut FoldDialect<'_>, stack: &mut Vec) -> Result<(), FoldError> { + call(d, POP_COUNT, [0; 3], stack) +} + /// `SUM(val_lane | mask)` for the mask on top of the stack. Pushes the sum. /// /// Calls: `LOAD`, `SUM`. diff --git a/crates/lance-graph-r2il/tests/procs.rs b/crates/lance-graph-r2il/tests/procs.rs index dc39944eb..366ee1636 100644 --- a/crates/lance-graph-r2il/tests/procs.rs +++ b/crates/lance-graph-r2il/tests/procs.rs @@ -5,7 +5,7 @@ //! selection, a partial one and a full one. use lance_graph_mask_risc::{words_for, Foreign, LaneRef, MaskOp, Planes}; -use lance_graph_r2il::procs::{filter_lt, group_sum, margin, select_lt, sum}; +use lance_graph_r2il::procs::{count, filter_lt, group_sum, margin, select_lt, select_u32, sum}; use lance_graph_r2il::{ BoolMode, FoldDialect, FoldError, Val, INT_AND, INT_NOT, INT_OR, POP_COUNT, }; @@ -81,6 +81,44 @@ fn filter_lt_keeps_the_rows_the_reference_keeps() { } } +/// FAILS IF: `select_u32` stops keeping exactly the live rows that pass every +/// condition. The live plane drops every seventh row, so a selection that +/// ignores the plane counts too many; `!=` and `=` are both exercised. +#[test] +fn select_u32_counts_live_rows_passing_every_condition() { + let f = fixture(); + let lanes = lanes(&f); + let mut live = vec![0u64; words_for(N)]; + for r in (0..N).filter(|r| r % 7 != 0) { + live[r / 64] |= 1 << (r % 64); + } + let planes = Planes { + n_rows: N, + masks: &[live.as_slice()], + lanes: &lanes, + }; + type Case = (&'static [(u16, u32, bool)], fn(usize) -> bool); + let cases: [Case; 4] = [ + (&[], |_| true), + (&[(KEY, 3, false)], |k| k == 3), + (&[(KEY, 3, true)], |k| k != 3), + (&[(KEY, 3, true), (KEY, 1, true)], |k| k != 3 && k != 1), + ]; + for (conds, keep) in cases { + let want = (0..N) + .filter(|&r| r % 7 != 0 && keep(f.key[r] as usize)) + .count() as i64; + let mut d = FoldDialect::new(&planes, &Foreign::NONE); + let mut stack = vec![]; + select_u32(&mut d, &mut stack, 0, conds).expect("select"); + count(&mut d, &mut stack).expect("count"); + assert_eq!(stack, vec![Val::Scalar(want)], "{conds:?}"); + assert_eq!(d.programs_run(), 1, "one fold"); + } + // Non-vacuity: the plane removes rows that pass the conditions. + assert!((0..N).any(|r| r % 7 == 0 && f.key[r] == 3)); +} + /// FAILS IF: `sum` stops matching `SUM(revenue | x < t)`. #[test] fn sum_matches_the_reference() { @@ -339,3 +377,52 @@ fn plane_and_plane_is_accepted_in_both_modes() { assert_eq!(stack, vec![Val::Scalar(want)], "{mode:?}"); } } + +/// FAILS IF: a fold refuses, or misreads, a resident plane as its mask. +/// A body with no condition hands the fold the live plane itself, not a +/// computed slot; SUM, GROUP_SUM and KEEP must read it where it stands, as +/// POP_COUNT does. +#[test] +fn every_fold_takes_a_resident_plane_as_its_mask() { + let f = fixture(); + let lanes = lanes(&f); + let mut live = vec![0u64; words_for(N)]; + for r in (0..N).filter(|r| r % 7 != 0) { + live[r / 64] |= 1 << (r % 64); + } + let planes = Planes { + n_rows: N, + masks: &[live.as_slice()], + lanes: &lanes, + }; + let kept: Vec = (0..N).filter(|r| r % 7 != 0).collect(); + let plane = || vec![Val::Address(lance_graph_r2il::Addr::Plane(0))]; + + let mut d = FoldDialect::new(&planes, &Foreign::NONE); + let mut stack = plane(); + sum(&mut d, &mut stack, REVENUE).expect("sum over a plane"); + let want: i64 = kept.iter().map(|&r| i64::from(f.revenue[r])).sum(); + assert_eq!(stack, vec![Val::Scalar(want)]); + + let mut want = [0i64; GROUPS]; + for &r in &kept { + want[f.key[r] as usize] += i64::from(f.revenue[r]); + } + let mut sink = [0i64; GROUPS]; + { + let mut d = FoldDialect::with_groups(&planes, &Foreign::NONE, &mut sink); + let mut stack = plane(); + group_sum(&mut d, &mut stack, KEY, REVENUE).expect("group_sum over a plane"); + } + assert_eq!(sink, want); + + let mut out = vec![0u64; words_for(N)]; + { + let mut d = FoldDialect::with_keep(&planes, &Foreign::NONE, &mut out); + let mut stack = plane(); + d.call(KEEP, [0; 3], &mut stack).expect("keep a plane"); + } + assert_eq!(rows(&out), kept); + // Non-vacuity: the plane drops rows, so reading every row would differ. + assert!(kept.len() < N); +}