relalg (D-RA-1): relational IR rewritten before it runs, lowered through R2IL — a D-CMR prototype

.claude/board/INTEGRATION_PLANS.md

@@ -1,3 +1,11 @@
+## 2026-10-13 (2) — relalg-v1 — relational algebra over coordinate populations, rewritten before it runs → `.claude/plans/relalg-v1.md`
+
+**Status:** D-RA-1 MEASURED (prototype crate); D-RA-2..4 PROPOSAL. A prototype for D-CMR, not a competing IR.
+
+- `Rel` describes a row set of one table; `normalize` applies De Morgan, unit/zero, complement, value-interval fusion per lane, interval union and absorption to a fixpoint; `∅` and (for a count) `U` are answered without a program; everything else lowers through R2IL over `FoldDialect`.
+- Measured: == row oracle on 400 generated descriptions × 5 measures; four comparisons on one lane 8 → 2 mask-risc ops; 70/2,000 small descriptions do not fit one fold as written (each negation takes a scratch slot), all fit after the rewrite.
+- D-RA-2 moves the rules into the contract set IR (D-CMR-1/-6); Datalog rules as algebra values with names only as codebook labels (D-RA-3) and bound recursion (D-RA-4) wait for it.
+
 ## 2026-10-13 (1) — d-cmr-concept-chain-normalization-v1 — preserve the equation, not the intermediate result → `.claude/plans/d-cmr-concept-chain-normalization-v1.md`
 
 **Status:** PROPOSAL D-CMR-0..9, revised to a set-theory IR (see the last bullet). Only D-CMR-0 (plan, board rows) is in the PR; no code, no dependency.

.claude/board/STATUS_BOARD.md

@@ -1,3 +1,14 @@
+## D-RA — Relational algebra over coordinate populations, a D-CMR prototype (2026-10-13)
+
+Plan: `.claude/plans/relalg-v1.md`. Crate: `crates/lance-graph-relalg` (own workspace root, excluded). Prototype for D-CMR-6 rules over residual comparisons and a lowering through R2IL.
+
+| D-id | scope | status | gate / falsifier |
+|---|---|---|---|
+| **D-RA-1** | `Rel` + `normalize` + `symbolic` + R2IL lowering | In PR (10 tests, 8 disable runs red; four comparisons on one lane 8 → 2 ops; 70/2,000 small descriptions refused as written, all run rewritten) | == row oracle over 400 generated descriptions × 5 measures |
+| **D-RA-2** | Rules into the contract set IR (D-CMR-1/-6); relalg keeps only lowering or is deleted | Queued (maintainer decision) | relalg tests pass unchanged against the contract IR |
+| **D-RA-3** | Datalog rules as algebra values; names are labels resolved through the label codebook, never strings in a rule | Queued (after D-RA-2) | codebook-built rules == hand-written `Rel`; unknown label refused at resolution |
+| **D-RA-4** | Bound recursion via external frontier driver over D-CMR-5 preimage | Queued (after D-CMR-5) | == row-oracle closure; frontier past bound refused |
+
 ## D-CMR — Concept-chain normalization: preserve the equation, not the intermediate result (2026-10-13)
 
 Plan: `.claude/plans/d-cmr-concept-chain-normalization-v1.md`. Parent: D-CMR-RESET. One new IR (set theory, plan §3.1 insufficiency proof); no method-body algebra, no dependency.

.claude/plans/relalg-v1.md

@@ -0,0 +1,72 @@
+# relalg-v1 — relational algebra over coordinate populations, rewritten before it runs
+
+**Status:** D-RA-1 MEASURED (prototype crate `crates/lance-graph-relalg`, own
+workspace root, excluded). D-RA-2..4 PROPOSAL.
+**Date:** 2026-10-13. **Repo:** lance-graph.
+**Relation to D-CMR:** a prototype for
+`d-cmr-concept-chain-normalization-v1.md`. It tests the rule catalogue (D-CMR-6)
+on residual comparisons and a lowering through R2IL (D-CMR-2's job, via the
+route NAL uses). It is not a third frontend syntax and not the home of the set
+algebra; D-CMR puts that in `lance-graph-contract` (D-CMR-1).
+
+## 1. What D-RA-1 built
+
+- `Rel`: a set of rows of one table, described (`All`, `Empty`, a resident
+  plane, one comparison on one `I32` or `U32` lane, `And` / `Or` / `Not`), and
+  a `Measure` (`Count`, `Sum`, `Min`, `Max`, `Keep`).
+- `normalize`: to a fixpoint — negation to the leaves; flattening, unit and
+  zero; duplicates; `p ∧ ¬p = ∅`, `p ∨ ¬p = U`; per `I32` lane in a
+  conjunction, one value interval plus the excluded values inside it, `∅` when
+  empty; per lane in a disjunction, overlapping or adjacent intervals merged,
+  `U` when they cover the lane; `U32` equalities agree or contradict;
+  absorption. Intervals are emitted as `<` / `>` / `==`, the forms
+  `IntSLess` / `IntEqual` lower without a negation pass.
+- `symbolic`: `∅` answers `Count 0`, `Sum 0`, `Min/Max None`; `U` answers
+  `Count n_rows`. No program runs.
+- `run`: R2IL calls over `FoldDialect` (`LOAD`, `IntSLess`, `IntEqual`,
+  `IntAnd`, `IntOr`, `IntNot`, `RANGE`, then `PopCount` / `SUM` / `MIN` /
+  `MAX` / `KEEP`). mask-risc executes; the crate has no row loop.
+
+In D-CMR terms every comparison here is a residual: the lanes are not a proven
+row ordering, so no row interval (`Runs`) is formed. The interval work is
+constraint propagation over value intervals of one lane (§3.5's first
+mechanism), not geometry.
+
+## 2. Measured (`cargo test --manifest-path crates/lance-graph-relalg/Cargo.toml`)
+
+| test | result |
+|---|---|
+| 400 generated descriptions × 5 measures: rewritten == as written == row oracle, idempotent | 273/400 rewritten; 70 of 2,000 as-written runs refused (below) |
+| four comparisons on one lane | rewritten 2 mask-risc ops, as written 8 |
+| `x=3 ∨ x=4 ∨ x=5` | rewritten 2 ops, as written 5 |
+| contradictions (`x<5 ∧ x>10`, `k=1 ∧ k=2`, `k=1 ∧ k≠1`, `p ∧ ¬p`, `x<MIN`, `x>MAX`) | answered with 0 programs, all four scalar measures |
+| tautologies (`x<5 ∨ x≥5`, `p ∨ ¬p`, `x≤MAX`, `x≥MIN`) | `Count = n_rows`, 0 programs |
+| ten comparisons on one lane | as written refused (`ScratchBufferTooSmall`), rewritten runs |
+
+Eight disable runs, each red: fusing across lanes, no symbolic answer, no
+absorption, union ignoring adjacency, no end tightening, `≤` negated to `≥`,
+no complement check, `<` off by one.
+
+**Finding:** normalization decides whether a description fits one fold, not
+only how fast it runs. `FoldDialect` gives one fold 8 scratch slots and every
+negation as written takes a fresh one, so 70 of 2,000 small descriptions (at
+most three leaves) do not lower as written at all. After the rewrite every one
+of them runs.
+
+## 3. Path
+
+| D-id | step | gate / falsifier |
+|---|---|---|
+| D-RA-1 | Prototype: `Rel`, `normalize`, `symbolic`, R2IL lowering | == row oracle on generated descriptions; each guard removal fails |
+| D-RA-2 | Move the rules into the contract set IR (D-CMR-1/-6) with a universe per set; relalg keeps only the R2IL lowering, or is deleted | relalg's tests pass unchanged against the contract IR |
+| D-RA-3 | Datalog rules as algebra values, not text: a rule is a `Rel` built from coordinates (lane indexes, numeric constants, classids). Names are labels only, resolved through the label codebook to coordinates once; no string is stored in a rule or used as an address. A text rendering is a label view of the value, never a query path | rules built from codebook coordinates == hand-written `Rel`; a rule naming an unknown label is refused at resolution, never kept as a string |
+| D-RA-4 | Bound recursion: `ancestor(c, Y)` as a frontier fixpoint in an external driver (option d′ of `entries/2026-10-03-coresearch-chained-hop.md`), each step a D-CMR-5 preimage; unbound closure refused | == row oracle closure; a step that grows the frontier past its bound is refused, never truncated |
+
+D-RA-2 is a decision for the maintainer (see the PR). D-RA-3 and D-RA-4 wait
+for it.
+
+## 4. Not built
+
+Joins (a hop is a preimage over coordinates, D-CMR-5, never a foreign key),
+recursion, row intervals, `Sum(A) − Sum(A∧B)`. Datalog text as a query path
+is excluded by design: strings are labels, the query is algebra.

Cargo.toml

@@ -111,6 +111,10 @@ exclude = [
     # needs the OGAR checkout too. Own workspace root; verify via
     # `cargo test --manifest-path crates/lance-graph-nal/Cargo.toml`.
     "crates/lance-graph-nal",
+    # Relational algebra over coordinate populations (D-RA-1): path-deps
+    # lance-graph-r2il like lance-graph-nal. Own workspace root; verify via
+    # `cargo test --manifest-path crates/lance-graph-relalg/Cargo.toml`.
+    "crates/lance-graph-relalg",
     "crates/symbiont",
     # OGAR (Open Graph of Active Record) activation crate — re-exports OGAR's full
     # AR surface (ogar-vocab Class/codebook + ogar-class-view impl ClassView +

crates/lance-graph-relalg/Cargo.toml

@@ -0,0 +1,26 @@
+[package]
+name = "lance-graph-relalg"
+version = "0.1.0"
+edition = "2021"
+publish = false
+description = "Relational algebra over coordinate populations: an IR that describes a query, proven rewrites before anything runs, lowering through R2IL into one mask-risc program per fold (D-RA-1)."
+
+# Its OWN workspace root, like `lance-graph-nal`: it path-deps the r2il crate,
+# which git-deps OGAR. Verify via
+# `cargo test --manifest-path crates/lance-graph-relalg/Cargo.toml`.
+[workspace]
+
+[dependencies]
+# The only bulk engine.
+lance-graph-mask-risc = { path = "../lance-graph-mask-risc" }
+# The lowering target: queries become R2IL call sequences over FoldDialect.
+lance-graph-r2il = { path = "../lance-graph-r2il" }
+# By git `main`, the coordinate `lance-graph-r2il` uses (one `ogar-loco`).
+ogar-loco = { git = "https://github.com/AdaWorldAPI/OGAR", branch = "main" }
+ogar-r2il = { git = "https://github.com/AdaWorldAPI/OGAR", branch = "main" }
+
+[patch."https://github.com/AdaWorldAPI/lance-graph"]
+lance-graph-contract = { path = "../lance-graph-contract" }
+
+[patch."https://github.com/AdaWorldAPI/ndarray"]
+ndarray = { path = "../../../ndarray" }

crates/lance-graph-relalg/src/ir.rs

@@ -0,0 +1,119 @@
+//! The IR. It describes a selection over one table's row space; it never
+//! holds rows and never holds a string: lanes are indexes, constants are
+//! numbers. A name a person reads is a label, resolved through the label
+//! codebook before a description is built, never part of it.
+
+// No string in the IR: an atom and a measure are `Copy`, which a `String`
+// field would break at compile time.
+const _: fn() = || {
+    fn copy<T: Copy>() {}
+    copy::<Atom>();
+    copy::<Cmp>();
+    copy::<Measure>();
+};
+
+/// An ordered comparison of an `I32` lane against a constant.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub enum Cmp {
+    /// `lane < t`
+    Lt,
+    /// `lane <= t`
+    Le,
+    /// `lane > t`
+    Gt,
+    /// `lane >= t`
+    Ge,
+    /// `lane == t`
+    Eq,
+    /// `lane != t`
+    Ne,
+}
+
+/// One predicate on one lane of the table.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub enum Atom {
+    /// `lanes[lane] op t` on an `I32` lane.
+    I32 { lane: u16, op: Cmp, t: i32 },
+    /// `lanes[lane] == v` (`eq`) or `!= v` (`!eq`) on a `U32` lane.
+    U32 { lane: u16, eq: bool, v: u32 },
+}
+
+/// A set of rows of the table, described, not computed.
+///
+/// `And` is intersection (σ of a conjunction, ∩), `Or` is union (∪), and
+/// `And[a, Not b]` is difference (−). Projection and aggregation are the
+/// [`Measure`] a [`Query`] ends with.
+#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub enum Rel {
+    /// Every row.
+    All,
+    /// No row.
+    Empty,
+    /// A resident mask plane, `Planes::masks[i]`.
+    Plane(u16),
+    /// The rows where one predicate holds.
+    Atom(Atom),
+    /// Intersection.
+    And(Vec<Rel>),
+    /// Union.
+    Or(Vec<Rel>),
+    /// Complement within the table.
+    Not(Box<Rel>),
+}
+
+impl Rel {
+    /// `a − b`.
+    pub fn diff(a: Rel, b: Rel) -> Rel {
+        Rel::And(vec![a, Rel::Not(Box::new(b))])
+    }
+
+    /// `lanes[lane] op t` on an `I32` lane.
+    pub fn i32(lane: u16, op: Cmp, t: i32) -> Rel {
+        Rel::Atom(Atom::I32 { lane, op, t })
+    }
+
+    /// `lanes[lane] == v` on a `U32` lane.
+    pub fn eq_u32(lane: u16, v: u32) -> Rel {
+        Rel::Atom(Atom::U32 { lane, eq: true, v })
+    }
+
+    /// `lanes[lane] != v` on a `U32` lane.
+    pub fn ne_u32(lane: u16, v: u32) -> Rel {
+        Rel::Atom(Atom::U32 { lane, eq: false, v })
+    }
+
+    /// How many predicates the description names (a lowering emits one
+    /// predicate pass per atom).
+    pub fn atoms(&self) -> usize {
+        match self {
+            Rel::Atom(_) => 1,
+            Rel::And(cs) | Rel::Or(cs) => cs.iter().map(Rel::atoms).sum(),
+            Rel::Not(c) => c.atoms(),
+            _ => 0,
+        }
+    }
+}
+
+/// What a query produces from its rows.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Measure {
+    /// How many rows.
+    Count,
+    /// Σ of an `I32` lane.
+    Sum(u16),
+    /// Minimum of an `I32` lane.
+    Min(u16),
+    /// Maximum of an `I32` lane.
+    Max(u16),
+    /// The rows themselves, written into a caller-owned mask.
+    Keep,
+}
+
+/// A selection and what to produce from it.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Query {
+    /// Which rows.
+    pub rel: Rel,
+    /// What to produce.
+    pub measure: Measure,
+}

crates/lance-graph-relalg/src/lib.rs

@@ -0,0 +1,37 @@
+//! `lance-graph-relalg` — relational algebra over coordinate populations
+//! (D-RA-1, `.claude/plans/relalg-v1.md`).
+//!
+//! A PROTOTYPE for D-CMR (`.claude/plans/d-cmr-concept-chain-normalization-v1.md`):
+//! rearrange the description of a query with proven identities before anything
+//! runs, and lower only the irreducible remainder.
+//!
+//! # What it holds
+//!
+//! - [`Rel`]: a set of rows of one table, described — `All`, `Empty`, a
+//!   resident plane, a comparison on one lane, `And` / `Or` / `Not`. In D-CMR
+//!   terms every comparison is a residual (`Res`): the lanes are not a proven
+//!   row ordering, so no row interval (`Runs`) is formed here.
+//! - [`normalize()`]: the rewrites, to a fixpoint (see [`mod@normalize`]'s
+//!   module doc). The interval work is constraint propagation over VALUE intervals of
+//!   one lane — narrow bounds, detect `∅` — not row intervals.
+//! - [`symbolic`]: a description that normalizes to `Empty` (or `All`, for a
+//!   count) is answered without running a program.
+//! - [`run`]: lowering into R2IL calls over
+//!   [`FoldDialect`](lance_graph_r2il::FoldDialect), which builds and runs one
+//!   mask-risc program per fold. This crate never loops over rows.
+//!
+//! # What it is not
+//!
+//! Not a third frontend syntax beside Quack's `Filter` and NAL's R2IL
+//! programs, and not the home of the rule catalogue: D-CMR puts the set
+//! algebra in `lance-graph-contract`. The rules and tests here are the
+//! evidence for that move; recursion (`μ`) and the hop (preimage) are not
+//! built.
+
+pub mod ir;
+pub mod lower;
+pub mod normalize;
+
+pub use ir::{Atom, Cmp, Measure, Query, Rel};
+pub use lower::{run, run_as_written, Answer, RelError, Run};
+pub use normalize::{normalize, symbolic};