Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ Plan: `.claude/plans/gremlin-composable-algebra-v1.md`.
| D-CMH-G1-3 | Gremlin-shaped runtime declarations | Shipped (#1508) | loads after the build; effects refused at load |
| D-CMH-G1-4a | functional relation carrier: `out(rel)`, `out(rel).has`, `count()` | Shipped (#1514; ConflictingUniverse fix follows) | row loop over the text at n ∈ {1,64,65,4097}; 6 disable runs red; R2IL refuses (no `GATHER`) |
| D-CMH-G1-4b | edge table as the anchor (`E()`, M:N one hop, `outE/inV`) + `Fold::Distinct` (`out(rel).dedup().count()`) | In PR | bag and set == row loop over the text at n ∈ {1,64,65,4097}; unordered key lane refused (`LaneNotOrdered`); 7 disable runs red |
| D-CMH-G1-4 | rest: a second hop (blocked on the chained-hop decision), `V(parent).in(rel)` spelling, `bothE` | Queued | — |
| D-CMH-G1-7 | chained hops folded: `MaskOp::GatherChain` + Quack `Filter::Chain` + glove `out(r1).out(r2)…` (one op, tests at every depth, no hop mask) | In PR (#1522; needs ndarray#358) | executor == oracle == row loop, gated/ungated, tiles; 0-byte L1; two-hop methods == row loop over text; 13 disable runs red |
| D-CMH-G1-4 | rest: `V(parent).in(rel)` spelling, `bothE` | Queued | — |
| D-CMH-G1-5 | FirstOfRun | Queued | insufficiency proof first; Quack + R2IL vs independent oracle |
| D-CMH-G1-6 | ordered `I32` conditions + `MIN`/`MAX` folds as method steps; R2IL `MIN`/`MAX` wired | In PR | both frontends == row loop incl. empty selection; 8 disable runs red |

Expand Down
36 changes: 36 additions & 0 deletions .claude/board/entries/2026-10-12-chained-hops-folded.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# 2026-10-12 — D-CMH-G1-7: chained hops are folded hops (mask-risc + Quack)

**Status:** In PR (needs AdaWorldAPI/ndarray#358). **Plan:** `.claude/plans/gremlin-composable-algebra-v1.md`.

`MaskOp::GatherChain { lane, chain, under, dst }` with `Program::chains`:
a key leads through every hop of the chain, each hop's reached row live in
its RESIDENT plane and passing its tests read in place, each hop's `next`
key naming the following hop's row. One op, one terminal, no mask over any
hop table, no program feeding another. Realised by
`ndarray::simd::mask_gather_chain_u32[_under]`. Quack's `Filter::Chain`
lowers to exactly one such op in both lowerings.

WHY a new op: `Gather` tests a target only through a plane over the
target's table, so a depth-2 predicate needed a depth-1 mask built first
and a `Keep` fed into another program's `Gather`, the shape the
`ScatterOrU32` survival condition forbids. (#1521, which built that
per-hop shape, was closed.)

MEASURED: `mask-risc/tests/chain.rs` (executor, oracle and a fixture row
loop agree gated and ungated across tile boundaries; one bare hop equals
`Gather`; eight refusals identical on both paths); `no_alloc.rs` (1000
gated two-hop executes, 0 bytes); Quack
`chain_folds_to_one_gather_chain_and_matches_a_per_row_reading`. Seven
disable runs red (gate dropped, tests dropped, oracle negate, chain shape
check, lowering negate, lowering to `Gather`, oracle stopping at depth 1).

DECISION: chains are stack-bounded (`MAX_CHAIN_HOPS = 8`,
`MAX_HOP_TESTS = 8`) so `execute` keeps L1 (no allocation). REVISIT WHEN: a
real query needs a deeper chain.

Glove (plan §11): `Path` + `Cond::ReachesPath` / `ViaPath`; every
condition on a path and its prefixes binds into one chain; Gremlin `out`
after `out` extends the path. `tests/two_hop.rs` against a row loop over the
text; six more disable runs red.

OPEN: the M:N two-edge-table hop (run-level OR) awaits a decision.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

319 entries, 2026-08-06 .. 2026-10-12.
320 entries, 2026-08-06 .. 2026-10-12.

| date | entry id | finding | file |
|---|---|---|---|
Expand All @@ -38,6 +38,7 @@ exactly one of them, never both.
| 2026-10-12 | `D-CMH-G1-4b` | | [2026-10-12-edge-table-anchor-distinct.md](2026-10-12-edge-table-anchor-distinct.md) |
| 2026-10-12 | `D-CMH-0` | Class × Schema × Methods activation connects HotPlug, WideFieldMask, Quack and R2IL without a second registry | [2026-10-12-d-cmh-0-class-method-hotplug-activation.md](2026-10-12-d-cmh-0-class-method-hotplug-activation.md) |
| 2026-10-12 | `D-CMH-1` | | [2026-10-12-class-method-polyfill-ledger.md](2026-10-12-class-method-polyfill-ledger.md) |
| 2026-10-12 | `D-CMH-G1-7` | | [2026-10-12-chained-hops-folded.md](2026-10-12-chained-hops-folded.md) |
| 2026-10-12 | `D-HL-0` | CE64 hypothesis layers: plateau of the design (opaque LE bytes, 64 sparse-delta layers merged on read, Narsese-parallel validation), 8 design-vs-code conflicts, 9 unification hypotheses | [2026-10-12-ce64-hypothesis-layers-plateau.md](2026-10-12-ce64-hypothesis-layers-plateau.md) |
| 2026-10-11 | `D-TELEPORT-0` | | [2026-10-11-teleport-probe-source-restriction.md](2026-10-11-teleport-probe-source-restriction.md) |
| 2026-10-11 | `synergy-ledger-classid-hotplug` | | [2026-10-11-synergy-ledger-classid-hotplug.md](2026-10-11-synergy-ledger-classid-hotplug.md) |
Expand Down
45 changes: 40 additions & 5 deletions .claude/plans/gremlin-composable-algebra-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Rows group steps of one mechanism, so percentages are of rows, not of step names
| in(rel) reverse fk | traversal | V->V' | re-anchor on child rows | — | pure | DIRECT (G1-4b) | spelled `V(child).out(rel)`, anchored on the edge or child concept (tests/distinct.rs); the `V(parent).in(rel)` spelling is still refused (`FanOut`) | composer re-anchoring of the `in` spelling |
| out/in(rel) M:N | traversal | V->V' | re-anchor on edge rows | — | pure | DIRECT (G1-4b) | `V(edge).where(p()).out(end)`: `count()` = bag, `dedup().count()` = set, both ends filtered at the source; anchored on the edge or child concept (tests/distinct.rs) | — |
| both(rel) | traversal | V->V' | union of two bags | — | pure | YET | no bag union across anchors | typed bag merge |
| second hop, functional | traversal | V->V'' | keeps | — | pure | YET | #1311 `functional_two_hop_is_refused…` | composed fk read (G1) |
| second hop, functional | traversal | V->V'' | keeps | — | pure | DIRECT (G1-7) | `out(r1)…out(rk)` folds into ONE `MaskOp::GatherChain` (§11): tests/two_hop.rs vs a row loop over the text; R2IL refuses (no `GATHER`) | R2IL `GATHER` |
| second fan-out (fan-in carried) | traversal | V->V'' | bag | — | pure | YET | #1311 `many_to_many_two_hop_is_refused…`; #1312 §D | barrier / fan-in operand (G2) |
| outE/inE/bothE | traversal | V->E | edge rows | — | pure | DIRECT (G1-4b) | `V(edge).where(p())` with `p` over `out(src)`; anchored on the edge or child concept (tests/distinct.rs); `bothE` needs the union below | — |
| outV/inV/otherV/bothV | traversal | E->V | fk read | — | pure | DIRECT (G1-4b) | `out(src)` / `out(dst)` from `V(edge)`: EqU32Via + liveness Semijoin; anchored on the edge or child concept (tests/distinct.rs) | — |
Expand Down Expand Up @@ -250,7 +250,7 @@ adapter, minus the relations that fan out.
test is emitted once per relation, wherever it is first named.
- `count()` after `out(rel)` counts anchor rows: a functional relation yields
at most one traverser per anchor row, so the anchor does not move.
- Refused by name: a second hop (`SecondHop`), `values(f)` on the target
- Refused by name (at §8; a second hop composes since §11): `values(f)` on the target
(`ValueThroughRelation`; no sum through a key in Quack), `dedup()` on the
target (`DedupThroughRelation`; a set count), `where(method())` on the
target (`WhereThroughRelation`), and `in`/`both` (`FanOut`).
Expand Down Expand Up @@ -330,8 +330,7 @@ Composer: `dedup()` after `out(rel)` moves to the distinct targets; `has`
commutes with it; `count()` folds; ending there, or any other step, stays
`DedupThroughRelation`. `V(parent).in(rel)` is still refused (`FanOut`); the
same population is spelled `V(child).out(rel)`. R2IL: `FrontendGap { R2il,
"GATHER" }`. A second hop stays refused: it needs the chained-hop decision
(`.claude/board/entries/2026-10-03-coresearch-chained-hop.md`).
"GATHER" }`. A second hop composes since §11.

Evidence: `tests/distinct.rs` (6 tests). FIXTURE: synthetic `method`,
`field` and `reads` tables, dangling keys and dead rows on each, two
Expand All @@ -342,6 +341,41 @@ test dropped, the test emitted twice, the fold lowered as `COUNT(*)`, the
R2IL refusal dropped, `has` after `dedup` dropped, ending on the distinct
targets accepted, the key left out of `required`.

## §11 — chained hops are folded hops (D-CMH-G1-7)

`out(r1).out(r2)…` over functional relations keeps the anchor and lowers to
ONE chained gather: `Filter::Chain` → `MaskOp::GatherChain` →
`ndarray::simd::mask_gather_chain_u32[_under]`. A condition on any hop is a
test read in place at the row the key reached; no mask over a hop table is
built and no program feeds another (the photolithography pattern: each hop
is one more aperture in the same pass, materialised once at the terminal).

Method layer: `Path` (1..=`MAX_PATH` = 8 relations) and
`Cond::ReachesPath` / `Cond::ViaPath` for a path of two or more; a one-hop
path is still `Cond::Reaches` / `Cond::Via`. Preparation walks each
relation on the previous one's target glove; every condition on a maximal
path and on its prefixes binds into one `BoundCond::Chain` (hop-1 tests
folded into hop 1, never a separate semijoin). The next key must be a
proven `Code` field of the hop table that the call supplies as a foreign
lane (`NoForeignLane` otherwise). Every target universe along the path is
checked at the call.

Composer: `out` after `out` extends the path (`PathTooDeep` past
`MAX_PATH`); `has` after it is a `ViaPath`; `dedup`, `values`, `min`/`max`
after a path of two are refused. Text form and awareness graph carry the
path.

Evidence: `tests/two_hop.rs` (5 tests): `line -billed_to-> partner
-located_in-> region`, dangling keys and dead rows at both depths, against a
row loop over the text at n ∈ {1, 64, 65, 4097}; the lowering is exactly one
chain; composer, text form, awareness, refusals. Six disable runs red:
prefix not absorbed, tests dropped, tests on the wrong hop, next key not
followed, only the first universe checked, the composer refusing the hop.
The executor side (mask-risc, Quack) has its own seven (board entry
`2026-10-12-chained-hops-folded.md`).

OPEN: the M:N two-edge-table hop (an edge concept per hop, run-level OR).

## D-ids

| D-id | scope | status |
Expand All @@ -352,6 +386,7 @@ targets accepted, the key left out of `required`.
| D-CMH-G1-3 | Gremlin-shaped runtime declarations | Shipped (#1508) |
| D-CMH-G1-4a | functional relation carrier: `out(rel)`, `out(rel).has`, `count()` after it (§8) | Shipped (#1514) |
| D-CMH-G1-4b | edge table as the anchor (`E()`, M:N, `outE/inV`, `in` spelled from the child), `Fold::Distinct` (§10) | In PR |
| D-CMH-G1-4 | the rest: a second hop (needs the chained-hop decision), the `V(parent).in(rel)` spelling, `bothE` | Queued |
| D-CMH-G1-4 | the rest: the `V(parent).in(rel)` spelling, `bothE` | Queued |
| D-CMH-G1-7 | chained hops folded: `out(r1).out(r2)…` → one `GatherChain` (§11) | In PR |
| D-CMH-G1-5 | FirstOfRun (insufficiency proof first; separate PR #3) | Queued |
| D-CMH-G1-6 | ordered `I32` conditions, `MIN`/`MAX` folds; R2IL `MIN`/`MAX` wired (§9) | In PR |
33 changes: 32 additions & 1 deletion crates/lance-graph-glove-parity/src/awareness.rs
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,15 @@ pub fn field_ident(concept: u16, seal: SchemaSeal, name: &str) -> String {
format!("field:{concept:#06x}/{:016x}/{name}", seal.0)
}

/// `out(a).out(b)…` for a relation path, as written.
fn outs(path: crate::method::Path<'_>) -> String {
path.rels()
.iter()
.map(|r| format!("out({r})"))
.collect::<Vec<_>>()
.join(".")
}

/// The primitive kind of a mask op, by variant.
#[must_use]
pub fn op_name(op: &MaskOp) -> String {
Expand All @@ -352,6 +361,7 @@ pub fn op_name(op: &MaskOp) -> String {
MaskOp::Not { .. } => "Not".into(),
MaskOp::Ternlog { .. } => "Ternlog".into(),
MaskOp::Gather { .. } => "Gather".into(),
MaskOp::GatherChain { .. } => "GatherChain".into(),
}
}

Expand Down Expand Up @@ -666,6 +676,25 @@ impl SoftwareGraph {
self.link(m, Rel::Filters, n, Tier::Declared, m, prov);
continue;
}
Cond::ReachesPath { path } => {
let n = self.intern(Kind::Cond, format!("cond:{}@r{r}", outs(path)));
self.link(m, Rel::Filters, n, Tier::Declared, m, prov);
continue;
}
Cond::ViaPath {
path,
field,
literal,
negate,
} => {
let op = if negate { "!=" } else { "=" };
let n = self.intern(
Kind::Cond,
format!("cond:{}.{field}{op}{literal}@r{r}", outs(path)),
);
self.link(m, Rel::Filters, n, Tier::Declared, m, prov);
continue;
}
};
let f = field(self, name);
let n = self.intern(Kind::Cond, format!("cond:{f}{op}{lit}@r{r}"));
Expand Down Expand Up @@ -1202,7 +1231,9 @@ fn gate_planes(p: &Program) -> Vec<u16> {
};
for op in &p.ops {
match *op {
MaskOp::Pred { under, .. } | MaskOp::Gather { under, .. } => {
MaskOp::Pred { under, .. }
| MaskOp::Gather { under, .. }
| MaskOp::GatherChain { under, .. } => {
if let Some(u) = under {
see(u);
}
Expand Down
45 changes: 44 additions & 1 deletion crates/lance-graph-glove-parity/src/decl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ use lance_graph_ogar::ogar_vocab::canonical_concept_id;
use crate::gremlin::{compose, GremlinRefusal, Pred, Step};
use crate::method::{
Cond, EffectClass, Fold, Glove, Law, MethodBody, MethodKey, MethodRef, MethodSpec, MethodTable,
Order, ReadingId, SchemaSeal,
Order, Path, ReadingId, SchemaSeal,
};

/// A method declared in text: a [`MethodSpec`] with owned storage.
Expand Down Expand Up @@ -73,6 +73,21 @@ enum DeclCond {
literal: String,
negate: bool,
},
/// `out(r1).out(r2)…`, two relations or more.
ReachesPath(Vec<String>),
/// `out(r1).out(r2)….has(field, literal)`.
ViaPath {
path: Vec<String>,
field: String,
literal: String,
negate: bool,
},
}

/// A loaded path as a [`Path`]; its length was checked when it was composed.
fn path_of(rels: &[String]) -> Path<'_> {
let rels: Vec<&str> = rels.iter().map(String::as_str).collect();
Path::try_of(&rels).expect("composed paths are 1..=MAX_PATH long")
}

#[derive(Debug, Clone, PartialEq, Eq)]
Expand Down Expand Up @@ -118,6 +133,20 @@ impl MethodDecl {
literal,
negate: *negate,
},
DeclCond::ReachesPath(path) => Cond::ReachesPath {
path: path_of(path),
},
DeclCond::ViaPath {
path,
field,
literal,
negate,
} => Cond::ViaPath {
path: path_of(path),
field,
literal,
negate: *negate,
},
})
.collect()
}
Expand Down Expand Up @@ -719,6 +748,20 @@ fn traversal(
literal: literal.to_owned(),
negate,
},
Cond::ReachesPath { path } => {
DeclCond::ReachesPath(path.rels().iter().map(|r| (*r).to_owned()).collect())
}
Cond::ViaPath {
path,
field,
literal,
negate,
} => DeclCond::ViaPath {
path: path.rels().iter().map(|r| (*r).to_owned()).collect(),
field: field.to_owned(),
literal: literal.to_owned(),
negate,
},
Cond::Method(MethodRef::Key(_)) => unreachable!("text names methods by label"),
})
.collect();
Expand Down
Loading
Loading