diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 194857f91..9bdceed36 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -8,7 +8,7 @@ Plan: `.claude/plans/software-awareness-graph-v1.md`. No new opcode, Quack varia | D-SAG-1-1 | typed schema + projection of prepared methods with tiers and provenance | In PR | `tests/awareness.rs` projection test; one identity, one kind | | D-SAG-1-2 | queries A–D through Quack; compiler verdicts as edges | In PR | answers == the programs; falsifiers 1–10, each disable-verified | | D-SAG-1-3 | H1 candidate + tier-labelled measurement; MDL measure with counterexample | In PR | `examples/awareness_loop.rs`; MDL test | -| D-SAG-1-4 | multi-hop dependency via relation carrier / factored `EqU32Via` | Queued (needs D-CMH-G1-4) | two-hop answer == closure computed from the programs | +| D-SAG-1-4 | multi-hop dependency: `SoftwareGraph::hops(start, path)`, one Quack program per hop, survivors passed as a value set (`in_u32`), never as a produced plane | In PR | 4 paths (up to 4 hops) == row loop; follows a reprojection; 4 disable runs red. Value-set transfer reads every edge row; skipping to survivors needs a Lance index or the order witness | | D-SAG-1-5 | G1 `CALLS` edges + method-reference refusals | In PR | `method_references_are_calls_edges_and_cycles_are_the_compilers`; callers via Quack; `MethodCycle` recorded, never re-derived | | D-SAG-1-6 | persist as a Lance dataset; source revision from git; canonical body digest | Queued | reload equals projection; toolchain change keeps digests | diff --git a/.claude/board/entries/2026-10-12-sag-multi-hop.md b/.claude/board/entries/2026-10-12-sag-multi-hop.md new file mode 100644 index 000000000..a09e31751 --- /dev/null +++ b/.claude/board/entries/2026-10-12-sag-multi-hop.md @@ -0,0 +1,20 @@ +# 2026-10-12 — D-SAG-1-4 — multi-hop questions over the software graph + +**Status:** In PR (stacked on #1519). + +`SoftwareGraph::hops(start, path)` walks `(relation, direction)` hops with set +semantics per hop. Each hop is one Quack program over the resident edge lanes, +`rel = R AND from IN (S)`, keeping its rows. The previous hop's survivors arrive +as a value set the size of those survivors; no hop reads a plane another +program produced. This is the operator's reading for chained hops: the masks +of one query fold into one materialisation, each hop filtered at the source. + +MEASURED (tests/awareness.rs): co-readers, field → claim, a four-hop claim +co-dependency and a four-hop shared-primitive path equal a row loop over the +edges; a hop path follows a reprojection; four disable runs red (relation +filter, direction, per-hop dedup, unknown start). + +OPEN: a value set still reads every edge row. Skipping straight to the +survivors' rows needs a Lance B-tree filter (D-TELEPORT-2 arm Db) or address +ranges over a key-ordered projection, which waits on +ISS-WITNESSED-RANGE-DOES-NOT-ATTEST-PLANE-ORDER. diff --git a/.claude/board/entries/2026-10-12-sag-reprojection-defects.md b/.claude/board/entries/2026-10-12-sag-reprojection-defects.md new file mode 100644 index 000000000..c45302ca2 --- /dev/null +++ b/.claude/board/entries/2026-10-12-sag-reprojection-defects.md @@ -0,0 +1,21 @@ +# 2026-10-12 — D-SAG-1R — SAG reprojection defects A and B, found by #1510 + +**Status:** In PR. **Found by:** #1510's audit of #1509. + +A: `SoftwareGraph::link` deduplicated on `(src, rel, dst, tier)`, so a fact two +methods share (the claim behind a field both read) belonged to whichever wrote +it first. When that method stopped reading the field, its reprojection +retracted the fact for both. `ctx` is now part of the key; `out` and +`capability` read each fact once. + +B: `project_compute` never retracted the DAG it replaced. Cyclic then acyclic +kept the `ComputeCycle` refusal and the stale `ComputesFrom` edges. It now +retracts its own class's compute edges and verdict first. + +MEASURED (tests/awareness.rs): for every method, reprojecting it with a body +that reads nothing gives the same facts as a fresh projection; three DAG +sequences give the expected edge count and verdict, the other class untouched. +Three disable runs red on the tests that name each guard. + +OPEN: C, `certify` accepts any test name. Who may issue a verified receipt is +an operator decision, not a code fix. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 5b445e0cb..6551453e8 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,11 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -317 entries, 2026-08-06 .. 2026-10-12. +319 entries, 2026-08-06 .. 2026-10-12. | date | entry id | finding | file | |---|---|---|---| | 2026-10-12 | `D-SAG-1` | | [2026-10-12-software-awareness-graph.md](2026-10-12-software-awareness-graph.md) | +| 2026-10-12 | `D-SAG-1R` | SAG reprojection defects A and B, found by #1510 | [2026-10-12-sag-reprojection-defects.md](2026-10-12-sag-reprojection-defects.md) | +| 2026-10-12 | `D-SAG-1-4` | multi-hop questions over the software graph | [2026-10-12-sag-multi-hop.md](2026-10-12-sag-multi-hop.md) | | 2026-10-12 | `opencog-pln-entropy-research-map` | Source inventory and falsifiable PLN/ECAN/Shannon/Moore attention comparison: entropy target, evidence overlap, graph projection, resource fairness | [2026-10-12-opencog-pln-entropy-research-map.md](2026-10-12-opencog-pln-entropy-research-map.md) | | 2026-10-12 | `nars-self-evidence-literature` | NARS internal trace versus evidence, Jøsang dependent/independent fusion and fission, and the snapshot-stable fold assumption; source-grounded red tests for merged #1496 | [2026-10-12-nars-self-evidence-literature.md](2026-10-12-nars-self-evidence-literature.md) | | 2026-10-12 | `hyperon-metta-lg-translation` | Hyperon/MeTTa to LG sibling map: atom/hypergraph roundtrip, binding and rewrite parity, typed folds, scoped effects and 15 open questions | [2026-10-12-hyperon-metta-lg-translation.md](2026-10-12-hyperon-metta-lg-translation.md) | diff --git a/crates/lance-graph-glove-parity/src/awareness.rs b/crates/lance-graph-glove-parity/src/awareness.rs index 431bc07fd..b65a60690 100644 --- a/crates/lance-graph-glove-parity/src/awareness.rs +++ b/crates/lance-graph-glove-parity/src/awareness.rs @@ -33,7 +33,7 @@ use lance_graph_mask_risc::{ Operand, Planes, Pred, Program, Scratch, Terminal, }; use lance_graph_quack::bind::{BindError, Binder, BoundField, Draft, FieldKind, TableId}; -use lance_graph_quack::{lower, Agg, Col, LowerError, Mask, Query}; +use lance_graph_quack::{lower, Agg, Cmp, Col, Filter, LowerError, Mask, Query}; use crate::basis::{FieldMap, Grade}; use crate::method::{ @@ -104,6 +104,14 @@ pub enum Rel { VerifiedBy, } +/// Which end of an edge a hop leaves from: `Out` follows `src -> dst`, +/// `In` follows `dst -> src`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Dir { + Out, + In, +} + /// How an edge is known. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum Tier { @@ -518,12 +526,18 @@ impl SoftwareGraph { (self.nodes.len() - 1) as u32 } - /// Add an edge unless the same `(src, rel, dst, tier)` is already there. + /// Add an edge unless the same `(src, rel, dst, tier, ctx)` is already + /// there. + /// + /// `ctx` is part of the key: a fact two methods contribute (the claim + /// behind a field both read) is one row per method, so a method's + /// reprojection retracts only its own row and the other method's + /// survives. Readers that ask for facts ([`Self::out`]) see one fact. pub fn link(&mut self, src: u32, rel: Rel, dst: u32, tier: Tier, ctx: u32, prov: u32) { let dup = self .edges .iter() - .any(|e| (e.src, e.rel, e.dst, e.tier) == (src, rel, dst, tier)); + .any(|e| (e.src, e.rel, e.dst, e.tier, e.ctx) == (src, rel, dst, tier, ctx)); if !dup { self.edges.push(Edge { src, @@ -536,17 +550,21 @@ impl SoftwareGraph { } } - /// The edges `src REL ?`, any tier. + /// The facts `src REL ?`, any tier, each once however many contexts + /// wrote it, in first-written order. #[must_use] pub fn out(&self, src: &str, rel: Rel) -> Vec<(&str, Tier)> { let Some(s) = self.node(src) else { return Vec::new(); }; - self.edges - .iter() - .filter(|e| e.src == s && e.rel == rel) - .map(|e| (self.nodes[e.dst as usize].ident.as_str(), e.tier)) - .collect() + let mut v: Vec<(&str, Tier)> = Vec::new(); + for e in self.edges.iter().filter(|e| e.src == s && e.rel == rel) { + let f = (self.nodes[e.dst as usize].ident.as_str(), e.tier); + if !v.contains(&f) { + v.push(f); + } + } + v } /// The current body digest of a projected method. @@ -804,6 +822,15 @@ impl SoftwareGraph { prov: u32, ) { let class = self.intern(Kind::Class, format!("class:{concept:#06x}")); + // A class's DAG projection replaces its earlier one: its edges and + // its cycle verdict described the DAG it had then. Other facts + // written in the class's context (a note from it) are kept. + let cycle = self.node("refusal:ComputeCycle"); + self.edges.retain(|e| { + e.ctx != class + || !(e.rel == Rel::ComputesFrom + || (e.rel == Rel::RefusedFor && Some(e.dst) == cycle)) + }); let name = |p: u8| { basis .get(usize::from(p)) @@ -939,6 +966,84 @@ impl SoftwareGraph { .collect()) } + /// The nodes reached from `start` along `path`, each hop over the edges + /// of one relation in one direction, any tier. Set semantics per hop: + /// each node once, in node order. + /// + /// Each hop is one Quack program over the resident edge lanes: + /// `rel = R AND from IN (S)`, keeping its rows. The previous hop's + /// survivors reach it as a VALUE set `S` the size of those survivors, + /// read from the kept rows' `to` lane; no hop reads a plane another + /// program produced. An empty set ends the walk without running a + /// program. + /// + /// # Errors + /// + /// [`QueryError::Bind`] for a start node the graph does not have (never + /// an empty answer); otherwise lowering or execution errors. + pub fn hops(&self, start: &[&str], path: &[(Rel, Dir)]) -> Result, QueryError> { + let mut set: Vec = start + .iter() + .map(|s| { + self.node(s).ok_or_else(|| { + QueryError::Bind(BindError::UnknownValue { + field: "src".into(), + value: (*s).into(), + }) + }) + }) + .collect::>()?; + set.sort_unstable(); + set.dedup(); + let (cols, live) = self.columns(); + let n = self.edges.len(); + for &(rel, dir) in path { + if set.is_empty() { + break; + } + let (from, to) = match dir { + Dir::Out => (SRC, DST), + Dir::In => (DST, SRC), + }; + let q = Query { + filter: Filter::and([ + Filter::plane(Mask(0)), + Filter::cmp(Col(REL), Cmp::EqU32(rel_code(rel))), + Filter::in_u32(Col(from), set.iter().copied()), + ]), + agg: Agg::Rows, + }; + let program = lower(&q).map_err(QueryError::Lower)?; + let lanes: Vec> = cols.iter().map(|c| LaneRef::U32(c)).collect(); + let planes = Planes { + n_rows: n, + masks: &[live.as_slice()], + lanes: &lanes, + }; + let mut scratch = Scratch::for_program(&program, n).map_err(QueryError::Exec)?; + let mut kept = vec![0u64; words_for(n)]; + execute_into( + &program, + &planes, + &Foreign::NONE, + &mut scratch, + lance_graph_mask_risc::Out::Mask(&mut kept), + ) + .map_err(QueryError::Exec)?; + let mut next: Vec = materialize_rows(&kept, n) + .into_iter() + .map(|r| cols[to as usize][r]) + .collect(); + next.sort_unstable(); + next.dedup(); + set = next; + } + Ok(set + .into_iter() + .map(|i| self.nodes[i as usize].ident.as_str()) + .collect()) + } + /// Query A: the methods whose compiled programs use `primitive`. /// /// # Errors @@ -996,11 +1101,14 @@ impl SoftwareGraph { return CapStatus::Unknown; }; let into = |p: &dyn Fn(&Edge) -> bool| -> Vec { - self.edges - .iter() - .filter(|e| e.dst == c && p(e)) - .map(|e| self.nodes[e.src as usize].ident.clone()) - .collect() + let mut v: Vec = Vec::new(); + for e in self.edges.iter().filter(|e| e.dst == c && p(e)) { + let s = &self.nodes[e.src as usize].ident; + if !v.contains(s) { + v.push(s.clone()); + } + } + v }; if !into(&|e| e.rel == Rel::UsesPrimitive && e.tier == Tier::Compiled).is_empty() { return CapStatus::Shipped; diff --git a/crates/lance-graph-glove-parity/tests/awareness.rs b/crates/lance-graph-glove-parity/tests/awareness.rs index 58265a961..862743e6a 100644 --- a/crates/lance-graph-glove-parity/tests/awareness.rs +++ b/crates/lance-graph-glove-parity/tests/awareness.rs @@ -7,8 +7,8 @@ use lance_graph_contract::class_view::ComputeEdge; use lance_graph_glove_parity::awareness::{ description_length, field_ident, method_ident, verdict, Candidate, CandidateKind, CapStatus, - CertError, Certificate, CostError, CpuTier, Measurement, Rel, SoftwareGraph, Tier, Verdict, - Workload, + CertError, Certificate, CostError, CpuTier, Dir, Measurement, Rel, SoftwareGraph, Tier, + Verdict, Workload, }; use lance_graph_glove_parity::basis::{canonical_fields, FieldMap, Grade, SAP_FIELDS}; use lance_graph_glove_parity::method::{ @@ -1131,3 +1131,256 @@ fn a_reprojection_replaces_the_methods_earlier_edges() { // Other methods' edges are untouched. assert_eq!(g.out(&method_ident(COUNT_BILLABLE), Rel::LowersTo).len(), 1); } + +/// The distinct `(src, rel, dst, tier)` facts of a graph, by identity. +fn facts(g: &SoftwareGraph) -> Vec<(String, String, String, Tier)> { + let id = |i: u32| g.nodes()[i as usize].ident.clone(); + let mut v: Vec<_> = g + .edges() + .iter() + .map(|e| (id(e.src), format!("{:?}", e.rel), id(e.dst), e.tier)) + .collect(); + v.sort(); + v.dedup(); + v +} + +/// Defect A (#1510). FAILS IF: reprojecting a subset of methods loses a +/// fact another, unchanged method still contributes. An edge two methods +/// share (the claim behind a field both read) belongs to each of them: one +/// method's reprojection may retract only its own copy. +#[test] +fn reprojecting_a_subset_keeps_the_facts_of_the_other_methods() { + let b = batch(65); + let basis = canonical_fields(); + let cats = CatsBinder::new(&b); + let claims = with_duration(); + let all = specs(); + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &all); + let mut full = SoftwareGraph::new(); + full.project(&t, &all, &claims, SOURCE, REV); + let want = facts(&full); + let billable = field_ident(BILLABLE_WORK_ENTRY, seal(), "billable"); + // Non-vacuity: `billable` is read by more than one method, so its claim + // edge is shared and the subset below would drop it under first-owner + // dedup. + let readers = all + .iter() + .filter(|(_, s)| { + full.out(&method_ident(s.key), Rel::ReadsField) + .iter() + .any(|(f, _)| *f == billable) + }) + .count(); + assert!(readers >= 2, "billable has {readers} readers"); + for i in 0..all.len() { + // Unchanged: a reprojection of one method changes nothing. + let mut g = full.clone(); + g.project(&t, &all[i..=i], &claims, SOURCE, REV); + assert_eq!(facts(&g), want, "reprojecting {} alone", all[i].0); + // Changed: the method stops reading any field. The graph must equal + // a fresh projection of the new declarations, so a fact another + // method still contributes survives. + let mut changed = all.clone(); + changed[i].1.body = query(&[], &[Fold::Count]); + let t2 = MethodTable::prepare(&glove(&basis, &cats, &claims), &changed); + let mut fresh = SoftwareGraph::new(); + fresh.project(&t2, &changed, &claims, SOURCE, REV); + let mut g = full.clone(); + g.project(&t2, &changed[i..=i], &claims, SOURCE, REV); + assert_eq!(facts(&g), facts(&fresh), "{} stops reading", all[i].0); + } + assert_eq!( + full.out(&billable, Rel::DependsOnClaim).len(), + 1, + "a shared fact reads as one fact" + ); +} + +/// Defect B (#1510). FAILS IF: reprojecting a class's compute DAG keeps +/// edges or the cycle verdict of the DAG it replaced, or touches another +/// class's DAG. +#[test] +fn reprojecting_a_compute_dag_replaces_the_earlier_one() { + let basis = canonical_fields(); + let cyclic = [ + ComputeEdge { + target: 2, + inputs: &[3], + }, + ComputeEdge { + target: 3, + inputs: &[2], + }, + ]; + let acyclic = [ComputeEdge { + target: 2, + inputs: &[3], + }]; + let state = |g: &SoftwareGraph, concept: u16| { + let class = g.node(&format!("class:{concept:#06x}")).unwrap(); + let from = g + .edges() + .iter() + .filter(|e| e.rel == Rel::ComputesFrom && e.ctx == class) + .count(); + let cycle = g + .out(&format!("class:{concept:#06x}"), Rel::RefusedFor) + .len(); + (from, cycle) + }; + for (first, then, want) in [ + (&cyclic[..], &acyclic[..], (1, 0)), + (&acyclic[..], &cyclic[..], (2, 1)), + (&cyclic[..], &cyclic[..], (2, 1)), + ] { + let mut g = SoftwareGraph::new(); + let p = g.note(SOURCE, REV); + g.project_compute(DOCUMENT, seal(), &basis, &cyclic, p); + g.project_compute(BILLABLE_WORK_ENTRY, seal(), &basis, first, p); + g.project_compute(BILLABLE_WORK_ENTRY, seal(), &basis, then, p); + assert_eq!( + state(&g, BILLABLE_WORK_ENTRY), + want, + "{first:?} then {then:?}" + ); + assert_eq!(state(&g, DOCUMENT), (2, 1), "the other class is untouched"); + } +} + +/// Row-loop oracle for a hop path: per hop, the set of nodes reached over +/// edges of `rel` in `dir` from the previous set, by identity. +fn oracle_hops(g: &SoftwareGraph, start: &[&str], path: &[(Rel, Dir)]) -> Vec { + let id = |i: u32| g.nodes()[i as usize].ident.clone(); + let mut set: Vec = start.iter().map(|s| (*s).to_owned()).collect(); + for &(rel, dir) in path { + let mut next: Vec = g + .edges() + .iter() + .filter(|e| e.rel == rel) + .filter_map(|e| { + let (from, to) = match dir { + Dir::Out => (e.src, e.dst), + Dir::In => (e.dst, e.src), + }; + set.contains(&id(from)).then(|| id(to)) + }) + .collect(); + next.sort(); + next.dedup(); + set = next; + } + set +} + +/// A named multi-hop question: start nodes and the path. +type HopCase<'a> = (&'a str, Vec<&'a str>, Vec<(Rel, Dir)>); + +fn sorted(v: Vec<&str>) -> Vec { + let mut v: Vec = v.into_iter().map(str::to_owned).collect(); + v.sort(); + v +} + +/// D-SAG-1-4. FAILS IF: a multi-hop question over the edge table disagrees +/// with the row loop. Each hop is one Quack program over the resident edge +/// lanes; the previous hop's survivors reach it as a value set, never as a +/// plane another program produced. +#[test] +fn multi_hop_questions_match_the_row_loop() { + let b = batch(65); + let w = world(&b, &with_duration(), &specs()); + let g = &w.graph; + let hours = hours_ident(); + let cases: Vec> = vec![ + // The methods that read a field `billable_hours` reads. + ( + "co-readers", + vec![&hours], + vec![(Rel::ReadsField, Dir::Out), (Rel::ReadsField, Dir::In)], + ), + // The claims behind the fields `billable_hours` reads. + ( + "claims", + vec![&hours], + vec![(Rel::ReadsField, Dir::Out), (Rel::DependsOnClaim, Dir::Out)], + ), + // Every method that depends on any claim behind any field it shares + // with `billable_hours`: four hops. + ( + "claim co-dependents", + vec![&hours], + vec![ + (Rel::ReadsField, Dir::Out), + (Rel::DependsOnClaim, Dir::Out), + (Rel::DependsOnClaim, Dir::In), + (Rel::ReadsField, Dir::In), + ], + ), + // The primitives of every program of a method that shares a field. + ( + "shared primitives", + vec![&hours], + vec![ + (Rel::ReadsField, Dir::Out), + (Rel::ReadsField, Dir::In), + (Rel::LowersTo, Dir::Out), + (Rel::UsesPrimitive, Dir::Out), + ], + ), + ]; + for (name, start, path) in &cases { + let want = oracle_hops(g, start, path); + let got = sorted(g.hops(start, path).unwrap()); + assert_eq!(got, want, "{name}"); + assert!(!got.is_empty(), "{name}: non-vacuous"); + } + // The hop filters: `count_by_duration` reads one field, so its + // co-readers are fewer than all methods (`billable_hours` reads both and + // shares a field with every method here). + let by_duration = method_ident(COUNT_BY_DURATION); + let co = g.hops(&[&by_duration], &cases[0].2).unwrap(); + assert_eq!( + sorted(co.clone()), + oracle_hops(g, &[&by_duration], &cases[0].2) + ); + let all = specs().len(); + assert!(!co.is_empty() && co.len() < all, "{} of {all}", co.len()); + // A start with no edge of the first relation reaches nothing; an empty + // path returns the start. + assert!(g + .hops(&["law:values"], &[(Rel::ReadsField, Dir::Out)]) + .unwrap() + .is_empty()); + assert_eq!(g.hops(&[&hours], &[]).unwrap(), vec![hours.as_str()]); + // An unknown start node is an error, never an empty answer. + assert!(g.hops(&["method:nope"], &cases[0].2).is_err()); +} + +/// FAILS IF: a hop path reads a stale edge after a reprojection (it runs on +/// the current edge table, the one defect A fixed). +#[test] +fn multi_hop_follows_a_reprojection() { + let b = batch(65); + let basis = canonical_fields(); + let cats = CatsBinder::new(&b); + let claims = with_duration(); + let all = specs(); + let t = MethodTable::prepare(&glove(&basis, &cats, &claims), &all); + let mut g = SoftwareGraph::new(); + g.project(&t, &all, &claims, SOURCE, REV); + let path = [(Rel::ReadsField, Dir::Out), (Rel::ReadsField, Dir::In)]; + let count = method_ident(COUNT_BILLABLE); + let before = sorted(g.hops(&[&count], &path).unwrap()); + assert!(before.contains(&hours_ident())); + // count_billable stops reading `billable`: it shares no field any more. + let mut changed = all.clone(); + changed[0].1.body = query(&[], &[Fold::Count]); + let t2 = MethodTable::prepare(&glove(&basis, &cats, &claims), &changed); + g.project(&t2, &changed[0..1], &claims, SOURCE, REV); + assert!(g.hops(&[&count], &path).unwrap().is_empty()); + assert_eq!( + sorted(g.hops(&[&count], &path).unwrap()), + oracle_hops(&g, &[&count], &path) + ); +}