diff --git a/.cargo/config.toml b/.cargo/config.toml index 8f2e83535..878fb7eba 100644 --- a/.cargo/config.toml +++ b/.cargo/config.toml @@ -53,7 +53,12 @@ # python/ sets RUSTFLAGS only on an x86_64 host. Note that the per-target # variable is JOINED with this file's entry, with `native` last (measured), so # it cannot pin a build that sees this file. Anything below v3 still SIGILLs on the -# `simd_avx2` intrinsics, so never build with no target-cpu at all. +# `simd_avx2` intrinsics, so never build with no target-cpu at all. A build with +# no target-cpu takes ndarray's AVX2 arm, and that arm is kept only as the +# validation oracle. Real builds go through the ndarray polyfill at a declared +# tier once its arms are validated against that oracle: `native` for local and +# host-bound builds, the explicit x86-64-v4 / `sapphirerapids` tier above for +# shipped artifacts. # # The two crypto cfgs ndarray's config also carries (`curve25519_dalek_backend`, # `poly1305_force_soft`) are deliberately NOT repeated: neither crate is in diff --git a/.claude/agents/cargo-substrate-architect.md b/.claude/agents/cargo-substrate-architect.md index 8c9d2b8ec..a3e069aec 100644 --- a/.claude/agents/cargo-substrate-architect.md +++ b/.claude/agents/cargo-substrate-architect.md @@ -79,7 +79,8 @@ Per workspace root: ```toml [workspace.dependencies] -ndarray = { git = "", rev = "", default-features = false, features = ["std"] } +ndarray = { git = "", branch = "master", default-features = false, features = ["std"] } +# never `rev = …`: a pin on an internal crate is forbidden; master is the go-to branch ``` Members become `ndarray = { workspace = true, features = [...] }`. diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index e03a2880e..f9cf4be20 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,7 @@ +## 2026-10-16 — main after #1581; in PR #1582 + +- **In PR #1582:** doc fixes (rev-pin template withdrawn, TD-POPCNT runtime-detect option withdrawn, the no-target-cpu AVX2 arm is the validation oracle only); the #1579 arc entry; and the OpenCog decisions in `entries/2026-10-16-opencog-oracle-scope.md` — OpenCog as a temporary validation oracle, the licence reading, `lance-graph-cog` as the Apache-2.0 API an AGPL OpenCog fork (`atomspace-lance`) would call, positional tables over any ordered KV; plus `TD-ONTOLOGY-REGISTRY-HASH-LOOKUP-REVIEW-1`. Nothing built. + ## 2026-10-16 — main after #1578 (merged `1f5322fe`) and #1579 (merged `7302f58c`); ledger update in PR #1581 - **On main:** #1578 (ledger revision after #1576, arc catch-up), #1579 (synergy map since 2026-10-11, ten STATUS rows set to Shipped, label codebook sizes measured), #1580 (hydrate: `zip` 2 → 9). diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index e9267da8e..d41743ba4 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,10 @@ +## 2026-10-16 — lance-graph PR #1579 (merged `7302f58c`, branch `ccr-9f3b8075-74u47l-synergy-map`, head `97b744ec`, 3 commits) — board: synergy map since 2026-10-11, status sweep, #1575 record + +- **Added:** entry `2026-10-16-synergy-map-since-1011.md` (the 2026-10-11 fleet listing revised against 123 merges: closed ends, dots D1–D8, fruits, Haiku ledger, §8 reconciliation with the recovery ledger, §9 label codebook sizes); ten STATUS rows moved from "In PR" to Shipped; arc entry for #1575. +- **Locked:** measured codebook sizes — ONA `ATOMS_MAX 255`; MeTTa stdlib 266 symbols, the 26 golden scripts 282 together (at most 26 per script outside the stdlib); cog vocabulary 214. No two corpora fit one 256-entry codebook, so codebooks are scoped per classid and space. D7 (two ndarray packages) closed: the root and relalg `[patch]` sections already resolve one package (Codex review). +- **Deferred / open:** D-RA-2, codebook scope, #1544 closure (decisions); the cross-repo doc fixes; the wasm `mul_add` test; the lgj `check_build_cpu` handshake. +- **Confidence:** n/a (board); counts by token regex over `.metta` files, not the Hyperon parser. + ## 2026-10-16 — lance-graph PR #1578 (merged `1f5322fe`, branch `ccr-b2e415d9-4jfvyk`, head `8b669321`) — recovery ledger revised against #1560–#1576 - **Added:** `.claude/research/2026-10-recovery/LEDGER-REVISION-2026-10-16.md` (closed C22–C30; open items merged and deduplicated; K2', K6, K7, K8; fruits; operator decisions with recommendations); entry `2026-10-16-ledger-revision-after-1576.md`; arc entries for #1560, #1570, #1571, #1574, #1575, #1576; LATEST_STATE section. diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 30ea73572..bcc1a0008 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -1860,7 +1860,7 @@ not necessarily a logic bug. NOTE: `lance-graph-arm-discovery/Cargo.toml` *recom U64x8 popcount kernel + the AMX re-exports), (b) decide whether ndarray's `U64x8` should runtime-detect VPOPCNTDQ/AMX rather than trust compile-time `target_feature`, and (c) correct the Cargo.toml comment to warn that `native` can SIGILL on virtualized hosts lacking the detected AVX-512 subset. Until then: -run the `ndarray-simd` probes on the default path. **Status: Open.** +run the `ndarray-simd` probes on the default path. **Status: Open; item (b) withdrawn 2026-10-16 — runtime detection in kernels is forbidden (dispatch is compile time, the only CPU check is ndarray's startup `cpu_guard`); (a) and (c) remain.** ### TD-CI-COVERAGE-MOLD-1 — `test-with-coverage` job lacks the mold linker the `test` job has (2026-06-12) @@ -4618,3 +4618,15 @@ as CAM search modes ("exact CAM search", "wide CAM scan", "DN-tree traversal") w (`causal_edge::isa::truth::code`, D-NALP-4) with the argument-order split recorded in `.claude/research/2026-10-15-nars-metta-ce64-mapping-inventory.md` §6. Doc-only; close by one PR that rewrites the five doc comments to the truth-function meaning and names the OpenNARS/ONA order convention. + +## TD-ONTOLOGY-REGISTRY-HASH-LOOKUP-REVIEW-1 (2026-10-16) — OPEN, review only + +`lance-graph-ontology`'s `OntologyRegistry` resolves names through string-keyed hash maps +(`registry.rs:56-73`: `by_bridge_name: HashMap<(String, String), u32>`, `by_uri: HashMap`, +`by_namespace`, `bundles`, `path_by_type` / `type_by_path`) and persists append-only to Lance +(`lance_cache.rs`). This predates the BindSpace DTO and the quack fold path. Review whether the hot +lookups can become label-codebook ids resolved once plus coordinate ranges and masks over baked SoA +lanes (`soa_bake`, quack `Binder::code`), leaving the string maps only at hydration. Not measured: +no lookup timing exists for this registry, so there is no claim it is slow yet. Also the precedent for +`lance-graph-cog` (`entries/2026-10-16-opencog-oracle-scope.md`), so a finding here carries over. Close +by: a measured comparison on one OGIT tenant (string-map resolve vs codebook + mask), recorded as an entry. diff --git a/.claude/board/entries/2026-10-16-opencog-oracle-scope.md b/.claude/board/entries/2026-10-16-opencog-oracle-scope.md new file mode 100644 index 000000000..7c6ae61fc --- /dev/null +++ b/.claude/board/entries/2026-10-16-opencog-oracle-scope.md @@ -0,0 +1,171 @@ +# 2026-10-16 — OpenCog is a temporary validation oracle, not a dependency + +**Status:** DECISION (docs only; nothing built, nothing linked). + +DECISION: OpenCog (Classic AtomSpace, PLN, ECAN, the pattern miner) is used only +as a temporary oracle, to validate that our MeTTa reading agrees with OpenCog's +on the same inputs. It is harvested for that check and not wired into any +lance-graph crate. + +SCOPE: `lance-graph-metta` stays the MeTTa front end and `Space` adapter +(D-MTP-0, `2026-10-14-metta-polyfill-map.md`). Oracle harnesses that run OpenCog +code live outside the workspace, like `datalog-oracle-probe` and `hyperon-probe`. + +BASIS: Classic AtomSpace is AGPL, so its code cannot enter an Apache/MIT crate, +and its value here is only as a reference answer. + +REVISIT WHEN: the validation is done. Then one of two outcomes is recorded: +OpenCog is redundant (MeTTa via Hyperon/PeTTa already covers it) and the harvest +is dropped, or it earns a separate `lance-graph-cog` crate under its own licence, +possibly in its own repository. + +OPEN: which OpenCog pieces the validation needs (PLN rules, ECAN, AtomSpace +queries); none is pinned yet. + +## Licence check: does an adapter to OpenCog code poison ours? (read 2026-10-16) + +Read: `opencog/atomspace` `LICENSE` at `master` (693 lines), the header and the +"Linking Exception" at lines 672-693; `opencog/cogserver` `LICENSE` has the same +header. Not read: PLN, attention (ECAN), the miner, and per-file headers, which +the licence says can override it ("Unless indicated otherwise (a license +reference in file headers, or a LICENSE file in a directory)"). + +The AtomSpace is AGPL v3 **with a linking exception** of the GPL-classpath kind: +"permission to link this library with independent modules to produce an +executable, regardless of the license terms of these independent modules", where +"An independent module is a module which is not derived from or based on this +library." + +| shape | effect on our licence | +|---|---| +| separate process (CogServer, socket, pipe) | none; no linking happens | +| our adapter linking the unmodified library, written against its API | none under the exception; the AtomSpace part stays AGPL and its source must be offered when we distribute it | +| we patch the AtomSpace (e.g. for speed) | our code stays ours; the patched library is AGPL and the exception is optional on it; AGPL §13 applies if users reach it over a network | +| we port or transcode AtomSpace code into Rust | **this is the risk**: a port is "derived from or based on this library", so it is AGPL, and so is any crate it lands in | + +So linking an adapter does not poison the licence; copying the code does. A +harvest for validation must keep OpenCog code and our code apart: OpenCog runs as +the oracle, and our side is written from its documented behaviour, never from its +source. This is not legal advice; check with counsel before distributing anything +that links OpenCog. + +OPEN: where a 200× speedup would come from has not been identified or measured. +The MeTTa-side engines measured so far are Hyperon (MIT) and PeTTa (MIT); the +local MORK checkout (`ed57c67`) has no licence file and no licence field, so its +licence is unknown. + +### Oracle and calibration use (same reading) + +Running unmodified AGPL software and comparing its answers with ours does not +license our code under AGPL; the AGPL grants running it without conditions. Two +things still need care: + +- **Fixtures:** OpenCog's own input files (rule libraries, test scripts) stay + AGPL wherever they are copied. Fetch them at a pinned commit outside the repo + (the `bench/fetch.sh` pattern in tesseract-rs) instead of committing them. Our + own generated inputs and the recorded answers (truth values, binding sets, + counts) are results, not OpenCog code, and can be committed. +- **Calibration:** matching our constants or truth formulas to OpenCog's answers + is fine. Taking a formula from the PLN papers or documentation is fine. + Transcribing it from their source code is the porting case above. + +### An OpenCog fork in its own repository that imports our crates + +Case: fork OpenCog (AtomSpace) in a separate repository and change it to use +ndarray, Datalog over folds, or a storage adapter backed by `lance-graph-quack`. + +- **Our crates keep their licences.** Dependencies flow OpenCog → ndarray / + quack, never back. ndarray is MIT or Apache-2.0, lance-graph is Apache-2.0 + (repository `LICENSE`; quack and mask-risc carry no own field), relalg is + MIT OR Apache-2.0. Both are compatible with AGPL v3, so a combined AGPL binary + may include them without changing them. +- **The fork is AGPL**, including every line we write inside it (the storage + backend, glue, patches), because that code is based on OpenCog. If the fork is + distributed, or modified and offered to users over a network (§13), its source + must be published; a public fork satisfies that. +- **The trap is reuse in the other direction.** Glue written in the fork cannot + later move into lance-graph without bringing AGPL with it. So general logic is + written in our repositories first, under our licence, and the fork only calls + it; the fork keeps the thinnest possible adapter. +- The combined binary is distributed under AGPL terms; our crates inside it are + still usable on their own under their own licences. + +OPEN: whether glue we author ourselves could be dual-licensed is a copyright +question for counsel, not settled here. + +### DECISION (same day): `lance-graph-cog` is our API for OpenCog to call + +DECISION: improving OpenCog for its community is welcome. `lance-graph-cog` is the +crate OpenCog would use: a lance-graph crate under lance-graph's own licence +(Apache-2.0) that exposes the storage, fold-execution and codebook API. The +AGPL side is a separate OpenCog fork that depends on it and keeps only the thin +adapter (for example an AtomSpace storage backend). + +SCOPE: `lance-graph-cog` contains no OpenCog code. It is written from OpenCog's +documented API and observed behaviour, never from its source, so it stays +reusable by any caller. This refines the D-MTP-0 naming note +(`2026-10-14-metta-polyfill-map.md`): "cog" there meant pulling OpenCog itself +into the crate, which stays ruled out; here it names the API OpenCog consumes. + +REVISIT WHEN: the oracle validation shows which surface OpenCog actually needs; +until then the crate is not created. + +**Precedent:** `lance-graph-ontology` (Apache-2.0) already has this shape. It +hydrates other people's formats into our registry and Lance cache without +containing their code: OGIT TTL (`oxttl`), OWL and FIBO RDF/XML (`oxrdfxml`), XSD +schemas (`quick-xml`), plus a Wikidata path, with `soa_bake` and `lance_cache` +as the fast side. Tenants then reach it through thin `NamespaceBridge` views. +`lance-graph-cog` would do the same for OpenCog: read AtomSpace content (Atomese +atoms) as input data, and give the AGPL fork's adapter the same bake-and-cache +surface. + +### Repositories (added 2026-10-16) + +AdaWorldAPI now has forks of `opencog/atomspace`, `atomspace-cog`, `atomspace-rocks`, +`cogserver`, `lg-atomese` and `learn`, plus an empty `AdaWorldAPI/atomspace-lance`, the +landing space for the AGPL-side adapter. Read: the atomspace-rocks fork at `6577a5b` +(`RocksStorage.h`, the key-schema comment in `RocksIO.cc:78-148`); same licence header and +linking exception as the AtomSpace; per-file headers `AGPL-3.0-or-later`. + +`atomspace-lance` is the analogue of `RocksStorageNode`: a `StorageNode` subclass (AGPL, it +is based on OpenCog) whose calls go down into `lance-graph-cog` (Apache-2.0). The StorageNode +API it must implement (`RocksStorage.h:142-171`): open/close, `getAtom`, `getLink`, +`fetchIncomingSet` / `fetchIncomingByType`, `storeAtom`, `removeAtom`, `storeValue` / +`updateValue` / `loadValue`, `loadType`, `loadAtomSpace` / `storeAtomSpace`, +`loadFrameDAG` / `storeFrameDAG` / `deleteFrame`, `barrier`. + +How Rocks lays out the data, and the candidate mapping on our side (not built, not measured): + +| Rocks key | meaning | our side | +|---|---|---| +| `n@satom`, `l@satom` → sid; `a@sid` → satom | atom s-expression ↔ 64-bit id | content → NodeGuid; names via the label codebook | +| `h@shash` → sid list | hash → ids | not needed if the address is content-derived | +| `i@sid:stype-sid` | incoming set (by type) | a hop: preimage over coordinate ranges | +| `k@sid:kid` → sval | values (truth value etc.) per key | lanes on the row | +| `d@`/`f@`/`o@` frames, `z` heights | Kripke frames (layered AtomSpaces), link heights | hypothesis layers / Lance versions | +| prefix scans (`n@(Concept`) | load all atoms of a type | class range over the classid prefix | + +OPEN: the atom identity mapping (content → NodeGuid) is the first design item; RocksIO's +s-expression text is the identity Rocks uses, which we would replace with a deterministic +address. + +### Positional tables and zipper merge on read work over any KV store + +The layout we use — `tableA[index]`, `tableB[index]`: parallel tables addressed by the same +row index, written sparsely, joined by a zipper merge on read — does not need Lance. It +needs only that each table can be read in index order. A KV store with ordered keys +(RocksDB iterates keys lexically) provides that, so the same layout runs on Rocks: one +key prefix per table, keys ending in the row index, and a read that walks the prefixes in +step and merges them in one pass instead of one `Get` per atom per key. + +Why this matters here: +- `atomspace-lance` and an improved `atomspace-rocks` can share one layout; Lance is one + backend for it, not a precondition (the 2026-10-05 storage-portability decision: + Quack owns the semantics, the backend supplies capabilities). +- Rocks already sorts by prefix (`k@sid:`, `i@sid:`); what changes is that `sid` becomes a + row index and the per-atom string keys stop being the join. +- It is the same pattern as the alpha layers (sparse dirty writes, merge on read). + +Not built, not measured: no timing exists for Rocks with this layout or for the current +`RocksStorageNode`. A first probe would load one AtomSpace both ways and compare reads +per element. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index ce02f0d77..7c074e541 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,11 +25,12 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -353 entries, 2026-08-06 .. 2026-10-16. +354 entries, 2026-08-06 .. 2026-10-16. | date | entry id | finding | file | |---|---|---|---| | 2026-10-16 | `synergy-map-since-1011` | | [2026-10-16-synergy-map-since-1011.md](2026-10-16-synergy-map-since-1011.md) | +| 2026-10-16 | `opencog-oracle-scope` | | [2026-10-16-opencog-oracle-scope.md](2026-10-16-opencog-oracle-scope.md) | | 2026-10-16 | `D-ASM-1` | | [2026-10-16-metta-datalog-assimilation-council-v3.md](2026-10-16-metta-datalog-assimilation-council-v3.md) | | 2026-10-16 | `ledger-update-dedup-1579` | | [2026-10-16-ledger-update-dedup-1579.md](2026-10-16-ledger-update-dedup-1579.md) | | 2026-10-16 | `ledger-revision-after-1576` | | [2026-10-16-ledger-revision-after-1576.md](2026-10-16-ledger-revision-after-1576.md) | diff --git a/.claude/knowledge/CARGO-COMPUTE-SUBSTRATE.md b/.claude/knowledge/CARGO-COMPUTE-SUBSTRATE.md index cccd113fa..7cf3d92cf 100644 --- a/.claude/knowledge/CARGO-COMPUTE-SUBSTRATE.md +++ b/.claude/knowledge/CARGO-COMPUTE-SUBSTRATE.md @@ -215,7 +215,7 @@ But this: every repo: ndarray = { workspace = true } workspace root: [workspace.dependencies] - ndarray = { git = "", rev = "" , ... } + ndarray = { git = "", branch = "master" , ... } # never a rev pin: internal pins are forbidden local supercheckout only: [patch.""] @@ -225,7 +225,7 @@ local supercheckout only: Which gives: ```text -CI / standalone clone -> pinned git ndarray +CI / standalone clone -> git ndarray, branch master local development -> same dep, transparently patched to ../ndarray release binary -> exactly ONE resolved ndarray package ```