Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,12 @@
# python/ sets RUSTFLAGS only on an x86_64 host. Note that the per-target
# variable is JOINED with this file's entry, with `native` last (measured), so
# it cannot pin a build that sees this file. Anything below v3 still SIGILLs on the
# `simd_avx2` intrinsics, so never build with no target-cpu at all.
# `simd_avx2` intrinsics, so never build with no target-cpu at all. A build with
# no target-cpu takes ndarray's AVX2 arm, and that arm is kept only as the
# validation oracle. Real builds go through the ndarray polyfill at a declared
# tier once its arms are validated against that oracle: `native` for local and
# host-bound builds, the explicit x86-64-v4 / `sapphirerapids` tier above for
# shipped artifacts.
#
# The two crypto cfgs ndarray's config also carries (`curve25519_dalek_backend`,
# `poly1305_force_soft`) are deliberately NOT repeated: neither crate is in
Expand Down
3 changes: 2 additions & 1 deletion .claude/agents/cargo-substrate-architect.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ Per workspace root:

```toml
[workspace.dependencies]
ndarray = { git = "<canonical>", rev = "<pinned>", default-features = false, features = ["std"] }
ndarray = { git = "<canonical>", branch = "master", default-features = false, features = ["std"] }
# never `rev = …`: a pin on an internal crate is forbidden; master is the go-to branch
```

Members become `ndarray = { workspace = true, features = [...] }`.
Expand Down
4 changes: 4 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 2026-10-16 — main after #1581; in PR #1582

- **In PR #1582:** doc fixes (rev-pin template withdrawn, TD-POPCNT runtime-detect option withdrawn, the no-target-cpu AVX2 arm is the validation oracle only); the #1579 arc entry; and the OpenCog decisions in `entries/2026-10-16-opencog-oracle-scope.md` — OpenCog as a temporary validation oracle, the licence reading, `lance-graph-cog` as the Apache-2.0 API an AGPL OpenCog fork (`atomspace-lance`) would call, positional tables over any ordered KV; plus `TD-ONTOLOGY-REGISTRY-HASH-LOOKUP-REVIEW-1`. Nothing built.

## 2026-10-16 — main after #1578 (merged `1f5322fe`) and #1579 (merged `7302f58c`); ledger update in PR #1581

- **On main:** #1578 (ledger revision after #1576, arc catch-up), #1579 (synergy map since 2026-10-11, ten STATUS rows set to Shipped, label codebook sizes measured), #1580 (hydrate: `zip` 2 → 9).
Expand Down
7 changes: 7 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## 2026-10-16 — lance-graph PR #1579 (merged `7302f58c`, branch `ccr-9f3b8075-74u47l-synergy-map`, head `97b744ec`, 3 commits) — board: synergy map since 2026-10-11, status sweep, #1575 record

- **Added:** entry `2026-10-16-synergy-map-since-1011.md` (the 2026-10-11 fleet listing revised against 123 merges: closed ends, dots D1–D8, fruits, Haiku ledger, §8 reconciliation with the recovery ledger, §9 label codebook sizes); ten STATUS rows moved from "In PR" to Shipped; arc entry for #1575.
- **Locked:** measured codebook sizes — ONA `ATOMS_MAX 255`; MeTTa stdlib 266 symbols, the 26 golden scripts 282 together (at most 26 per script outside the stdlib); cog vocabulary 214. No two corpora fit one 256-entry codebook, so codebooks are scoped per classid and space. D7 (two ndarray packages) closed: the root and relalg `[patch]` sections already resolve one package (Codex review).
- **Deferred / open:** D-RA-2, codebook scope, #1544 closure (decisions); the cross-repo doc fixes; the wasm `mul_add` test; the lgj `check_build_cpu` handshake.
- **Confidence:** n/a (board); counts by token regex over `.metta` files, not the Hyperon parser.

## 2026-10-16 — lance-graph PR #1578 (merged `1f5322fe`, branch `ccr-b2e415d9-4jfvyk`, head `8b669321`) — recovery ledger revised against #1560–#1576

- **Added:** `.claude/research/2026-10-recovery/LEDGER-REVISION-2026-10-16.md` (closed C22–C30; open items merged and deduplicated; K2', K6, K7, K8; fruits; operator decisions with recommendations); entry `2026-10-16-ledger-revision-after-1576.md`; arc entries for #1560, #1570, #1571, #1574, #1575, #1576; LATEST_STATE section.
Expand Down
14 changes: 13 additions & 1 deletion .claude/board/TECH_DEBT.md
Original file line number Diff line number Diff line change
Expand Up @@ -1860,7 +1860,7 @@ not necessarily a logic bug. NOTE: `lance-graph-arm-discovery/Cargo.toml` *recom
U64x8 popcount kernel + the AMX re-exports), (b) decide whether ndarray's `U64x8` should runtime-detect
VPOPCNTDQ/AMX rather than trust compile-time `target_feature`, and (c) correct the Cargo.toml comment to
warn that `native` can SIGILL on virtualized hosts lacking the detected AVX-512 subset. Until then:
run the `ndarray-simd` probes on the default path. **Status: Open.**
run the `ndarray-simd` probes on the default path. **Status: Open; item (b) withdrawn 2026-10-16 — runtime detection in kernels is forbidden (dispatch is compile time, the only CPU check is ndarray's startup `cpu_guard`); (a) and (c) remain.**

### TD-CI-COVERAGE-MOLD-1 — `test-with-coverage` job lacks the mold linker the `test` job has (2026-06-12)

Expand Down Expand Up @@ -4618,3 +4618,15 @@ as CAM search modes ("exact CAM search", "wide CAM scan", "DN-tree traversal") w
(`causal_edge::isa::truth::code`, D-NALP-4) with the argument-order split recorded in
`.claude/research/2026-10-15-nars-metta-ce64-mapping-inventory.md` §6. Doc-only; close by one PR that rewrites the five doc
comments to the truth-function meaning and names the OpenNARS/ONA order convention.

## TD-ONTOLOGY-REGISTRY-HASH-LOOKUP-REVIEW-1 (2026-10-16) — OPEN, review only

`lance-graph-ontology`'s `OntologyRegistry` resolves names through string-keyed hash maps
(`registry.rs:56-73`: `by_bridge_name: HashMap<(String, String), u32>`, `by_uri: HashMap<String, u32>`,
`by_namespace`, `bundles`, `path_by_type` / `type_by_path`) and persists append-only to Lance
(`lance_cache.rs`). This predates the BindSpace DTO and the quack fold path. Review whether the hot
lookups can become label-codebook ids resolved once plus coordinate ranges and masks over baked SoA
lanes (`soa_bake`, quack `Binder::code`), leaving the string maps only at hydration. Not measured:
no lookup timing exists for this registry, so there is no claim it is slow yet. Also the precedent for
`lance-graph-cog` (`entries/2026-10-16-opencog-oracle-scope.md`), so a finding here carries over. Close
by: a measured comparison on one OGIT tenant (string-map resolve vs codebook + mask), recorded as an entry.
171 changes: 171 additions & 0 deletions .claude/board/entries/2026-10-16-opencog-oracle-scope.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# 2026-10-16 — OpenCog is a temporary validation oracle, not a dependency

**Status:** DECISION (docs only; nothing built, nothing linked).

DECISION: OpenCog (Classic AtomSpace, PLN, ECAN, the pattern miner) is used only
as a temporary oracle, to validate that our MeTTa reading agrees with OpenCog's
on the same inputs. It is harvested for that check and not wired into any
lance-graph crate.

SCOPE: `lance-graph-metta` stays the MeTTa front end and `Space` adapter
(D-MTP-0, `2026-10-14-metta-polyfill-map.md`). Oracle harnesses that run OpenCog
code live outside the workspace, like `datalog-oracle-probe` and `hyperon-probe`.

BASIS: Classic AtomSpace is AGPL, so its code cannot enter an Apache/MIT crate,
and its value here is only as a reference answer.

REVISIT WHEN: the validation is done. Then one of two outcomes is recorded:
OpenCog is redundant (MeTTa via Hyperon/PeTTa already covers it) and the harvest
is dropped, or it earns a separate `lance-graph-cog` crate under its own licence,
possibly in its own repository.

OPEN: which OpenCog pieces the validation needs (PLN rules, ECAN, AtomSpace
queries); none is pinned yet.

## Licence check: does an adapter to OpenCog code poison ours? (read 2026-10-16)

Read: `opencog/atomspace` `LICENSE` at `master` (693 lines), the header and the
"Linking Exception" at lines 672-693; `opencog/cogserver` `LICENSE` has the same
header. Not read: PLN, attention (ECAN), the miner, and per-file headers, which
the licence says can override it ("Unless indicated otherwise (a license
reference in file headers, or a LICENSE file in a directory)").

The AtomSpace is AGPL v3 **with a linking exception** of the GPL-classpath kind:
"permission to link this library with independent modules to produce an
executable, regardless of the license terms of these independent modules", where
"An independent module is a module which is not derived from or based on this
library."

| shape | effect on our licence |
|---|---|
| separate process (CogServer, socket, pipe) | none; no linking happens |
| our adapter linking the unmodified library, written against its API | none under the exception; the AtomSpace part stays AGPL and its source must be offered when we distribute it |
| we patch the AtomSpace (e.g. for speed) | our code stays ours; the patched library is AGPL and the exception is optional on it; AGPL §13 applies if users reach it over a network |
| we port or transcode AtomSpace code into Rust | **this is the risk**: a port is "derived from or based on this library", so it is AGPL, and so is any crate it lands in |

So linking an adapter does not poison the licence; copying the code does. A
harvest for validation must keep OpenCog code and our code apart: OpenCog runs as
the oracle, and our side is written from its documented behaviour, never from its
source. This is not legal advice; check with counsel before distributing anything
that links OpenCog.

OPEN: where a 200× speedup would come from has not been identified or measured.
The MeTTa-side engines measured so far are Hyperon (MIT) and PeTTa (MIT); the
local MORK checkout (`ed57c67`) has no licence file and no licence field, so its
licence is unknown.

### Oracle and calibration use (same reading)

Running unmodified AGPL software and comparing its answers with ours does not
license our code under AGPL; the AGPL grants running it without conditions. Two
things still need care:

- **Fixtures:** OpenCog's own input files (rule libraries, test scripts) stay
AGPL wherever they are copied. Fetch them at a pinned commit outside the repo
(the `bench/fetch.sh` pattern in tesseract-rs) instead of committing them. Our
own generated inputs and the recorded answers (truth values, binding sets,
counts) are results, not OpenCog code, and can be committed.
- **Calibration:** matching our constants or truth formulas to OpenCog's answers
is fine. Taking a formula from the PLN papers or documentation is fine.
Transcribing it from their source code is the porting case above.

### An OpenCog fork in its own repository that imports our crates

Case: fork OpenCog (AtomSpace) in a separate repository and change it to use
ndarray, Datalog over folds, or a storage adapter backed by `lance-graph-quack`.

- **Our crates keep their licences.** Dependencies flow OpenCog → ndarray /
quack, never back. ndarray is MIT or Apache-2.0, lance-graph is Apache-2.0
(repository `LICENSE`; quack and mask-risc carry no own field), relalg is
MIT OR Apache-2.0. Both are compatible with AGPL v3, so a combined AGPL binary
may include them without changing them.
- **The fork is AGPL**, including every line we write inside it (the storage
backend, glue, patches), because that code is based on OpenCog. If the fork is
distributed, or modified and offered to users over a network (§13), its source
must be published; a public fork satisfies that.
- **The trap is reuse in the other direction.** Glue written in the fork cannot
later move into lance-graph without bringing AGPL with it. So general logic is
written in our repositories first, under our licence, and the fork only calls
it; the fork keeps the thinnest possible adapter.
- The combined binary is distributed under AGPL terms; our crates inside it are
still usable on their own under their own licences.

OPEN: whether glue we author ourselves could be dual-licensed is a copyright
question for counsel, not settled here.

### DECISION (same day): `lance-graph-cog` is our API for OpenCog to call

DECISION: improving OpenCog for its community is welcome. `lance-graph-cog` is the
crate OpenCog would use: a lance-graph crate under lance-graph's own licence
(Apache-2.0) that exposes the storage, fold-execution and codebook API. The
AGPL side is a separate OpenCog fork that depends on it and keeps only the thin
adapter (for example an AtomSpace storage backend).
Comment thread
AdaWorldAPI marked this conversation as resolved.

SCOPE: `lance-graph-cog` contains no OpenCog code. It is written from OpenCog's
documented API and observed behaviour, never from its source, so it stays
reusable by any caller. This refines the D-MTP-0 naming note
(`2026-10-14-metta-polyfill-map.md`): "cog" there meant pulling OpenCog itself
into the crate, which stays ruled out; here it names the API OpenCog consumes.

REVISIT WHEN: the oracle validation shows which surface OpenCog actually needs;
until then the crate is not created.

**Precedent:** `lance-graph-ontology` (Apache-2.0) already has this shape. It
hydrates other people's formats into our registry and Lance cache without
containing their code: OGIT TTL (`oxttl`), OWL and FIBO RDF/XML (`oxrdfxml`), XSD
schemas (`quick-xml`), plus a Wikidata path, with `soa_bake` and `lance_cache`
as the fast side. Tenants then reach it through thin `NamespaceBridge` views.
`lance-graph-cog` would do the same for OpenCog: read AtomSpace content (Atomese
atoms) as input data, and give the AGPL fork's adapter the same bake-and-cache
surface.

### Repositories (added 2026-10-16)

AdaWorldAPI now has forks of `opencog/atomspace`, `atomspace-cog`, `atomspace-rocks`,
`cogserver`, `lg-atomese` and `learn`, plus an empty `AdaWorldAPI/atomspace-lance`, the
landing space for the AGPL-side adapter. Read: the atomspace-rocks fork at `6577a5b`
(`RocksStorage.h`, the key-schema comment in `RocksIO.cc:78-148`); same licence header and
linking exception as the AtomSpace; per-file headers `AGPL-3.0-or-later`.

`atomspace-lance` is the analogue of `RocksStorageNode`: a `StorageNode` subclass (AGPL, it
is based on OpenCog) whose calls go down into `lance-graph-cog` (Apache-2.0). The StorageNode
API it must implement (`RocksStorage.h:142-171`): open/close, `getAtom`, `getLink`,
`fetchIncomingSet` / `fetchIncomingByType`, `storeAtom`, `removeAtom`, `storeValue` /
`updateValue` / `loadValue`, `loadType`, `loadAtomSpace` / `storeAtomSpace`,
`loadFrameDAG` / `storeFrameDAG` / `deleteFrame`, `barrier`.

How Rocks lays out the data, and the candidate mapping on our side (not built, not measured):

| Rocks key | meaning | our side |
|---|---|---|
| `n@satom`, `l@satom` → sid; `a@sid` → satom | atom s-expression ↔ 64-bit id | content → NodeGuid; names via the label codebook |
| `h@shash` → sid list | hash → ids | not needed if the address is content-derived |
| `i@sid:stype-sid` | incoming set (by type) | a hop: preimage over coordinate ranges |
| `k@sid:kid` → sval | values (truth value etc.) per key | lanes on the row |
| `d@`/`f@`/`o@` frames, `z` heights | Kripke frames (layered AtomSpaces), link heights | hypothesis layers / Lance versions |
| prefix scans (`n@(Concept`) | load all atoms of a type | class range over the classid prefix |

OPEN: the atom identity mapping (content → NodeGuid) is the first design item; RocksIO's
s-expression text is the identity Rocks uses, which we would replace with a deterministic
address.

### Positional tables and zipper merge on read work over any KV store

The layout we use — `tableA[index]`, `tableB[index]`: parallel tables addressed by the same
row index, written sparsely, joined by a zipper merge on read — does not need Lance. It
needs only that each table can be read in index order. A KV store with ordered keys
(RocksDB iterates keys lexically) provides that, so the same layout runs on Rocks: one
key prefix per table, keys ending in the row index, and a read that walks the prefixes in
step and merges them in one pass instead of one `Get` per atom per key.

Why this matters here:
- `atomspace-lance` and an improved `atomspace-rocks` can share one layout; Lance is one
backend for it, not a precondition (the 2026-10-05 storage-portability decision:
Quack owns the semantics, the backend supplies capabilities).
- Rocks already sorts by prefix (`k@sid:`, `i@sid:`); what changes is that `sid` becomes a
row index and the per-atom string keys stop being the join.
- It is the same pattern as the alpha layers (sparse dirty writes, merge on read).

Not built, not measured: no timing exists for Rocks with this layout or for the current
`RocksStorageNode`. A first probe would load one AtomSpace both ways and compare reads
per element.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,12 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

353 entries, 2026-08-06 .. 2026-10-16.
354 entries, 2026-08-06 .. 2026-10-16.

| date | entry id | finding | file |
|---|---|---|---|
| 2026-10-16 | `synergy-map-since-1011` | | [2026-10-16-synergy-map-since-1011.md](2026-10-16-synergy-map-since-1011.md) |
| 2026-10-16 | `opencog-oracle-scope` | | [2026-10-16-opencog-oracle-scope.md](2026-10-16-opencog-oracle-scope.md) |
| 2026-10-16 | `D-ASM-1` | | [2026-10-16-metta-datalog-assimilation-council-v3.md](2026-10-16-metta-datalog-assimilation-council-v3.md) |
| 2026-10-16 | `ledger-update-dedup-1579` | | [2026-10-16-ledger-update-dedup-1579.md](2026-10-16-ledger-update-dedup-1579.md) |
| 2026-10-16 | `ledger-revision-after-1576` | | [2026-10-16-ledger-revision-after-1576.md](2026-10-16-ledger-revision-after-1576.md) |
Expand Down
4 changes: 2 additions & 2 deletions .claude/knowledge/CARGO-COMPUTE-SUBSTRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ But this:
every repo: ndarray = { workspace = true }
workspace root:
[workspace.dependencies]
ndarray = { git = "<canonical>", rev = "<pinned>" , ... }
ndarray = { git = "<canonical>", branch = "master" , ... } # never a rev pin: internal pins are forbidden

local supercheckout only:
[patch."<canonical>"]
Expand All @@ -225,7 +225,7 @@ local supercheckout only:
Which gives:

```text
CI / standalone clone -> pinned git ndarray
CI / standalone clone -> git ndarray, branch master
local development -> same dep, transparently patched to ../ndarray
release binary -> exactly ONE resolved ndarray package
```
Expand Down
Loading