research(nightly): structural-time-gated memory compaction scheduling — REJECT, evidence retained

[ruvnet]

Nightly RuVector Research — 2026-09-18

Topic: Structural-Time-Gated Memory Compaction Scheduling
Slug: structural-time-gated-memory-compaction
Acceptance: REJECT (for production use as calibrated) — real measured evidence retained, per the nightly process's own rule that a well-characterized falsified hypothesis is a successful run.

Hypothesis

ruvector-agent-memory has three CompactionPolicy implementations that decide what survives a compaction pass, but nothing in the crate has ever decided when to run one — every existing benchmark compacts a store exactly once, on demand. This experiment adds a CompactionTrigger trait and asks whether emergent-time's existing, independently-tested StructuralProperTime clock ("time = arc length through a system's own state manifold"), reused unmodified, can gate compaction scheduling better than a naive fixed-interval or capacity-ceiling trigger — staying quiet through near-duplicate writes and firing promptly on bursts of genuinely new material.

Full Given/When/Then and four pre-registered numeric acceptance thresholds: ADR-346.

Architecture

• crates/ruvector-agent-memory/src/structural_gate.rs (new, feature-gated structural-gate, off by default): CompactionTrigger trait + FixedIntervalTrigger, CapacityTrigger, StructuralGateTrigger (bounded O(window*dims) per write, not O(store.len()*dims) — a direct response to the prior nightly's, 2026-09-05 mincut-gated-forgetting, rejection for an O(n)-or-worse structural signal).
• New optional path dependency on emergent-time (existing workspace crate, reused as-is — no reimplementation).
• examples/structural_gated_compaction_bench.rs: the full deterministic, seeded comparison benchmark.
• A genuine, previously-unreachable correctness bug found and fixed along the way: MemoryStore::insert assigned id = entries.len(), which collides with a surviving entry's id once compaction and further insertion interleave (every prior call site only ever compacted once, terminally). Fixed to a monotonic counter — pure correctness fix, independent of which trigger wins, full existing test suite passes unchanged.

Benchmark

cargo run --release -p ruvector-agent-memory --features structural-gate \
  --example structural_gated_compaction_bench

trigger           compactions excess_size_integral      recall@10      wall_ms final_size
FixedInterval              24                30551         1.0000        6.065        200
Capacity                    5               120615         1.0000        2.437        200
StructuralGate            100                18913         1.0000       22.031        200

=== Diagnostic: fire location (1200 Quiet writes, 200 Burst writes) ===
trigger             fires@quiet    fires@burst
FixedInterval                20              4
Capacity                      4              1
StructuralGate               30             70

=== Acceptance (vs. FixedInterval, thresholds fixed pre-run) ===
compaction-call reduction:    -316.7%  (need >= 20%)
excess-size-integral reduction:    38.1%  (need >= 20%)
recall@10 gap:                 0.0000  (need <= 0.02)
wall-clock ratio (struct/fixed):    3.63x (need <= 2.0x)

ACCEPTANCE RESULT: REJECT

3 repeated release-mode runs: algorithmic columns bit-identical (fully deterministic, seeded), wall-clock varied 6.2–6.3ms (fixed) / 22.2–23.5ms (structural) — REJECT is robust to run-to-run timing noise, not a single unlucky measurement.

Root cause (full writeup in the ADR/report): StructuralProperTime's coherence channel accumulates only on loss, which is correct for genuine irreversible drift but means a small sliding-window coherence estimate's sampling noise — half of which is downward fluctuation — reads as monotone drift. A threshold calibrated from the quiet baseline's mean tick was still within that baseline's own variance. The diagnostic shows the signal is real, not noise-only (70 of 100 fires land on the 14.3% of writes that are genuine bursts — a ~14x fire-density skew), but the tested calibration wasn't conservative enough to also win on absolute call count.

Darwin / Flywheel / MetaHarness

Per this run's tool-discovery pass (Step 0 of the nightly process): npx metaharness --help resolves to an installed package that scaffolds new harness projects — not an in-repo research-orchestration layer usable against this repo. npx ruvector harness doctor --json does not resolve to any executable in this environment. No Darwin/Flywheel tool output is fabricated; the research → hypothesis → implementation → measurement → critique → promotion-or-rejection loop was carried out directly in this session, with the diagnostic fire-location breakdown serving as the adversarial root-cause pass. No Darwin evolutionary sweep was run this session — a single pre-registered calibration was tested and rejected clearly enough that a same-session parameter retune would have violated the "don't move the goalposts" rule; a follow-up with an independently-registered calibration hypothesis is the correct next step (see ADR-346 §Next Research).

Security review

No new attack surface: the trigger only reads MemoryEntry vectors already resident in the store and compares a scalar accumulator to a scalar threshold — no untrusted parsing, no new serialization format, no witness/signature involvement. structural-gate is an additive, opt-in Cargo feature (off by default) with no default-path exposure.

Main limitations

• Single synthetic dataset shape; not validated against a real write trace.
• Wall-clock comparison conflates per-write trigger cost and the cost of 100 vs. 24 actual compact() calls — not decomposed.
• No deletes, no concurrent writers, no adversarial streams tested.
• Two of the clock's five channels (graph, prediction error) left at 0.0 rather than fed a fabricated signal.

Production recommendation

Do not promote as calibrated. Keep the CompactionTrigger trait and the MemoryStore id fix (unconditionally useful); do not wire StructuralGateTrigger into any default path. Next attempt should calibrate from both quiet and burst reference data (not quiet alone) and isolate whether the loss-only coherence asymmetry or the entropy-histogram noise is the larger contributor before re-testing against the same four thresholds.

Docs

• ADR-346
• Nightly research report
• Public technical gist

Build/test

cargo fmt clean, cargo build --release clean, cargo clippy --features structural-gate --examples --tests -- -D warnings clean, cargo test (all feature combinations: structural-gate,mincut-forget,proof-gate) — 71 tests, 0 failures.

---

🤖 Generated with claude-flow

https://claude.ai/code/session_01W9AzJk52mCWmwLy4tkHtVd

---

Generated by Claude Code