Skip to content

Nightly research: ApproxMincutForgetting fixes ADR-345's latency bottleneck, not its correctness gap (ADR-346) - #1000

Draft
ruvnet wants to merge 2 commits into
mainfrom
claude/focused-darwin-y4wp8i
Draft

ruvnet wants to merge 2 commits into
mainfrom
claude/focused-darwin-y4wp8i

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

Autonomous nightly research run. Direct follow-up to ADR-345 (2026-09-05, mincut-gated-forgetting), which rejected MincutGatedForgetting (a ruvector-agent-memory compaction policy protecting structural "bridge" memories via ruvector-mincut::RuVectorGraphAnalyzer) for two reasons: no measured bridge-survival benefit, and RuVectorGraphAnalyzer::partition() measured 1,800-2,700x slower than the scalar baseline.

This run reads ruvector-mincut's source for the alternative that ADR-345's own "Alternatives Considered" left untried, and finds ApproxMinCut — a distinct, already-implemented "(1+ε)-approximate min-cut" entry point (docstring cites SODA 2025, arXiv:2412.15069) — sitting unused. Before running any benchmark, reading ApproxMinCut::compute_partition's source raised a specific, falsifiable concern: it never uses the min-cut value it computes to build its returned partition. The hypothesis was fixed at that point (speed will likely pass, correctness will likely fail) and then measured.

Result: both halves of the pre-registered prediction were confirmed, plus a bonus finding.

  1. Speed bottleneck: fixed. ApproxMinCut is ~22-24x faster than RuVectorGraphAnalyzer::partition() on the identical 84-memory corpus — a large, reproducible improvement over ADR-345's 1,800-2,700x slowdown.
  2. Correctness gap: confirmed, and root-caused. ApproxMinCut::compute_partition (crates/ruvector-mincut/src/algorithm/approximate.rs) BFS-walks from an arbitrary start vertex and stops once exactly half the total vertex count is visited — unrelated to the cut it reports. A new executable probe (examples/approx_mincut_partition_probe.rs) demonstrates this directly: correct on a balanced two-triangle graph (coincidence), wrong on an unbalanced 9-clique+bridge+3-clique graph, while the reported cut value is correct in both.
  3. Bonus finding: non-determinism. ApproxMinCut's internal HashSet<VertexId> uses Rust's default randomized hasher, so its output varies run-to-run against byte-identical input. Confirmed at the unit-test level (a test passed in isolation, failed 2/6 separate process invocations) and at the corpus level (ApproxMincutForgetting-Soft bridge survival ranged 8.3%-58.3% across repeated trials, never once reaching the 66.7% do-nothing baseline).

Acceptance: REJECT (for production use as a bridge-protection mechanism) — full evidence in the ADR and research doc below.

Architecture

  • crates/ruvector-agent-memory/src/graph_forget_approx.rsApproxMincutForgetting (Soft/Hard modes), mirroring ADR-345's MincutGatedForgetting with only the mincut engine swapped, reusing the existing mincut-forget feature flag.
  • crates/ruvector-agent-memory/examples/approx_mincut_partition_probe.rs — balanced vs. unbalanced graph correctness probe.
  • crates/ruvector-agent-memory/examples/approx_mincut_forgetting_bench.rs — corpus-level benchmark (baseline, ADR-345's exact engine, this ADR's approximate engine), using a mean of 10 in-process trials for the non-deterministic rows.

Files Changed

  • crates/ruvector-agent-memory/src/graph_forget_approx.rs (new)
  • crates/ruvector-agent-memory/examples/approx_mincut_forgetting_bench.rs (new)
  • crates/ruvector-agent-memory/examples/approx_mincut_partition_probe.rs (new)
  • crates/ruvector-agent-memory/Cargo.toml, crates/ruvector-agent-memory/src/lib.rs (wiring, no new feature flag)
  • docs/adr/ADR-346-approx-mincut-forgetting-partition-gap.md (new)
  • docs/research/nightly/2026-09-19-approx-mincut-forgetting/README.md, gist.md (new)
  • docs/adr/INDEX.md (regenerated via node scripts/adr-index.mjs; next available ADR-347)

Benchmark Command & Real Results

cargo run --release -p ruvector-agent-memory --example approx_mincut_partition_probe --features mincut-forget
cargo run --release -p ruvector-agent-memory --example approx_mincut_forgetting_bench --features mincut-forget
Policy                            Bridge Surv.   Recall@10  Compaction (us)
------------------------------------------------------------------------------
CoherenceWeighted                        66.7%      100.0%               58
MincutGatedForgetting-Soft               66.7%      100.0%           105018
MincutGatedForgetting-Hard               66.7%      100.0%           106380
ApproxMincutForgetting-Soft             39.2%*       97.2%             4425
ApproxMincutForgetting-Hard             66.7%*      100.0%             4607
  * mean of 10 in-process trials; ranges: Soft min=8.3% max=58.3%, Hard min=max=66.7%

Acceptance test (candidate B = ApproxMincutForgetting)
  Soft speedup vs. exact (23.7x) >= 10x : PASS
  Hard speedup vs. exact (23.1x) >= 10x : PASS
  Soft bridge-survival gap (-27.5pp) >= 15pp : FAIL
  Hard bridge-survival gap (-0.0pp) >= 15pp  : FAIL
  Soft |recall delta| (2.80pp) <= 2pp        : FAIL
  Hard |recall delta| (0.00pp) <= 2pp        : PASS

Acceptance Result

REJECT for production use as a bridge-protection mechanism. Speed gate passes decisively; the mandatory bridge-survival gate fails for both modes. Per the nightly process's own rule, a falsified hypothesis with a well-characterized, code-level root cause is a successful research outcome — this converts "mincut-based bridge protection is too slow" (ADR-345) into a much more specific, cheaply-reproducible claim: "this crate's fast min-cut entry point has a one-function partition-extraction bug, plus a hasher-order non-determinism bug, both independent of the speed question."

Darwin / Flywheel

Not run: no candidate here clears its own acceptance bar, so there is nothing to evolve, and no Darwin/Flywheel CLI was found installed in this environment (checked via npx metaharness --help — a generic scaffold generator, not applicable — and npx ruvector harness doctor/status — not a resolvable package). Evidence retention is via this PR's ADR and nightly docs, following the same in-repo convention ADR-345 used.

Security Review

No new cryptographic primitive. ApproxMincutForgetting's output is an advisory ranking signal only; it cannot corrupt the existing witnessed_compaction eviction-witness chain or bypass target_size. That chain is generic over any CompactionPolicy and was already validated 20/20 by ADR-345's own benchmark, so it was not re-exercised here (documented rationale in the research doc).

Main Limitations

  • Baseline/exact-engine rows are single runs (no measured variance found); approximate-engine rows are means of 10 in-process trials given the newly-found non-determinism — see the research doc's "Limitations" for what that does and does not characterize.
  • Single dataset seed (341, chosen for direct comparability with ADR-345).
  • DynamicMinCut (a third engine in the same crate) was inspected but not benchmarked — its approximate config flag was found unwired into its actual computation path, so it was deprioritized as very unlikely to differ from RuVectorGraphAnalyzer's profile; flagged as an open question instead.

Production Recommendation

Do not enable ApproxMincutForgetting (either mode) in production. Soft mode is actively harmful (worse than doing nothing in every sampled trial); Hard mode is merely inert (no better than doing nothing). The concrete, scoped next step if this capability is still wanted: patch ApproxMinCut::compute_partition upstream to derive its partition from the same computation as its cut value (and make its arbitrary-start-vertex choice deterministic), then re-run approx_mincut_forgetting_bench.rs completely unchanged.

Test Plan

  • cargo test --release -p ruvector-agent-memory --features mincut-forget — 34/34 unit tests green (3 new)
  • cargo clippy --release -p ruvector-agent-memory --features mincut-forget --examples --tests — no new warnings
  • cargo fmt -p ruvector-agent-memory -- --check — clean
  • Both new examples run to completion and print the acceptance table above
  • node scripts/adr-index.mjs --check — passes, no duplicate ADR numbers

Read the full ADR (docs/adr/ADR-346-approx-mincut-forgetting-partition-gap.md) and nightly research doc (docs/research/nightly/2026-09-19-approx-mincut-forgetting/README.md) for full methodology, raw output, and the standalone gist.


🤖 Generated with claude-flow

https://claude.ai/code/session_015x5MSakF9UGbWojMAexddu


Generated by Claude Code

claude and others added 2 commits September 19, 2026 07:37
…y policy (ADR-346)

Nightly follow-up to ADR-345: swaps ruvector-mincut's RuVectorGraphAnalyzer
(measured 1,800-2,700x slower than scalar compaction) for its unused
ApproxMinCut entry point in a new CompactionPolicy variant, isolating the
mincut engine as the only variable versus the existing MincutGatedForgetting.

Includes two executable probes: a balanced/unbalanced min-cut partition
correctness check, and a corpus-level benchmark (mean of repeated in-process
trials, since ApproxMinCut's partition turned out to be non-deterministic
against identical input) comparing baseline, the exact engine, and this one.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_015x5MSakF9UGbWojMAexddu
…etting

Records the measured evidence: ApproxMinCut is ~22-24x faster than
RuVectorGraphAnalyzer on the ADR-345 corpus (fixing that ADR's rejected
bottleneck), but its compute_partition never derives its returned partition
from the cut it computes, so bridge-survival stays at baseline parity (Hard
mode) or gets actively worse (Soft mode) across every sampled trial. Also
documents a second, independent finding surfaced while writing this
experiment's tests: ApproxMinCut's boundary output is itself non-
deterministic against identical input, traced to a randomized-hasher-order
dependency in its internal vertex set.

Regenerates docs/adr/INDEX.md via `node scripts/adr-index.mjs` to register
ADR-346 (next available: 347); the check gate passes.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_015x5MSakF9UGbWojMAexddu

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants