Skip to content

Reduce exact downset replay time with integer dot products - #5002

Merged
Mister-Meeseeks merged 2 commits into
CrocSwap:mainfrom
aliromman:codex/exact-matrix-product-20261023
Oct 23, 2026
Merged

Mister-Meeseeks merged 2 commits into
CrocSwap:mainfrom
aliromman:codex/exact-matrix-product-20261023

Conversation

@aliromman

@aliromman aliromman commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Change and mathematical scope

Reduces the full downset proof replay from 19.814 to 16.715 seconds in a final local comparison, about 16% less time, by replacing repeated Fraction operations in matrix multiplication with scaled integer dot products. Every original scientific evidence field and rejection-witness count stays equal. This is a verifier runtime improvement; the proof and complete kappa calculation do not change.

Arithmetic identity and evidence, implementation and direct product oracle, and unchanged source pins.

Each row of the left matrix and each column of the right matrix is scaled by its denominator LCM. Their dot product uses integers, then one Fraction reduction per output. There is no cache or floating point arithmetic. The valid rectangular rational input contract and fixed matrix capacities are unchanged.

Task and overlap

Takes the non-caching multiplication follow-up from PR #4947. Base: f259dabc7d70e0db8d6d36bfda888f008bbfecc1, priority 7. That main-branch file still contains the original inverse algorithm. This PR leaves it unchanged; the timing comparison is against those exact main bytes, not PR #4947's inverse-cache version. The original package metadata, frozen proof README and historical provenance pins remain byte-exact. Runtime notes are in a separate file because the Full-allocation package authenticates the proof document.

The final search covered 300 open PRs, 4,098 Discussion records and 69 of 6,710 local notes. It found the separate inverse-cache and Full-allocation work. Sparse omitted notes, arbitrary diffs and unpublished work remain limits.

Validation

CPython 3.12.13, standard library:

python3.12 -B scripts/research_package.py --package research/downset-block-solve-bound

Expected: both commands print PASS_DOWNSET_BLOCK_SOLVE, followed by PASS_REGISTERED_RESEARCH_PACKAGE. Normal and optimized replay passed. All original counts remain equal: 46 trees, 2,380 states, 9,520 matrix identities, 23,504 spectral certificates, 9,372 partition blocks and eleven original control groups. The added product_controls: 93 field records signed/zero/rectangular Fraction-oracle comparisons, ownership/type checks and invalid-input controls. Four actual arithmetic mutations fail against that oracle.

Both registered normal/optimized replays also passed for the dependent Full-allocation package (63 contracts, six fixtures, 24 trajectories and seven controls). Its frozen source bindings are unchanged.

python3.12 -B scripts/research_package.py --package research/full-downset-allocation

Expected: PASS_FULL_DOWNSET_ALLOCATION actual_factory_same_stored_inverse no_new_kappa and PASS_REGISTERED_RESEARCH_PACKAGE.

Exact source-pin checks, committed-byte checks for both packages and Git-free read-only optimized export replay passed. The full-history wrapper was not run in this shallow checkout; all four historical blob/SHA-256 pins were checked directly. No selected cold replay, Lean, Docker or archive-wide sweep ran.

Timing uses sequential full subprocess replays on the same shared macOS host. Two earlier trials measured 19.488/21.431 seconds before and 16.765/16.696 seconds after; the final pair includes the new controls. This is local timing evidence, not a cross-platform guarantee.

Credit

Nick Petalas supplied the mathematics, fixtures and original checker; see the original source notice. Brian (JoyboyBrian) identified the remaining product bottleneck in PR #4947. Their predecessor credits are retained. Prepared for Ali Romman with substantial OpenAI Codex assistance; Apache-2.0.

Open next questions

The effect when combined with the separate inverse-cache implementation has not been measured. Other rational verifiers need their own bottleneck profile and exact equivalence checks before using this change.

@jwr404

jwr404 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Independent check at head afe5f0d8, on Linux with CPython 3.12.14.

  • Identity. (AB)_ij = Σ_k (d_i A_ik)(e_j B_kj) / (d_i e_j), with d_i and e_j the row and column denominator LCMs, is exact. The implementation also accepts integer entries (int.numerator and int.denominator exist), and lcm() of an empty row is 1. The shape check is unchanged.
  • Equivalence. I ran main f259dabc and this head back to back and compared the printed receipts. Every top-level field except source_sha256 is identical, including the whole finite_evidence object (46 trees, 2,380 states, 23,504 spectral certificates, all rejection-witness counts). The only addition is product_controls: 93. The only hash changes are README.md and verify.py.
  • Speed-up. Two runs each: 26.84 / 26.75 s on main versus 22.89 / 22.97 s with this PR, about 14.5% less. That is consistent with the reported 16% on macOS.

No coverage is dropped. This is an author-independent check, not maintainer acceptance.

Reviewed by James Reed with substantial Anthropic Claude (Claude Code) assistance.

@Mister-Meeseeks
Mister-Meeseeks merged commit 5948b87 into CrocSwap:main Oct 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants