Tooling: exact Gram prefix reuse with pinned equivalence and timing evidence

.github/workflows/gram-prefix-performance.yml

@@ -0,0 +1,40 @@
+name: Exact Gram prefix performance
+
+on:
+  workflow_dispatch:
+  pull_request:
+    paths:
+      - 'research/gram-prefix-performance/**'
+      - '.github/workflows/gram-prefix-performance.yml'
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    strategy:
+      matrix:
+        python: ['3.11', '3.14']
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+        with:
+          persist-credentials: false
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
+        with:
+          python-version: ${{ matrix.python }}
+      - name: Exact checks and full frame regeneration
+        run: python -B research/gram-prefix-performance/verify.py
+      - name: Mutated kernels and enabled arithmetic under optimized Python
+        run: |
+          python -B research/gram-prefix-performance/test_kernel.py --mutations
+          python -O -B research/gram-prefix-performance/test_kernel.py
+      - name: Complete checker comparison without a timing threshold
+        run: python -B research/gram-prefix-performance/benchmark.py --pairs 7 --out "$RUNNER_TEMP/gram-prefix.json"
+      - name: Preserve frozen inputs
+        run: git diff --exit-code
+      - uses: actions/upload-artifact@v4
+        with:
+          name: gram-prefix-python-${{ matrix.python }}
+          path: ${{ runner.temp }}/gram-prefix.json

research/gram-prefix-performance/CLAIM.md

@@ -0,0 +1,43 @@
+# Sprint claim
+
+- Track: verification engineering.
+- Assignment seed: 88, drawn with `secrets.randbelow(100)` from operating-system randomness.
+- Task: `01a13378-812e-75b3-9b4b-0dd927f1901a`.
+- Start: `2026-10-13T04:15:36.018560+00:00`.
+- Base: `66849e1475c930fd7732b1a0067ffee7e1e95f60`.
+- Expected finish: `2026-10-13T05:00:00Z`.
+
+## Candidates
+
+1. Reuse exact leading principal minors in the selected bit Gram checker.
+2. Replace the selected owner checker's repeated rational exponential operations.
+3. Accelerate exact state evaluation in the selected helper-interface checker.
+
+PR672 covers candidate 2. PR662 covers candidate 3.
+Choose candidate 1. Use seed 88 for the matrix controls and benchmark order.
+
+## Scope
+
+Optimize determinant evaluation in the pinned `gram_frames.py` checker for the selected R420 bit word.
+Compute the leading principal minors in one exact Bareiss pass.
+Retain the original pivoted determinant for zero-pivot cases.
+
+Compare every requested determinant with the original implementation.
+Run the complete Gram checker on identical exported frames with both engines.
+Retain rank, annihilator, prime-witness, positivity, and failure checks.
+Provide exact source pins, negative controls, raw timings, and an optional adapter.
+
+The deliverable is a tooling improvement. It makes no new kappa claim.
+Reserve the final quarter of the session for verification and the handoff.
+
+## Coordination and predecessors
+
+GitHub reports no pinned issues at the initial inspection.
+The local claim follows the sprint instructions for that case.
+The overlap scan covers 513 open PRs through PR693 and the review ledger.
+PR239 uses dual Gram matrices for large frames.
+PR367 changes rational rank and canonical basis calculations for a historical verifier.
+
+PR452 shares producer prefixes and changes scalar admission in another historical pipeline.
+PR154 changes concurrency and binary frame operations.
+Those contributions use different mechanisms. None is a code dependency.

research/gram-prefix-performance/NOTICE.md

@@ -0,0 +1,25 @@
+# Attribution and license
+
+This contribution uses the repository's Apache-2.0 license.
+Prepared for Charles Bethin with substantial OpenAI Codex assistance.
+The implementation, tests, mathematical explanation, review, and packaging use that assistance.
+Internal agent review is not independent human peer review.
+
+The reference Gram checker and record exporter come from Dylan Crow
+(`utcorvusvolat-dotcom`), with Anthropic Claude assistance, through PR370 and
+its later retained copies in eumemic's carrier-skeleton package.
+The prime witness and determinant code comes from DreamingOfClouds through PR315.
+The selected R420 bit word is eumemic's PR543 contribution, with Anthropic Claude assistance.
+The new fixture contains the unchanged export of that word.
+All original files and their notices remain unchanged.
+
+The current selected composition combines that bit supplier with the PR626
+owner allocation by Chafik Boukhalfa, on Jeremy Kothe's PR622/623 suppliers.
+This package checks only the bit Gram stage and does not claim those other results.
+The retained package notices give the complete earlier construction lineage.
+
+Related work includes Richard Hoekstra's PR239 dual-Gram acceleration,
+Aman's PR367 frame-rank and canonical-basis optimization, and Aman's PR452
+shared-prefix replay work. PR154 improves concurrency and binary frame algebra.
+Concurrent PR662, PR667, and PR672 optimize other exact verification workloads.
+These contributions are credited context, not code dependencies.

research/gram-prefix-performance/PROOF.md

@@ -0,0 +1,108 @@
+# Exact determinant-prefix reuse
+
+## Claim and contract
+
+`PrefixDeterminant(reference_det)` returns the exact determinant of a square integer matrix.
+The claim assumes that `reference_det` returns an exact determinant when the fallback calls it.
+The selected reference is `det` in the pinned `prime_witnesses.py` source.
+The kernel accepts built-in Python integers and rejects other scalar types.
+The empty matrix has determinant 1.
+
+This result concerns determinant evaluation only.
+It does not prove the surrounding mathematical theorem or change its parameters.
+It does not independently verify the selected word, frame generation, or prime range.
+
+## Identity over the integers
+
+Let `A` be an `n` by `n` integer matrix.
+Let `D_r` denote its leading principal determinant of order `r`, with `D_0 = 1`.
+For indices `i,j >= k`, define the bordered minor:
+
+```text
+P_k(i,j) = det A[(0,...,k-1,i), (0,...,k-1,j)].
+P_0(i,j) = A[i,j].
+P_k(k,k) = D_(k+1).
+```
+
+The determinant identity of Sylvester gives:
+
+```text
+D_k P_(k+1)(i,j)
+    = D_(k+1) P_k(i,j) - P_k(i,k) P_k(k,j),
+    for i,j > k.
+```
+
+One proof first assumes that the leading block of order `k` is invertible over the rationals.
+Expand the bordered minors through the Schur complement of that block.
+The displayed identity becomes the formula for a determinant of order 2.
+Both sides are polynomials in the entries of `A`.
+The polynomial identity therefore holds for every integer matrix, including singular matrices.
+For `k=0`, it is directly the same formula with the empty determinant equal to 1.
+
+## Elimination invariant
+
+Before elimination step `k`, each active entry `work[i][j]` equals `P_k(i,j)`.
+The variable `previous` equals `D_k`.
+The initialization satisfies these statements at `k=0`.
+The pivot equals `D_(k+1)`.
+
+When the pivot is nonzero, the kernel applies the displayed identity to each trailing entry.
+The divisor `D_k` is nonzero because each earlier pivot is nonzero.
+The identity proves that each division has zero remainder and preserves the invariant.
+The kernel checks divisibility with `divmod` and raises `ArithmeticError` for a nonzero remainder.
+It also explicitly rejects a zero divisor.
+These checks remain active under `python -O`.
+
+The kernel records each pivot as the corresponding leading principal determinant.
+After the final elimination, `work[n-1][n-1]` equals `D_n`.
+The kernel records this value as well.
+For `n=1`, the only entry directly equals the determinant.
+This argument does not require symmetry, positive definiteness, or positive entries.
+Negative pivots and negative determinants cause no change to the identity.
+
+## Zero pivots and singular matrices
+
+A zero elimination pivot remains a valid value of its leading principal determinant.
+However, subsequent elimination can require division by that zero value.
+The kernel stops before this can occur.
+It calls the supplied reference on a separate copy of the full requested matrix.
+It retains only the leading minors already proved by the invariant and the returned full determinant.
+It does not infer any later leading minor from the interrupted pass.
+
+The fallback covers nonsingular matrices with a singular leading block, including matrices that require row swaps.
+It also covers singular matrices with an early zero pivot.
+A zero final entry needs no further elimination or division and directly gives determinant zero.
+
+## Cache validity and bounds
+
+Each key is an immutable tuple of rows containing every entry of one matrix.
+Python dictionary lookup checks full key equality, even when hashes collide.
+Thus, a hit identifies precisely the matrix whose determinant the kernel records.
+The kernel does not use mutable object identities as keys.
+Later mutation of the caller's matrix cannot create a stale hit.
+The kernel also leaves the caller's matrix unchanged.
+
+A miss discards the previous cache and creates one cache for the requested matrix.
+A hit retains this cache, so requests for successive prefixes reuse the same elimination.
+The cache stores at most `n` determinants after a miss of dimension `n`.
+The empty matrix creates no cache entries.
+An interrupted pass stores fewer prefixes plus the full determinant, still at most `n` entries.
+
+The bound counts entries, not bytes.
+The complete prefix keys contain at most a constant times `n^3` scalar references in total.
+Integer bit lengths and arithmetic costs depend on the input.
+The uncached elimination uses at most a constant times `n^3` exact arithmetic operations.
+This statement is not a constant-time or unit-bit-cost claim.
+
+## Scope of equivalence
+
+For each supported input, a cache hit returns a previously established exact determinant.
+On a miss, the invariant proves the result unless the reference handles a zero pivot.
+Under the reference contract, that fallback also returns the exact determinant.
+These cases establish equivalence to the reference for square integer inputs.
+
+The optimization helps when the caller requests a full matrix before its leading prefixes.
+The selected Gram checker uses this order with its positivity option.
+The kernel does not change rank, annihilator, factor, residual, or positivity checks.
+Correct determinant values alone do not establish the correctness of those surrounding checks.
+The cache and counters support sequential calls; the implementation makes no thread-safety claim.

research/gram-prefix-performance/README.md

@@ -0,0 +1,139 @@
+# Exact Gram prefix reuse
+
+This optional adapter reuses exact leading principal determinants in the selected R420 bit Gram checker.
+The checker first asks for a full determinant, then for each leading principal determinant.
+One Bareiss pass supplies those values.
+A zero pivot invokes the original pivoted determinant on the full matrix.
+
+**Status: tooling improvement; no new kappa.**
+The exact base is `66849e1475c930fd7732b1a0067ffee7e1e95f60`.
+The selected conditional result at that base is `0.001635527380375392`.
+This contribution preserves the selected certificate, frozen sources, default commands, and `upstream/`.
+
+## Reproduce
+
+Use Python 3.11 or newer and a full repository checkout.
+All dependencies belong to the Python standard library.
+The benchmark uses the POSIX `resource` module on macOS or Linux.
+No network connection is required.
+
+```sh
+python3 -B research/gram-prefix-performance/verify.py
+python3 -B research/gram-prefix-performance/test_kernel.py --mutations
+python3 -B research/gram-prefix-performance/benchmark.py --pairs 31 --out /tmp/gram-prefix-result.json
+```
+
+Use a fresh benchmark output path.
+Expected results: 23 passing tests, four detected kernel mutations, and `PASS_GRAM_PREFIX_BENCHMARK`.
+The first command prints `PASS_GRAM_PREFIX_VERIFICATION`.
+The mutation command reports the passing baseline before the four rejected mutations.
+
+Run either complete checker directly with the same pinned frame fixture:
+
+```sh
+python3 -B research/gram-prefix-performance/adapter.py --engine reference
+python3 -B research/gram-prefix-performance/adapter.py --engine prefix
+python3 -B research/gram-prefix-performance/adapter.py --engine compare --trace --out /tmp/gram-prefix-trace.json
+```
+
+All three print:
+
+```text
+PASS e8ham Gram: 672 frames checked, all nondegenerate and positive definite; prime witnesses valid; max determinant bits 17; primes [2]; by dim {0: 1, 1: 120, 2: 180, 4: 70, 6: 180, 7: 120, 8: 1}
+```
+
+## Exact coverage
+
+The adapter checks source hashes before execution.
+It inserts one assignment after the pinned checker imports `prime_witnesses.py`.
+That assignment replaces only `pw.det`.
+The checker retains every frame, rank, annihilator, prime-witness, residual, and positivity check.
+The reference mode uses the original determinant.
+The comparison mode compares every requested determinant with that original.
+
+All 3,360 determinant requests on the selected frames agree exactly.
+Their ordered matrix/value transcript has SHA-256:
+
+```text
+441d68e330a1b8b159fd914ceb2eab302d38eb50c87147ce62a513a02796dbef
+```
+
+The fixture contains all 672 exported frames, including frame zero.
+The verification command regenerates it from the pinned R420 word through the original exporter.
+It compares the full uncompressed payload byte for byte.
+Compressed container bytes do not determine mathematical equality.
+`SOURCE.json` binds the word, exporter, checker, determinant source, and fixture payload.
+
+The kernel suite includes an independent permutation determinant oracle through dimension five.
+It exhausts 530 binary matrices of dimensions one through three.
+Seed 88 also gives 240 small signed matrices, 108 positive Gram matrices, and 144 larger matrices through dimension sixteen.
+Larger cases use the original determinant and independent determinant identities.
+Named cases cover negative pivots, singular leading blocks, singular matrices, large integers, and mutable inputs.
+
+The four kernel mutations corrupt signs, cached minor values, cache keys, or zero-pivot handling.
+Each mutation causes assertion failures, without execution errors.
+Package controls reject altered dimensions, dependent bases, wrong annihilators, oversized prime residuals, singular Gram matrices, and failed positivity.
+The tested rejection arguments agree across all three engines.
+Source tampering, disabled checker assertions, and existing output paths also fail.
+The retained `--new` frame filter produces matching results in all engines.
+
+[PROOF.md](PROOF.md) gives the integer identity and cache argument.
+The proof assumes an exact reference determinant for the fallback.
+Finite tests corroborate the implementation; they are not a formal proof of the multiplication theorem.
+
+## Measurements
+
+Each evidence file records 31 alternating fresh-process pairs.
+The first runs measure wall time only.
+The follow-up runs also measure child user and system CPU time.
+Both sets remain in `evidence/`; no samples are removed.
+Both engines receive identical exported frames.
+Every measured complete process must produce the same output.
+Each separate determinant batch follows the actual 3,360 requests captured from the checker.
+Every measured batch compares its full result list with the original afterward.
+A fresh kernel starts each batch, so no previous batch supplies cached results.
+
+| Python | Run | Full process wall, reference → prefix | Child CPU, reference → prefix | Determinant batch, reference → prefix |
+|---|---|---:|---:|---:|
+| 3.13 | First | 113.79 → 115.15 ms | Not collected | 19.49 → 9.69 ms (2.01×) |
+| 3.13 | CPU follow-up | 161.59 → 148.07 ms | 150.98 → 138.00 ms | 21.61 → 11.11 ms (1.94×) |
+| 3.14 | First | 206.13 → 246.14 ms | Not collected | 18.83 → 10.16 ms (1.85×) |
+| 3.14 | CPU follow-up | 142.55 → 147.88 ms | 135.07 → 138.29 ms | 15.31 → 8.30 ms (1.84×) |
+
+The determinant batches improve consistently.
+Complete-process wall results vary in direction, so no reliable end-to-end wall speedup is claimed.
+
+Complete process timings include adapter startup and source/fixture validation.
+The determinant batches exclude imports, input collection, and result comparison.
+Filesystem caches are not flushed.
+Other applications can affect local timings; these are paired medians, not guaranteed speedups.
+The benchmark has no timing threshold and retains every sample.
+`wall-only-benchmark.py.txt` preserves the first measurement script.
+Its hash matches the earlier receipts; the current script matches the CPU receipts.
+
+These measurements do not include word construction, record export, full supplier replay, or integer multiplication.
+No full-frontier speedup follows from this small stage measurement.
+
+## Review map and limits
+
+- `kernel.py`: exact determinant implementation with a bounded prefix cache.
+- `adapter.py`: source-pinned execution of the retained checker.
+- `test_kernel.py`: independent arithmetic tests and kernel mutations.
+- `test_package.py`: full-frame reproduction, equivalence, and package controls.
+- `benchmark.py`: reproducible paired measurements with source hashes.
+- `evidence/`: raw local measurements and validation receipts.
+- `CLAIM.md`: assignment, candidates, and overlap check.
+- `NOTICE.md`: predecessor attribution, license, and AI disclosure.
+
+The cache stores at most one requested matrix's leading prefixes.
+It uses exact immutable matrix values and does not retain successful checker results.
+The public kernel accepts square matrices of built-in integers.
+It does not support arbitrary scalar types or concurrent calls on one instance.
+The adapter rejects optimized Python because the retained checker uses assertions.
+The kernel's arithmetic guards remain active under optimized Python.
+
+The local checks cover only this optional adapter and the pinned Gram stage.
+The full `make verify`, supplier cold replay, assembly, and Lean checks are not rerun for this addition.
+Dedicated Linux CI is supplied; its remote status remains separate from local results.
+All inherited compiler, hosting, precision, tape, and analytic assumptions remain.
+Direct activation in a frozen production package requires separate integration and affected source-pin updates.

research/gram-prefix-performance/SOURCE.json

@@ -0,0 +1,12 @@
+{
+  "base_commit": "66849e1475c930fd7732b1a0067ffee7e1e95f60",
+  "assignment_seed": 88,
+  "source_sha256": {
+    "research/e8-bit-carrier-skeleton/code/bit/checkers/gram_frames.py": "223a5a07731fc315a90a3e48177796babc155c1d9985c7ae5017eff6af2a0608",
+    "research/e8-bit-carrier-skeleton/code/pricing/prime_witnesses.py": "98cd03bc9582e7bd0ab194c191ea8eea940539e510051443ab9fd3e572a364ab",
+    "research/e8-bit-carrier-skeleton/code/bit/export_records.py": "8a5999601f917dbb492bb9ae9a344611f18cc7cedc53dc3c76496a369e530eb3",
+    "research/e8-bit-r420/data/bitE8mut-r420.json.gz": "9721a3819efd68307f66f00d84a952eb28eb7c3ad2ce79653ce295c98a764784"
+  },
+  "frames_payload_sha256": "46f49b700fa3ecd59ba6815b58234c0b67b94360f7af52a718b275b8198c1c74",
+  "frames": 672
+}