Reduce E8 recursive cost with four coupled frame changes

.github/workflows/e8-coupled-retiming.yml

@@ -0,0 +1,29 @@
+name: E8 coupled retiming
+
+on:
+  pull_request:
+    paths:
+      - 'research/e8-coupled-retiming/**'
+      - '.github/workflows/e8-coupled-retiming.yml'
+  push:
+    paths:
+      - 'research/e8-coupled-retiming/**'
+      - '.github/workflows/e8-coupled-retiming.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
+        with:
+          python-version: '3.14'
+      - run: python3 -B research/e8-coupled-retiming/verify.py
+      - run: git diff --exit-code

research/e8-coupled-retiming/NOTICE.md

@@ -0,0 +1,78 @@
+# Attribution and scope of new work
+
+Jacob Sussman supplied the E8 label family, circuit, scatter, frame scheduler,
+checkers, price routines, and general proof framework in
+`jacobalansussman/wht-power-saving-lean`, commit
+`9c94857bbaee886f8649edee0bc2d3c738f9555e`.
+
+`data/original.json.gz` is an unchanged copy from that commit. The new
+certificate derives from that file. This package uses the repository's
+[Apache-2.0 license](../../LICENSE). The source notices that apply to the
+retained certificate are copied below.
+
+DaysSky's CrocSwap PR352 supplies the connection to the complex branch and the
+exact coarse-moment convention used here. Its commit
+`04b4c3c478b5bd8797d2663d89d8db4dfd0ad9f2`, source paths, and hashes are
+recorded in `sources.json`. Prior frame-retiming contributions
+include CrocSwap PR328 and PR350. No claim of priority for the general method is
+made.
+
+The new work consists of four explicit frame changes, a deterministic
+generator, independent scalar and frame checks, exact rational cost bounds,
+and the short cost proof. OpenAI Codex and its research agents supplied
+substantial assistance. New contributions are
+offered under the repository's Apache-2.0 license.
+
+Attribution does not imply review or endorsement by the credited authors.
+
+## Upstream notice excerpts
+
+The following text is copied from the copyright header and sections 4 and 8
+of the [upstream NOTICE](https://github.com/jacobalansussman/wht-power-saving-lean/blob/9c94857bbaee886f8649edee0bc2d3c738f9555e/NOTICE).
+References within these excerpts refer to that source repository.
+
+```text
+Walsh-Hadamard power saving (wht-power-saving-lean)
+Copyright 2026 Jacob Sussman
+
+Repository: https://github.com/jacobalansussman/wht-power-saving-lean
+
+4. How the new files were made.
+   Jacob Sussman made the new files on 2026-10-08, 2026-10-09 and 2026-10-10
+   by directing a team of AI agents (Claude). The agents wrote the new Lean
+   files, the certificate data and the generator scripts. They made the
+   rebuild scripts under tools/rebuild/, in part by adapting programs of
+   CrocSwap/integer-mult-bounds (section 6): ten of the thirteen on
+   2026-10-09, tidied for this repository on 2026-10-10, and three on
+   2026-10-10. On 2026-10-10 they also wrote the search programs under
+   tools/e8/ and found with them the certificate of the fourth result
+   (section 8). See README.md, section 4.
+
+8. The certificate of the fourth result and the search programs.
+   tools/certificate/gcert1-e8-r783.json.gz, the Lean data modules
+   generated from it (Work/GCert/Data/Gen/E8*.lean, Work/GCert/Data/Gen/E8/
+   and the modules of Work/GCert/Data/ with E8 or B2Ge8 in their names) and
+   the Python programs under tools/e8/ were made by the agents for this
+   repository on 2026-10-10 (section 4). The circuit in that certificate
+   is not the circuit of section 6, and no data file of
+   CrocSwap/integer-mult-bounds was loaded to make it. It uses three
+   devices that were published by others in pull requests to that
+   repository:
+   - the in-place pair, a + b and a - b formed on the two arrays that held
+     a and b: icekylinx (#184), carried in #191 and #193 (ikeboy);
+   - the re-use of a finished helper with a compensating read: jamesyc
+     (#124) and eumemic (#143);
+   - the late pairing rule, the choice of which finished helper a new
+     value takes: Chafik Boukhalfa (the account chafreaky), #200 and #233,
+     and pull request #2 of this repository. The search programs apply the
+     idea with a greedy rule of their own; his program, his pairs and his
+     certificate were not used.
+   No program of that repository is included in tools/e8/, imported or
+   run by it. The programs of tools/e8/ have not been compared statement
+   by statement with the programs of that repository, as those of
+   tools/rebuild/ were (section 6). The integer programme of
+   tools/e8/ is solved with the open-source solver HiGHS, called through
+   SciPy (scipy.optimize.milp), with NumPy; none of the three is included
+   here. The credits, with links, are in README.md ("Whose ideas the new
+   unit uses") and in RELATED-WORK.md, section 8.
+```

research/e8-coupled-retiming/PROOF.md

@@ -0,0 +1,170 @@
+# Four E8 frame changes
+
+All gate and register indices below are zero-based indices in the original
+certificate. Binary frames are given by integer bit masks. The new certificate
+has uncompressed SHA-256
+`3191011e0cb189665c8e0a9b97137dbed50241a0704c8be01c11b1b5a9518fc2`.
+
+## Circuit changes
+
+1. Remove `helper388 -= source64` and `helper771 += source64` from A65.
+   Insert those two additions just before A187, at frame 285 with basis
+   `[196,56]`. No gate A66 through A186 touches any of these three registers.
+   Source 64 has label 56.
+2. Remove `helper693 += source98` and `helper839 -= source98` from A100.
+   Insert those two additions just before A198, at frame 296 with basis
+   `[257,243,14]`. No gate A101 through A197 touches any of these three
+   registers. Source 98 has label 253, and `253 = 243 XOR 14`.
+3. In B193, B916, and B917, replace frame 599, with basis
+   `F=[276,148,84,51,15]`, by `G=[256,128,64,39,20,15]`.
+   These gates add helper807 to helper766, distribute plus or minus half of
+   helper807 to targets149 and155, and add helper766 to helper807.
+4. In B873 and B874, replace frame 585, with basis `[161,81,9,5,3]`,
+   by `[305,161,81,9,5,3]`. The first gate adds half of helper550 to target198
+   and subtracts half of helper550 from target202. The second gate adds
+   helper521 to helper550.
+
+The first two changes move additions only across gates that touch disjoint
+registers. Such scalar operations commute, for all initial register values.
+Neither change crosses the scatter. The last two changes keep all scalar
+operations and their order. Thus the complete scalar operator and the retained
+totals at scatter stay the same. The verifier also compares the complete
+1023-by-1023 rational scalar matrices.
+
+All paths remain nested. The verifier checks actual binary subspace inclusion,
+not only dimensions. For the third change, `G=F+span(256)` has dimension six.
+The identities
+
+```
+276 = 256 XOR 20
+148 = 128 XOR 20
+84  = 64 XOR 20
+51  = 39 XOR 20
+```
+
+show that F is contained in G. The preceding helper frames are contained in F.
+The next helper frames are
+
+```
+helper766: [256,128,64,37,20,13,2]
+helper807: [256,128,64,38,20,14,1]
+```
+
+They contain G because `39=37 XOR 2=38 XOR 1` and
+`15=13 XOR 2=14 XOR 1`. The final target frames also contain G:
+
+```
+target149: [256,128,64,32,18,8,6,1]
+target155: [256,128,64,34,16,10,4,1]
+```
+
+For target149, use `39=32 XOR 6 XOR 1`, `20=18 XOR 6`, and
+`15=8 XOR 6 XOR 1`. For target155, use `39=34 XOR 4 XOR 1`,
+`20=16 XOR 4`, and `15=10 XOR 4 XOR 1`.
+
+For the fourth change, let `F=span(161,81,9,5,3)` and
+`G=F+span(305)`. These frames have dimensions five and six. All preceding
+frames are contained in F. The next frame of helper521 is
+`span(257,145,81,48,9,5,3)`. The next frame of helper550 and both targets is
+`span(304,160,80,8,4,2,1)`. Both contain F. They contain the added vector
+because `305=257 XOR 48=304 XOR 1`. Thus both contain G, and the paths
+remain nested.
+
+## Child ranks
+
+Each climb contributes a child whose rank is the increase in frame dimension.
+The four changes affect disjoint sets of registers. Thus their histogram
+differences add.
+
+| Change | Register | Old child ranks | New child ranks |
+| --- | --- | --- | --- |
+| 1 | source64 | 5,3 | 1,4,3 |
+| 1 | helper388 | 1,1,2,2,3 | 2,2,2,3 |
+| 1 | helper771 | 1,1,1,3,1,2 | 2,1,3,1,2 |
+| 2 | source98 | 5,3 | 2,3,3 |
+| 2 | helper693 | 1,2,1,5 | 3,1,5 |
+| 2 | helper839 | 1,2,1,2,1,1,1 | 3,1,2,1,1,1 |
+| 3 | helper766 and helper807, each | 3,2,2,1,1 | 3,3,1,1,1 |
+| 3 | target149 and target155, each | 5,3 | 6,2 |
+| 4 | target198 and target202, each | 5,2,1 | 6,1,1 |
+| 4 | helper521 | 2,1,2,2,1,1 | 2,1,2,1,1,1,1 |
+| 4 | helper550 | 2,1,2,2,2 | 2,1,3,1,2 |
+
+The combined invocation histogram difference is
+
+```
+rank:     1   2   3   4   5   6
+change:  +2  -6  +4  +1  -6  +4
+```
+
+Its rank mass is zero and its child count is minus one. Copy cost remains 72.
+The five-stage histogram is five times the invocation histogram, plus 240
+children at each rank 4, 8, 16, and 20. Therefore its rank mass remains 56,715,
+with dimension m=45, stock W=1263, and deficit mW-56715=120.
+
+## Strict cost inequality
+
+Let a be a saving, with 0<a<1, and put p=1-a. A rank-r child costs r^p, apart
+from the common factor 1/(W m^p). Put `s_r=r^p`, with `s_0=0`. The combined
+invocation price difference is
+
+```
+D(p) = 2*s_1 - 6*s_2 + 4*s_3 + s_4 - 6*s_5 + 4*s_6
+     = sum_{k=1}^5 c_k*(s_{k-1} - 2*s_k + s_{k+1}),
+c = [1,4,1,2,4].
+```
+
+For `0<p<1`, the function `x^p` is strictly concave on `[0,6]`.
+Each second difference is strictly negative. All five weights are positive,
+so `D(p)<0`. The verifier checks the coefficient identity against the actual
+histogram difference. The raw five-stage moment decreases by
+`5*D(p)/(W*m^p)<0`. This is a proof for the combined changes. It does not
+claim that each separate change improves the cost for every saving.
+
+## Fallback and exact savings
+
+Use the [PR352 moment convention](https://github.com/DaysSky/integer-mult-bounds/blob/04b4c3c478b5bd8797d2663d89d8db4dfd0ad9f2/research/w3-p10b-e8-complex/code/pricing/moment.py#L55)
+at commit `04b4c3c478b5bd8797d2663d89d8db4dfd0ad9f2`. Its
+[E8 application](https://github.com/DaysSky/integer-mult-bounds/blob/04b4c3c478b5bd8797d2663d89d8db4dfd0ad9f2/research/w3-p10b-e8-complex/code/complex/certify_complex.py#L34)
+and both source hashes are recorded in `sources.json`. If H is the five-stage
+histogram, the moment is
+
+```
+M(a) = (1/W) sum_r H_r (r/m)^(1-a)
+       + (32*m*sum_r H_r)/(10^16*W) * m^a.
+```
+
+The second term charges the rare bad classes. The candidate removes five
+five-stage children, so this term also strictly decreases. The improved
+moment is smaller for every 0<a<1, independently of any decimal estimate.
+
+`reconstruct.py` uses rational logarithm and exponential enclosures to prove
+
+```
+M_original(876248285600677 / 10^18) < 1
+M_original(876248285600678 / 10^18) > 1
+M_candidate(876412559609473 / 10^18) < 1
+M_candidate(876412559609474 / 10^18) > 1.
+```
+
+The logarithm uses the atanh series with a positive geometric tail bound.
+Arguments are reduced to [1,2]. The exponential uses a Taylor polynomial of degree 20 with
+a geometric tail bound. All arithmetic in these enclosures is rational.
+Decimals in the saved result are only displays of the proved margins.
+
+## Limits
+
+This proves the finite circuit and recursive-cost improvement under the
+stated convention. The general frame-transfer and recursion theorems remain
+dependencies. A complete scalar identity is not a proof of the tape compiler
+or the integer-multiplication assembly. No global optimum or larger final
+integer-multiplication saving is claimed.
+
+The local verifier checks the finite rational circuit, its frame paths, and
+the stated moment. It does not run the upstream checkers or their Lean
+representation checks for separate positive and negative parts and digit
+limits. The disjoint operations in the first two changes commute in those
+representations as well: each moved addition crosses only operations on other
+registers. The last two changes leave the scalar operations in place. This
+preserves the scalar computation from the pinned baseline; it does not replace
+a build of new Lean data.

research/e8-coupled-retiming/README.md

@@ -0,0 +1,80 @@
+# E8 complex cost from four coupled frame changes
+
+This package changes Jacob Sussman's E8 circuit at commit
+`9c94857bbaee886f8649edee0bc2d3c738f9555e`. It gives a smaller recursive moment with
+the same 783 helpers, copy cost 72, rank mass 56,715, and deficit 120.
+
+| Quantity | Original | Candidate |
+| --- | ---: | ---: |
+| Conditional complex saving under PR352's fallback convention | 876248285600677 / 10^18 | 876412559609473 / 10^18 |
+| Children in one invocation | 3974 | 3973 |
+| Children in five stages | 20830 | 20825 |
+| Scalar additions, including scatter | 3876 | 3876 |
+
+The conditional saving increases by about 0.01875%.
+
+Four phase-A additions move to two later frames. Two sets of phase-B gates use
+larger frames. The original frame table gains two entries. All scalar coefficients,
+retained totals, and scatter coefficients stay the same. The complete rational
+scalar operator also stays the same, including its dependence on every initial
+helper value.
+
+[PROOF.md](PROOF.md) gives the exact edits and a strict cost inequality for every
+saving between zero and one. It does not rely on the approximate moment root.
+
+## Reproduce
+
+From the repository root, run:
+
+```sh
+python3 -B research/e8-coupled-retiming/verify.py
+```
+
+The script uses Python and its standard library. It checks source hashes,
+rebuilds the candidate, checks every frame path, reconstructs the scalar source
+map and histogram, and compares all 1,046,529 entries of the complete scalar maps.
+A coefficient identity checks the
+strict-concavity proof for the full saving interval. Exact rational bounds prove both
+reported savings. In each case the next 10^-18 grid point fails. Four damaged
+inputs must fail for their stated reasons.
+
+To write the candidate to another file, run:
+
+```sh
+python3 -B research/e8-coupled-retiming/retime.py /tmp/e8-retimed.json.gz
+```
+
+The saved verification result is `verification.json`. Run the script to obtain
+a fresh result; the script does not use that saved result as an input.
+
+## Scope
+
+This is a finite complex-circuit improvement. It does not change the selected
+integer-multiplication result, the bit circuit, or its saving.
+
+The new certificate has not been compiled in Lean. The original certificate's
+Lean data and theorem do not certify this changed file. A formal extension must
+generate and compile new data. The general frame-transfer, helper cleanup,
+recursive construction, and multiplication assembly remain inherited
+dependencies. The complete scalar-map comparison does not prove those steps.
+The script does not run the upstream checkers or their Lean representation
+checks for separate positive and negative parts and digit limits.
+
+## Source and search
+
+The baseline is Sussman's `tools/certificate/gcert1-e8-r783.json.gz`. The exact
+commit and original-data hash are in `sources.json`. The E8 complex
+connection and fallback convention follow CrocSwap PR352 at
+`04b4c3c478b5bd8797d2663d89d8db4dfd0ad9f2`. The exact reference paths and
+hashes are recorded as mathematical provenance in `sources.json`; those files
+are not imported or run. Earlier retiming
+work includes Sussman's `tools/e8/sched.py` and CrocSwap PR328 and PR350.
+This package supplies a new E8 witness; it does not claim to invent retiming.
+
+The candidate came from bounded searches over pairs of moves and common
+intermediate frames. Frames came from existing stops and their joins or
+intersections. A later search moved two more complete gates to one common
+intermediate frame. These searches do not establish
+global optimality. The search is not needed to reproduce or verify the witness.
+
+See [NOTICE.md](NOTICE.md) for attribution and AI assistance.