diff --git a/research/e8-decoder-floors-and-cold14-notes/README.md b/research/e8-decoder-floors-and-cold14-notes/README.md new file mode 100644 index 0000000000..36177f741f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/README.md @@ -0,0 +1,229 @@ +# Research notes (no κ claim): E8 decoder floors and #615/#617 cold-end levers + +Negative results and one open lead, recorded so others don't repeat them. Scripts are included as they were run (paths reference a local scratch dir and `~/imb-src` pinned sources; Python 3.11 with numpy, scipy, sympy, python-sat, ortools). SAT results come from CaDiCaL via python-sat **without proof logs**. Nothing here is packaged with a verify.sh. + +## 1. Unit screen (`ade_screen.py`; **superseded by `s3-unit-families/` §1**) + +> **Erratum.** As #630 (vectorsss) points out, the "E6" row was D6 (30 lines). True E6 (the complement of an A2: v 36, h 6) has exact F2 minrank 7 and label copy cost 35 (x = 35/36), so X ≤ 5.69e-5: it is still useless. The c column here was rank_F2(J+G), which is only an upper bound; the exact minranks are in `s3-unit-families/`. E7 and E8 are unchanged. + +Real ADE root systems as units (v lines, window h = rank, c = F2 rank of (J+G) mod 2, proxy x = c·h/v, ceiling via #595's `skel.X`): only E8 gives a useful ceiling. E7 (v 63, h 7, c 7) reaches X* ≈ 1.84e-3; A_n, D_n and E6 have x ≥ 1 and give no saving. X scales roughly as 1/h, consistent with the Leech (#598) and Niemeier (#606) losses. + +## 2. E8 decoder floors (`s2-decoder/`, details in `REPORT.md`) +- **P111 (shipped bit puncture 39,52,57,71,76,78,103,109,114):** the 9-total F2 decoder is unique (= B0 on the live lines); 8 totals is UNSAT (`uniq.py`). +- **#446 Theorem C floor = star floor** on all realised units checked: P120 bit 72 (all 511 reader sets span Q^8, `t1.py`); P111 bit 63 (exact SAT over all label-decoder decompositions: cost ≤ 62 UNSAT, 63 SAT; `run62.py`, `dec9sat.py`, `p111c.py`); P112 complex frame decoder 64 (`cxp.py`). Decoders differing from B0 off-pattern (rank ≥ 10) are not covered. +- Where slack would have to come from: a total whose readers lie in a hyperplane orthogonal to a punctured line can drop to that line (cost 6 for orthogonal punctured pairs). It does not get P111 below 63. +- Deeper punctures (annealing, 9–26 lines removed), sub-root-system units (D8, E7+A1, A8, …) and mixed units with non-root ports (≥ 14 lines must be removed; best x ≥ 0.67) are all worse. Search results, not exhaustive. Exact W(E8) hyperplane orbit counts: E7 120, D7 1080, E6+A1 3360, A7 8640 (`hyper.py`). +- Asymmetric sinks on P112: 0 of 28,560 target/sink pairs relax for free (floating-point rank, `asym.py`); one extra copy unit frees 2 sinks (matches #595 C.6). Pricing (argued): ≤ 2 × 4.4 helpers × 0.143% ≈ 1.3% b < 2.82% per copy unit, so net negative. + +## 3. Levers on #615 / #617 (`r1-cold14/`, details in `REPORT.md`) +Measured on #615's words (#615 re-verified locally: κ 0.001626010676080313): +- **"e2" entering ends are infeasible** at all three Lean (E,U) pairs of #593: the row-cone left-nullity equals the number of helpers already ended (12/24/12) on jp8 and atm3. Cold-mixing directions (542/543/554/555, 566/567/578/579) have no column-cone solution (`nulls.py`, `broad2.py`, `probe2.py`). +- Extra cold dim-8 hyperplane ends all lose (best −5.67e-9 b per call). E1 partners 438/440 can't be replaced by already-lifted registers. +- No PR583-type frame coalescences on jp8, with or without commuting gate moves (the same scan finds #583's six on atm3) (`coal*.py`, `movecoal.py`). +- #615's owner map + dispatch equals the aggregate LP optimum over {jp8, atm3}, even with unlimited windows (`aglp.py`). + +**Open lead on #617 (κ 0.001635183729255778):** #617's owner solve stopped at its time limit with remainders {1:24, 2:2, 3:2, 7:2}. The aggregate LP bound (`aglp617.py`) is **+119.3e-9 b** above #617's map at fractional dispatch jp8 60.9 / atm3 11.1. An integer packing at jp8×61 / atm3×11 that meets the bound would give about **+1.1e-7 κ**. `mip617.py` (free-dispatch MIP with Hall constraints) and `expand617.py` (to OWNERS/PLAN) are included; the integer packing is not yet solved or verified. + +## Credits +Builds on jeremykothe2 (#591–#601, #607–#617), DreamingOfClouds (#595 format limits, #604, #610), chafreaky (#583, #616), #446 (Theorems A–C), eumemic (#543) and the contributors listed in those PRs. Prepared by Joel Pulikkan (GamingPuzzled) with Claude Opus 5.5 (Anthropic) via Claude Code. Apache-2.0. + +## 4. E8 over extension rings and K12 (`s1-extension-rings/`, details in `REPORT.md`) +On the bit side ranks are counted over Q, so a K-line unit is a rank-k unit with k = [K:Q] and h = k·w_K (argued). Exact F2 minrank of the decoder pattern, asymmetric decoders allowed (SAT, no proof logs): + +| view | lines v | w | largest non-orthogonal set | minrank | v/w | +|---|---|---|---|---|---| +| Gaussian E8 | 60 | 4 | 12 | 15 (13, 14 UNSAT) | 15 | +| Eisenstein E8 (Witting) | 40 | 4 | 7 | 10 (7–9 UNSAT) | 10 | +| Hurwitz E8 | 10 | 2 | 5 | 5 | 5 | + +- In every extension view, minrank = v/w. Real E8's cross-field gain (minrank 9 < 120/8) disappears. Under the full-rank-totals premise that #595 uses for E8's 72, x_K = 1.0, so D ≤ 0: no saving. +- Copy-cost searches (CP-SAT, 900 s, not proven optimal): Gaussian best 60 (proven lower bound only x_K ≥ 0.15); Eisenstein best 40 (x_K ≥ 0.225). Hurwitz symmetric decoders: optimum 10 (x_K = 1). Hurwitz asymmetric decoders: open. +- **Open question:** a structural lower bound Σ_j dim span A_j ≥ ~0.71·v for these units (the threshold to beat bit E8's ceiling at w = 4, float screen). This has the same open status as E8's 72. +- G34 / Coxeter–Todd K12: 126 lines, w = 6, largest non-orthogonal set 15, so x_K ≥ 0.714 even at minrank 15; it loses. +- Complex side (argued): F4-Hermitian (Eisenstein) labels give an alternating F2 trace form, so there are no odd-norm residuals and the phase calculus fails. Gaussian labels reduce mod (1+i) to F2, so they bring no extension. +- Side finding (float screen only, not certified): the paper's original triple family has a higher X* than E8 (~7.4e-3 bit, ~7.2e-3 complex at h = 24, s = 3), but 630 orthogonal neighbours per target (vs 63 for E8) makes its helper cost the issue. X* alone does not rank unit families. + +## 5. Beyond E8: triple and Lorentzian families (`s3-unit-families/`, own README and `verify.sh`, 114 PASS) +- Certified ceilings above E8's: the paper's triple family T_h (bit cst h(h−1), max X* 7.7735e-3 at h = 21; complex cst h², max 7.4444e-3 at h = 22) and the Lorentzian units of #634: I_{11,1} (minrank exactly 11, cst 110, x = 1/2, X* 7.1715e-3) and I_{13,1} (cst 156, x = 3/7, X* 8.6247e-3). +- Helper-adjusted (model, calibrated to #555 within 0.3%, not a bound): no candidate reaches E8's estimate, even at E8's 3.95 helpers per port. To break even, a design would need ≤ 2.6–3.4 helpers per port for I_{n,1} (≤ 2.5 for bit triples). The best lead is I_{11,1}; next steps are its punctures (the analogue of E8's 120 → 111) and a real helper design. +- #595 §D helper floors don't separate the families: the triple family has cover number 2 (exact certificates up to h = 24). + +## 6. Dispatch and cold-end pricing on the #639 lineage (`r2-dispatch/`, details in `REPORT.md`) +- Exact prices at concentrated idle (PR568 `moment.certify`): #639 b = 0.0016391439889600812; 72×jp8 b = 0.0016392558 (not admissible, since the global controls need two dispatched providers); 71 jp8 + 1 st2 (#635's jp8 word as the second provider) b = 0.0016392519, i.e. **+1.08e-7 b** from removing the forced ck call. #643 (jeremykothe2) later realised this twin dispatch independently. +- Exact sub-MIP obstruction: with jp8 at t = 5 and every window not touching t ≤ 7 held fixed, the optimum leaves 64 idle-3 owners, so no perfect packing exists in that neighbourhood. Once ck's 9+7 pairs are gone, the early windows are the tight ones. +- Mass-neutral V pricing per end per call (V639.json): u4-type premix-only dim-8 ends ({1:+2, 2:−1, 4:+1, 5:−1}, width −1) ≈ −0.025, essentially neutral, consistent with #637's 23 LP-negative candidates. **Dim-7 factored end pairs (#627 type) ≈ +2.25 per pair: the positive direction.** This should carry over to the gauge44 lineage (#645 onward) and was not searched here. + +## 7. I_{11,1} / I_{13,1}: punctures and helper structure (`s4-lorentzian-helpers/`) +Status: `expected/` is frozen from the complete run1 (all steps finished; the k = 19 SAT runs take ~2 h). No second fresh run has been done yet. +- **Punctures never help the centre decoder** (exact matroid-partition certificate). The 220 lines of I_{11,1} split into 22 independent sets, 2 inside each centre reader set, so the cost is ≥ |L|/2 for every live set L. For I_{13,1}, three copies of the lines split into 91 sets, giving ≥ (3/7)|L|. Every label decoder satisfies x ≥ 1/2 for |P| ≤ 8 (x ≥ 3/7 for |P| ≤ 10 on I_{13,1}). +- **Minrank stays 11** after ≤ 19 punctures for **all 5** maximum-clique S_11-orbits (SAT, no proof logs; orbit 0 UNSAT in 7,484 s). So any 10-total decoder needs ≥ 20 punctures. The 38,786 maximum cliques fall into 5 orbits, and a union bound shows any ≤ 21 punctures leave one alive. +- **Helpers.** E8's R = 439 is 240 carriers (3 branches per source, forced by E7 being a 4-design) + 180 cell registers + 9 totals + 10 puncture carriers. In I_{11,1}: + - triple sources also need ≥ 3 branches in any frame design (exact quadratic-form nullspace); + - the cell cover number is 3; + - 20,790 good 4-frames exist, but **no partition into good 4-frames exists**: every good frame holds 0 or 2 pairs, and there are 55 pairs. + - Needers cover 56–70 lines, against 30 in E8. + - The transferred cell skeleton has 787 helpers (3.58 per port), or 732 (3.33 per port) with 2-branch pairs. The calibrated model gives a_est 1.58–1.81e-3, against E8's 1.806e-3: no robust gain without a new middle network. + +## 8. Why "no new kappa" construction savings cannot move κ (R5 triage, 2026-10-13) +κ depends only on (i) the complex paid rank profile H = calls × local_paid + idle and the live stock W (b is the moment root of Σ_r r·n_r·(73/r)^b = 73W), and (ii) the bit profile (c). Primitive, ADD and XOR counts enter only finite admission checks, all with large slack: local_primitive_bound < 2^48, prefix < payload, row < 20161, and so on. The outer row gap also uses the fixed 20161. +- #883, #916, #901, #917, #919, #922, #882, #932, #937 and #945 each leave the paid children, helper widths and phase events unchanged, so the gain is exactly 0. Several act on historical CK/JP8/X4 words rather than the selected U8 word. +- Bit savings (#914, #915, #908, #904, #891, #713, #884, #936, #941) leave c unchanged. c = 0.0017334 > b ≈ 0.0016647, so the bit side doesn't bind anyway. +- #892 (rank-8 coded helper) does change the local profile, but it is excluded by its own packing obstruction on the old word. +- The only path from these savings to κ is a lower local prefix bound, which allows more replicas. Raising the charged payload does that directly: row slack 3,692 allows payload up to ~2,350, which is what T37 (#942) uses. +- **Next real lever:** dim-7 factored end pairs (#627 type, local {1:+2, 2:+2, 5:−2}, width −4) on the U8 word. Priced at ideal packing, each is worth about **+1.7e-7 b per pair per call**, about 60× the T37 gain. + +## 9. Exhaustive local-rewrite search on the fused X54 words (`r7-fusion-search/`, 2026-10-13) +**Search target:** the composed words of #1239/#1218: #1193's mixed T37 construction with #1195's source-holder fusions and #1130's X54 slot. JFX is the joint-entrance provider and D4FX its group-627 variant. Prices are exact, from #1186's frame toolkit (b = #1094 cost units): X54 = −1,788 and the two fusions = −4,338 on the joint word. + +**Frames:** the exact frame optimum of both words equals their published profiles, so frame changes have no slack left. + +| family | candidates per word | feasible | gains | +|---|---|---|---| +| every read fusion × {B-after, B-before-next, tail, prepend} restores × {none, single/pair swaps, full exchange} (#1186 `repair.py`) | 2,238 fusions / 2,850 pricings | 969 | **0** | +| standalone SW4 read-set exchanges (all non-empty subsets per copy pair) | 50 | 0 (cone cap "L not inside U" at end-of-B target vertices) | 0 | +| #1186 `joint.py` triangle-hybrid family | 161 (JFX/D4FX); 259 on the fusions-only word | 24 | 0 | +| `joint2.py`: generalised slots over all donation/restore gap classes, with keep/swap/fuse per movable read (reproduces X54 exactly in 120 equivalent variants), non-target holders | 3,924 | 0 | 0 | +| a fusion analogue at the X54 site (in place of the slot) | 45 touching 54/589 | 4 | 0 (best +956, worse than the slot) | +| `joint2` with target holders (+2,940 per word) and `fusion2.py` (10,603 generalised fusions on JFX) | — | **partial**: stopped before completion | none found | + +**Conclusion (scoped):** on these words, no further width-neutral source-holder fusion, read exchange or triangle slot of the searched families gains. The remaining κ gains in the lineage come from other mechanisms, notably #1274's helper-donor cold kernels. (**Correction, 2026-10-13:** an earlier version also cited the idle-overlay exchanges of #1268/#1280. The author has since closed #1268, #1280, #1305 and #1324 as invalid, because those moves shift axes between idle children of different (pair, side) families whose DATA-PHASES fix the Z4 multiplier; see #1286. The valid frontier is #1319/#1424.) Scripts `joint2.py` and `fusion2.py` extend #1186's exact-frame toolkit (credit mdumitrean #1186/#1195), which is not re-vendored here. + +## 10. Extension-ring E8: copy-cost floors over every decoder (`s5-extension-ring-floors/`, `verify.sh` ALL MATCH) +This follows up §4, where S1 obtained x_K = 1 only under the full-rank-totals premise. Here every F2 decoder on the orthogonality pattern is covered: any number of totals, asymmetric pairings, no star premise. Costs are #446 Theorem C costs Σ_j β_j. +- **Eisenstein / Witting (40 lines, w = 4): Σβ ≥ 40 = v for every decoder, so it is excluded** (checked plus proof). + - Sum-rank step: Σβ ≥ rank(G∘N), where N counts the totals covering each pair. + - If every line lattice span(T) ∩ E8 has odd determinant, G∘N reduces mod 2 to an invertible block-diagonal matrix. The Witting line lattices are A2 (det 3); the Hermitian diagonal 2+ω is a unit at the inert prime 2. + - Real E8 escapes this because its line lattices are √2·Z (det 2). That is consistent with 72 < 120. +- **Hurwitz (10 lines, w = 2): minimum over all decoders = 10 = v, so it is excluded** (checked plus a König/rank proof). The reduced SAT is UNSAT at cost ≤ 9, and the frame decoder attains 10. +- **Gaussian (60 lines, w = 4): Σβ ≥ 40 for every decoder (x_K ≥ 2/3, checked).** Not yet excluded: the float screen says x = 42/60 would still beat bit E8's ceiling (X* 6.33e-3), while 43/60 would not (5.66e-3). + - How the 40 is proved: 640 twelve-line cliques each force B[C,C] = I₁₂, and hits ≤ 192·rank holds on all 2271² flat pairs. + - The 2-adic method caps at 12, and an LP over clique families stays at 40. + - The best known decoder costs 60. A decoder at ≤ 42 would need ≥ 16 totals with at most 10 of full rank. + - **Open:** close the window 40–42. A pending SAT (`gauss_c15.py`) would show that every 15-total decoder costs exactly 60. + +## 11. Helper-donor cold-kernel census on #1319's word (**scope limited; see §12**) (`r8-cold-kernel-census/`, 2026-10-13) +**Base:** #1297/#1319's four-kernel base word (byte-identical to the one #1319 uses), κ 0.001676855229071321. #1305 and #1324 were withdrawn as invalid (#1286), so the idle-axis overlay is not used. + +**What a kernel is:** a cold pivot q (still at its rank-one start frame) has early B reads `t += k·q`. A kernel deletes them and repairs exactness with one or two shocks: a cut (`h += a·q`, undone in early cleanup) or a hook (`s ± c·q` around a donor read). The coefficients come from an exact rational equation. + +**Admissibility:** the scalar map is unchanged mod p, the cone caps hold, and #1186's exact frame solver finds feasible frames. + +**Why kernel 555 gains:** deleting reads merges the consecutive climbs of targets 187 and 483 (H1 −3, H3 +1, cost −11,676). + +**Control:** restricted to pivots 531/543/555/567 on the #1286 word, the census recovers exactly #1297's 5 + 5 relations, each −11,676. + +| family | candidates | admissible | gains | +|---|---|---|---| +| #1297's family, all 376 pivots with early uses | 254 | 99 (all Δ0) | **0** | +| G: cuts after any read or at B start, hooks around any early add gate, 1–2 shocks, every subset of early uses | 9,916 at 248 pivots | 1,435 (Δ0 or +4,913) | **0** | +| K2: two shocks of the same type, 15,507 new (**13,402 priced**) | 13,402 | 4,346 (Δ0 or +4,913) | **0** | +| 3-shock spot check (pivots 554, 448) | 12 | 0 | 0 | + +The dominant failure is "L not inside U", at the early-cleanup restore or at the end-of-B cone caps. The (output, output, helper) three-read pattern that the four kernels use occurs only at 531/543/555/567, which are already spent. + +**Not done:** 2,105 K2 candidates, pricing on the extra word, a full 3-shock census, and 3,066 gate-retiming candidates (generated by `retime.py`, unpriced; they target the same climb-merging mechanism directly). + +## 12. Supplier census on the current pivot-566 + 554 T74 word, and why §11 missed those kernels (`r9-supplier-census/`, 2026-10-14) +**Why this matters:** at #1800's ε → 1 outer point, κ ≈ the complex supplier endpoint b, so only supplier gains move κ. + +**Word and controls.** The words are #1800's `new/{gauge44,extra}-word-566.json.gz`, identical to #1672/#1957. Our pricer reproduces their profiles, gauge44 {1:2348, 2:500, 3:416, 4:130, 5:82} and extra {1:2340, 2:508, 3:420, 4:126, 5:82}, and prices #1672's k554 edit at Δ −1,850. + +**Correction to §11.** Its families G and K2 allowed only paired shocks, at most two per relation: a cut restored at the end of early cleanup, or a hook around one gate. The 566 and signed-554 kernels are 5–7-term relations built from three things those families lacked: +- unpaired direct reads `x += c·q` that are never undone; +- windows closed in early cleanup right after q's own restoration (the "signed restore" reads q while it holds −q0); +- donor windows spanning many gates. + +The right family is the span of reads `x += λ·y` wherever y holds a multiple of the deleted value; this is #1957's gencomp2. Positive controls: it recovers 566 (delete B755 on the #1319 word) and signed 554 (delete B342 on step 0), each at −1,850. + +#1957 (rohanarun) already ran that family on the current word, with 0 gains. The extensions here also find nothing: + +| run | coverage | gains | +|---|---|---| +| E1: several reads of the same value deleted together | 128 / 330 | 0 (all infeasible) | +| E2: relaxed frame-cost budget (+4,913) on 24 small-gain reads, incl. 858/876 not in #1957 | 24 / 24 | 0 (only trivial Δ0 slot moves) | +| E3: retime every B add to both ends of its commuting window | 3,074 / 3,074 | 0 (2,068 ok at Δ0, 1,006 infeasible) | +| E4: new two-value holder family (`gencomp3.py`; reads of holders of v+u with u cancelled, exactness on matrix entries, no insert feeding a later insert) | 144 / 1,002 (small-gain, mid and largest-gain bands) | 0 (126 shape-infeasible, 18 CP-infeasible) | + +E4's control recovers 566 plus new exact two-value variants at −1,850. A reversed-constraint bug in gencomp3 was found and fixed mid-run, and the affected results were rerun. + +**Not covered:** the rest of E1 and E4, three-value holders, multi-deletions across different values, A-section edits, and the extra word. The negative results suggest further supplier gains need changes outside B-local kernels, such as the A-section or the parent/centre interface. + +## 13. Gaussian E8: every 15-total decoder costs exactly 60 (`s6-gaussian-c15/`, 2026-10-14) +This answers §10's open question G2. + +**Reduction (argued).** Take a valid 15-total decoder B = W·Uᵀ over F2. If supp(Wx) lies in a 3-flat H for some x ≠ 0, then on O = the lines outside H we get W[O]x = 0. So B[O,O] has rank ≤ 14 and still obeys the decoder pattern. It therefore suffices to show that the induced pattern on O has F2 minrank ≥ 15 for every 3-flat H; the U side follows by transposition. Then every total spans K⁴, so β_j = 4 and the cost is 60. + +**Symmetry (checked).** The 60 complex reflections preserve the lines and their non-orthogonality, and generate a group of order **11,520**. The 1,500 3-flats fall into **3 orbits**: 60 of size 15, 480 of size 6 and 960 of size 4. + +**SAT (checked, no proof logs).** On the complement of each orbit representative (45, 54 and 56 lines), c = 14 is **UNSAT** with all three of CaDiCaL 1.5.3, Glucose 4 and MapleChrono, in 5–123 s. Calibrations: c = 15 is SAT on every O, and on all 60 lines c = 14 is UNSAT and c = 15 is SAT. The previous monolithic encoding ran for 18 h without an answer. + +**Group-free cross-check** (every 3-flat individually, no symmetry used): **complete: all 1,500 UNSAT** ("ALL AS EXPECTED", `logs/prove_all.log`). + +**Side lemma (argued).** For a c-total decoder and any family of flats, at most c − minrank(pattern on the complement of their union) totals have R_j inside the union (and likewise for A_j). + +**Consequence:** a Gaussian decoder costing ≤ 42, which would beat real E8, needs **≥ 16 totals**. The lower bound of 40 from §10 is unchanged. The 16-total case is being checked with this multi-flat argument. + +## 14. Gaussian E8: no 16-total decoder costs ≤ 42 (`s7-gaussian-c16/`, `verify.sh` reproduces `expected.txt`, 2026-10-14) +Together with §13, a Gaussian E8 decoder that beats real E8 (cost ≤ 42) needs **≥ 17 totals**. + +**Proof chain** (k = number of cheap totals, β_j < 4): +1. **Cost types (argued).** β_j = dim X − dim(X ∩ Y^⊥). The cheap types are: + - *a*: a side of dim ≤ 2 (β ≥ 0); + - *p*: a (3,3) "dumbbell" with β = 2; + - *q*: everything else (β ≥ 3). + + Cost ≤ 42 at c = 16 forces **3a + p ≥ 22 − k**. +2. **Multi-flat rank drop (argued; generalises §13).** Choose one legal 3-flat per cheap total. Then for every subset J, the minrank of the pattern on the complement of their union is ≤ 16 − |J| (a "level-k" set). +3. **Level tables (checked; orbit reps under the order-11,520 group).** + + | k | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | + |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| + | reps | 3 | 28 | 120 | 368 | 498 | 464 | 315 | 206 | 124 | 74 | 32 | 24 | 10 | 6 | 1 | **0** | + +4. **(A) checked:** *a* = 0 whenever k ≥ 5. +5. **(B) checked:** the most jointly valid dumbbells are p* = 5, 4, 3, 2, 1 for k = 11..15, against the 11, 10, 9, 8, 7 needed. +6. **Arithmetic.** For k ≤ 4, 3a + p ≤ 3k < 22 − k. For 5 ≤ k ≤ 10, p ≤ k < 22 − k. For 11 ≤ k ≤ 15, p ≤ p* < 22 − k. For k = 16, no level-16 set exists. + +**Checks.** +- All 200 UNSAT prunings were re-solved with Glucose 4: 0 mismatches. +- The fast canonicaliser matches the slow one on levels 2–4. +- A from-scratch `verify.sh` matches exactly; `CROSS=1` adds the Glucose re-solve. +- Calibrations: c = 16 without a cost cap is SAT, and the decoder found prices at 64; c = 15 reproduces §13. +- Side result: at most 2 totals can have a side of dim ≤ 2. +- Related scoped results: #767, #2066. + +**Caveat (found in §15):** the fast canonical key in `levels2.py` can overflow int64 for k > 10, so in principle a pruning could be wrong. No mismatch was observed, and §15's Theorem 1 re-proves the c = 16 result independently (cost ≥ 44) without level tables or SAT. + +## 15. Gaussian E8 for every c: cost ≥ max(40, 60 − c), and c = 18 excluded (`s8-gaussian-all-c/`; `verify.sh` about 6 min, exact arithmetic) +- **Theorem 1 (checked).** + - On all 2271² flat pairs, the maximum clique hits by rank are 0, 192, 384, 512 and 640 for ranks 0–4. Their concave envelope has two facets: h ≤ 192·R and **h ≤ 128·R + 128**. + - Summing the second facet over the c totals: 7680 ≤ 128·cost + 128·c, so **cost ≥ 60 − c**. + - Hence every decoder with c ≤ 17 costs ≥ 43. This **settles c = 17** and re-proves §14 (c = 16 gives ≥ 44) without SAT. +- **Theorem 2 (checked): c = 18 is excluded.** + - At c = 18 and cost ≤ 42, every clique is hit exactly 12 times and every total is tight. + - All 1,081 tight flat pairs miss exactly the cliques M·x_t, where x_t is a set of pairwise-orthogonal lines. + - So Mμ = 6·**1** with μ integral. M is the 640×60 clique/line incidence matrix, of exact rank 60, so μ = ½·**1**: a contradiction. +- **Theorem 3 (checked): the tight face cost = 60 − c never occurs.** It would need μ = (c − 12)/12. Hence c = 19 requires cost ≥ 42 (only cost = 42 is open) and c = 20 requires cost ≥ 41. +- **Facts for all c (argued).** + - Cost-0 totals can be dropped, so WLOG c ≤ 42. + - At most 3 totals are full rank and at most 6 have β ≥ 3. + - At most 2(c − 18) totals have rank 1. +- **Why counting alone stops here.** + - The LP over cliques of every size, lines, flat complements and a free c-term gives exactly max(40, 60 − c). + - An **integral** cover exists from c = 21: 21 dumbbells (ℓ^⊥, m^⊥) with ℓ ⊥ m, of total cost 42, hit every clique 12–16 times (checked). + - So excluding c ≥ 21 needs the F2 decoder pattern itself, not counting. + - CP-SAT covering ILPs for c = 19 and c = 20 returned UNKNOWN. + +**Status of the Gaussian question.** A Gaussian E8 decoder that beats real E8 (cost ≤ 42) needs **19 ≤ c ≤ 42** totals of nonzero cost. At c = 19 only cost exactly 42 is open, and at c = 20 only 41–42. + +## 16. Gaussian E8: c = 19 excluded; c = 20 partial (`s9-gaussian-c19/`, 2026-10-15) +- **μ-packing theorem (checked: HiGHS exact-optimal, and CP-SAT for capacities 7 and 8).** Tight totals miss exactly the cliques M·x_t (§15). Over μ ∈ ℤ≥0^60 with Mμ ≤ cap, the maximum |μ| for cap = 6..12 is **28, 32, 36, 42, 48, 52, 60**; the LP allows 5·cap. Consequences: + - an all-tight decoder costs ≥ 44 for every c ≤ 20; + - at c = 21 the requirement equals the maximum (42), which is exactly S8's 21-dumbbell cover. +- **c = 19 is excluded (checked, HiGHS MIP dual bounds; floating point, not exact certificates).** Non-tight totals are handled by a case split over the 27 rank-maximal type orbits with deficit ≤ 256. All 20 lifted cases are excluded: each best value is ≤ 32 with a dual bound ≤ 33, against the 34 needed. **Hence no 19-total decoder costs ≤ 42.** +- **c = 20 is partial.** + - At cost ≤ 41: 6 cases are excluded and cases 6 and 9 timed out. + - At cost ≤ 42: cases −1 and 21 are excluded; cases 19, 22 and 23 timed out, and the heavy cases (20k–54k columns) were not run. +- **c ≥ 21 (argued, partly checked).** Counting is exhausted. However, the c = 21 covers that counting allows are **F2-infeasible** (0.1 s SAT). The minimal core is a non-orthogonal pair {a, b}: one total covers both diagonal entries while another gives only an off-diagonal entry. S9 classified the exact 2×2 support lemma into 17 minimal feasible patterns; `pair_ilp.py` (all-dumbbell plus the lemma) is written but not run. + +**Status of the Gaussian question:** a decoder that beats real E8 needs **20 ≤ c ≤ 42**. c = 20 is partly excluded, and c ≥ 21 has a candidate F2 route. diff --git a/research/e8-decoder-floors-and-cold14-notes/ade_screen.py b/research/e8-decoder-floors-and-cold14-notes/ade_screen.py new file mode 100644 index 0000000000..6e73b0d052 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/ade_screen.py @@ -0,0 +1,40 @@ +import sys, itertools, numpy as np +from fractions import Fraction as Fr +sys.path.insert(0, '/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from e8units import f2_rank +from skel import X + +def roots_A(n): # in R^{n+1} + R=[] + for i in range(n+1): + for j in range(i+1,n+1): + v=[0]*(n+1); v[i]=1; v[j]=-1; R.append(v) + return np.array(R) +def roots_D(n): + R=[] + for i in range(n): + for j in range(i+1,n): + for s in (1,-1): + v=[0]*n; v[i]=1; v[j]=s; R.append(v) + return np.array(R) +def roots_E8(): + R=[list(r) for r in roots_D(8)] + for signs in itertools.product((1,-1),repeat=8): + if signs[0]==1 and signs.count(-1)%2==0: R.append([s*0.5 for s in signs]) + return np.array(R) +def E7(): + R=roots_E8(); s=np.ones(8)*0.5 + return R[np.abs(R@s)<1e-9] +def E6(): + R=roots_E8(); a=np.ones(8)*0.5; b=np.zeros(8); b[6]=1;b[7]=1 # pick two orthogonal roots + b=R[[i for i in range(len(R)) if abs(R[i]@a)<1e-9][0]] + R7=R[np.abs(R@a)<1e-9]; return R7[np.abs(R7@b)<1e-9] +def stats(name,R): + G=np.rint(R@R.T).astype(int); v=int(len(R)); h=int(np.linalg.matrix_rank(G)) + B=(1+G)%2; c=int(f2_rank(B)) + x=Fr(c*h, v) + vals=[(s,X(s,h,1,x)) for s in (3,5,7,9)]; vals=[(s,float(r["hi"])) for s,r in vals if r]; best=max(vals,key=lambda t:t[1]) if vals else (0,0.0) + print(f"{name:6s} v={v:4d} h={h:2d} c={c:3d} x~c*h/v={float(x):.3f} X*={best[1]:.4e}@s{best[0]}") +for n in (4,5,6,7,8): stats(f"A{n}",roots_A(n)) +for n in (4,5,6,7,8,9,10,12): stats(f"D{n}",roots_D(n)) +stats("E6",E6()); stats("E7",E7()); stats("E8",roots_E8()) diff --git a/research/e8-decoder-floors-and-cold14-notes/ade_screen_fixed.py b/research/e8-decoder-floors-and-cold14-notes/ade_screen_fixed.py new file mode 100644 index 0000000000..f9a9bc338b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/ade_screen_fixed.py @@ -0,0 +1,131 @@ +"""Corrected ADE unit screen (replaces #620's ade_screen.py; erratum found by #630). + +Per unit (real root lines, window h = Gram rank, k = 1): + v, h exact + c0 = rank_F2(J + G) the label decoder B0 used by #620 (an UPPER bound on the F2 minrank of the pattern) + omega largest pairwise non-orthogonal set (identity block of the pattern) -> minrank >= omega + c = minrank exact F2 minrank of the pattern (asymmetric decoders allowed, SAT; c-1 UNSAT, c SAT) + x_proxy = c*h/v #620's full-rank-totals premise (each total charged the window), now with the exact minrank + cst_lab star cost attainable by label decoders on the B0 labels: min over bases {p_j} of F2^c0 of + sum_j dim_Q span{lines T : u_T . p_j = 1} (matroid greedy, exact); an UPPER bound on cst_min + X* certified ceiling of #595 (skel.X, rational intervals) maximised over odd s in 3..15 + +What changed against #620: + * E6 was built as the complement of two ORTHOGONAL roots (= D6, 30 lines); it is now the complement of an A2 (36 lines). + * the column 'c' was rank_F2(J+G) of one decoder; it is now the exact minrank (both are printed). + * an attainable label-decoder star cost is added next to the proxy. +Usage: python ade_screen_fixed.py REPO_ROOT [--json out.json] +""" +import json +import sys +import time +from fractions import Fraction as Fr + +import numpy as np + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X, profile # noqa: E402 (#595, certified ceiling) +from cert import cost_iv # noqa: E402 +from minrank_sat import solve_asym # noqa: E402 (#595 SAT encoding, GL(c) gauge on a clique) +from e8units import sym_factor_f2 # noqa: E402 + +import units as U # noqa: E402 + +S_RANGE = range(3, 17, 2) +TIMEOUT = 600 # seconds per SAT query (A8, D9 hit it: minrank left as a range) + + +def xstar(h, x): + best = None + for s in S_RANGE: + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi'], r['lo']) + return best + + +def minrank(G, omega_set, c0): + lo, hi, log = U.minrank_f2(G, omega_set, c0, solve_asym, timeout=TIMEOUT) + return (lo if lo == hi else '%d..%d' % (lo, hi)), log, lo + + +def label_cost(L, G): + B0 = (1 + G) % 2 + Uf = sym_factor_f2(B0).astype(np.int64) + c0 = Uf.shape[1] + costs = [] + for p in range(1, 1 << c0): + pv = np.array([(p >> j) & 1 for j in range(c0)]) + R = np.nonzero((Uf @ pv) % 2)[0] + costs.append((U.rank_q(L[R].tolist()), p)) + costs.sort() + basis, tot = {}, 0 + for cst, p in costs: + x = p + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + if x: + tot += cst + if len(basis) == c0: + break + return tot, c0 + + +def unit(name, L, G=None): + if G is None: + G = U.gram_doubled(L) + v = len(L) + h = U.rank_q(G.tolist()) + c0 = U.f2_rank((1 + G) % 2) + om = U.nonorth_clique(G) + c, log, clo = minrank(G, om, c0) + cst_lab, _ = label_cost(L, G) + xp = Fr(clo * h, v) # proxy at the certified lower end of the minrank + xl = Fr(cst_lab, v) + row = dict(unit=name, v=v, h=h, c0=c0, omega=len(om), minrank=c, sat_log={k: v_ for k, v_ in log.items()}, + x_proxy=str(xp), cst_lab=cst_lab, x_lab=str(xl)) + for key, x in (('proxy', xp), ('lab', xl)): + b = xstar(h, x) + row['Xstar_' + key] = None if b is None else dict(s=b[0], hi=str(b[1]), hi_float=float(b[1])) + if row['Xstar_lab'] is None and xl < 1: + # positive deficit only at s > 1/(1 - x): bound the ceiling by D_s / Cost_s (#595 A.2) for odd s up to 801 + best = None + for s_ in range(3, 802, 2): + prof, m, D = profile(s_, h, 1, xl) + if D <= 0: + continue + b = D / cost_iv(prof, m)[0] + if best is None or b > best[1]: + best = (s_, b) + row['DoverCost_lab'] = dict(s=best[0], hi=str(best[1]), hi_float=float(best[1])) + print(' %s: x_lab < 1, deficit > 0 only for s > %s; max over odd s <= 801 of D_s/Cost_s = %.4e at s = %d' + % (name, 1 / (1 - xl), float(best[1]), best[0])) + return row + + +def main(): + rows = [] + cases = [('A%d' % n, U.roots_A(n)) for n in (4, 5, 6, 7, 8)] + cases += [('D%d' % n, U.roots_D(n)) for n in (4, 5, 6, 7, 8, 9, 10, 12)] + cases += [('D6(archived "E6")', U.roots_D6_archived()), ('E6', U.roots_E6()), ('E7', U.roots_E7()), ('E8', U.roots_E8())] + for name, L in cases: + t = time.time() + r = unit(name, L) + rows.append(r) + f = lambda k: ('%.4e@s%d' % (r[k]['hi_float'], r[k]['s'])) if r[k] else 'D<=0' + print('%-18s v=%4d h=%2d c0=%2d omega=%2d minrank=%-5s x_proxy=%-7s cst_lab=%4d x_lab=%-8s X*proxy=%-14s X*lab=%-14s (%.0fs)' + % (name, r['v'], r['h'], r['c0'], r['omega'], r['minrank'], r['x_proxy'], r['cst_lab'], r['x_lab'], + f('Xstar_proxy'), f('Xstar_lab'), time.time() - t), flush=True) + if '--json' in sys.argv: + json.dump(rows, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/REPORT.md b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/REPORT.md new file mode 100644 index 0000000000..a1dadf10ee --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/REPORT.md @@ -0,0 +1,18 @@ +# R1 resume (record track) — no strict gain yet +Worktrees: ~/imb-r1 (branch research/cold14-e2, base pr615, WIP commit with tools in research/r1-wip-tools/), +~/imb-r1b (branch research/cold14-b3-r1, base pr617, unchanged). Sources: ~/imb-src/pr{568,569,571,575,583}. +Verify: PATH=$SP/venv/bin:$PATH sh research//verify.sh WORK ~/imb-src (system python3 is too old). #615 baseline PASS kappa 0.001626010676080313 (615 s). +Tools ($SP/r1/tools == research/r1-wip-tools): price.py (exact b via PR568 moment.certify), V.json/V617.json (mass-neutral 1e-9 b per rank-r child), +nulls.py/broad2.py/hypsurvey.py/cands.py (cone capacity), coal.py/coalf.py/movecoal.py (frame-coalescence scans), aglp*.py (aggregate LP), mip617.py + expand617.py (free-dispatch MIP -> OWNERS/PLAN). +## Findings (on #615 words; #617 keeps E1) +- e2 infeasible: for all 3 Lean pairs (E=[30],[6],[24] raw) left-nullity of row-cone matrix over same-E helpers == #already-ended (12/24/12) on jp8 and atm3. Cold-mixing directions exist only via cold helpers 542/543/554/555 (E30), 566/567/578/579 (E24) but their column cone is unfixable (col-null space has no vector on them); other violators (315,347, data ports) are blocked by monotonicity. +- Cold dim-8 (bot,U) extras: best single end net -5.67 (V units), all negative. +- E1 partners 438/440 cannot be replaced by already-lifted registers (nullity drops 12->11 per dropped partner). +- Frame coalescence (PR583-type) on jp8 joint-prefix word: 0 single-gate (incl. final frames), 0 with commuting moves W=6 (validation on atm3 reproduces PR583's 6). +- #615 owner/dispatch: aggregate LP (any remainders, even unlimited windows) == current solution -> no owner/dispatch gain on #615. +- #617 (kappa 0.001635183729255778): its MIP hit time limit with remainders {1:24,2:2,3:2,7:2}. My LP (any remainders) bound is +119.3e-9 b over #617's map at fractional dispatch jp8 60.9/atm3 11.1 -> an integer perfect packing at jp8x61/atm3x11 would be ~+115e-9 b (~+1.1e-7 kappa). THIS IS THE NEXT LEVER. +## Resume +cd $SP/r1/tools && ../../venv/bin/python mip617.py 3 1800 ../m617 # free-dispatch Hall-mask MIP (was running, 900 s) +mkdir ../m617-out && ../../venv/bin/python expand617.py ../m617.pkl ../m617-out +../../venv/bin/python price.py "$(cat ../m617-profile.json)" # compare to b of #617 = 0.0016378619344417451 +Then: copy OWNERS/PLAN into ~/imb-r1b/research/complex-571-cold14-b3/data, update SPEC.json (data hashes, dispatch, expected global_profile/provider_counts/kappa/idle remainders), WITNESS/EXPECTED (spec.py generate), MANIFEST, rerun verify.sh with --reuse-baseline/--reuse-local from $SP/r1/b3-base for iteration, then full fresh run. diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp.py new file mode 100644 index 0000000000..d791089261 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp.py @@ -0,0 +1,80 @@ +"""aggregate free-dispatch LP (first-order, mass-neutral values V) over providers; patterns with any remainder.""" +import json,sys,itertools +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import linprog +from scipy.sparse import coo_matrix +V={int(k):v for k,v in json.load(open('V.json')).items()} +PLAN=json.load(open('/Users/joelpulikkan/imb-r1/research/complex-571-cold14-e1/data/PLAN.json')) +reg=PLAN['provider_registry'] +provs=sys.argv[1].split(',') if len(sys.argv)>1 else ['jp8','atm3'] +fixed=json.loads(sys.argv[2]) if len(sys.argv)>2 else None # {time:provider} optional +inv={p:Counter(reg[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in reg[p]['actual_paid_profile'].items()) for p in provs} +WID=sorted({w for p in provs for w in inv[p]}) +print('values',val,'widths',WID) +def pats(cap,L): + out=[] + def rec(pre,i,tot): + out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)1e-7: rem[PLAN['windows'][key][0]-sum(q)]+=x[i] +print('remainders',{k:round(v,3) for k,v in sorted(rem.items())}) +cur=68*val.get('jp8',0)+4*val.get('atm3',0) +print('current solution value',cur,'LP gain over current (1e-9 b)',-res.fun-cur) +json.dump({'sel':{f'{t},{p}':x[i] for (t,p),i in sel.items()},'pv':{f'{k}|{list(q)}':x[i] for (k,q),i in pv.items() if x[i]>1e-9}},open('aglp-out.json','w')) +du=res.eqlin.marginals +d=defaultdict(dict) +for i,k in enumerate(ek): + if k[0]=='inv': d[k[2]][k[1]]=-du[i] +for w in WID: + vals=[d[w][t] for t in range(72)] + print('dual width',w,'min %.2f max %.2f mean %.2f'%(min(vals),max(vals),sum(vals)/72)) +print('per-time dual(9)-dual(8):',[round(d[9][t]-d[8][t],2) for t in range(72)]) +print('per-time dual(8)-dual(7):',[round(d[8][t]-d[7][t],2) for t in range(72)]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp617.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp617.py new file mode 100644 index 0000000000..342abdbe4e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp617.py @@ -0,0 +1,80 @@ +"""aggregate free-dispatch LP (first-order, mass-neutral values V) over providers; patterns with any remainder.""" +import json,sys,itertools +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import linprog +from scipy.sparse import coo_matrix +V={int(k):v for k,v in json.load(open('V617.json')).items()} +PLAN=json.load(open('/Users/joelpulikkan/imb-r1b/research/complex-571-cold14-b3/data/PLAN.json')) +reg=PLAN['provider_registry'] +provs=sys.argv[1].split(',') if len(sys.argv)>1 else ['jp8','atm3'] +fixed=json.loads(sys.argv[2]) if len(sys.argv)>2 else None # {time:provider} optional +inv={p:Counter(reg[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in reg[p]['actual_paid_profile'].items()) for p in provs} +WID=sorted({w for p in provs for w in inv[p]}) +print('values',val,'widths',WID) +def pats(cap,L): + out=[] + def rec(pre,i,tot): + out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)1e-7: rem[PLAN['windows'][key][0]-sum(q)]+=x[i] +print('remainders',{k:round(v,3) for k,v in sorted(rem.items())}) +cur=60*val.get('jp8',0)+12*val.get('atm3',0)+V[1]*24+V[2]*2+V[3]*2+V[7]*2 +print('current solution value',cur,'LP gain over current (1e-9 b)',-res.fun-cur) +json.dump({'sel':{f'{t},{p}':x[i] for (t,p),i in sel.items()},'pv':{f'{k}|{list(q)}':x[i] for (k,q),i in pv.items() if x[i]>1e-9}},open('aglp617-out.json','w')) +du=res.eqlin.marginals +d=defaultdict(dict) +for i,k in enumerate(ek): + if k[0]=='inv': d[k[2]][k[1]]=-du[i] +for w in WID: + vals=[d[w][t] for t in range(72)] + print('dual width',w,'min %.2f max %.2f mean %.2f'%(min(vals),max(vals),sum(vals)/72)) +print('per-time dual(9)-dual(8):',[round(d[9][t]-d[8][t],2) for t in range(72)]) +print('per-time dual(8)-dual(7):',[round(d[8][t]-d[7][t],2) for t in range(72)]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp_allwin.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp_allwin.py new file mode 100644 index 0000000000..36c9cae3ac --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/aglp_allwin.py @@ -0,0 +1,82 @@ +"""aggregate free-dispatch LP (first-order, mass-neutral values V) over providers; patterns with any remainder.""" +import json,sys,itertools +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import linprog +from scipy.sparse import coo_matrix +V={int(k):v for k,v in json.load(open('V.json')).items()} +PLAN=json.load(open('/Users/joelpulikkan/imb-r1/research/complex-571-cold14-e1/data/PLAN.json')) +reg=PLAN['provider_registry'] +provs=sys.argv[1].split(',') if len(sys.argv)>1 else ['jp8','atm3'] +ALLWIN=True +fixed=json.loads(sys.argv[2]) if len(sys.argv)>2 else None # {time:provider} optional +inv={p:Counter(reg[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in reg[p]['actual_paid_profile'].items()) for p in provs} +WID=sorted({w for p in provs for w in inv[p]}) +print('values',val,'widths',WID) +def pats(cap,L): + out=[] + def rec(pre,i,tot): + out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)1e-7: rem[PLAN['windows'][key][0]-sum(q)]+=x[i] +print('remainders',{k:round(v,3) for k,v in sorted(rem.items())}) +cur=68*val.get('jp8',0)+4*val.get('atm3',0) +print('current solution value',cur,'LP gain over current (1e-9 b)',-res.fun-cur) +json.dump({'sel':{f'{t},{p}':x[i] for (t,p),i in sel.items()},'pv':{f'{k}|{list(q)}':x[i] for (k,q),i in pv.items() if x[i]>1e-9}},open('aglp-out.json','w')) +du=res.eqlin.marginals +d=defaultdict(dict) +for i,k in enumerate(ek): + if k[0]=='inv': d[k[2]][k[1]]=-du[i] +for w in WID: + vals=[d[w][t] for t in range(72)] + print('dual width',w,'min %.2f max %.2f mean %.2f'%(min(vals),max(vals),sum(vals)/72)) +print('per-time dual(9)-dual(8):',[round(d[9][t]-d[8][t],2) for t in range(72)]) +print('per-time dual(8)-dual(7):',[round(d[8][t]-d[7][t],2) for t in range(72)]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad.py new file mode 100644 index 0000000000..910e4d5198 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad.py @@ -0,0 +1,24 @@ +import sys,sympy +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +tau=lambda x: x^(43 if bin(x&43).count('1')%2 else 0) +LEAN={(30,):(256,128,64,32,18,10,6,1),(45,):(256,128,64,33,16,9,5,2),(51,):(256,128,64,33,17,8,4,3)} +LEANRAW={span([tau(x) for x in E]):span([tau(x) for x in U]) for E,U in LEAN.items()} +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +for E,U in LEANRAW.items(): + QU={g for g in range(n) if inside(endB[g],U)} + # any helper whose start frame is inside E (cold or E itself) + H=sorted(h for h in range(2*v,n) if inside(start[h],E)) + ended=[h for h in H if h in hf] + colsD=sorted({g for h in H for g,x in Pinv[h].items() if x and g not in QU}) + S=set(colsD);rk=modrank([{g:x for g,x in Pinv[h].items() if g in S} for h in H]) + Hp=[h for h in H if not (start[h]==E and h not in hf)] + rk2=modrank([{g:x for g,x in Pinv[h].items() if g in S} for h in Hp]) + print(' without nonended E helpers: |H|',len(Hp),'nullity',len(Hp)-rk2) + Hc=[h for h in H if start[h]==()] + rk3=modrank([{g:x for g,x in Pinv[h].items() if g in S} for h in Hc]);print(' cold only',len(Hc),'nullity',len(Hc)-rk3,'cold ended',len([h for h in Hc if h in hf])) + print(E,'|H|',len(H),'ended',len(ended),'left-nullity',len(H)-rk) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad2.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad2.py new file mode 100644 index 0000000000..6ef94ddfac --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/broad2.py @@ -0,0 +1,31 @@ +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +first={} +for part in ('A','B'): + for e in w[part]: + for r in regs(e): first.setdefault(r,span(e[-1])) +tau=lambda x: x^(43 if bin(x&43).count('1')%2 else 0) +LEAN={(30,):(256,128,64,32,18,10,6,1),(45,):(256,128,64,33,16,9,5,2),(51,):(256,128,64,33,17,8,4,3)} +LEANRAW={span([tau(x) for x in E]):span([tau(x) for x in U]) for E,U in LEAN.items()} +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +for E,U in LEANRAW.items(): + QU={g for g in range(n) if inside(endB[g],U)} + Hc=[h for h in range(2*v,n) if start[h]==() and h not in hf and inside(E,first.get(h,span([511]*0) or E))] + HE=[h for h in range(2*v,n) if start[h]==E] + H=HE+Hc + S={g for h in H for g,x in Pinv[h].items() if x and g not in QU} + rk=modrank([{g:x for g,x in Pinv[h].items() if g in S} for h in H]) + rk2=modrank([{g:x for g,x in Pinv[h].items() if g in S} for h in [h for h in HE if h in hf]+Hc]) + print(E,'E-group',len(HE),'ended',len([h for h in HE if h in hf]),'mixable cold',len(Hc),'nullity',len(H)-rk,'nullity without nonended E',len(H)-len([h for h in HE if h not in hf])-rk2) + print(' cold mixable:',[(h,first.get(h),endB[h],h in hf) for h in Hc]) + if not Hc: continue + rb=nullspace_rows([Pinv[h] for h in H],S) + for pv,r in rb.items(): + hs={H[j]:str(c) for j,c in r.items()} + if any(H[j] in HE and H[j] not in hf for j in r): + print(' null vec',hs) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/cands.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/cands.py new file mode 100644 index 0000000000..7676aba354 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/cands.py @@ -0,0 +1,38 @@ +import sys +from common import * +V={0:0,1:-3.987,2:-6.531,3:-8.532,4:-10.181,5:-11.569,6:-12.748,7:-13.755,8:-14.614,9:-15.344} +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]); avals=[int(x) for x in sys.argv[3].split(',')] +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +H0=[h for h in range(2*v,n) if start[h]==() and h not in hf] +def dy(q): return q.denominator&(q.denominator-1)==0 +for a in avals: + U=span([x for x in range(1,512) if bin(x&a).count('1')%2==0]) + QU={g for g in range(n) if inside(endB[g],U)} + keepR={g for h in H0 for g,x in Pinv[h].items() if x and g not in QU} + rb=nullspace_rows([Pinv[h] for h in H0],keepR) + # column null: m over H0 with sum_j P[g][H0[j]] m_j = 0 for g not in QU -> left null of columns + rowsC=[g for g in range(n) if g not in QU] + cols=[{g:P[g][h] for g in rowsC if P[g].get(h)} for h in H0] + cb=nullspace_rows(cols,set(rowsC)) + print('U a=%d'%a,U,'rownull',len(rb),'colnull',len(cb)) + for k,r in sorted(rb.items()): + hk=H0[k] + if not inside(endB[hk],U): continue + row={} + for j,c in r.items(): + for g,x in Pinv[H0[j]].items(): + y=row.get(g,0)+c*x + if y:row[g]=y + else:row.pop(g,None) + # column: try m=e_k + colk={g:P[g][hk] for g in range(n) if P[g].get(hk)} + okcol=all(g in QU for g in colk if g!=hk) + partners=(set(row)|(set(colk) if okcol else set()))-{hk} + tg=[g for g in partners if v<=g<2*v] + cost=sum(V[8-len(endB[g])]+V[1]-V[9-len(endB[g])] if len(endB[g])<8 else 0 for g in partners) + gain=V[8-len(endB[hk])]-V[9-len(endB[hk])] + print(' k',hk,'endB',len(endB[hk]),'premix',{H0[j]:str(c) for j,c in r.items() if j!=k},'selfcoef',row.get(hk),'colOK',okcol,'partners',sorted((g,len(endB[g])) for g in partners),'targets',tg,'dyadic',all(dy(x) for x in row.values()),'net %.2f'%(gain+cost)) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coal.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coal.py new file mode 100644 index 0000000000..4ba14acf79 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coal.py @@ -0,0 +1,44 @@ +import sys,json,gzip,copy,time +from collections import Counter +GK='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold/vendor/pr558/vendor/pr552/complex-local/static44/gk' +sys.path.insert(0,GK) +import gk_word as G,gk_common as C +G.V=C.V=112;G.DATA=C.DATA=7168 +V={int(k):v for k,v in json.load(open('V.json')).items()} +w=json.loads(gzip.open(sys.argv[1]).read()) +regs=lambda e:[e[1]]+([e[2]] if e[0]=='add' else []) +base=G.profile(w) +val=lambda p:sum(V[r]*c for r,c in p.items()) +print('base',dict(base),val(base)) +# next frame per register occurrence +occ={} +seq=[('A',i,e) for i,e in enumerate(w['A'])]+[('B',i,e) for i,e in enumerate(w['B'])] +nxt={} +last={} +for k in range(len(seq)-1,-1,-1): + part,i,e=seq[k] + for r in regs(e): + nxt[(k,r)]=last.get(r) + last[r]=k +found=[] +t0=time.time() +for k,(part,i,e) in enumerate(seq): + F=G.basis(e[-1]) + cands=set() + nf=[] + for r in regs(e): + j=nxt[(k,r)] + if j is not None: nf.append(G.basis(seq[j][2][-1])) + for f in nf: cands.add(G.basis(list(F)+list(f))) + if len(nf)==2: cands.add(G.basis(list(F)+list(nf[0])+list(nf[1]))) + cands.discard(F) + for Fp in cands: + z=copy.deepcopy(w[part][i]);old=w[part][i];w[part][i]=z;z[-1]=list(Fp) + try: + p=G.profile(w);d=val(p)-val(base) + if d>1e-9: found.append((d,part,i,old,list(Fp),{r:p[r]-base[r] for r in set(p)|set(base) if p[r]!=base[r]})) + except AssertionError: pass + w[part][i]=old +print('scan s',time.time()-t0,'found',len(found)) +for f in sorted(found,reverse=True)[:40]: print(f) +json.dump(found,open('coal-found.json','w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalf.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalf.py new file mode 100644 index 0000000000..9b14ca2377 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalf.py @@ -0,0 +1,45 @@ +import sys,json,gzip,copy,time +from collections import Counter +GK='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold/vendor/pr558/vendor/pr552/complex-local/static44/gk' +sys.path.insert(0,GK) +import gk_word as G,gk_common as C +G.V=C.V=112;G.DATA=C.DATA=7168 +V={int(k):v for k,v in json.load(open('V.json')).items()} +w=json.loads(gzip.open(sys.argv[1]).read()) +regs=lambda e:[e[1]]+([e[2]] if e[0]=='add' else []) +base=G.profile(w) +val=lambda p:sum(V[r]*c for r,c in p.items()) +print('base',dict(base),val(base)) +# next frame per register occurrence +occ={} +seq=[('A',i,e) for i,e in enumerate(w['A'])]+[('B',i,e) for i,e in enumerate(w['B'])] +nxt={} +last={} +for k in range(len(seq)-1,-1,-1): + part,i,e=seq[k] + for r in regs(e): + nxt[(k,r)]=last.get(r) + last[r]=k +found=[] +t0=time.time() +for k,(part,i,e) in enumerate(seq): + F=G.basis(e[-1]) + cands=set() + nf=[] + for r in regs(e): + j=nxt[(k,r)] + if j is not None: nf.append(G.basis(seq[j][2][-1])) + else: nf.append(G.FULL if (r<112 or r>=224) else G.perp(w['ports'][r-112])) + for f in nf: cands.add(G.basis(list(F)+list(f))) + if len(nf)==2: cands.add(G.basis(list(F)+list(nf[0])+list(nf[1]))) + cands.discard(F) + for Fp in cands: + z=copy.deepcopy(w[part][i]);old=w[part][i];w[part][i]=z;z[-1]=list(Fp) + try: + p=G.profile(w);d=val(p)-val(base) + if d>1e-9: found.append((d,part,i,old,list(Fp),{r:p[r]-base[r] for r in set(p)|set(base) if p[r]!=base[r]})) + except AssertionError: pass + w[part][i]=old +print('scan s',time.time()-t0,'found',len(found)) +for f in sorted(found,reverse=True)[:40]: print(f) +json.dump(found,open('coal-found-final.json','w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalfinal.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalfinal.py new file mode 100644 index 0000000000..d0af182bb1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coalfinal.py @@ -0,0 +1,34 @@ +import sys,json,gzip,copy,time +from collections import Counter +GK='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold/vendor/pr558/vendor/pr552/complex-local/static44/gk' +sys.path.insert(0,GK) +import gk_word as G,gk_common as C +G.V=C.V=112;G.DATA=C.DATA=7168 +V={int(k):v for k,v in json.load(open('V.json')).items()} +w=json.loads(gzip.open(sys.argv[1]).read()) +regs=lambda e:[e[1]]+([e[2]] if e[0]=='add' else []) +base=G.profile(w);val=lambda p:sum(V[r]*c for r,c in p.items()) +P=w['ports'] +def finalframe(r): + if r<112 or r>=224: return G.FULL + return G.perp(P[r-112]) +seq=[('A',i,e) for i,e in enumerate(w['A'])]+[('B',i,e) for i,e in enumerate(w['B'])] +lastk={} +for k,(part,i,e) in enumerate(seq): + for r in regs(e): lastk[r]=k +found=[] +for k,(part,i,e) in enumerate(seq): + rs=regs(e) + if not any(lastk[r]==k for r in rs): continue + F=G.basis(e[-1]);fin=[finalframe(r) for r in rs if lastk[r]==k] + Fp=G.basis(list(F)+list(min(fin,key=len))) + if Fp==F: continue + old=w[part][i];w[part][i]=old[:-1]+[list(Fp)] + try: + p=G.profile(w);d=val(p)-val(base) + found.append((round(d,3),part,i,old,len(Fp),{r:p[r]-base[r] for r in set(p)|set(base) if p[r]!=base[r]})) + except AssertionError as ex: found.append(('ERR',part,i,old,str(ex)[:80])) + w[part][i]=old +print(len(found)) +c=Counter((x[0] if x[0]=='ERR' else 'ok') for x in found);print(c) +for f in found[:30]:print(f) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coldsurvey.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coldsurvey.py new file mode 100644 index 0000000000..b5849ab78c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/coldsurvey.py @@ -0,0 +1,18 @@ +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +res=Counter();cands=[] +for h in range(2*v,n): + if h in hf: continue + supp={g for g,x in Pinv[h].items() if x}|{g for g in range(n) if P[g].get(h)}|{h} + J=span([x for g in supp for x in endB[g]]) + kind='cold' if start[h]==() else 'E%d'%len(start[h]) + res[(kind,len(J))]+=1 + if len(J)<=8: cands.append((h,kind,len(J),endB[h],len(supp))) +print(sorted(res.items())) +for c in cands: print(c) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/common.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/common.py new file mode 100644 index 0000000000..3b29c24869 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/common.py @@ -0,0 +1,99 @@ +import json,gzip,sys,copy +from fractions import Fraction as Q +from collections import Counter,defaultdict +PKG='/Users/joelpulikkan/imb-r1/research/complex-571-cold14-e1' +sys.path.insert(0,PKG+'/code') +import build_cold_end as BCE +import hpe_audit_and_transport as ATC +span=BCE.span +def rd(p): + b=open(p,'rb').read() + return json.loads(gzip.decompress(b) if str(p).endswith('.gz') else b) +def regs(e): return [e[1]]+([e[2]] if e[0]=='add' else []) +def inside(a,b): return span(list(a)+list(b))==span(b) +def frames_endB(w): + """raw-coordinate frame of every register after A, B (before tail)""" + v,R=w['report']['v'],w['report']['R'];n=2*v+R + cur=[() for _ in range(n)] + for i,p in enumerate(w['ports']):cur[i]=span([p]) + for r,f in w.get('start_extra',{}).items():cur[int(r)]=span(f) + start=list(cur) + for part in ('A','B'): + for e in w[part]: + for r in regs(e):cur[r]=span(e[-1]) + return start,cur +def main_maps(w): + v,R=w['report']['v'],w['report']['R'];n=2*v+R + main=w['A']+w['cut']+w['B'] + m=[{r:Q(1)} for r in range(n)] + for e in main:ATC.apply(e,m) + dirty=[{} for _ in range(n)] + for r in range(v,2*v):dirty[r]={j:x for j,x in m[r].items() if j>=2*v} + for e in w['precondition']:ATC.apply(e,dirty) + comp=[['add',r,j,-q.numerator,q.denominator,[]] for r in range(v,2*v) for j,q in sorted(dirty[r].items())] + P,Pinv=ATC.state_maps(w['precondition']+comp+main,n) + return P,Pinv,comp +PR=2**61-1 +def modrank(rows): + """rank mod a large prime of a list of dict rows {col: Fraction}""" + piv={} + r=0 + for row in rows: + x={c:(q.numerator*pow(q.denominator,-1,PR))%PR for c,q in row.items() if q} + x={c:a for c,a in x.items() if a} + while x: + c=min(x) + if c in piv: + p=piv[c];f=x[c] + for cc,a in p.items(): + x[cc]=(x.get(cc,0)-f*a)%PR + if not x[cc]:del x[cc] + else: + inv=pow(x[c],-1,PR);piv[c]={cc:a*inv%PR for cc,a in x.items()};r+=1;break + return r +def nullspace_rows(rows, keep): + """left null space of the matrix whose rows are rows[i] (dict col->Q), restricted to columns in keep. + returns RREF basis vectors as dicts i->Q (indices into rows).""" + # augment: row i -> (restricted row, identity tag) + aug=[({c:x for c,x in r.items() if c in keep and x},{i:Q(1)}) for i,r in enumerate(rows)] + piv={} + null=[] + for a,t in aug: + a=dict(a);t=dict(t) + while a: + c=min(a) + if c in piv: + pa,pt=piv[c];f=a[c] + for cc,x in pa.items(): + y=a.get(cc,0)-f*x + if y:a[cc]=y + else:a.pop(cc,None) + for cc,x in pt.items(): + y=t.get(cc,0)-f*x + if y:t[cc]=y + else:t.pop(cc,None) + else: + f=a[c];piv[c]=({cc:x/f for cc,x in a.items()},{cc:x/f for cc,x in t.items()});break + if not a:null.append(t) + # RREF the null basis on tag coordinates (prefer high index pivots? use min) + basis={} + for t in null: + t=dict(t) + for c in sorted(basis): + if c in t: + f=t[c]; + for cc,x in basis[c].items(): + y=t.get(cc,0)-f*x + if y:t[cc]=y + else:t.pop(cc,None) + if not t:continue + c=max(t);f=t[c];t={cc:x/f for cc,x in t.items()} + for c2 in basis: + if c in basis[c2]: + g=basis[c2][c] + for cc,x in t.items(): + y=basis[c2].get(cc,0)-g*x + if y:basis[c2][cc]=y + else:basis[c2].pop(cc,None) + basis[c]=t + return basis # pivot index -> vector diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/e1probe.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/e1probe.py new file mode 100644 index 0000000000..acf505bf16 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/e1probe.py @@ -0,0 +1,37 @@ +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +P,Pinv,comp=main_maps(w) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +# frames after tail +cur=list(endB) +for i,f in w['early_cleanup']: + for r in regs(w['cleanup'][i]):cur[r]=span(f) +U=span([256,128,64,32,18,10,6,1]) +liftedU={g for g in range(n) if cur[g]==U} +print('registers at U after tails:',len(liftedU),sorted(liftedU)) +print('endB==U:',sorted(g for g in range(n) if endB[g]==U)) +for k in (608,610): + print(k,'row',{g:(str(x),len(endB[g]),g in liftedU) for g,x in Pinv[k].items()}) + print(k,'col',{g:(str(P[g][k]),len(endB[g])) for g in range(n) if P[g].get(k)}) +# per tail gate partner listing +tl=defaultdict(list) +for i,f in w['early_cleanup']: + e=w['cleanup'][i] + for r in regs(e): tl[r].append(i) +E=(30,) +H=[h for h in range(2*v,n) if start[h]==E] +first={} +for part in ('A','B'): + for e in w[part]: + for r in regs(e): first.setdefault(r,span(e[-1])) +Hc=[h for h in range(2*v,n) if start[h]==() and h not in hf and first.get(h)==E] +for drop in ([438],[440],[438,440]): + F=(liftedU-set(drop)) + for HH,name in ((H,'E-group'),(H+Hc,'E-group+cold')): + S={g for h in HH for g,x in Pinv[h].items() if x and g not in F} + rb=nullspace_rows([Pinv[h] for h in HH],S) + print('drop',drop,name,'nullity',len(rb),[{HH[j]:str(c) for j,c in r.items()} for r in rb.values() if any(HH[j] in (608,610) for j in r) or len(r)>1][:6]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/expand617.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/expand617.py new file mode 100644 index 0000000000..d71268dcc8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/expand617.py @@ -0,0 +1,96 @@ +"""expand an aggregated MIP solution to a literal owner map + PLAN (PR569 nominator expansion logic).""" +import json,sys,gzip,hashlib,random,pickle,copy +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import milp,LinearConstraint,Bounds +from scipy.sparse import coo_matrix,csr_matrix +from scipy.sparse.csgraph import maximum_flow +PKG='/Users/joelpulikkan/imb-r1b/research/complex-571-cold14-b3' +PLAN=json.load(open(PKG+'/data/PLAN.json'));reg=PLAN['provider_registry'] +S=pickle.load(open(sys.argv[1],'rb'));OUT=sys.argv[2] +groups,x,disp=S['groups'],S['x'],{int(k):v for k,v in S['disp'].items()} +WID=[4,5,7,8,9] +def expand(plist,flows,seed): + times=sorted({t for t,w in flows});counts=[Counter(p) for p in plist];rng=random.Random(seed) + for attempt in range(60): + used=[set() for _ in plist];result=[[] for _ in plist] + colors=sorted(WID,key=lambda w:(sum(w in p for p in plist),w)) + if attempt:rng.shuffle(colors) + ok=True + for width in colors: + active=[p for p in range(len(counts)) if counts[p][width]] + if not active:continue + rng.shuffle(active);start=1+len(times);sink=start+len(active);rr,cc,vv=[],[],[] + for ti,t in enumerate(times): + d=flows.get((t,width),0) + if d: + rr.append(0);cc.append(1+ti);vv.append(d) + for pi,port in enumerate(active): + if t not in used[port]:rr.append(1+ti);cc.append(start+pi);vv.append(1) + for pi,port in enumerate(active):rr.append(start+pi);cc.append(sink);vv.append(counts[port][width]) + f=maximum_flow(csr_matrix((vv,(rr,cc)),shape=(sink+1,sink+1),dtype=np.int64),0,sink) + if f.flow_value!=sum(flows.get((t,width),0) for t in times):ok=False;break + F=f.flow.tocoo() if hasattr(f.flow,'tocoo') else coo_matrix(f.flow) + for a,b,v in zip(F.row,F.col,F.data): + if v>0 and 1<=a3 else 'free_dispatch_exact_window_pattern_milp',gap=None, + source_plan_sha256=hashlib.sha256(open(PKG+'/data/PLAN.json','rb').read()).hexdigest(),source_owners_sha256=hashlib.sha256(open(PKG+'/data/OWNERS.json.gz','rb').read()).hexdigest(),assignment_sha256=hashlib.sha256(packed).hexdigest()) +open(OUT+'/PLAN.json','w').write(json.dumps(plan,indent=2)+'\n') +print('profile',dict(sorted(paid.items())),'remainders',dict(rem),'dispatch',Counter(disp.values())) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/hypsurvey.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/hypsurvey.py new file mode 100644 index 0000000000..d4088fe7dd --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/hypsurvey.py @@ -0,0 +1,28 @@ +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +H0=[h for h in range(2*v,n) if start[h]==() and h not in hf] +colsupp={h:[g for g in range(n) if P[g].get(h)] for h in H0} +out=[] +for a in range(1,512): + U=span([x for x in range(1,512) if bin(x&a).count('1')%2==0]) # hyperplane = a-perp + QU={g for g in range(n) if inside(endB[g],U)} + K=[h for h in H0 if h in QU] + if not K: continue + S=sorted({g for h in H0 for g,x in Pinv[h].items() if x and g not in QU}) + Sset=set(S) + rr=modrank([{g:x for g,x in Pinv[h].items() if g in Sset} for h in H0]) + Nr=len(H0)-rr + # column condition: m over H0 with P[g,:]m=0 for g not in QU: nullity = |H0| - rank of the (rows not in QU) restricted to H0 columns + rowsC=sorted({g for h in H0 for g in colsupp[h] if g not in QU}) + H0s=set(H0) + rc=modrank([{h:x for h,x in P[g].items() if h in H0s} for g in rowsC]) + Nc=len(H0)-rc + out.append((min(Nr,Nc),Nr,Nc,len(K),a,U)) +out.sort(reverse=True) +for o in out[:25]: print(o) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/mip617.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/mip617.py new file mode 100644 index 0000000000..4c9f5721a8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/mip617.py @@ -0,0 +1,85 @@ +"""free-dispatch exact-window pattern MIP (first-order mass-neutral values), Hall masks, then literal expansion.""" +import json,sys,gzip,hashlib,random,time,itertools +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import milp,LinearConstraint,Bounds +from scipy.sparse import coo_matrix,csr_matrix +from scipy.sparse.csgraph import maximum_flow +V={int(k):v for k,v in json.load(open('V617.json')).items()} +PKG='/Users/joelpulikkan/imb-r1b/research/complex-571-cold14-b3' +PLAN=json.load(open(PKG+'/data/PLAN.json')) +reg=PLAN['provider_registry'];provs=['jp8','atm3'] +TIGHT=int(sys.argv[1]);SECS=float(sys.argv[2]);OUT=sys.argv[3] +inv={p:Counter(reg[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in reg[p]['actual_paid_profile'].items()) for p in provs} +WID=[4,5,7,8,9] +def pats(cap,L): + out=[] + def rec(pre,i,tot): + r=cap-tot + if r<=TIGHT or (r==cap==48): out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)>b&1} + req=tuple(i for (tt,w),i in fl.items() if tt==t and w in cols) + pres=tuple(j for j,c in zip(pv,cnts) if any(c[w] for w in cols)) + if req and (req,pres) not in seen: + seen.add((req,pres));row={i:1 for i in req} + for j in pres:row[j]=row.get(j,0)-1 + con(row,-np.inf,0) + for w in WID: + row={i:1 for (t,ww),i in fl.items() if ww==w} + for j,c in zip(pv,cnts): + if c[w]:row[j]=row.get(j,0)-c[w] + if row:con(row,0,0) + groups[key]=dict(cap=cap,times=times,ps=ps,pv=pv,fl=fl) +for t in range(72): + for w in WID: + row=dict(invrow[t,w]) + for p in provs: + if inv[p][w]:row[sel[t,p]]=-inv[p][w] + con(row,0,0) +rr,cc,vv=[],[],[] +for i,row in enumerate(R): + for j,x in row.items():rr.append(i);cc.append(j);vv.append(x) +A=coo_matrix((vv,(rr,cc)),shape=(len(R),len(ub))).tocsr() +print('vars',len(ub),'cons',len(R),flush=True) +t0=time.time() +res=milp(np.array(cost),integrality=np.ones(len(ub)),bounds=Bounds(np.zeros(len(ub)),np.array(ub)),constraints=LinearConstraint(A,lo,hi),options={'time_limit':SECS,'mip_rel_gap':1e-9,'disp':False}) +print(res.status,res.message,'obj',-res.fun if res.x is not None else None,'secs',time.time()-t0,flush=True) +x=np.round(res.x).astype(int) +disp={t:p for (t,p),i in sel.items() if x[i]} +rem=Counter() +for key,g in groups.items(): + for q,i in zip(g['ps'],g['pv']): + if x[i]:rem[g['cap']-sum(q)]+=x[i] +print('dispatch',Counter(disp.values()),'remainders',dict(rem)) +json.dump(dict(disp=disp,x=x.tolist(),obj=-res.fun),open(OUT+'-sol.json','w')) +# profile +prof=Counter() +for t,p in disp.items():prof.update({int(r):c for r,c in reg[p]['actual_paid_profile'].items()}) +prof.update(rem) +print('profile',dict(sorted(prof.items())),'mass',sum(r*c for r,c in prof.items())) +json.dump({str(k):v for k,v in sorted(prof.items())},open(OUT+'-profile.json','w')) +import pickle;pickle.dump(dict(groups=groups,x=x,disp=disp),open(OUT+'.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/movecoal.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/movecoal.py new file mode 100644 index 0000000000..af903a995d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/movecoal.py @@ -0,0 +1,48 @@ +import sys,json,gzip,copy,time +from multiprocessing import Pool +GK='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold/vendor/pr558/vendor/pr552/complex-local/static44/gk' +sys.path.insert(0,GK) +import gk_word as G,gk_common as C +G.V=C.V=112;G.DATA=C.DATA=7168 +V={int(k):v for k,v in json.load(open('V.json')).items()} +w0=json.loads(gzip.open(sys.argv[1]).read()) +W=int(sys.argv[2]) +def regs(e):return(e[1],e[2])if e[0]=='add'else(e[1],) +def commute(x,y): + if x[0]=='add'and y[0]=='add':return x[1]!=y[2]and y[1]!=x[2] + return not set(regs(x))&set(regs(y)) +val=lambda p:sum(V[r]*c for r,c in p.items()) +base=G.profile(w0);bv=val(base) +def nextframe(L,j,r): + for k in range(j+1,len(L)): + if r in regs(L[k]):return G.basis(L[k][-1]) + return None +def work(args): + part,k=args + w=copy.deepcopy(w0);L=w[part];out=[] + for d in list(range(1,W+1))+[-x for x in range(1,W+1)]: + j=k+d + if not 0<=jk else range(j,k) + if not all(commute(L[k],L[q]) for q in passed):continue + z=copy.deepcopy(w);Z=z[part];Z.insert(j,Z.pop(k)) + e=Z[j];F=G.basis(e[-1]);nf=[f for f in (nextframe(Z,j,r) for r in regs(e)) if f] + cands={G.basis(list(F)+list(f)) for f in nf} + if len(nf)==2:cands.add(G.basis(list(F)+list(nf[0])+list(nf[1]))) + cands.discard(F) + for Fp in cands: + old=Z[j];Z[j]=old[:-1]+[list(Fp)] + try: + p=G.profile(z);dv=val(p)-bv + if dv>1e-9:out.append((dv,part,k,j,old,list(Fp),{r:p[r]-base[r] for r in set(p)|set(base) if p[r]!=base[r]})) + except AssertionError:pass + Z[j]=old + return out +if __name__=='__main__': + tasks=[(p,k) for p in ('A','B') for k in range(len(w0[p]))] + t=time.time() + with Pool(9) as pool:res=pool.map(work,tasks,chunksize=50) + found=[x for r in res for x in r] + print('scan',time.time()-t,'found',len(found)) + for f in sorted(found,reverse=True)[:40]:print(f) + json.dump(found,open('movecoal-'+sys.argv[3]+'.json','w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/nulls.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/nulls.py new file mode 100644 index 0000000000..5f3927d784 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/nulls.py @@ -0,0 +1,26 @@ +import sys,sympy +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +tau=lambda x: x^(43 if bin(x&43).count('1')%2 else 0) +LEAN={(30,):(256,128,64,32,18,10,6,1),(45,):(256,128,64,33,16,9,5,2),(51,):(256,128,64,33,17,8,4,3)} +LEANRAW={span([tau(x) for x in E]):span([tau(x) for x in U]) for E,U in LEAN.items()} +hf={int(k):span(f) for k,f in w['helper_final'].items()} +P,Pinv,comp=main_maps(w) +for E,U in LEANRAW.items(): + H=sorted(int(k) for k,f in w['start_extra'].items() if span(f)==E ) + QU={g for g in range(n) if inside(endB[g],U)} + K=[h for h in H if h in QU] + rowsC=sorted({g for g in range(n) if g not in QU for h in H if P[g].get(h)}) + C=sympy.Matrix([[P[g].get(h,0) for h in H] for g in rowsC]) if rowsC else sympy.zeros(0,len(H)) + colsD=sorted({g for h in H for g,x in Pinv[h].items() if x and g not in QU}) + D=sympy.Matrix([[Pinv[h].get(g,0) for g in colsD] for h in H]) + ncol=C.nullspace() if rowsC else [sympy.eye(len(H))[:,i] for i in range(len(H))] + nrow=D.T.nullspace() + print('E',E,'U',U,'|H|',len(H),'candidates K',K,'ended',[h for h in H if h in hf]) + print(' C rows',len(rowsC),'nullity',len(ncol),' D cols',len(colsD),colsD,'left-nullity',len(nrow)) + def supp(vs): return [[H[i] for i in range(len(H)) if v[i]!=0] for v in vs] + print(' col-null supports',supp(ncol)[:12]) + print(' row-null supports',supp(nrow)[:12]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/price.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/price.py new file mode 100644 index 0000000000..cf550736a3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/price.py @@ -0,0 +1,7 @@ +import sys,json,importlib.util,time +from fractions import Fraction as Q +S='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold' +s=importlib.util.spec_from_file_location('moment',S+'/engines/moment.py');M=importlib.util.module_from_spec(s);s.loader.exec_module(M) +def price(H,live=7168):return Q(M.certify({int(k):v for k,v in H.items()},73,live,False)['lower']) +if __name__=='__main__': + H=json.loads(sys.argv[1]);t=time.time();b=price(H);print(float(b),b,time.time()-t) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe.py new file mode 100644 index 0000000000..ecbe54f535 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe.py @@ -0,0 +1,19 @@ +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +P,Pinv,comp=main_maps(w) +U30=span([256,128,64,32,18,10,6,1]);U24=span([256,128,64,32,24,4,2,1]) +first={} +for part in ('A','B'): + for e in w[part]: + for r in regs(e): + first.setdefault(r,span(e[-1])) +for U,ks in ((U30,[508,532,509,533]),(U24,[484,556,485,557])): + for k in ks: + print('k',k,'start',start[k],'endB',endB[k],{g:str(x) for g,x in Pinv[k].items()}) + for g,x in Pinv[k].items(): + if not inside(endB[g],U): + print(' viol',g,'start',start[g],'first',first.get(g),'endB',endB[g],'row',{gg:str(xx) for gg,xx in Pinv[g].items()},'inQU?',[inside(endB[gg],U) for gg in Pinv[g]]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe2.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe2.py new file mode 100644 index 0000000000..c625f604f6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/probe2.py @@ -0,0 +1,48 @@ +import sys +from common import * +V={0:0,1:-3.987,2:-6.531,3:-8.532,4:-10.181,5:-11.569,6:-12.748,7:-13.755,8:-14.614,9:-15.344} +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +P,Pinv,comp=main_maps(w) +U30=span([256,128,64,32,18,10,6,1]);U24=span([256,128,64,32,24,4,2,1]) +for U,combos in ((U30,[(542,518,Q(1,2)),(543,519,Q(-1)),(554,530,Q(-1)),(555,531,Q(-1))]),(U24,[(566,494,Q(-1)),(567,495,Q(-1)),(578,506,Q(-1)),(579,507,Q(-1))])): + QU={g for g in range(n) if inside(endB[g],U)} + for k,j,c in combos: + row=dict(Pinv[k]) + for g,x in Pinv[j].items(): + y=row.get(g,0)+c*x + if y:row[g]=y + else:row.pop(g) + col={g:P[g][k] for g in range(n) if P[g].get(k)} + colj={g:P[g][j] for g in range(n) if P[g].get(j)} + bad=[g for g in row if g not in QU]; badc=[g for g in col if g!=k and g not in QU] + partners=(set(row)|set(col))-{k} + cost=sum(V[8-len(endB[g])]+V[1]-V[9-len(endB[g])] if len(endB[g])<8 else 0 for g in partners) + gain=V[8-len(endB[k])]-V[9-len(endB[k])] + print(k,j,'row',{g:(str(x),len(endB[g])) for g,x in row.items()},'bad',bad,'col',{g:str(x) for g,x in col.items()},'badc',badc,'net %.2f'%(gain+cost)) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +first={} +for part in ('A','B'): + for e in w[part]: + for r in regs(e): first.setdefault(r,span(e[-1])) +print('--- column fixes') +for E,U,ks in (((30,),U30,[542,543,554,555]),((24,),U24,[566,567,578,579])): + QU={g for g in range(n) if inside(endB[g],U)} + H=[h for h in range(2*v,n) if h not in hf and (start[h]==E or (start[h]==() and first.get(h)==E))] + rowsC=[g for g in range(n) if g not in QU] + cols=[{g:P[g][h] for g in rowsC if P[g].get(h)} for h in H] + cb=nullspace_rows(cols,set(rowsC)) + print(E,'H',len(H),'colnull',len(cb)) + for pv,m in cb.items(): + if any(H[j] in ks for j in m): + col={} + for j,c in m.items(): + for g in range(n): + x=P[g].get(H[j]) + if x: + y=col.get(g,0)+c*x + if y:col[g]=y + else:col.pop(g) + print(' m',{H[j]:str(c) for j,c in m.items()},'col support',{g:(str(x),len(endB[g])) for g,x in col.items()}) diff --git a/research/e8-decoder-floors-and-cold14-notes/r1-cold14/survey.py b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/survey.py new file mode 100644 index 0000000000..8f60de6768 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r1-cold14/survey.py @@ -0,0 +1,28 @@ +"""survey entering helpers: start frame, end frame, cone violations for Lean (E,U) pairs.""" +import sys +from common import * +hpe_word=rd(sys.argv[1]); recipe=rd(sys.argv[2]) +w=BCE.apply(copy.deepcopy(hpe_word),recipe) +v,R=w['report']['v'],w['report']['R'];n=2*v+R +start,endB=frames_endB(w) +tau=lambda x: x^(43 if bin(x&43).count('1')%2 else 0) +LEAN={(30,):(256,128,64,32,18,10,6,1),(45,):(256,128,64,33,16,9,5,2),(51,):(256,128,64,33,17,8,4,3)} +LEANRAW={span([tau(x) for x in E]):span([tau(x) for x in U]) for E,U in LEAN.items()} +print('raw lean pairs',LEANRAW) +hf={int(k):span(f) for k,f in w['helper_final'].items()} +ent=Counter();groups=defaultdict(list) +for k,f in w['start_extra'].items(): + E=span(f);h=int(k);ent[(len(E),E in LEANRAW, h in hf)]+=1 + if len(E)==1 and h not in hf: groups[E].append(h) +print(ent) +for E,hs in groups.items(): print('E',E,'lean' if E in LEANRAW else '', len(hs), hs) +P,Pinv,comp=main_maps(w) +premixed={r for e in recipe['premix'] for r in regs(e)} +for E,hs in groups.items(): + if E not in LEANRAW: continue + U=LEANRAW[E] + QU={g for g in range(n) if inside(endB[g],U)} + for h in hs: + rowv=[g for g,x in Pinv[h].items() if x and g not in QU] + colv=[g for g in range(n) if g!=h and P[g].get(h) and g not in QU] + print(h,'premixed' if h in premixed else '','row',len(rowv),rowv[:8],'col',len(colv),colv[:8],'endB',endB[h]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/REPORT.md b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/REPORT.md new file mode 100644 index 0000000000..bc0cea6222 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/REPORT.md @@ -0,0 +1,15 @@ +# R2 progress (record track) +Worktree ~/imb-r2 (branch research/cold14-r2, base pr639 cf2f6d77). Tools ~/imb-harness/r2 (load.py, price.py, disp.py, alt*.py, lns*.py, aug.py, mip.py, V639.json). +## Findings so far +- #639 global profile = sum provider profiles + idle; idle = 387072 - total widths. jp8(u4) widths 3890, ck 3854, st2-jp8 (#635 word) 3894. +- Concentrated-idle bound: 72 jp8 -> b 0.0016392558 (+1.118e-7 b over #639 0.0016391440). BUT global assembly controls need 2 dispatched providers (README: ck pinned at t=5 for that reason). +- 2nd-provider options (concentrated bound): st2 jp8 word (#635) x1: b 0.0016392519 (+1.08e-7); jp8 minus one u4 end: 0.0016392548. +- Literal packing: LNS (CP-SAT) for 72 jp8 stalls at ~24 partial owners (b 0.0016389 < base). Perfect packing not found yet. +- Running: mip.py (aggregate free-dispatch exact-window MIP, Hall masks) LP bound over {jp8,ck,st2}. +## Update (handback) +- V639.json: mass-neutral b values (1e-9 b) vs #639 profile. Provider net vs jp8(u4): ck -247.4 profile (+36 width -> net ~-101 => the 1.1e-7 gap); st2 +16.37 profile, +4 width (~-16.3 idle) => ~neutral. +- Lever 1 accounting: u4-type end {1:+2,2:-1,4:+1,5:-1}, width -1 => net -0.025 per end per call (neutral/negative). Dim-7 factored end pair ({1:+2,2:+2,5:-2}, width -4) => +2.25 per pair per call: further dim-7 ends would be the real lever 1. +- Sub-MIP (submip.py, windows touching t<=7, jp8 at t=5, rest fixed): optimum forces 64 owners (9,9,9,9,9) idle 3 in window 5,8 => perfect packing not available locally. +- Launched (may still be running, ~/imb-harness/r2): c1.log = cpagg.py jp8,ck,st2 free dispatch CP-SAT hinted by #639 (only hint solution found so far); f72.log = cpfeas.py pure feasibility of perfect 72-jp8 packing; mip2.log = HiGHS free-dispatch MIP. +- Package skeleton: ~/imb-r2/research/complex-571-cold14-r2 (copy of witness638 + data/recipes/k492-st2-cold_end.json.gz = #635 jp8 recipe, sha 90952fde..). Not wired up; no commit. +Resume: cd ~/imb-harness/r2; cat c1.log f72.log mip2.log. If a dispatch {jp8 x(72-k), st2 x k} with idle profile beating b 0.0016391439889600812 is found: expand (expand_fn.py), wire st2 provider into SPEC/WITNESS (build entry = #635 SPEC build.providers.jp8, providers entry = #635 SPEC providers.jp8, registry = st2-PLAN.json jp8 with directory providers/st2), drop ck/ck_504, fix extras/check_native_owners.py + check_u4_portable provider lists, drop PR638 owner-replay steps from verify.sh, run verify. diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/V639.json b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/V639.json new file mode 100644 index 0000000000..10c5172220 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/V639.json @@ -0,0 +1 @@ +{"1": 0.0, "2": 1.460005797133559, "3": 3.469932932242822, "4": 5.836723075555035, "5": 8.46874739837603, "6": 11.312062510844147, "7": 14.331037175716805, "8": 17.500354290769017, "9": 20.801089049900753, "10": 24.21855529378407, "11": 27.7410230271743, "12": 31.358906530427365, "13": 35.064225132528996, "14": 38.85023095040961, "15": 42.71114356550622, "16": 46.641955786833186, "17": 50.63828817463157, "18": 54.696277919861934, "19": 58.81249249872061, "20": 62.98386155943487, "21": 67.20762246873477, "22": 71.48127625626488, "23": 75.80255158749605, "24": 80.16937501554928, "25": 84.57984620087221, "26": 89.0322171031147, "27": 93.52487437982572, "28": 98.05632439702877, "29": 102.62518038446753, "30": 107.23015136515151, "31": 111.87003256303241, "32": 116.54369705005661, "33": 121.25008843867404, "34": 125.98821446119275, "35": 130.75714130539652, "36": 135.55598859825827, "37": 140.38392494763218, "38": 145.2401639664379, "39": 150.1239607157839, "40": 155.03460851326403, "41": 159.9714360607355, "42": 164.93380485257933, "43": 169.92110683102177, "44": 174.93276225976217, "45": 179.9682177910782, "46": 185.0269447048926, "47": 190.1084373010941, "48": 195.21221142879605} \ No newline at end of file diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/a1.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/a1.py new file mode 100644 index 0000000000..c9460855a5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/a1.py @@ -0,0 +1,15 @@ +from load import * +print(len(rows), rows[0]) +occ=defaultdict(list) +for r in rows: occ[(r['owner_replica'],r['child'],r['port'])].append(r) +fill={k:sum(r['width'] for r in v) for k,v in occ.items()} +part=[(k,fill[k],P['windows'][f'{k[0]},{k[1]}'][0]) for k in fill if fill[k]4 else 0 # owners that must end with exactly 4 used (one 4-role) +random.seed(int(sys.argv[5]) if len(sys.argv)>5 else 0) +W=P['windows'];CAP={};TIM={} +for k,(c,ts) in W.items(): + a,b=map(int,k.split(',')) + for p in range(112):CAP[a,b,p]=c;TIM[a,b,p]=set(ts) +calls=[c['provider'] for c in P['provider_calls']] +def wid(t,h):return NEWW[h] if t==T else reg[calls[t]]['widths'][h] +own={} # (t,h)->owner +oldw=reg[calls[T]]['widths'] +pool=defaultdict(list) +for h in range(492):pool[NEWW[h]].append(h) +pend=[] +for r in sorted(rows,key=lambda r:(r['time'],r['helper'])): + o=(r['owner_replica'],r['child'],r['port']) + if r['time']!=T:own[r['time'],r['helper']]=o + elif pool[r['width']]:own[T,pool[r['width']].pop()]=o + else:pend.append((o,r['width'])) +newroles=[(T,h) for w,v in pool.items() for h in v] +# pair the leftover old slots with new roles (widths bigger): greedy by sorting +pend.sort(key=lambda x:x[1]);newroles.sort(key=lambda r:NEWW[r[1]]) +assert len(pend)==len(newroles) +for (o,w),r in zip(pend,newroles):own[r]=o +at=defaultdict(dict) +for r,o in own.items():at[o][r[0]]=r +fill={o:sum(wid(*r) for r in d.values()) for o,d in at.items()} +exc={o:fill[o]-CAP[o] for o in fill if fill[o]>CAP[o]} +print('excess',sum(exc.values()),len(exc)) +# partial owners +part=[o for o in CAP if 0=9 or any(wid(*c)==left for c in cands)==False else next(c for c in cands if wid(*c)==left) + plan.append(r);tot+=wid(*r) +for r in plan: + d=own[r];move(r,Z) +print('Z filled',fill[Z],CAP[Z]) +# extra 4-owners: move a 4-role into an empty 48 owner +empties=[o for o in CAP if CAP[o]==48 and fill.get(o,0)==0] +for k in range(EXTRA4): + for _ in range(10000): + r=random.choice([r for r,o in own.items() if wid(*r)==4]) + e=[o for o in empties if r[0] in TIM[o]] + if e:move(r,e[0]);empties.remove(e[0]);break +bal={o:fill.get(o,0)-CAP[o] for o in at if at[o]} +# owners that must be exactly 4 used are not CAP-balanced: treat target fill +target={o:(4 if fill.get(o,0)==4 and CAP[o]==48 and len(at[o])==1 else CAP[o]) for o in at if at[o]} +bal={o:fill[o]-target[o] for o in target} +src=[o for o,b in bal.items() if b>0];snk={o for o,b in bal.items() if b<0} +print('sources',sum(bal[o] for o in src),'sinks',-sum(bal[o] for o in snk)) +idx=defaultdict(list) # (t,width)->roles +for r in own:idx[r[0],wid(*r)].append(r) +def bfs(s): + prev={s:None};q=deque([s]) + while q: + x=q.popleft() + rs=list(at[x].values());random.shuffle(rs) + for r in rs: + w=wid(*r) + for y in idx.get((r[0],w-1),()): + Y=own[y] + if Y in prev:continue + prev[Y]=(x,r,y) + if Y in snk and bal[Y]<0:return Y,prev + q.append(Y) + return None,prev +def swap(r,y): + X,Y=own[r],own[y];t=r[0] + own[r],own[y]=Y,X;at[X][t]=y;at[Y][t]=r + fill[X]+=wid(*y)-wid(*r);fill[Y]+=wid(*r)-wid(*y) +npath=0 +while any(bal[o]>0 for o in src): + s=next(o for o in src if bal[o]>0) + Y,prev=bfs(s) + if Y is None:print('stuck at',s);break + path=[];cur=Y + while prev[cur] is not None:x,r,y=prev[cur];path.append((r,y));cur=x + for r,y in reversed(path):swap(r,y) + bal[s]-=1;bal[Y]+=1;npath+=1 +print('paths',npath,'remaining',sum(b for b in bal.values() if b>0)) +assert all(fill[o]<=CAP[o] for o in fill) +pickle.dump({r:o for r,o in own.items()},open(OUT,'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpagg.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpagg.py new file mode 100644 index 0000000000..f2a6742c13 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpagg.py @@ -0,0 +1,87 @@ +"""Aggregate exact-window pattern model (Hall masks) in CP-SAT with free dispatch over provs, >=MINALT non-jp8, +hinted with PR639's literal map. Objective: maximize sum of mass-neutral V (1e-9 b, scaled x1000).""" +import json,sys,time,pickle,gzip +from collections import Counter,defaultdict +from ortools.sat.python import cp_model +from load import P,reg,rows +V={int(k):v for k,v in json.load(open('V639.json')).items()};V[0]=0.0 +st2=json.load(open('st2-PLAN.json'))['provider_registry']['jp8'] +REG={'jp8':reg['jp8'],'ck':reg['ck'],'st2':st2} +provs=sys.argv[1].split(',');TIGHT=int(sys.argv[2]);SECS=float(sys.argv[3]);OUT=sys.argv[4];MINALT=int(sys.argv[5]) +FIX=json.loads(sys.argv[6]) if len(sys.argv)>6 else {} # {time:provider} +SC=1000 +inv={p:Counter(REG[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in REG[p]['actual_paid_profile'].items()) for p in provs} +WID=[4,5,7,8,9] +# hint from PR639 +own=defaultdict(list) +for r in rows:own[r['owner_replica'],r['child'],r['port']].append((r['time'],r['width'])) +hpat=defaultdict(Counter);hflow=defaultdict(Counter) +for k in P['windows']: + a,b=map(int,k.split(',')) + for p in range(112): + L=own.get((a,b,p),[]);hpat[k][tuple(sorted(w for _,w in L))]+=1 + for t,w in L:hflow[k][t,w]+=1 +def pats(cap,L,extra): + out=set() + def rec(pre,i,tot): + r=cap-tot + if r==0 or r<=TIGHT or r==cap: out.add(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)=MINALT) +invrow=defaultdict(list);groups={} +for key,(cap,times) in P['windows'].items(): + ps=pats(cap,len(times),list(hpat[key]));cnts=[Counter(q) for q in ps] + pv=[] + for q in ps: + v=m.NewIntVar(0,112,'');pv.append(v);obj.append(int(round(V[cap-sum(q)]*SC))*v);m.AddHint(v,hpat[key][q]) + m.Add(sum(pv)==112) + fl={} + for t in times: + for w in WID: + if not any(c[w] for c in cnts):continue + v=m.NewIntVar(0,112,'');fl[t,w]=v;invrow[t,w].append(v);m.AddHint(v,hflow[key][t,w]) + seen=set() + for mask in range(1,32): + cols={w for b,w in enumerate(WID) if mask>>b&1} + req=tuple((t,w) for (tt,w) in fl if tt==t and w in cols) + pres=tuple(j for j,c in enumerate(cnts) if any(c[w] for w in cols)) + if req and (req,pres) not in seen: + seen.add((req,pres));m.Add(sum(fl[k] for k in req)<=sum(pv[j] for j in pres)) + for w in WID: + fs=[v for (t,ww),v in fl.items() if ww==w] + m.Add(sum(fs)==sum(c[w]*pv[j] for j,c in enumerate(cnts) if c[w])) + groups[key]=dict(cap=cap,times=times,ps=ps,pv=pv,fl=fl) +for t in range(72): + for w in WID: + m.Add(sum(invrow[t,w])==sum(inv[p][w]*sel[t,p] for p in provs)) +m.Maximize(sum(obj)) +s=cp_model.CpSolver();s.parameters.max_time_in_seconds=SECS;s.parameters.num_workers=10;s.parameters.log_search_progress=False +base=71*val['jp8']+(val['ck'] if 'ck' in val else 0)+V[36]+2229*V[48] +class CB(cp_model.CpSolverSolutionCallback): + def __init__(s):super().__init__();s.t0=time.time() + def on_solution_callback(s):print('sol %.1fs obj %.3f gain_vs_639 %.3f bound %.3f'%(time.time()-s.t0,s.ObjectiveValue()/SC,s.ObjectiveValue()/SC-base,s.BestObjectiveBound()/SC),flush=True) +st=s.Solve(m,CB()) +print(s.StatusName(st),'obj',s.ObjectiveValue()/SC,'bound',s.BestObjectiveBound()/SC,'base639',base,flush=True) +if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + disp={t:p for (t,p),v in sel.items() if s.Value(v)} + rem=Counter() + x={} + for key,g in groups.items(): + g2=dict(cap=g['cap'],times=g['times'],ps=g['ps'],pv=[s.Value(v) for v in g['pv']],fl={k:s.Value(v) for k,v in g['fl'].items()}) + x[key]=g2 + for q,n in zip(g['ps'],g2['pv']): + if n and g['cap']-sum(q) and not(g['cap']==48 and not q):rem[g['cap']-sum(q)]+=n + print('dispatch',Counter(disp.values()),{t:p for t,p in disp.items() if p!='jp8'},'remainders',dict(rem)) + pickle.dump(dict(groups=x,disp=disp),open(OUT+'.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpfeas.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpfeas.py new file mode 100644 index 0000000000..d96d2d7629 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/cpfeas.py @@ -0,0 +1,87 @@ +"""Aggregate exact-window pattern model (Hall masks) in CP-SAT with free dispatch over provs, >=MINALT non-jp8, +hinted with PR639's literal map. Objective: maximize sum of mass-neutral V (1e-9 b, scaled x1000).""" +import json,sys,time,pickle,gzip +from collections import Counter,defaultdict +from ortools.sat.python import cp_model +from load import P,reg,rows +V={int(k):v for k,v in json.load(open('V639.json')).items()};V[0]=0.0 +st2=json.load(open('st2-PLAN.json'))['provider_registry']['jp8'] +REG={'jp8':reg['jp8'],'ck':reg['ck'],'st2':st2} +provs=sys.argv[1].split(',');TIGHT=int(sys.argv[2]);SECS=float(sys.argv[3]);OUT=sys.argv[4];MINALT=int(sys.argv[5]) +FIX=json.loads(sys.argv[6]) if len(sys.argv)>6 else {} # {time:provider} +SC=1000;HINT=__import__('os').environ.get('HINT','1')=='1' +inv={p:Counter(REG[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in REG[p]['actual_paid_profile'].items()) for p in provs} +WID=[4,5,7,8,9] +# hint from PR639 +own=defaultdict(list) +for r in rows:own[r['owner_replica'],r['child'],r['port']].append((r['time'],r['width'])) +hpat=defaultdict(Counter);hflow=defaultdict(Counter) +for k in P['windows']: + a,b=map(int,k.split(',')) + for p in range(112): + L=own.get((a,b,p),[]);hpat[k][tuple(sorted(w for _,w in L))]+=1 + for t,w in L:hflow[k][t,w]+=1 +def pats(cap,L,extra): + out=set() + def rec(pre,i,tot): + r=cap-tot + if r==0 or r<=TIGHT or (r==cap and cap==48): out.add(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)=MINALT) +invrow=defaultdict(list);groups={} +for key,(cap,times) in P['windows'].items(): + ps=pats(cap,len(times),[q for q in hpat[key] if (cap-sum(q))<=TIGHT or not q] if not HINT else list(hpat[key]));cnts=[Counter(q) for q in ps] + pv=[] + for q in ps: + v=m.NewIntVar(0,112,'');pv.append(v);obj.append(int(round(V[cap-sum(q)]*SC))*v);m.AddHint(v,hpat[key][q]) if HINT else None + m.Add(sum(pv)==112) + fl={} + for t in times: + for w in WID: + if not any(c[w] for c in cnts):continue + v=m.NewIntVar(0,112,'');fl[t,w]=v;invrow[t,w].append(v);m.AddHint(v,hflow[key][t,w]) if HINT else None + seen=set() + for mask in range(1,32): + cols={w for b,w in enumerate(WID) if mask>>b&1} + req=tuple((t,w) for (tt,w) in fl if tt==t and w in cols) + pres=tuple(j for j,c in enumerate(cnts) if any(c[w] for w in cols)) + if req and (req,pres) not in seen: + seen.add((req,pres));m.Add(sum(fl[k] for k in req)<=sum(pv[j] for j in pres)) + for w in WID: + fs=[v for (t,ww),v in fl.items() if ww==w] + m.Add(sum(fs)==sum(c[w]*pv[j] for j,c in enumerate(cnts) if c[w])) + groups[key]=dict(cap=cap,times=times,ps=ps,pv=pv,fl=fl) +for t in range(72): + for w in WID: + m.Add(sum(invrow[t,w])==sum(inv[p][w]*sel[t,p] for p in provs)) +m.Maximize(sum(obj)) +s=cp_model.CpSolver();s.parameters.max_time_in_seconds=SECS;s.parameters.num_workers=int(__import__("os").environ.get("NW","10"));s.parameters.log_search_progress=False +base=71*val['jp8']+(val['ck'] if 'ck' in val else 0)+V[36]+2229*V[48] +class CB(cp_model.CpSolverSolutionCallback): + def __init__(s):super().__init__();s.t0=time.time() + def on_solution_callback(s):print('sol %.1fs obj %.3f gain_vs_639 %.3f bound %.3f'%(time.time()-s.t0,s.ObjectiveValue()/SC,s.ObjectiveValue()/SC-base,s.BestObjectiveBound()/SC),flush=True) +st=s.Solve(m,CB()) +print(s.StatusName(st),'obj',s.ObjectiveValue()/SC,'bound',s.BestObjectiveBound()/SC,'base639',base,flush=True) +if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + disp={t:p for (t,p),v in sel.items() if s.Value(v)} + rem=Counter() + x={} + for key,g in groups.items(): + g2=dict(cap=g['cap'],times=g['times'],ps=g['ps'],pv=[s.Value(v) for v in g['pv']],fl={k:s.Value(v) for k,v in g['fl'].items()}) + x[key]=g2 + for q,n in zip(g['ps'],g2['pv']): + if n and g['cap']-sum(q) and not(g['cap']==48 and not q):rem[g['cap']-sum(q)]+=n + print('dispatch',Counter(disp.values()),{t:p for t,p in disp.items() if p!='jp8'},'remainders',dict(rem)) + pickle.dump(dict(groups=x,disp=disp),open(OUT+'.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/dbg.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/dbg.py new file mode 100644 index 0000000000..d792c348a9 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/dbg.py @@ -0,0 +1,8 @@ +import sys +sys.argv=['aug.py','5','-','/dev/null'] +src=open('aug.py').read().split("npath=0")[0] +exec(src) +s=(2,2,43);print(at[s],[wid(*r) for r in at[s].values()],bal[s]) +for r in at[s].values():print(r,wid(*r),len(idx.get((r[0],wid(*r)-1),()))) +Y,prev=bfs(s);print(Y,len(prev)) +print([ (o,bal[o],[ (r,wid(*r)) for r in at[o].values()]) for o in snk]) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/disp.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/disp.py new file mode 100644 index 0000000000..32710655d3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/disp.py @@ -0,0 +1,18 @@ +import json,sys +from collections import Counter +from fractions import Fraction as Q +from price import price +P=json.load(open('/Users/joelpulikkan/imb-r2/research/pr638-improvement-20261013/witness638/data/PLAN.json')) +reg=P['provider_registry'] +W={p:sum(reg[p]['widths']) for p in reg} +print(W) +base=Counter({int(k):v for k,v in P['profile'].items()}) +print('base b',float(price(base))) +for nck in range(0,6): + prof=Counter() + for p,n in (('jp8',72-nck),('ck',nck)): + for r,c in reg[p]['actual_paid_profile'].items(): prof[int(r)]+=n*c + idle=107028+(W['jp8']-W['ck'])*(nck-1) + prof[48]+=idle//48 + if idle%48: prof[idle%48]+=1 + b=price(prof);print(nck,idle,float(b)) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/evalst.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/evalst.py new file mode 100644 index 0000000000..a3eaaa9814 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/evalst.py @@ -0,0 +1,20 @@ +import sys,json,pickle +from load import * +from price import price +st=pickle.load(open(sys.argv[1],'rb'));T5=int(sys.argv[3]) if len(sys.argv)>3 else 5 +jw=json.load(open(sys.argv[2])) if len(sys.argv)>2 and sys.argv[2]!='-' else reg['jp8']['widths'] +prof5=json.load(open(sys.argv[4])) if len(sys.argv)>4 else reg['jp8']['actual_paid_profile'] +W=P['windows'];fill=Counter() +for (t,h),o in st.items(): + fill[o]+= jw[h] if t==T5 else reg[P['provider_calls'][t]['provider']]['widths'][h] +idle=Counter() +for k,(c,ts) in W.items(): + a,b=map(int,k.split(',')) + for p in range(112): + f=fill.get((a,b,p),0);assert f<=c + if f0 and 1<=a1:m.AddAtMostOne(v) +s=cp_model.CpSolver();s.parameters.max_time_in_seconds=TL;s.parameters.num_workers=10 +st=s.Solve(m);print('status',s.StatusName(st)) +if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + assign=[(freer[i],O[j]) for (i,j),v in x.items() if s.Value(v)] + out=[dict(time=t,helper=h,width=w,owner=list(o)) for (t,h,w),o in assign]+[dict(time=t,helper=h,width=w,owner=list(o)) for t,h,w,o in fixedrows] + json.dump(out,open(f'sol-{seed}.json','w')) + print('saved',len(out)) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns2.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns2.py new file mode 100644 index 0000000000..a0cfec0962 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns2.py @@ -0,0 +1,85 @@ +"""LNS: dispatch all-jp8 (call 5 ck->jp8); minimise #partial owners with CP-SAT neighbourhoods.""" +import sys,random,json,gzip,pickle,os +from load import * +from ortools.sat.python import cp_model +seed=int(sys.argv[1]);ITERS=int(sys.argv[2]);NB=int(sys.argv[3]);TL=float(sys.argv[4]);ST=sys.argv[5] +random.seed(seed) +W=P['windows'] +CAP={};TIM={} +allown=[] +for k,(c,ts) in W.items(): + o,ch=map(int,k.split(',')) + for p in range(112): allown.append((o,ch,p));CAP[o,ch,p]=c;TIM[o,ch,p]=set(ts) +FAM={o:fam(*o) for o in allown} +jw=reg['jp8']['widths'] +if os.path.exists(ST): + asg=pickle.load(open(ST,'rb')) +else: + asg={} # (t,h) -> owner + byw=defaultdict(list) + for h in range(492):byw[jw[h]].append(h) + for r in sorted(rows,key=lambda r:(r['time'],r['helper'])): + o=(r['owner_replica'],r['child'],r['port']) + if r['time']!=5: asg[r['time'],r['helper']]=o + elif byw[r['width']]: asg[5,byw[r['width']].pop()]=o + for w,v in byw.items(): + for h in v: asg[5,h]=None +def width(role):return reg['jp8']['widths'][role[1]] if role[0]==5 else reg[P['provider_calls'][role[0]]['provider']]['widths'][role[1]] +def state(): + occ=defaultdict(list) + for r,o in asg.items(): + if o is not None:occ[o].append(r) + return occ +def score(occ): + fill={o:sum(width(r) for r in v) for o,v in occ.items()} + part=[o for o,f in fill.items() if f1:m.AddAtMostOne(v) + m.Minimize(sum(pen)) + s=cp_model.CpSolver();s.parameters.max_time_in_seconds=TL;s.parameters.num_workers=10 + st=s.Solve(m) + if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + new=dict(asg) + for (i,j),v in x.items(): + if s.Value(v):new[freer[i]]=O[j] + old=(un,len(part)) + asg_bak=asg;asg=new + un2,part2,_=score(state()) + if (un2,len(part2))<=old: pickle.dump(asg,open(ST,'wb')) + else: asg=asg_bak + print(it,'owners',len(O),'roles',len(freer),s.StatusName(st),'obj',s.ObjectiveValue(),'->',un2,len(part2),flush=True) + else: print(it,s.StatusName(st),flush=True) +occ=state();un,part,fill=score(occ) +print('final',un,len(part),sorted(Counter(CAP[o]-fill[o] for o in part).items())) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns3.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns3.py new file mode 100644 index 0000000000..5a4f83c821 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns3.py @@ -0,0 +1,86 @@ +"""LNS: dispatch all-jp8 (call 5 ck->jp8); minimise #partial owners with CP-SAT neighbourhoods.""" +import sys,random,json,gzip,pickle,os +from load import * +from ortools.sat.python import cp_model +seed=int(sys.argv[1]);ITERS=int(sys.argv[2]);NB=int(sys.argv[3]);TL=float(sys.argv[4]);ST=sys.argv[5] +random.seed(seed) +W=P['windows'] +CAP={};TIM={} +allown=[] +for k,(c,ts) in W.items(): + o,ch=map(int,k.split(',')) + for p in range(112): allown.append((o,ch,p));CAP[o,ch,p]=c;TIM[o,ch,p]=set(ts) +FAM={o:fam(*o) for o in allown} +jw=json.load(open(sys.argv[6])) if len(sys.argv)>6 else reg['jp8']['widths'] +if os.path.exists(ST): + asg=pickle.load(open(ST,'rb')) +else: + asg={} # (t,h) -> owner + byw=defaultdict(list) + for h in range(492):byw[jw[h]].append(h) + for r in sorted(rows,key=lambda r:(r['time'],r['helper'])): + o=(r['owner_replica'],r['child'],r['port']) + if r['time']!=5: asg[r['time'],r['helper']]=o + elif byw[r['width']]: asg[5,byw[r['width']].pop()]=o + for w,v in byw.items(): + for h in v: asg[5,h]=None +def width(role):return jw[role[1]] if role[0]==5 else reg[P['provider_calls'][role[0]]['provider']]['widths'][role[1]] +def state(): + occ=defaultdict(list) + for r,o in asg.items(): + if o is not None:occ[o].append(r) + return occ +def score(occ): + fill={o:sum(width(r) for r in v) for o,v in occ.items()} + part=[o for o in allown if fill.get(o,0)0 or CAP[o]<48)] + un=sum(1 for o in asg.values() if o is None) + return un,part,fill +occ=state();un,part,fill=score(occ) +print('start unassigned',un,'partial',len(part),flush=True) +for it in range(ITERS): + occ=state();un,part,fill=score(occ) + # neighbourhood: partial owners + owners of unassigned-time + random others sharing times + O=set(part) + unr=[r for r,o in asg.items() if o is None] + seedt={r[0] for r in unr}|{t for o in part for t in TIM[o]} + cand=[o for o in allown if TIM[o]&seedt and o not in O] + full=[o for o in cand if o in occ];emp=[o for o in cand if o not in occ] + O|=set(random.sample(full,min(NB,len(full)))) + O|=set(random.sample(emp,min(NB//6,len(emp)))) + O=sorted(O);Oi={o:j for j,o in enumerate(O)} + freer=[r for r,o in asg.items() if o is None or o in Oi] + used=set((r[0],FAM[o]) for r,o in asg.items() if o is not None and o not in Oi) + m=cp_model.CpModel();x={} + byown=defaultdict(list);byfam=defaultdict(list) + for i,r in enumerate(freer): + c=[] + for j,o in enumerate(O): + if r[0] in TIM[o] and (r[0],FAM[o]) not in used and width(r)<=CAP[o]: + v=m.NewBoolVar('');x[i,j]=v;c.append(v);byown[j].append((i,v));byfam[r[0],FAM[o]].append(v) + if asg[r]==o:m.AddHint(v,1) + m.AddExactlyOne(c) + pen=[] + for j,o in enumerate(O): + s=sum(width(freer[i])*v for i,v in byown[j]) + f=m.NewBoolVar('');e=m.NewBoolVar('') + m.Add(s<=CAP[o]);m.Add(s==CAP[o]).OnlyEnforceIf(f);m.Add(s==0).OnlyEnforceIf(e) + if CAP[o]<48: m.Add(e==0) + p=m.NewBoolVar('');m.AddBoolOr([f,e,p]);pen.append(p) + for k,v in byfam.items(): + if len(v)>1:m.AddAtMostOne(v) + m.Minimize(sum(pen)) + s=cp_model.CpSolver();s.parameters.max_time_in_seconds=TL;s.parameters.num_workers=10 + st=s.Solve(m) + if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + new=dict(asg) + for (i,j),v in x.items(): + if s.Value(v):new[freer[i]]=O[j] + old=(un,len(part)) + asg_bak=asg;asg=new + un2,part2,_=score(state()) + if (un2,len(part2))<=old: pickle.dump(asg,open(ST,'wb')) + else: asg=asg_bak + print(it,'owners',len(O),'roles',len(freer),s.StatusName(st),'obj',s.ObjectiveValue(),'->',un2,len(part2),flush=True) + else: print(it,s.StatusName(st),flush=True) +occ=state();un,part,fill=score(occ) +print('final',un,len(part),sorted(Counter(CAP[o]-fill.get(o,0) for o in part).items())) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns4.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns4.py new file mode 100644 index 0000000000..19edbfcaeb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/lns4.py @@ -0,0 +1,92 @@ +"""LNS: dispatch all-jp8 (call 5 ck->jp8); minimise #partial owners with CP-SAT neighbourhoods.""" +import sys,random,json,gzip,pickle,os +from load import * +from ortools.sat.python import cp_model +seed=int(sys.argv[1]);ITERS=int(sys.argv[2]);NB=int(sys.argv[3]);TL=float(sys.argv[4]);ST=sys.argv[5] +random.seed(seed) +W=P['windows'] +CAP={};TIM={} +allown=[] +for k,(c,ts) in W.items(): + o,ch=map(int,k.split(',')) + for p in range(112): allown.append((o,ch,p));CAP[o,ch,p]=c;TIM[o,ch,p]=set(ts) +FAM={o:fam(*o) for o in allown} +jw=json.load(open(sys.argv[6])) if len(sys.argv)>6 else reg['jp8']['widths'] +if os.path.exists(ST): + asg=pickle.load(open(ST,'rb')) +else: + asg={} # (t,h) -> owner + byw=defaultdict(list) + for h in range(492):byw[jw[h]].append(h) + for r in sorted(rows,key=lambda r:(r['time'],r['helper'])): + o=(r['owner_replica'],r['child'],r['port']) + if r['time']!=5: asg[r['time'],r['helper']]=o + elif byw[r['width']]: asg[5,byw[r['width']].pop()]=o + for w,v in byw.items(): + for h in v: asg[5,h]=None +def width(role):return jw[role[1]] if role[0]==5 else reg[P['provider_calls'][role[0]]['provider']]['widths'][role[1]] +def state(): + occ=defaultdict(list) + for r,o in asg.items(): + if o is not None:occ[o].append(r) + return occ +def score(occ): + fill={o:sum(width(r) for r in v) for o,v in occ.items()} + part=[o for o in allown if fill.get(o,0)0 or CAP[o]<48)] + un=sum(1 for o in asg.values() if o is None) + return un,part,fill +occ=state();un,part,fill=score(occ) +print('start unassigned',un,'partial',len(part),flush=True) +for it in range(ITERS): + occ=state();un,part,fill=score(occ) + # neighbourhood: partial owners + owners of unassigned-time + random others sharing times + O=set(part) + unr=[r for r,o in asg.items() if o is None] + seedt={r[0] for r in unr}|{t for o in part for t in TIM[o]} + # structured: owners holding roles at the same times as problem owners' roles + bytime=defaultdict(list) + for r,o in asg.items(): + if o is not None: bytime[r[0]].append(o) + probt=[r[0] for o in part for r in occ[o]]+[r[0] for r in unr] + per=max(1,NB//max(1,len(probt))) + for t in probt: + O|=set(random.sample(bytime[t],min(per,len(bytime[t])))) + emp=[o for o in allown if o not in occ and CAP[o]==48 and TIM[o]&seedt] + O|=set(random.sample(emp,min(NB//10,len(emp)))) + O=sorted(O);Oi={o:j for j,o in enumerate(O)} + freer=[r for r,o in asg.items() if o is None or o in Oi] + used=set((r[0],FAM[o]) for r,o in asg.items() if o is not None and o not in Oi) + m=cp_model.CpModel();x={} + byown=defaultdict(list);byfam=defaultdict(list) + for i,r in enumerate(freer): + c=[] + for j,o in enumerate(O): + if r[0] in TIM[o] and (r[0],FAM[o]) not in used and width(r)<=CAP[o]: + v=m.NewBoolVar('');x[i,j]=v;c.append(v);byown[j].append((i,v));byfam[r[0],FAM[o]].append(v) + if asg[r]==o:m.AddHint(v,1) + m.AddExactlyOne(c) + pen=[] + for j,o in enumerate(O): + s=sum(width(freer[i])*v for i,v in byown[j]) + f=m.NewBoolVar('');e=m.NewBoolVar('') + m.Add(s<=CAP[o]);m.Add(s==CAP[o]).OnlyEnforceIf(f);m.Add(s==0).OnlyEnforceIf(e) + if CAP[o]<48: m.Add(e==0) + p=m.NewBoolVar('');m.AddBoolOr([f,e,p]);pen.append(p) + for k,v in byfam.items(): + if len(v)>1:m.AddAtMostOne(v) + m.Minimize(sum(pen)) + s=cp_model.CpSolver();s.parameters.max_time_in_seconds=TL;s.parameters.num_workers=10 + st=s.Solve(m) + if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + new=dict(asg) + for (i,j),v in x.items(): + if s.Value(v):new[freer[i]]=O[j] + old=(un,len(part)) + asg_bak=asg;asg=new + un2,part2,_=score(state()) + if (un2,len(part2))<=old: pickle.dump(asg,open(ST,'wb')) + else: asg=asg_bak + print(it,'owners',len(O),'roles',len(freer),s.StatusName(st),'obj',s.ObjectiveValue(),'->',un2,len(part2),flush=True) + else: print(it,s.StatusName(st),flush=True) +occ=state();un,part,fill=score(occ) +print('final',un,len(part),sorted(Counter(CAP[o]-fill.get(o,0) for o in part).items())) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/load.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/load.py new file mode 100644 index 0000000000..393c1397ae --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/load.py @@ -0,0 +1,7 @@ +import json,gzip +from collections import Counter,defaultdict +D='/Users/joelpulikkan/imb-r2/research/pr638-improvement-20261013/witness638/data' +P=json.load(open(D+'/PLAN.json')) +rows=json.loads(gzip.decompress(open(D+'/OWNERS.json.gz','rb').read())) +reg=P['provider_registry'] +def fam(owner,child,port): return owner*896+(2*P['pairs'][str(child)]+P['sides'][str(child)])*112+port diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mip.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mip.py new file mode 100644 index 0000000000..423b06ee72 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mip.py @@ -0,0 +1,90 @@ +"""free-dispatch exact-window pattern MIP over {jp8(u4), st2, ck}; >=MINALT non-jp8 calls; Hall masks; V639 values.""" +import json,sys,time,pickle +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import milp,LinearConstraint,Bounds +from scipy.sparse import coo_matrix +from load import P,reg +V={int(k):v for k,v in json.load(open('V639.json')).items()};V[0]=0.0 +st2=json.load(open('st2-PLAN.json'))['provider_registry']['jp8'] +REG={'jp8':reg['jp8'],'ck':reg['ck'],'st2':st2} +provs=sys.argv[1].split(',');TIGHT=int(sys.argv[2]);SECS=float(sys.argv[3]);OUT=sys.argv[4];MINALT=int(sys.argv[5]);LPONLY=len(sys.argv)>6 and sys.argv[6]=='lp' +# check sibling families have disjoint times +fams=defaultdict(list) +for k,(c,ts) in P['windows'].items(): + o,ch=map(int,k.split(','));fams[o,P['pairs'][str(ch)],P['sides'][str(ch)]].append(set(ts)) +for f,L in fams.items(): + for i in range(len(L)): + for j in range(i):assert not L[i]&L[j],f +inv={p:Counter(REG[p]['widths']) for p in provs} +val={p:sum(V[int(r)]*c for r,c in REG[p]['actual_paid_profile'].items()) for p in provs} +print('vals',{p:val[p]-val['jp8'] for p in provs}) +WID=[4,5,7,8,9] +def pats(cap,L): + out=[] + def rec(pre,i,tot): + r=cap-tot + if cap<48 or r<=TIGHT or r==48: out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)>b&1} + req=tuple(i for (tt,w),i in fl.items() if tt==t and w in cols) + pres=tuple(j for j,c in zip(pv,cnts) if any(c[w] for w in cols)) + if req and (req,pres) not in seen: + seen.add((req,pres));row={i:1 for i in req} + for j in pres:row[j]=row.get(j,0)-1 + con(row,-np.inf,0) + for w in WID: + row={i:1 for (t,ww),i in fl.items() if ww==w} + for j,c in zip(pv,cnts): + if c[w]:row[j]=row.get(j,0)-c[w] + if row:con(row,0,0) + groups[key]=dict(cap=cap,times=times,ps=ps,pv=pv,fl=fl) +for t in range(72): + for w in WID: + row=dict(invrow[t,w]) + for p in provs: + if inv[p][w]:row[sel[t,p]]=-inv[p][w] + con(row,0,0) +rr,cc,vv=[],[],[] +for i,row in enumerate(R): + for j,x in row.items():rr.append(i);cc.append(j);vv.append(x) +A=coo_matrix((vv,(rr,cc)),shape=(len(R),len(ub))).tocsr() +print('vars',len(ub),'cons',len(R),flush=True) +base=71*val['jp8']+val['ck']+V[36]+2229*V[48] if 'ck' in provs else None +t0=time.time() +res=milp(np.array(cost),integrality=np.zeros(len(ub)) if LPONLY else np.ones(len(ub)),bounds=Bounds(np.zeros(len(ub)),np.array(ub)),constraints=LinearConstraint(A,lo,hi),options={'time_limit':SECS,'mip_rel_gap':1e-12,'disp':False}) +print(res.status,res.message,'obj',-res.fun if res.x is not None else None,'base639',base,'gain(1e-9b)',(-res.fun-base) if res.x is not None and base else None,'secs',time.time()-t0,flush=True) +if res.x is None:sys.exit() +if LPONLY: + d=Counter() + for (t,p),i in sel.items():d[p]+=res.x[i] + print('lp dispatch',dict(d));sys.exit() +x=np.round(res.x).astype(int) +disp={t:p for (t,p),i in sel.items() if x[i]} +rem=Counter() +for key,g in groups.items(): + for q,i in zip(g['ps'],g['pv']): + if x[i] and g['cap']-sum(q):rem[g['cap']-sum(q)]+=x[i] +print('dispatch',Counter(disp.values()),{t:p for t,p in disp.items() if p!='jp8'},'remainders',dict(rem)) +pickle.dump(dict(groups=groups,x=x,disp=disp,REG=REG),open(OUT+'.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mkV.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mkV.py new file mode 100644 index 0000000000..1e604f98ce --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/mkV.py @@ -0,0 +1,11 @@ +from price import price +from load import * +import json +base=Counter({int(k):v for k,v in P['profile'].items()}) +b0=price(base) +V={} +for r in range(1,49): + p=Counter(base);p[r]+=1;p[1]-=r + V[r]=float((price(p)-b0)*10**9) +print({r:round(v,4) for r,v in V.items()}) +json.dump(V,open('V639.json','w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/price.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/price.py new file mode 100644 index 0000000000..cf550736a3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/price.py @@ -0,0 +1,7 @@ +import sys,json,importlib.util,time +from fractions import Fraction as Q +S='/Users/joelpulikkan/imb-src/pr568/research/complex-multicold' +s=importlib.util.spec_from_file_location('moment',S+'/engines/moment.py');M=importlib.util.module_from_spec(s);s.loader.exec_module(M) +def price(H,live=7168):return Q(M.certify({int(k):v for k,v in H.items()},73,live,False)['lower']) +if __name__=='__main__': + H=json.loads(sys.argv[1]);t=time.time();b=price(H);print(float(b),b,time.time()-t) diff --git a/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/submip.py b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/submip.py new file mode 100644 index 0000000000..31bcacd1bb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r2-dispatch/submip.py @@ -0,0 +1,104 @@ +"""Window-subset exact repack: free every role hosted in windows S (plus unplaced roles), re-solve S with the +aggregate pattern MIP (Hall masks; cap<48 exact full; cap48 full/empty or remainder<=TIGHT), expand literally.""" +import sys,json,pickle,random,time +import numpy as np +from collections import Counter,defaultdict +from scipy.optimize import milp,LinearConstraint,Bounds +from scipy.sparse import coo_matrix +from load import P,reg +sys.path.insert(0,'.') +V={int(k):v for k,v in json.load(open('V639.json')).items()};V[0]=0.0 +WID=[4,5,7,8,9] +def build_state(T,neww): + import gzip + rows=json.loads(gzip.decompress(open('/Users/joelpulikkan/imb-r2/research/pr638-improvement-20261013/witness638/data/OWNERS.json.gz','rb').read())) + own={};pool=defaultdict(list) + for h in range(492):pool[neww[h]].append(h) + for r in sorted(rows,key=lambda r:(r['time'],r['helper'])): + o=(r['owner_replica'],r['child'],r['port']) + if r['time']!=T:own[r['time'],r['helper']]=o + elif pool[r['width']]:own[T,pool[r['width']].pop()]=o + for w,v in pool.items(): + for h in v:own[T,h]=None + return own +def solve(own,widthof,S,TIGHT,SECS,seed=0): + W=P['windows'] + free=[r for r,o in own.items() if o is None or f'{o[0]},{o[1]}' in S] + dem=Counter((r[0],widthof(r)) for r in free) + ub=[];cost=[];R=[];lo=[];hi=[] + def var(c,u=np.inf):ub.append(u);cost.append(-c);return len(ub)-1 + def con(row,l,h):R.append(row);lo.append(l);hi.append(h) + groups={};invrow=defaultdict(dict) + for key in S: + cap,times=W[key] + def pats(cap,L): + out=[] + def rec(pre,i,tot): + r=cap-tot + if r==0 or (cap==48 and (r<=TIGHT or r==48)): out.append(tuple(pre)) + for j in range(i,len(WID)): + w=WID[j] + if tot+w<=cap and len(pre)>b&1} + req=tuple(i for (tt,w),i in fl.items() if tt==t and w in cols) + pres=tuple(j for j,c in zip(pv,cnts) if any(c[w] for w in cols)) + if req and (req,pres) not in seen: + seen.add((req,pres));row={i:1 for i in req} + for j in pres:row[j]=row.get(j,0)-1 + con(row,-np.inf,0) + for w in WID: + row={i:1 for (t,ww),i in fl.items() if ww==w} + for j,c in zip(pv,cnts): + if c[w]:row[j]=row.get(j,0)-c[w] + if row:con(row,0,0) + groups[key]=dict(cap=cap,times=times,ps=ps,pv=pv,fl=fl) + for (t,w),n in dem.items(): + if not invrow[t,w]:print('no window for',t,w);return None + con(dict(invrow[t,w]),n,n) + rr,cc,vv=[],[],[] + for i,row in enumerate(R): + for j,x in row.items():rr.append(i);cc.append(j);vv.append(x) + A=coo_matrix((vv,(rr,cc)),shape=(len(R),len(ub))).tocsr() + t0=time.time() + res=milp(np.array(cost),integrality=np.ones(len(ub)),bounds=Bounds(np.zeros(len(ub)),np.array(ub)),constraints=LinearConstraint(A,lo,hi),options={'time_limit':SECS,'mip_rel_gap':1e-9}) + print('submip',len(S),'windows vars',len(ub),'status',res.status,res.message[:60],'obj',None if res.x is None else -res.fun,'secs',round(time.time()-t0,1),flush=True) + if res.x is None:return None + x=np.round(res.x).astype(int) + rem=Counter() + for key,g in groups.items(): + for q,i in zip(g['ps'],g['pv']): + if x[i] and g['cap']-sum(q) and not (g['cap']==48 and not q):rem[g['cap']-sum(q)]+=x[i];print(' win',key,q,x[i]) + print(' remainders',dict(rem)) + # expand + exec(open('expand_fn.py').read(),globals()) + pools=defaultdict(list) + for r in free:pools[r[0],widthof(r)].append(r) + new=dict(own) + for gi,(key,g) in enumerate(sorted(groups.items())): + plist=[q for q,i in zip(g['ps'],g['pv']) for _ in range(x[i])] + flows={k:x[i] for k,i in g['fl'].items() if x[i]} + slots=expand(plist,flows,gi) + a,b=map(int,key.split(',')) + for port,s in enumerate(slots): + for t,wd in s:new[pools[t,wd].pop()]=(a,b,port) + assert not any(pools.values()) + return new,rem +if __name__=='__main__': + T=int(sys.argv[1]);neww=json.load(open(sys.argv[2])) if sys.argv[2]!='-' else reg['jp8']['widths'] + S=sys.argv[3].split(';');TIGHT=int(sys.argv[4]);SECS=float(sys.argv[5]);OUT=sys.argv[6] + own=build_state(T,neww) + calls=[c['provider'] for c in P['provider_calls']] + widthof=lambda r:neww[r[1]] if r[0]==T else reg[calls[r[0]]]['widths'][r[1]] + out=solve(own,widthof,S,TIGHT,SECS) + if out:pickle.dump(out[0],open(OUT,'wb'));print('saved',OUT) diff --git a/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/REPORT.md b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/REPORT.md new file mode 100644 index 0000000000..3b607b4710 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/REPORT.md @@ -0,0 +1,8 @@ +# R7 progress +- [start] reading R6, PR list +- [setup] no newer kappa head than #1218/#1239 (0.001675809683500501). Using #1186 exact-frame-toolkit (~/imb-harness/r7/tk, IMB_ROOT=~/imb-r6c). Exact frame optimum of JFX/D4FX = published profiles (frames already optimal). Running repair.py fusion+repair sweep on both (~/imb-harness/r7/sweep). +- [understand] source-holder fusion (#1195): at a two-donation triangle slot (E->H2 for t2, X->H1 for t1), keep only X->H1 donation (+FULL restore) and replace E delivery by full SW4 read-set exchange of copy pair (E,X) (t2 reads X, pre-read of E deleted). X54 slot (#1130) = donations 367+=589 (t223), 335+=54 (t159) + swaps t219/t155. Toolkit joint.py family = {1..3 fused reads per copy pair} x {rest kept/swapped/partial}; repair.py = every read fusion + single/pair swap repairs + full exchange. Plan: repair+joint sweeps on JFX, D4FX and on F (pre-X54) to test 'fusion at X54 site' (X54 analog of #1195). +- [result] cost units (b=#1094 default): J 11683162362, X(J+X54) -1788, F(J+fusions) -4338, JFX = F-1788. X54-site source-holder fusion analog (t159 A54 H335 + exchange 589) on F: best +956 (worse than slot's -1788); 45 fusions touching 54/589 tried, only 4 feasible, none gain. So no 'X54 -> fusion' upgrade. +- [result] repair.py sweep (every read fusion x {B-after,B-before-next,tail,prepend restores} x {none, single/pair swaps, full exchange}) on JFX and D4FX: 2238 fusions each, 2850 exact pricings each, 969 feasible, 0 gains (1911s, 5 workers). Standalone SW4 exchange subsets (50 per word, all non-empty subsets per copy pair): all 50 infeasible (cone caps: 'L not inside U' at end-of-B target vertices). Next: joint.py triangle-hybrid family on JFX/D4FX/F. +- [result] joint.py family (#1186) on JFX/D4FX: 161 cands, 24 ok, 0 gains; on F 259, 0 gains -- but it does NOT contain the X54 slot (donation position fixed at lwH+1). Wrote joint2.py (generalized: all donation/restore gap classes, keep/swap/fuse per movable read). Validated: on F it reproduces X54 slot = -1788 (120 equivalent variants ok). Counts: JFX 3924, D4FX 3924, F 42132 (38264 at pair 589). Running JFX/D4FX. +- [result] joint2 (non-target holders) JFX/D4FX: 3924 each, ALL infeasible (exact frame solver). Extended to target holders (as joint.py allows): +2940 candidates each, running (joint2b). fusion2.py (generalized fusion, all gap classes, alone + with full exchange): 10603 cands/2305 (t,A,H) on JFX, queued. diff --git a/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/fusion2.py b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/fusion2.py new file mode 100644 index 0000000000..35f990d3ed --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/fusion2.py @@ -0,0 +1,74 @@ +"""fusion2: generalized read fusion sweep (superset of fusion.py + repair.py's base fusions). +A target t reads A (coef a, at iA) and H (coef h, at iH), H any non-target register (entrance pre-read reads included). +Donate q*A into H (q = a/h dyadic) at EVERY gap class p in the valid window, delete t's read of A, restore H -= q*A at +every B gap class up to H's next read/scale or A's next write, every early-tail class, or as FULL prepend. +Each fusion is priced alone and with the full read-set exchange of the copy pair containing A or H (as in #1195). +usage: python -B fusion2.py WORD.json.gz OUT.json [--workers N] [--count]""" +import argparse, collections, json, multiprocessing as mp, time +from fractions import Fraction as Q +import harness as C +from fusion import index, dyadic, any_in +from exchange import copy_pairs, movable_reads, is_target +from joint import swap_edit, merge +from joint2 import gap_classes +def candidates(w): + I = index(w); B = I['B']; nB = len(B); out = [] + by_t = collections.defaultdict(list) + for i, e in enumerate(B): + if e[0] == 'add' and is_target(e[1]): by_t[e[1]].append((i, e[2], Q(e[3], e[4]))) + for t, rs in by_t.items(): + for (iA, A, a) in rs: + for (iH, H, ch) in rs: + if H == A or iH == iA or t in (A, H): continue + if any_in(I['RD'][t] + I['SC'][t], min(iA, iH), max(iA, iH) + 1): continue + q = a / ch + if not dyadic(q): continue + othersH = [x for x in I['RD'][H] + I['SC'][H] if x != iH] + lo = max([x + 1 for x in othersH if x < iH], default=0) + ev = set(I['W'][H]) | set(I['W'][A]) | set(I['RD'][A]) | set(othersH) + for p in gap_classes(ev, lo, iH): + if any_in(I['W'][A], min(p, iA + 1), max(p, iA + 1)): continue # A holds the value t read + g = ['add', H, A, q.numerator, q.denominator, []]; gr = ['add', H, A, -q.numerator, q.denominator, []] + nxt = min([x for x in othersH if x > iH] + [x for x in I['W'][A] if x >= p] + [nB]) + rests = [] + for rp in gap_classes(ev, iH + 1, nxt): rests.append(("B%d" % rp, dict(insert=[(p, g), (rp, gr)]))) + if not any_in(I['W'][A], p, nB) and not any_in(othersH, iH + 1, nB): + jmax = min(I['TW'][A] + I['TR'][H] + [len(I['T'])]) + tev = {k for k, e in enumerate(I['T']) if H in C.regs(e) or A in C.regs(e)} + for tp in sorted({0, jmax} | {k + 1 for k in tev if k + 1 <= jmax}): rests.append(("T%d" % tp, dict(insert=[(p, g)], tail=[(tp, gr)]))) + if A not in I['low'] and H not in I['low'] and not I['TW'][A] and not I['TR'][H]: rests.append(("P", dict(insert=[(p, g)], prepend=[gr]))) + for tag, e in rests: + ed = dict(delete=[iA], insert=e.get('insert', []), tail=e.get('tail', []), prepend=e.get('prepend', [])) + out.append(dict(t=t, A=A, H=H, iA=iA, iH=iH, p=p, rest=tag, ed=ed)) + return out +P = None; PAIRS = None +def init(path): + global P, PAIRS + P = C.Pricer(path); PAIRS = copy_pairs(P.base) +def run(c): + best = None; tried = [] + eds = [((), c['ed'])] + for h, (r, ic) in PAIRS.items(): + if c['A'] in (h, r) or c['H'] in (h, r): + rest = [m for m in movable_reads(P.base, h, r, ic) if m[0] not in c['ed']['delete']] + ss = [swap_edit(P.base, h, r, ic, m) for m in rest] + if rest and all(ss): + m2 = merge([c['ed']] + ss) + if m2: eds.append(((("exchange", h),), m2)) + for key, ed in eds: + x = P.price(ed, timeout=120); tried.append((key, x.get('status'))) + if x.get('delta') is not None and (best is None or x['delta'] < best['delta']): best = dict(delta=x['delta'], steps=key, profile=x['profile']) + return dict({k: v for k, v in c.items() if k != 'ed'}, tried=tried, best=best) +if __name__ == "__main__": + ap = argparse.ArgumentParser(); ap.add_argument("word"); ap.add_argument("out"); ap.add_argument("--workers", type=int, default=4); ap.add_argument("--count", action="store_true") + a = ap.parse_args(); cs = candidates(C.load(a.word)); print("fusion2 candidates:", len(cs), "distinct (t,A,H):", len({(c['t'], c['A'], c['H']) for c in cs}), flush=True) + if a.count: raise SystemExit + t0 = time.time(); res = [] + with mp.get_context("fork").Pool(a.workers, initializer=init, initargs=(a.word,), maxtasksperchild=40) as pool: + for n, r in enumerate(pool.imap_unordered(run, cs, chunksize=1), 1): + res.append(r) + if n % 500 == 0: print(f" {n}/{len(cs)} {time.time()-t0:.0f}s feasible {sum(1 for x in res if x['best'])} gains {sum(1 for x in res if x['best'] and x['best']['delta'] < 0)}", flush=True) + gains = sorted([x for x in res if x['best'] and x['best']['delta'] < 0], key=lambda x: x['best']['delta']) + print(f"done {len(res)} in {time.time()-t0:.0f}s feasible {sum(1 for x in res if x['best'])} gains {len(gains)}") + for x in gains[:30]: print(" gain", -x['best']['delta'], x['t'], x['A'], x['H'], x['p'], x['rest'], x['best']['steps'], x['best']['profile']) + json.dump(res, open(a.out, "w")) diff --git a/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/joint2.py b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/joint2.py new file mode 100644 index 0000000000..320139dc44 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/joint2.py @@ -0,0 +1,100 @@ +"""joint2: generalized triangle-hybrid / source-holder family per copy pair (superset of joint.py). + +Differences from joint.py: (1) the holder donation position p ranges over EVERY gap class (positions between consecutive +B events touching H or src) in the window where src keeps the value t read and H is neither read by another register nor +scaled before t's read of H; (2) restores range over every B gap class after t's read of H up to the next read/scale of H or +write of src, every early-tail position class, and the FULL prepend; (3) each movable read independently chooses +keep / swap / fuse(option) (cartesian product). Exactness is re-checked by harness.Pricer (mod p) and frames solved exactly. +usage: python -B joint2.py WORD.json.gz OUT.json [--workers N] [--count]""" +import argparse, itertools, json, multiprocessing as mp, time, collections +from fractions import Fraction as Q +import harness as C +from fusion import index, dyadic, any_in +from exchange import copy_pairs, movable_reads, is_target +from joint import swap_edit, merge +def reads_of(B, t): + return [(i, e[2], Q(e[3], e[4])) for i, e in enumerate(B) if e[0] == 'add' and e[1] == t] +def gap_classes(events, lo, hi): + """representative insertion positions in [lo, hi]: one per gap between consecutive events (position = event index + 1).""" + pts = sorted({lo} | {x + 1 for x in events if lo <= x + 1 <= hi}) + return pts +def holder_options(w, I, m, h, r, ic): + j, t, src, (a0, b0) = m; a = Q(a0, b0); B = I['B']; nB = len(B); out = [] + for (iH, H, cH) in reads_of(B, t): + if iH == j or H in (h, r, t): continue + q = a / cH + if not dyadic(q): continue + lo_t, hi_t = min(j, iH), max(j, iH) + if any_in(I['RD'][t] + I['SC'][t], lo_t, hi_t + 1): continue + othersH = [x for x in I['RD'][H] + I['SC'][H] if x != iH] + # donation window: p <= iH, no other read/scale of H in [p, iH), src unchanged between p and j + lo = max([x + 1 for x in othersH if x < iH] + [ic + 1 if src == h else 0]) + lo = max(lo, max([x + 1 for x in I['W'][src] if x < min(j, iH)], default=0)) if j < iH else lo + if j < iH: lo = max(lo, max([x + 1 for x in I['W'][src] if x < j], default=0)) + else: lo = max(lo, j + 1) if any_in(I['W'][src], j + 1, iH) else lo + if lo > iH: continue + ev = set(I['W'][H]) | set(I['W'][src]) | set(I['RD'][src]) | set(othersH) + for p in gap_classes(ev, lo, iH): + if j < iH and p <= j and any_in(I['W'][src], p, j): continue + if (j < iH and p > j and any_in(I['W'][src], j + 1, p)) or (j > iH and any_in(I['W'][src], p, j)): continue + g = ['add', H, src, q.numerator, q.denominator, []]; gr = ['add', H, src, -q.numerator, q.denominator, []] + nxt = min([x for x in othersH if x > iH] + [x for x in I['W'][src] if x > iH] + [nB]) + rs = [] + for rp in gap_classes(ev, iH + 1, nxt): + if any_in(I['W'][src], p, rp): continue + rs.append(("B%d" % rp, dict(insert=[(p, g), (rp, gr)]))) + if not any_in(I['W'][src], p, nB) and not any_in(othersH, iH + 1, nB): + jmax = min(I['TW'][src] + I['TR'][H] + [len(I['T'])]) + tev = {k for k, e in enumerate(I['T']) if H in C.regs(e) or src in C.regs(e)} + for tp in sorted({0, jmax} | {k + 1 for k in tev if k + 1 <= jmax}): rs.append(("T%d" % tp, dict(insert=[(p, g)], tail=[(tp, gr)]))) + if src not in I['low'] and H not in I['low'] and not I['TW'][src] and not I['TR'][H]: rs.append(("P", dict(insert=[(p, g)], prepend=[gr]))) + for tag, e in rs: + ed = dict(delete=[j], insert=e.get('insert', []), tail=e.get('tail', []), prepend=e.get('prepend', [])) + out.append((dict(H=H, iH=iH, p=p, rest=tag), ed)) + return out +def candidates(w): + I = index(w); out = [] + for h, (r, ic) in copy_pairs(w).items(): + M = movable_reads(w, h, r, ic) + if not M: continue + choices = [] + for m in M: + ch = [("keep", None, dict(delete=[], insert=[], tail=[], prepend=[]))] + s = swap_edit(w, h, r, ic, m) + if s: ch.append(("swap", None, s)) + ch += [("fuse", o, e) for o, e in holder_options(w, I, m, h, r, ic)] + choices.append(ch) + for combo in itertools.product(*choices): + if all(c[0] == "keep" for c in combo): continue + if not any(c[0] == "fuse" for c in combo): continue # pure exchanges: xsweep.py + ed = merge([c[2] for c in combo]) + if ed is None: continue + out.append(dict(h=h, r=r, choice=[(m[0], m[1], m[2], c[0], c[1]) for m, c in zip(M, combo)], ed=ed)) + return out +P = None +def init(path): + global P + P = C.Pricer(path) +def run(c): + x = P.price(c['ed'], timeout=120) + return dict(c, status=x.get('status'), delta=x.get('delta'), profile=x.get('profile')) +if __name__ == "__main__": + ap = argparse.ArgumentParser(); ap.add_argument("word"); ap.add_argument("out"); ap.add_argument("--workers", type=int, default=4) + ap.add_argument("--count", action="store_true"); ap.add_argument("--pairs", default=""); ap.add_argument("--skip", default="") + a = ap.parse_args(); w = C.load(a.word); cs = candidates(w) + if a.pairs: cs = [c for c in cs if str(c['h']) in a.pairs.split(",")] + if a.skip: + done = {json.dumps(r["ed"], sort_keys=True) for r in json.load(open(a.skip))}; n0 = len(cs) + cs = [c for c in cs if json.dumps(json.loads(json.dumps(c["ed"])), sort_keys=True) not in done]; print("skipped", n0 - len(cs)) + per = collections.Counter(c['h'] for c in cs); print("joint2 candidates:", len(cs), dict(per), flush=True) + if a.count: raise SystemExit + t0 = time.time(); res = [] + with mp.get_context("fork").Pool(a.workers, initializer=init, initargs=(a.word,), maxtasksperchild=40) as pool: + for n, r in enumerate(pool.imap_unordered(run, cs, chunksize=1), 1): + res.append(r) + if n % 200 == 0: print(f" {n}/{len(cs)} {time.time()-t0:.0f}s gains {sum(1 for x in res if x['delta'] is not None and x['delta'] < 0)}", flush=True) + st = collections.Counter(r['status'] for r in res) + gains = sorted([r for r in res if r['delta'] is not None and r['delta'] < 0], key=lambda r: r['delta']) + print(f"done {len(res)} in {time.time()-t0:.0f}s status {dict(st)} gains {len(gains)}") + for g in gains[:40]: print(" gain", -g['delta'], "pair", g['h'], g['r'], g['choice'], g['profile']) + json.dump(res, open(a.out, "w")) diff --git a/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/x54site.py b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/x54site.py new file mode 100644 index 0000000000..e6cf6fe50d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r7-fusion-search/x54site.py @@ -0,0 +1,13 @@ +import sys; sys.path.insert(0,'.') +import harness as C, repair as R, fusion as G +from exchange import copy_pairs, movable_reads +F=sys.argv[1] +R.init(F); w=R.P.base +h=589; r,ic=copy_pairs(w)[h]; print('pair',h,r,ic, 'movable', movable_reads(w,h,r,ic)) +cs=[c for c in G.candidates(w) if 54 in (c['A'],c['H']) or 589 in (c['A'],c['H'])] +print('fusions touching 54/589:',len(cs)) +best=[] +for c in cs: + x=R.repair(c) + if x['best']: best.append((x['best']['delta'],c['t'],c['A'],c['H'],c['p'],c['rest'],x['best']['steps'])) +for b in sorted(best)[:15]: print(b) diff --git a/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/PROGRESS.md b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/PROGRESS.md new file mode 100644 index 0000000000..2399290c3f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/PROGRESS.md @@ -0,0 +1,18 @@ +# R8 progress +- Tue Oct 13 20:42:47 EDT 2026 started +- FINDING: #1324, #1305 (and #1280) CLOSED by author 23:22Z as INVALID (idle-overlay moves axes between idle children of different (pair,side) families; DATA-PHASES fix each family's Z4 multiplier). Flagged by rohanarun #1286. Overlay-free: #1261, #1274. +- Valid frontier therefore: #1319 rohanarun 0.001676855229071321 (four kernels exact fill, 74 replicas), #1377 jk2 regrid ...886144, #1424 rohanarun draft regrid ...886206. Retarget: beat #1319 construction (κ > 0.0016768552290713219), lever = more helper-donor cold kernels; NO idle overlay. +- [setup] worktree ~/imb-r8 = pr1319 head 603defe5. Final #1297 words rebuilt to $SP/r8/words/{base,extra}.json.gz (recount matches published profiles). Pricer (r7/tk harness, IMB_ROOT=~/imb-r6c) reproduces -11676 for 555/567 kernels on PR1286 input word (positive control). +- [running] tools/relations.py: generalized census (all 376 registers with early uses; two-shock + one-shock helper/hook + 'free' deletions), then tools/price_all.py exact pricing. +- [result] full-deletion census on final #1297 base word: 376 pivots, 254 relation candidates at 48 pivots (131 hook, 87 two-shock, 36 helper); exact pricing: 99 ok all delta 0, 80 infeasible (frame/cone), 75 inexact mod p. ZERO gains. +- [control] same census on PR1286 input word restricted to 531/543/555/567 recovers exactly PR1297's 5+5 relations, all -11676 (matches SEARCH-RESULT.json). +- [running] subsets-of-early-uses census on base (~1950 jobs). +- [tool] tools/relations2.py = family G (cuts after ANY read of ANY register h, or at B start; hooks around ANY early add gate, any reader/donor; one- and two-shock; all subsets of early uses), solved by hashing images in V/. Contains PR1297's family (control: all 10 original relations recovered, +2 new on 1286 word). +- [running] G on final base word: 1952 (pivot,subset) jobs -> 9916 candidates at 248 pivots (7632 two, 2116 hook, 168 helper); exact pricing in progress (cands/G-base-sub-price.log). +- [result] G census on final base word fully priced: 9916 cands -> 1435 ok (1311 delta 0, 124 delta +4913), 7287 infeasible (exact frame solver/cone caps), 1194 inexact mod p. ZERO negative-delta (gaining) kernels. Extra word pricing running (same scalar program, different frames/entrances). +- [tool] tools/relations3.py: any-type 2-shock (cut+cut, hook+hook, cut+hook) and 3-shock census via Gaussian quotient hashing. K=2 subsets on base: 23139 cands; 15507 new beyond G (11220 hook+hook, 4229 cut+cut, 58 hook+cut). Pricing on base running (cands/K2-base-new-price.log). Extra-word full pricing paused (CPU). +- [result] K2 (any-type 2-shock) on base: first 9601/15507 priced (no timeouts): 1407 ok (1230 Δ0, 177 Δ+4913), 8125 infeasible, 69 inexact, 0 gains. Remainder resumed with 30 s timeout. +- [result] 3-shock spot (554,448): 12 cands, all infeasible. Full 3-shock ~4 CPU-min/pivot -> not run. +- [mechanism] kernel 555 gain = climb merging at read-into targets 187/483 ((1,1)->2, (1,2)->3). +- [notes] draft at $SP/r8/notes/README.md, scripts in notes/scripts. +- [final] K2 on base: 13402/15507 priced: 4346 ok (4169 Δ0, 177 Δ+4913), 8808 infeasible, 248 inexact, 0 gains; 2105 unpriced (budget). Extra-word pricing not completed. No admissible negative-cost kernel found -> NEGATIVE RESULT, notes at $SP/r8/notes/README.md. No branch/commit (nothing to realise). diff --git a/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/README.md b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/README.md new file mode 100644 index 0000000000..4350d661d3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/README.md @@ -0,0 +1,69 @@ +# Helper-donor cold kernels: exhaustive census on the four-kernel words (negative result, no new κ) + +Scope: the two local provider words of PR1297 (pivots 531/543/555/567 already applied), which are byte-identical +to the words used by PR1319's T74 exact-fill construction (current valid κ head 0.001676855229071321; #1377/#1424 +only refine the outer grid). PR1305/PR1324 were withdrawn by their author (idle-overlay invalid), so their idle +reassignment step is not used here. + +## What a helper-donor cold kernel is + +A *cold pivot* q is a register whose value is still its start value (start frame `start_extra[q]`, here usually a +rank-one frame such as `[24]` or `[30]`) when it is read by some early B gates `t += k*q` ("early uses", before +q's first B write). A kernel deletes a set D of those early uses and restores the scalar program with *shocks*: + +* **cut** `h += a*q` inserted in B (PR1274/PR1297: right after an early output read of helper h), undone by + `h -= a*q` at the end of early cleanup; +* **hook** `s += c*q` / `s -= c*q` around an early gate `t += k*s` (the donor s is already at q's frame). + +Exactness is the rational equation `d = Σ coeff_i · response_i`, where `d` is the change of q's final column +caused by deleting D and each response is the backward column of the shocked register. Admissibility is then: +scalar map unchanged (mod p check of B + early tails + prepends), the cone caps (`audit_local` inverse-row and +dirt-column cones as end-of-B caps), and a feasible frame annotation; the profile is the exact optimum of the +#1186 frame solver (integer duality certificate). Widths, ports, centres and entrances are unchanged, so any +admitted kernel would reuse the owners/dispatch unchanged. + +Why the four kernels pay: per-register climb diff of pivot 555 on the PR1286 word +(`tools/climbs.py`): `(187,1)-2, (187,2)+1, (483,1)-1, (483,2)-1, (483,3)+1`. Deleting the rank-one early reads +of q into 187/483 merges two consecutive climbs of those targets into one larger climb; the new gates land on +registers (donor 620, helper 383) that are already at the needed frame. Paid profile H1 −3, H3 +1, cost −11676. + +## Census (all exact) + +| Family | Generator | (pivot, D) jobs | candidates | priced: ok / infeasible / inexact | gains | +|---|---|---|---|---|---| +| PR1297 family, D = all early uses | `relations.py full` | 376 | 254 (48 pivots) | 99 (all Δ=0) / 80 / 75 | 0 | +| G: cut after any read of any h (or B start), hook around any early add, 1- or 2-shock cut+hook, all subsets D | `relations2.py subsets` | 1952 | 9916 (248 pivots) | 1435 (1311 Δ=0, 124 Δ=+4913) / 7287 / 1194 | 0 | +| K2: any two shocks (cut+cut, hook+hook), all subsets D, beyond G | `relations3.py 2 subsets` | 1952 | 15507 | 13402 priced: 4346 (4169 Δ=0, 177 Δ=+4913) / 8808 / 248; 2105 unpriced (time budget; 38 pivots, 1547 hook+hook, 557 cut+cut, 1 hook+cut) | 0 | +| 3-shock spot check (pivots 554, 448) | `relations3.py 3 full` | 2 | 12 | 0 / 12 / 0 | 0 | + +Controls: on the PR1286 input word, `relations.py` restricted to 531/543/555/567 recovers exactly PR1297's +5 + 5 relations, all priced −11676 (= its SEARCH-RESULT.json); `relations2.py` contains all ten and finds two more +(same pivots). So the generators and the pricer reproduce the published kernels. + +Per pivot (family G): 376 registers have early uses; 128 admit no exact dyadic relation at all; of the 248 with +relations, the best outcome is Δ=0 for 171, Δ=+4913 for 6, infeasible for 67, inexact for 4. +Main infeasibility reason: `L not inside U at mutable vertex` (3566 at the early-cleanup restore of a cut, the +rest at end-of-B cone caps) — the restore `h -= a*q` or the hook pair forces a register below its cone. + +Cold-pivot inventory on the final base word: 192 registers have start frames; 152 have early uses +(rank-one start: 24 with 1 use, 12 with 2, 5 with 3, 53 with 4, 2 with 5; rank 4: 44 with 3; rank 5: 12 with 3). +The (out, out, helper) three-use pattern of the four kernels occurs only at 531/543/555/567, which are spent. + +## Conclusion + +On the current four-kernel words there is **no further admissible helper-donor cold kernel with negative cost** +in families PR1297 ⊂ G (fully priced) and in the 13402 priced members of K2 (all subsets of early uses, any single or paired cut/hook shocks). PR1297's +four kernels are the only positive ones in this neighbourhood. Not covered: three or more shocks in general +(cost ~4 CPU-min per pivot; only spot-checked), cuts at gaps not adjacent to a read, and gate reordering. +Only the base provider word was priced exhaustively. The extra word has the identical scalar program (same 1844 B gates; it differs only in frames, report and one start_extra entrance group), so the census produces the identical 9916 / 23139 candidate sets; its pricing was started and paused for CPU and is not claimed. 2105 K2 candidates on base remain unpriced (resume: `TMO=30 price_all.py words/base.json.gz cands/K2-base-new.json cands/K2-base-new-price.json`, it skips priced keys). + +## Reproduce + +```sh +export IMB_ROOT= # #1186 toolkit roots +python build_word.py --base vendor/pr1297/inputs/base.json.gz --recipe vendor/pr1297/RECIPE-base.json.gz --out words/base.json.gz +python -B scripts/relations2.py words/base.json.gz cands/G-base-sub.json subsets +python -B scripts/price_all.py words/base.json.gz cands/G-base-sub.json cands/G-base-sub-price.json 10 +python -B scripts/relations3.py words/base.json.gz cands/K2-base-sub.json 2 subsets +``` +`price_all.py` imports the #1186 exact frame toolkit (`harness.Pricer`). diff --git a/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/climbs.py b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/climbs.py new file mode 100644 index 0000000000..85c990f523 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/climbs.py @@ -0,0 +1,27 @@ +import sys,gzip,json,collections +sys.path.insert(0,'/Users/joelpulikkan/imb-harness/r7/tk') +import word as Wd +def climbs(w): + ev=collections.Counter() + orig=Wd.dimension + cur=None + # re-implement recount with per-register tally + v,helpers=w['report']['v'],w['report']['R'];n=2*v+helpers + full=frozenset(range(512));span=Wd.span;dim=Wd.dimension + cur=[frozenset((0,p)) for p in w['ports']]+[frozenset((0,))]*(n-v) + for r,vs in w['start_extra'].items(): cur[int(r)]=span(tuple(vs)) + def climb(r,t,tag): + k=dim(t)-dim(cur[r]) + if k: ev[(r,k)]+=1 + cur[r]=t + for sec in ('A','B'): + for e in w[sec]: + t=span(tuple(e[-1]));rs=e[1:3] if e[0]=='add' else e[1:2] + for r in rs: climb(r,t,sec) + for i,vs in w['early_cleanup']: + e=w['cleanup'][i];rs=e[1:3] if e[0]=='add' else e[1:2] + for r in rs: climb(r,span(tuple(vs)),'E') + return ev +a=climbs(json.load(gzip.open(sys.argv[1])));b=climbs(json.load(gzip.open(sys.argv[2]))) +diff={k:b[k]-a[k] for k in set(a)|set(b) if a[k]!=b[k]} +print(sorted(diff.items())) diff --git a/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/price_all.py b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/price_all.py new file mode 100644 index 0000000000..b7d6d51346 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/price_all.py @@ -0,0 +1,25 @@ +"""Price every census candidate exactly with the #1186 exact frame toolkit (scalar exactness mod p, cone caps, exact frames). +usage: price_all.py WORD CANDS.json OUT.json [workers]""" +import sys,json,time +sys.path.insert(0,'/Users/joelpulikkan/imb-harness/r7/tk') +from multiprocessing import Pool +P=None +def init(w): + global P + import harness as H; P=H.Pricer(w) +def job(a): + k,c=a + r=P.price(c['ed'],timeout=int(__import__('os').environ.get('TMO','120'))); r.pop('word',None) + return k,r +if __name__=='__main__': + w,cp,op=sys.argv[1:4]; nw=int(sys.argv[4]) if len(sys.argv)>4 else 10 + C=json.load(open(cp)); t=time.time(); import os; out=json.load(open(op)) if os.path.exists(op) else {}; out={int(k):v for k,v in out.items()} + with Pool(nw,initializer=init,initargs=(w,),maxtasksperchild=200) as pool: + for n,(k,r) in enumerate(pool.imap_unordered(job,[(k,c) for k,c in enumerate(C) if k not in out],chunksize=1)): + out[k]=r + if r.get('status')=='ok' and r['delta']<0: print('GAIN',k,C[k]['pivot'],C[k]['kind'],r['delta'],r['profile'],flush=True) + if n%200==0: print('progress',n,len(C),round(time.time()-t),flush=True); json.dump(out,open(op,'w')) + json.dump(out,open(op,'w')) + import collections + print('status',collections.Counter(r['status'] for r in out.values())) + print('ok deltas',collections.Counter(r['delta'] for r in out.values() if r['status']=='ok').most_common(20)) diff --git a/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/relations.py b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/relations.py new file mode 100644 index 0000000000..830bfaf3ca --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r8-cold-kernel-census/scripts/relations.py @@ -0,0 +1,109 @@ +"""Exhaustive helper-donor cold-kernel relation census (generalizes PR1297 search_relations.py). +For every register q with early uses (reads before q's first B write) and every non-empty subset D of those early uses, +solve d = a*u + b*v exactly over Q, where d is the change of q's final column caused by deleting D, u is a helper cut +shock (h += a*q right after an early output read of helper h; restored at end of early cleanup), v is a donor hook +(donor += c*q / donor -= c*q around an output read of donor). Also one-shock relations (a only / b only). +Dyadic coefficients only. Output: list of candidate edit dicts in harness format (indices into the given word).""" +import gzip,json,sys,itertools,time +from fractions import Fraction as Q +from multiprocessing import Pool +def coef(g): return Q(g[3],g[4]) if g[0]=='add' else Q(g[2],g[3]) +def add(a,b,c): + d=dict(a) + for k,v in b.items(): + x=d.get(k,Q(0))+c*v + if x: d[k]=x + else: d.pop(k,None) + return d +def col(seq,q): + r={q:Q(1)} + for g in seq: + t=g[1] + if g[0]=='add': + x=r.get(t,Q(0))+coef(g)*r.get(g[2],Q(0)) + else: x=r.get(t,Q(0))*coef(g) + if x: r[t]=x + else: r.pop(t,None) + return r +dy=lambda c: not (c.denominator&(c.denominator-1)) +def solve2(u,v,d): + keys=set(u)|set(v)|set(d) + # least squares-free exact: pick two independent rows + ks=sorted(keys) + for i,j in itertools.combinations(ks,2): + det=u.get(i,0)*v.get(j,0)-u.get(j,0)*v.get(i,0) + if det: + a=Q(d.get(i,0)*v.get(j,0)-d.get(j,0)*v.get(i,0))/det;b=Q(u.get(i,0)*d.get(j,0)-u.get(j,0)*d.get(i,0))/det + if all(a*u.get(k,0)+b*v.get(k,0)==d.get(k,0) for k in keys): return a,b + return None + return None # u,v dependent: covered by one-shock +def solve1(u,d): + k=next(iter(d)); + if not u.get(k): return None + a=Q(d[k])/u[k] + if set(u)==set(d) and all(a*u[x]==d[x] for x in d): return a + return None +W=None +def init(path): + global W; W=json.load(gzip.open(path)) +def work(args): + q,dele=args; w=W; B=w['B']; dele=set(dele) + first=next((i for i,g in enumerate(B) if g[1]==q),len(B)) + tail=[w['cleanup'][i] for i,U in w['early_cleanup']]; seq=B+tail + new=[g for i,g in enumerate(B) if i not in dele]+tail + d=add(col(seq,q),col(new,q),-1) + if not d: return [dict(pivot=q,dele=sorted(dele),kind='free',ed=dict(delete=sorted(dele),insert=[],tail=[],prepend=[]))] + cols={};hs=[];ts=[] + get=lambda j: cols.get(j) or {j:Q(1)} + for i in range(len(seq)-1,-1,-1): + g=seq[i] + if i=224 and g[2]!=q and i not in dele: + h=g[2];eff=add(get(h),{h:Q(1)},-1) + if eff: hs.append((i+1,h,eff)) + ts.append((i,g,dict(get(g[1])))) + if i in dele: continue + if g[0]=='add': + t,s=g[1:3];cols[s]=add(get(s),get(t),coef(g)) + else: cols[g[1]]={k:v*coef(g) for k,v in get(g[1]).items()} + out=[];NE=len(w['early_cleanup']) + def hookins(i,g,c): return [[i,['add',g[2],q,c.numerator,c.denominator,[]]],[i+1,['add',g[2],q,-c.numerator,c.denominator,[]]]] + def cutins(cut,h,a): return [[cut,['add',h,q,a.numerator,a.denominator,[]]]],[[NE,['add',h,q,-a.numerator,a.denominator,[]]]] + for cut,h,u in hs: + a=solve1(u,d) + if a is not None and dy(a): + ins,tl=cutins(cut,h,a);out.append(dict(pivot=q,dele=sorted(dele),kind='helper',helper=h,cut=cut,amount=str(a),ed=dict(delete=sorted(dele),insert=ins,tail=tl,prepend=[]))) + for i,g,v in ts: + b=solve1(v,d) + if b is not None: + c=b/coef(g) + if dy(c): out.append(dict(pivot=q,dele=sorted(dele),kind='hook',donor=g[2],hook=i,output=g[1],correction=str(b),ed=dict(delete=sorted(dele),insert=hookins(i,g,c),tail=[],prepend=[]))) + for cut,h,u in hs: + for i,g,v in ts: + if h==g[2]: continue + s=solve2(u,v,d) + if s: + a,b=s;c=b/coef(g) + if not a or not b or not dy(a) or not dy(c): continue + ins,tl=cutins(cut,h,a) + out.append(dict(pivot=q,dele=sorted(dele),kind='two',helper=h,cut=cut,amount=str(a),donor=g[2],hook=i,output=g[1],correction=str(b),ed=dict(delete=sorted(dele),insert=hookins(i,g,c)+ins,tail=tl,prepend=[]))) + return out +if __name__=='__main__': + path,outp=sys.argv[1],sys.argv[2]; mode=sys.argv[3] if len(sys.argv)>3 else 'full' + init(path);B=W['B'];first={} + for i,g in enumerate(B): first.setdefault(g[1],i) + early={} + for i,g in enumerate(B): + if g[0]=='add' and i (projectively normalized); dyadic a, c only. +All candidates are then screened exactly (mod p map, cones, exact frames) by price_all.py.""" +import gzip,json,sys,itertools,time,os +from fractions import Fraction as Q +from multiprocessing import Pool +from relations import coef,add,col,dy +W=None +def init(path): + global W; W=json.load(gzip.open(path)) +def canon(x,d,k0): + if not x: return None + r=Q(x.get(k0,0))/d[k0] + y={k:v-r*d.get(k,0) for k,v in x.items()} + for k,v in d.items(): + if k not in x and r: y[k]=-r*v + y={k:v for k,v in y.items() if v} + if not y: return ('D',) # x proportional to d + ks=sorted(y); n=y[ks[0]] + return tuple((k,y[k]/n) for k in ks) +def work(args): + q,dele=args; w=W; B=w['B']; dele=set(dele) + first=next((i for i,g in enumerate(B) if g[1]==q),len(B)) + tail=[w['cleanup'][i] for i,U in w['early_cleanup']]; seq=B+tail + new=[g for i,g in enumerate(B) if i not in dele]+tail + d=add(col(seq,q),col(new,q),-1) + if not d: return [dict(pivot=q,dele=sorted(dele),kind='free',ed=dict(delete=sorted(dele),insert=[],tail=[],prepend=[]))] + k0=min(d) + cols={}; get=lambda j: cols.get(j) or {j:Q(1)} + U=[];Vv=[] + for i in range(len(seq)-1,-1,-1): + g=seq[i] + if i20000: return out + return out +if __name__=='__main__': + path,outp,k,mode=sys.argv[1],sys.argv[2],int(sys.argv[3]),sys.argv[4]; minu=int(sys.argv[5]) if len(sys.argv)>5 else 1 + init(path,k);B=W['B'];first={} + for i,g in enumerate(B): first.setdefault(g[1],i) + early={} + for i,g in enumerate(B): + if g[0]=='add' and i=0 and comm(g,B[k-1]): k-=1 + gate=list(g[:5])+[[]] + if j>i: out.append(dict(pivot=g[2],kind='late',gate=i,to=j,ed=dict(delete=[i],insert=[[j+1,gate]],tail=[],prepend=[]))) + if k 5-term kernel delta -1850 ok; on step0 word dels [342] (482 -= 554) -> 5-term kernel -1850 ok. => positive control passes. +- [why R8 missed] R8 families (G/K2) only had: cut = h += a*q after a read, restored at END of early cleanup (+q0), hook = +/- pair around ONE gate, <=2 shocks. The 566/554 kernels are 5-7 term relations using UNPAIRED direct reads (x += q at B0..q's first write, never undone: response col_p[x], not col-e or hook form), helper windows closed in early cleanup right after q's OWN restoration (holder windows, possibly with alpha=-1 = signed restore), and donor windows spanning many gates. Every term sits in a holder window of q's initial value; so the right family is the full linear span of holder-read responses (PR1957 gencomp2), not a pairwise cut/hook family. +- [overlap] PR1957 (rohanarun, open) already ran that family on the CURRENT word for 101 single deletions + '0,1,2', '3,4' (budget -ddel-1, oracle-pruned candidates), plus single-gate commuting moves at 4461 positions: 0 gains. R9 extension therefore targets: (E1) multi-deletion same-value subsets, (E2) relaxed oracle budget (oracle is not additive; price exactly), (E3) retiming of ALL B adds (not only gain-carrying), running. +- [running] E1: 330 same-value multi-deletion subsets (pairs..quads from 172 value groups of gain reads), gencomp2 maxterms 8, budget -ddel-1. First ~15: all shape-infeasible/INFEASIBLE. +- [running] E2: relaxed oracle budget (+4913) on 24 small-gain reads (16 one-split, 783/802 -6763, 822/840/858/876 -6100 (858/876 not run by PR1957), 857/875 -3050): first 8 -> only the trivial "move the read one slot" (delta 0), else infeasible/shape-infeasible. +- [running] E3: retime census (3074 commuting extreme moves, all B adds) on current gauge44 word, exact pricing. +- [next] E4 two-value holder family: compensation reads of registers holding v+u combos cancelled by holders of u (matrix-valued exactness); not in PR1957 or R8. +- [tool] tools/gencomp3.py: two-value holder family (holders of v, or of v+u where >=2 windows share residual u, incl. pure-u holders), matrix-valued exactness, chain-exclusion (no insert target feeds a later insert), holder values from the DELETED program. Control on #1319 word dels [755]: recovers the pure 566 kernel (-1850) AND new exact two-value variants (e.g. 210 += 1/2*676 at 293 / -1/2 at 237 replacing 210 -= q), all -1850 (same cost). +- [result] E2 complete (24 jobs, slack +4913): 0 gains (only delta-0 trivial slot moves). +- [running] E4 = gencomp3 on all 1002 gain-carrying single reads of current gauge44 word, ordered by smallest gain; E1 (330 multi-deletion) and retime (3074) still running. +- [result] E3 retime census (3074 commuting-window extreme moves of every B add, current gauge44 word): 2068 ok all delta 0, 1006 infeasible, 0 gains. +- [running] E4b = E4 in reverse order (largest-gain delivery reads first), 3 more workers. +- [load] advisor cap: stopped E1 (resumable; SUMMARY skips done) to keep <=6 workers (E4 3 + E4b 3). +- [fix] gencomp3 chain-exclusion direction was reversed (inexact solutions at 783/802: 506 += 66 then 530 += 506). Fixed (exclude reads of x at positions >= insert); E4/E4b relaunched, keeping only shape-infeasible results (independent of the chain constraint). +- [notes] draft at $SP/r9/notes/README.md (scripts in notes/scripts) +- [result] E4 stopped at 144/1002 (time/CPU cap): 126 shape-infeasible, 18 CP-infeasible, 0 gains. Covered: all small-gain reads (-3050..-6763), the -8857/-9567/-9826 reads done so far, and the largest-gain reads (-22039..-26389). Control rerun with fixed tool: 566 kernel + two-value variant -1850 ok. +- [result] E1 stopped at 128/330: 0 solutions. +- [final] NO GAIN. Notes: $SP/r9/notes/README.md. No branch/commit/PR. diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/README.md b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/README.md new file mode 100644 index 0000000000..b4e4ae9e3e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/README.md @@ -0,0 +1,67 @@ +# Why the R8 census missed the pivot-566 / signed-554 kernels, and an extended census on the current T74 supplier (no new κ) + +Words: the current supplier words carried by PR1800 / PR1839 (`new/{gauge44,extra}-word-566.json.gz`, git blobs +76c01583… / e86142b3…, byte-identical to PR1672 and to PR1957's inputs). Exact frame pricing (PR1186 toolkit) +reproduces their local paid profiles: gauge44 {1:2348,2:500,3:416,4:130,5:82}, extra {1:2340,2:508,3:420,4:126,5:82}. +PR1672's EDITS-1 (pivot-554 kernel) on the step-0 word prices at Δ = −1850 (ok), same as published. + +## 1. The missed mechanism + +R8's families (PR1297 ⊂ G ⊂ K2) were built from two shock types, both paired: +* cut `h += a·q` right after a read of h, undone by `h −= a·q` at the **end** of early cleanup; +* hook `s ± c·q` around **one** existing gate; +with at most two shocks per relation. + +The pivot-566 (PR1625) and pivot-554 (PR1672/PR1718) kernels are 5–7-term relations that lie outside this: +* **unpaired direct reads** `x += c·q` anywhere q still holds its start value; never undone (response `col_p[x]`, + neither the cut form `col − e` nor the hook form); +* **windows closed in early cleanup right after q's own restoration** (q holds ±q0 again there: the "signed restore" + of PR1718 is a read at a position where q's holder coefficient is −1), not at the end of cleanup; +* donor windows spanning many gates. + +The common description: every term is a read `x += λ·y` at a position where some register y holds a multiple of the +deleted read's source value (a "holder window", in B or in early-cleanup tails), and the relation is the full linear +span of these responses. This is PR1957's `gencomp2` family. Positive controls (this run): +* PR1297/#1319 gauge44 word, delete B755 (`494 −= 566`): gencomp2 and our `gencomp3` find the 5-term kernel + `98+=q@1963(tail), 98−=q@866, 210−=q@514, 150+=q@484, 146+=q@480`, Δ = −1850 ok; +* step-0 word, delete B342 (`482 −= 554`): 5-term kernel with signed tail `74 −= q`, Δ = −1850 ok. +So R8's families were too narrow (paired shocks, ≤2 terms, restore only at the end, only `+q0` value). + +## 2. Extended census on the current supplier word (gauge44) + +PR1957 (open) already ran gencomp2 on these words for 101 single-read deletions (budget −Δdel−1), and single-gate +commuting moves at 4461 positions: 0 gains. The extensions here: + +| Ext | Family | Jobs | Result | +|---|---|---|---| +| E1 | same-value **multi-deletion** subsets (pairs–quads within 172 value groups of gain-carrying reads), gencomp2, ≤8 terms | 128 / 330 run (stopped for CPU cap) | 0 solutions (74 shape-infeasible, rest CP-infeasible) | +| E2 | relaxed oracle budget (+4913; the frame-cost oracle is not additive) on 24 small-gain reads: 16 one-split (−4913), 783/802 (−6763), 822/840/858/876 (−6100; 858/876 not in PR1957), 857/875 (−3050) | 24 / 24 | 0 gains: only the trivial "read moved one slot" (Δ 0), else infeasible / shape-infeasible | +| E3 | commuting-window retiming of **every** B add to its earliest/latest slot | 3074 / 3074 priced exactly | 2068 ok (all Δ 0), 1006 infeasible, 0 gains | +| E4 | NEW **two-value holder** family (`gencomp3.py`): reads of registers holding v + u, with u cancelled by other holders of u (incl. pure-u registers); matrix-valued exactness on (input, output) entries + B-only target rows; chain exclusion; holder values taken in the deleted program | 144 / 1002 single deletions: all 24 small-gain reads (−3050…−9826 band complete: 54 of the −9567/−9826 reads) + 90 largest-gain reads (−22039…−26389) | 126 shape-infeasible, 18 CP-infeasible, 0 gains | + +E4 control: on the #1319 word it recovers the 566 kernel and new exact two-value variants (e.g. `210 += ½·676 @293`, +`210 −= ½·676 @237` replacing `210 −= q`), all Δ = −1850. So the family is strictly larger and still prices +correctly. On the current word: 144 single deletions done (smallest-gain first, and largest-gain first in parallel): +all 24 small-gain reads (−3050…−6763) are shape-infeasible or CP-infeasible within budget −Δdel−1 even with +two-value holders (e.g. 783/802: 1857 candidates, 4 two-value classes, INFEASIBLE; one-split reads 1/3: ~2400 +candidates, 8–11 classes, INFEASIBLE). All large-gain delivery reads finished so far are shape-infeasible: +some effect row has no candidate under the budget. + +## 3. Conclusion + +The R8 negative was an artifact of a too-narrow relation family, not of the words. With the right family +(holder-window spans, PR1957) and two further generalizations (multi-deletion, two-value holders), plus relaxed +oracle budgets and full retiming, the current pivot-566+554 T74 supplier word has **no admissible negative-cost +kernel in the covered sets**. Not covered: the remaining E1/E4 jobs (CPU budget), three-or-more-value holders, +cross-value multi-deletions, A-section edits (see PR1957 §5), and the extra word (it has the same scalar program, +so it yields the same candidate sets, but frames differ). + +## Reproduce +```sh +export IMB_ROOT= OPENBLAS_NUM_THREADS=1 +python scripts/ext3.py words/gauge44.json.gz E4-jobs.json out/E4 10 4 40 0 3 # two-value census +python scripts/ext.py words/gauge44.json.gz E1-jobs.json out/E1 8 4 30 0 3 # multi-deletion (gencomp2) +python scripts/ext.py words/gauge44.json.gz E2-jobs.json out/E2 10 6 40 4913 2 # relaxed budget +python scripts/retime.py words/gauge44.json.gz retime.json && python scripts/price_all.py words/gauge44.json.gz retime.json out.json 6 +``` +`gencomp3.py` imports PR1957's `fc3.solve`, `costorc.CostOracle` and the vendored PR1186 toolkit. diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/base.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/base.py new file mode 100644 index 0000000000..5add0ef610 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/base.py @@ -0,0 +1,9 @@ +import sys,json,time +sys.path.insert(0,'/Users/joelpulikkan/imb-harness/r7/tk') +import harness as H +SP='/private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/r9/' +for w in ['gauge44','extra']: + t=time.time();P=H.Pricer(SP+f'words/{w}.json.gz');print(w,P.base_cost,P.base_profile,round(time.time()-t),flush=True) +t=time.time();P=H.Pricer(SP+'words/step0-gauge44.json.gz');print('step0',P.base_cost,P.base_profile) +ed=json.load(open(SP+'src/EDITS-1.json'))['edit'];print(ed) +r=P.price(ed);print('k554 on step0',r) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext.py new file mode 100644 index 0000000000..cbb7e058b2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext.py @@ -0,0 +1,39 @@ +"""Extended value-holder kernel census driver (wraps PR1957 gencomp2.run unchanged). +usage: ext.py WORD JOBS.json OUTDIR maxterms K tl slack nworkers +JOBS = [[dels...], ...]; budget = -ddel - 1 + slack (slack>0 relaxes the non-additive frame-cost oracle; every +solution is priced exactly by the PR1186 frame solver).""" +import os, sys, json, time +sys.path.insert(0, '/private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/r9/src/pr1957/research/t74-k554-local-exhaustion/tools') +import _paths, contextlib, io +from multiprocessing import Pool +P = None; ARGS = None +def init(word, args): + global P, ARGS + import harness as H; P = H.Pricer(word); ARGS = args +def job(dels): + import gencomp2 as G + from fc3 import solve + mt, K, tl, slack, od = ARGS + t = time.time() + try: + _, r = solve(P, dict(delete=dels)); ddel = r['cost'] - P.base_cost + if ddel >= 0: return dels, dict(ddel=ddel, skip=True) + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + ddel, res = G.run(P, dels, mt, K, tl, os.path.join(od, 'x-%s.json' % '-'.join(map(str, dels))), budget=-ddel - 1 + slack) + best = min([x['delta'] for x in res if x.get('delta') is not None], default=None) + return dels, dict(ddel=ddel, nsol=len(res), best=best, st=[x['status'] for x in res], log=buf.getvalue()[-600:], sec=round(time.time() - t, 1)) + except Exception as e: + return dels, dict(err=repr(e)[:200]) +if __name__ == '__main__': + word, jp, od = sys.argv[1:4]; mt, K, tl, slack, nw = int(sys.argv[4]), int(sys.argv[5]), float(sys.argv[6]), int(sys.argv[7]), int(sys.argv[8]) + os.makedirs(od, exist_ok=True); jobs = json.load(open(jp)); summ = os.path.join(od, 'SUMMARY.json') + done = json.load(open(summ)) if os.path.exists(summ) else {} + jobs = [j for j in jobs if '-'.join(map(str, j)) not in done] + t = time.time() + with Pool(nw, initializer=init, initargs=(word, (mt, K, tl, slack, od)), maxtasksperchild=40) as pool: + for n, (dels, r) in enumerate(pool.imap_unordered(job, jobs)): + done['-'.join(map(str, dels))] = r + if r.get('best') is not None and r['best'] < 0: print('GAIN', dels, r, flush=True) + print(n, len(jobs), dels, r.get('ddel'), r.get('nsol'), r.get('best'), r.get('err', ''), r.get('sec'), round(time.time() - t), flush=True) + json.dump(done, open(summ, 'w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext3.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext3.py new file mode 100644 index 0000000000..22ead22719 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/ext3.py @@ -0,0 +1,39 @@ +"""Extended value-holder kernel census driver (wraps PR1957 gencomp2.run unchanged). +usage: ext.py WORD JOBS.json OUTDIR maxterms K tl slack nworkers +JOBS = [[dels...], ...]; budget = -ddel - 1 + slack (slack>0 relaxes the non-additive frame-cost oracle; every +solution is priced exactly by the PR1186 frame solver).""" +import os, sys, json, time +sys.path.insert(0, '/private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/r9/tools') +import contextlib, io, gencomp3 +from multiprocessing import Pool +P = None; ARGS = None +def init(word, args): + global P, ARGS + import harness as H; P = H.Pricer(word); ARGS = args +def job(dels): + import gencomp3 as G + from fc3 import solve + mt, K, tl, slack, od = ARGS + t = time.time() + try: + _, r = solve(P, dict(delete=dels)); ddel = r['cost'] - P.base_cost + if ddel >= 0: return dels, dict(ddel=ddel, skip=True) + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + info = G.run(P, dels, mt, K, tl, os.path.join(od, 'x-%s.json' % '-'.join(map(str, dels))), slack=slack, verbose=False); res = info['res']; info.pop('res') + best = min([x['delta'] for x in res if x.get('delta') is not None], default=None) + return dels, dict(info, nsol=len(res), best=best, st=[(x['status'], x['delta'], x['two_value']) for x in res], sec=round(time.time() - t, 1)) + except Exception as e: + return dels, dict(err=repr(e)[:200]) +if __name__ == '__main__': + word, jp, od = sys.argv[1:4]; mt, K, tl, slack, nw = int(sys.argv[4]), int(sys.argv[5]), float(sys.argv[6]), int(sys.argv[7]), int(sys.argv[8]) + os.makedirs(od, exist_ok=True); jobs = json.load(open(jp)); summ = os.path.join(od, 'SUMMARY.json') + done = json.load(open(summ)) if os.path.exists(summ) else {} + jobs = [j for j in jobs if '-'.join(map(str, j)) not in done] + t = time.time() + with Pool(nw, initializer=init, initargs=(word, (mt, K, tl, slack, od)), maxtasksperchild=40) as pool: + for n, (dels, r) in enumerate(pool.imap_unordered(job, jobs)): + done['-'.join(map(str, dels))] = r + if r.get('best') is not None and r['best'] < 0: print('GAIN', dels, r, flush=True) + print(n, len(jobs), dels, r.get('ddel'), r.get('nsol'), r.get('best'), r.get('err', ''), r.get('sec'), round(time.time() - t), flush=True) + json.dump(done, open(summ, 'w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/gencomp3.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/gencomp3.py new file mode 100644 index 0000000000..07bcf63b34 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/gencomp3.py @@ -0,0 +1,181 @@ +"""Two-value holder kernel search (extends PR1957 gencomp2): delete B read(s) of a value v, compensate with inserted +reads x += lam*y where y holds v, OR holds v + u for a residual direction u shared by >=2 holder windows (so the u parts +can cancel). Exactness is matrix-valued: sum_j lam_j val_j (x) col_j == Delta on every (input, output) entry, over B+tails, +plus the B-only target-row (dirt) rows. Frame-cost oracle = PR1957 CostOracle; every solution is priced exactly.""" +import os, sys, json, time, math +from fractions import Fraction as Q +from collections import defaultdict +sys.path.insert(0, '/private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/r9/src/pr1957/research/t74-k554-local-exhaustion/tools') +import _paths +from ortools.sat.python import cp_model +from fc3 import solve +from costorc import CostOracle +V = 112 +coef = lambda g: Q(g[3], g[4]) if g[0] == 'add' else Q(g[2], g[3]) + +def windows(seq, nreg, skip=frozenset()): + rows = {}; get = lambda x: rows.get(x, {x: Q(1)}) + start = [0] * nreg; cur = [None] * nreg; out = [] + for i, g in enumerate(seq): + if i in skip: continue + t = g[1] + if g[0] == 'add': + a = dict(get(t)); c = coef(g) + for k, v in get(g[2]).items(): + a[k] = a.get(k, 0) + c * v + if not a[k]: del a[k] + else: a = {k: v * coef(g) for k, v in get(t).items()} + old = get(t); rows[t] = a + if a != old: + out.append((t, start[t], i + 1, old)); start[t] = i + 1 + for x in range(nreg): out.append((x, start[x], len(seq) + 1, get(x))) + return out + +def resid(val, v, k0): + r = Q(val.get(k0, 0)) / v[k0] + y = {k: x - r * v.get(k, 0) for k, x in val.items()} + for k, x in v.items(): + if k not in val and r: y[k] = -r * x + y = {k: x for k, x in y.items() if x} + if not y: return (), r + ks = sorted(y); n = y[ks[0]] + return tuple((k, y[k] / n) for k in ks), r + +_W = None +def run(P, dels, maxterms, K, tl, out=None, slack=0, lamr=8, maxcand=6000, verbose=True): + global _W + w = P.base; B = w['B']; NB = len(B); tails = [w['cleanup'][i] for i, U in w['early_cleanup']]; seq = B + tails; L = len(seq) + nreg = 2 * V + w['report']['R'] + W = windows(seq, nreg, frozenset(dels)) # holder values of the DELETED program + # values at deletion + rows = {}; get = lambda x: rows.get(x, {x: Q(1)}); srcval = {} + for i, g in enumerate(seq): + if i > max(dels): break + if i in dels: srcval[i] = dict(get(g[2])) + t = g[1] + if g[0] == 'add': + a = dict(get(t)); c = coef(g) + for k, v in get(g[2]).items(): + a[k] = a.get(k, 0) + c * v + if not a[k]: del a[k] + else: a = {k: v * coef(g) for k, v in get(t).items()} + rows[t] = a + v = srcval[dels[0]]; k0 = min(v) + for i in dels: assert resid(srcval[i], v, k0)[0] == (), 'deleted reads must share a value' + w2, r = solve(P, dict(delete=dels)); ddel = r['cost'] - P.base_cost + bud = -ddel - 1 + slack + orc = CostOracle(w2, P.costs.c); ds = sorted(dels) + def to2(p): return p - sum(1 for t in ds if t < p) + vin = set(v) + cls = defaultdict(list) + ov = set() + for wid, (y, a, b, val) in enumerate(W): + key, _ = resid(val, v, k0) + cls[key].append(wid) + if set(val) & vin: ov.add(wid) + keep = [wid for key, L_ in cls.items() if key == () or (len(L_) >= 2 and any(x in ov for x in L_)) for wid in L_] + holders = defaultdict(list) + for wid in keep: + y, a, b, val = W[wid] + for p in range(a, min(b, L + 1)): holders[p].append(wid) + colsF, colsB = {}, {} + SF = lambda x: colsF.get(x, {x: Q(1)}); SB = lambda x: colsB.get(x, {x: Q(1)}) + D, DB = defaultdict(Q), defaultdict(Q); dset = set(dels); seen = {}; cand = [] + def step(cols, Sf, g): + if g[0] == 'add': + t, s = g[1], g[2]; c = coef(g); a = dict(Sf(s)) + for k, x in Sf(t).items(): + a[k] = a.get(k, 0) + c * x + if not a[k]: del a[k] + cols[s] = a + else: x = g[1]; cols[x] = {k: y * coef(g) for k, y in Sf(x).items()} + for i in range(L, -1, -1): + g = seq[i] if i < L else None + if g is None: pass + elif i in dset: + t = g[1]; c = coef(g) + for kin, vv in srcval[i].items(): + for k, x in SF(t).items(): D[(kin, k)] += c * vv * x + if i < NB: + for k, x in SB(t).items(): + if V <= k < 2 * V: DB[(kin, k)] += c * vv * x + else: + step(colsF, SF, g) + if i < NB: step(colsB, SB, g) + if i in holders and i not in dset: + p2 = to2(i) if i < NB else i - len(ds) + for wid in holders[i]: + y, a, b, val = W[wid] + for x in range(nreg): + if x == y: continue + cst = orc.cost(x, y, p2) + if cst > bud: continue + vf = SF(x); vb = {k: z for k, z in SB(x).items() if V <= k < 2 * V} if i < NB else {} + kk = (x, wid, tuple(sorted(vf.items())), tuple(sorted(vb.items())), i < NB) + if kk in seen and seen[kk] <= cst: continue + seen[kk] = cst + rf = {(ki, k): vi * z for ki, vi in val.items() for k, z in vf.items()} + rb = {(ki, k): vi * z for ki, vi in val.items() for k, z in vb.items()} + cand.append((x, y, i, rf, rb, cst, wid)) + D = {k: x for k, x in D.items() if x}; DB = {k: x for k, x in DB.items() if x} + nwin0 = len(cls.get((), [])); ncls = sum(1 for k, L_ in cls.items() if k and len(L_) >= 2 and any(x in ov for x in L_)) + info = dict(dels=dels, ddel=ddel, bud=bud, pure_windows=nwin0, two_value_classes=ncls, cand=len(cand), D=len(D), DB=len(DB)) + if verbose: print(info, flush=True) + if len(cand) > maxcand or ddel >= 0: return dict(info, skipped='too many candidates' if ddel < 0 else 'no deletion gain', res=[]) + den = 1 + for c_ in cand: + for z in list(c_[3].values()) + list(c_[4].values()): den = den * z.denominator // math.gcd(den, z.denominator) + for z in list(D.values()) + list(DB.values()): den = den * z.denominator // math.gcd(den, z.denominator) + SC = 2 * den + m = cp_model.CpModel(); lam = []; zz = [] + for j in range(len(cand)): + l = m.NewIntVar(-lamr, lamr, ''); b = m.NewBoolVar(''); m.Add(l == 0).OnlyEnforceIf(b.Not()); m.Add(l != 0).OnlyEnforceIf(b); lam.append(l); zz.append(b) + byF = defaultdict(list); byB = defaultdict(list) + for j, c_ in enumerate(cand): + for e, z in c_[3].items(): byF[e].append((j, z)) + for e, z in c_[4].items(): byB[e].append((j, z)) + ok = True + for E, T, by in ((D, set(D) | set(byF), byF), (DB, set(DB) | set(byB), byB)): + for e in T: + rhs = E.get(e, Q(0)) * 2 * SC; terms = [lam[j] * int(z * SC) for j, z in by.get(e, [])] + if not terms: + if rhs: ok = False + continue + m.Add(sum(terms) == int(rhs)) + byY = defaultdict(list) + for j, c_ in enumerate(cand): byY[c_[1]].append(j) + npair = 0 + for j, c_ in enumerate(cand): # an insert into x must not feed a later (or same-slot) insert reading x + for k in byY.get(c_[0], []): + if cand[k][2] >= c_[2]: m.Add(zz[j] + zz[k] <= 1); npair += 1 + info['chain_pairs'] = npair + m.Add(sum(zz) <= maxterms) + m.Add(sum(zz[j] * cand[j][5] for j in range(len(cand))) <= bud) + m.Minimize(sum(zz[j] * (cand[j][5] + 50) for j in range(len(cand)))) + info['shape'] = ok; res = [] + if ok: + for it in range(K): + s = cp_model.CpSolver(); s.parameters.max_time_in_seconds = tl; s.parameters.num_workers = 2 + st = s.Solve(m) + if st not in (cp_model.OPTIMAL, cp_model.FEASIBLE): info['stop'] = s.StatusName(st); break + supp = [j for j in range(len(cand)) if s.Value(zz[j])] + terms = [(cand[j][0], cand[j][1], cand[j][2], Q(s.Value(lam[j]), 2)) for j in supp] + m.Add(sum(zz[j] for j in supp) <= len(supp) - 1) + ed = dict(delete=list(dels), insert=[], tail=[], prepend=[]) + for x, y, p, l in terms: + gte = ['add', x, y, l.numerator, l.denominator, []] + if p < NB: ed['insert'].append((p, gte)) + else: ed['tail'].append((p - NB, gte)) + ed['insert'].sort(key=lambda t: t[0]); ed['tail'].sort(key=lambda t: t[0]) + rr = P.price(ed, timeout=240) + two = any(resid(W[cand[j][6]][3], v, k0)[0] != () for j in supp) + res.append(dict(terms=[(x, y, p, str(l)) for x, y, p, l in terms], two_value=two, est=sum(cand[j][5] for j in supp), status=rr.get('status'), delta=rr.get('delta'), profile=rr.get('profile'), ed=ed)) + if verbose: print(it, res[-1]['terms'], two, rr.get('status'), rr.get('delta'), flush=True) + info['res'] = res + if out: json.dump(info, open(out, 'w'), indent=1, default=str) + return info + +if __name__ == '__main__': + import harness as H + P = H.Pricer(sys.argv[1]); jobs = json.loads(sys.argv[2]) + for dels in jobs: run(P, dels, int(sys.argv[3]), int(sys.argv[4]), float(sys.argv[5]), slack=int(sys.argv[6]) if len(sys.argv) > 6 else 0) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/groups.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/groups.py new file mode 100644 index 0000000000..05178543a0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/groups.py @@ -0,0 +1,27 @@ +"""Group B reads by normalized source value at read time; report groups of gain-carrying reads (PR1957 relaxation).""" +import gzip,json,sys,collections +from fractions import Fraction as Q +w=json.load(gzip.open(sys.argv[1])); rel=json.load(open(sys.argv[2])) +coef=lambda g: Q(g[3],g[4]) if g[0]=='add' else Q(g[2],g[3]) +B=w['B']; tails=[w['cleanup'][i] for i,U in w['early_cleanup']]; seq=B+tails +rows={}; get=lambda x: rows.get(x,{x:Q(1)}) +def norm(r): + k0=min(r);a=r[k0];return tuple(sorted((k,v/a) for k,v in r.items())) +G=collections.defaultdict(list) +for i,g in enumerate(seq): + if g[0]=='add': + if i=2: out.append(dict(key=str(key)[:60],reads=L,gain=gl,deltas=[rel[str(i)]['delta'] for i in gl],src=sorted({B[i][2] for i in gl}))) +out.sort(key=lambda r:-len(r['gain'])) +print(len(out),'groups with >=2 gain reads; sizes',collections.Counter(len(r['gain']) for r in out)) +for r in out[:60]: print(r['src'],r['gain'],r['deltas']) +json.dump(out,open(sys.argv[3],'w')) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/price_all.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/price_all.py new file mode 100644 index 0000000000..b7d6d51346 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/price_all.py @@ -0,0 +1,25 @@ +"""Price every census candidate exactly with the #1186 exact frame toolkit (scalar exactness mod p, cone caps, exact frames). +usage: price_all.py WORD CANDS.json OUT.json [workers]""" +import sys,json,time +sys.path.insert(0,'/Users/joelpulikkan/imb-harness/r7/tk') +from multiprocessing import Pool +P=None +def init(w): + global P + import harness as H; P=H.Pricer(w) +def job(a): + k,c=a + r=P.price(c['ed'],timeout=int(__import__('os').environ.get('TMO','120'))); r.pop('word',None) + return k,r +if __name__=='__main__': + w,cp,op=sys.argv[1:4]; nw=int(sys.argv[4]) if len(sys.argv)>4 else 10 + C=json.load(open(cp)); t=time.time(); import os; out=json.load(open(op)) if os.path.exists(op) else {}; out={int(k):v for k,v in out.items()} + with Pool(nw,initializer=init,initargs=(w,),maxtasksperchild=200) as pool: + for n,(k,r) in enumerate(pool.imap_unordered(job,[(k,c) for k,c in enumerate(C) if k not in out],chunksize=1)): + out[k]=r + if r.get('status')=='ok' and r['delta']<0: print('GAIN',k,C[k]['pivot'],C[k]['kind'],r['delta'],r['profile'],flush=True) + if n%200==0: print('progress',n,len(C),round(time.time()-t),flush=True); json.dump(out,open(op,'w')) + json.dump(out,open(op,'w')) + import collections + print('status',collections.Counter(r['status'] for r in out.values())) + print('ok deltas',collections.Counter(r['delta'] for r in out.values() if r['status']=='ok').most_common(20)) diff --git a/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/retime.py b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/retime.py new file mode 100644 index 0000000000..f9018d10b4 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/r9-supplier-census/scripts/retime.py @@ -0,0 +1,20 @@ +"""Commuting-window retiming census: move one B add gate g=(t += c*s) to the far end (earliest / latest) of its maximal +window of commuting neighbours (h commutes with g iff h neither reads t nor writes s; scales on t or s block). +Exact by construction (checked again mod p by the pricer). usage: retime.py WORD OUT.json""" +import gzip,json,sys +w=json.load(gzip.open(sys.argv[1]));B=w['B'];out=[] +def comm(g,h): + if g[0]!='add': return False + t,s=g[1],g[2] + if h[0]=='scale': return h[1] not in (t,s) + return h[2]!=t and h[1]!=s +for i,g in enumerate(B): + if g[0]!='add': continue + j=i + while j+1=0 and comm(g,B[k-1]): k-=1 + gate=list(g[:5])+[[]] + if j>i: out.append(dict(pivot=g[2],kind='late',gate=i,to=j,ed=dict(delete=[i],insert=[[j+1,gate]],tail=[],prepend=[]))) + if k 4), but v falls by the same factor: 120 real lines become 60 Q(i)-lines. +- **What decides the comparison.** It comes down to (w_K, x_K = Σ rank_K / v), with the F2 decoder pattern taken from K-orthogonality. +- **Q(i) is free on the bit side.** No ring rule (Lemma λ) blocks Q(i) there: the payload is F2 and the frames are Q-linear. + +## Numbers (exact) +- **Gaussian E8** (J = i on coordinate pairs; preserves E8; checked) + - 60 lines, w = 4. + - Each line is non-orthogonal to 44 others. + - α (largest pairwise non-orthogonal set) = 12. + - F2 minrank (asymmetric allowed, SAT with clique gauge) = 15: c = 13 and 14 are UNSAT, c = 15 is SAT. 15 = v/w, the frame-partition decoder. + - Hyperplane flats: 60 of size 15, 480 of size 6, 960 of size 4. +- **Eisenstein E8** (fixed-point-free order-3 element from 4 orthogonal A2's; Witting configuration) + - 40 lines, w = 4, non-orthogonal degree 27, α = 7. + - F2 minrank = 10: c = 7, 8, 9 are UNSAT, c = 10 is SAT. 10 = v/w. +- **Hurwitz E8** (left multiplication by i, j) + - 10 lines in H^2, w = 2, each line orthogonal to exactly one other, α = 5, minrank 5 = v/w. +- **Copy cost.** Symmetric decoders, exact CP-SAT with cost = dim span A_j, searched 900 s (not proven optimal): + - Gaussian (c ≤ 20): best found 60, so x_K = 1.0. Proven lower bound only 9 (x_K ≥ 0.15). + - Eisenstein (c ≤ 16): best found 40, so x_K = 1.0. Lower bound 9 (x_K ≥ 0.225). + - Hurwitz: + - Symmetric optimum is 10 (x_K = 1.0, proven OPTIMAL). + - The asymmetric D_j-relaxation gives only 2, and that relaxation is too weak. + - The exact SAT run with the Theorem-C cost (costs 8/9/10) did not finish, so the Hurwitz asymmetric case is **open**. +- **Under the README's own E8 premise** (cst = minrank × full in-cap clique rank, the same premise as E8's 72): Gaussian, Eisenstein and Hurwitz all have x_K = minrank·w/v = **1.0**. Then D/k = (s−1)v − s·v < 0 at every s, so there is **no saving at all**. +- **Threshold to beat bit E8** (X* = 6.0005e-3, float screen fx.py): x_K < ~0.71 at w = 4, < ~0.86 at w = 2, < ~0.64 at w = 6. +- **Structural finding.** The cross-field gain of E8 (minrank 9 < v/w = 15) disappears over Q(i), Q(ω) and H: in every extension view minrank = v/w exactly. +- **G34 / Coxeter–Todd K12.** Built exactly as {x ∈ Z[ω]^6 : x_i ≡ x_j mod θ, Σ x_i ≡ 0 mod 3}: 756 minimal vectors, 126 lines. + - w = 6, non-orthogonal degree 80, α = 15. + - Even at minrank = α, x_K ≥ 90/126 = 0.714 > 0.64, so it loses under the premise. The minrank itself was not run. +- **Complex side** (argued). Rules out F4/Eisenstein F2-labels and Gaussian F2-labels: + - **F4 / Eisenstein labels.** A Hermitian form over F4 has an alternating F2 trace form (Tr h(x,x) = 0), so there are no odd-norm residuals. This breaks the phase calculus (orthonormal kernel basis; Lemma λ consequence). It is the first broken rule. + - **Gaussian labels.** Z[i]/(1+i) = F2, so there is no extension. + - **A Q(i)-valued decoder** (Hermitian Gram, 4 centres) forces F2 window = the Gaussian-pattern minrank = 15 on 60 ports, or ≥ 24 on the 120 real ports (clique of r, ir pairs). Not evaluated further. +- **Side finding (b), checked with the float screen only.** The upstream Alon triple family (v = C(h,3), cst = h² bit / h(h+1) complex) has a higher **ceiling** than E8: + - X* = 7.39e-3 (bit, h = 24) and 7.16e-3 (complex, h = 24), at s = 3. + - Generalised Alon sets (7-subsets, λ = 3) would push the ceiling higher still. + - So X* alone does not rank unit families; helper cost decides (per-target neighbour count z = 3·C(h−3,2) = 630 at h = 24, against 63 for E8). + - This was the pre-E8 family (h = 23/25/28 audits in docs/research). + - Task (c), the helper-floor estimate, was not done. + +## Reproduce +- explore4.py: v, degree, α for the three views. +- minrank_any.py {Gauss|Eisen|Hurwitz|K12} [cmin]: minrank SAT. +- costexact.py {Gauss|Eisen} cmax 1 secs: exact cost CP-SAT (needs flats_*.pkl, built by costlb.py). +- k12.py: G34. +- fx.py: float X screen (certify with skel.X before publishing). + +## Open and next steps +1. A real lower bound Σ_j dim span A_j ≥ 0.71·v (Gaussian) for decoders with more than minrank totals. CP-SAT bounds stay at 9; a structural argument is needed. Same open status as E8's 72. +2. Finish the Hurwitz asymmetric SAT (hurwitz_exact.py, cost ≤ 8/9/10). +3. Certify the triple-family X* with skel.X, and estimate helpers via d1 cover numbers. The triple cover number is 2 by the argued two-hyperplane cover. diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/algebra_lines.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/algebra_lines.py new file mode 100644 index 0000000000..6ecaf65779 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/algebra_lines.py @@ -0,0 +1,77 @@ +"""E8 roots grouped into lines over an algebra A acting on R^8 (A spanned over Q by integer matrices preserving E8). +line = A-span of a root (a real subspace of dim = dim A); two lines are A-orthogonal iff = 0 for all basis a.""" +import sys, itertools, numpy as np +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from e8units import standard_e8, rank_q +import networkx as nx + +def e8_roots(): + L,G=standard_e8(); return np.vstack([L,-L]) # doubled coords, norm 8 + +def left_quat(): + """left multiplication by i, j, k on each H-block of R^8 = H^2 (coords 1,i,j,k).""" + Li=np.array([[0,-1,0,0],[1,0,0,0],[0,0,0,-1],[0,0,1,0]]) + Lj=np.array([[0,0,-1,0],[0,0,0,1],[1,0,0,0],[0,-1,0,0]]) + Lk=Li@Lj + blk=lambda M: np.kron(np.eye(2,dtype=np.int64),M).astype(np.int64) + return blk(Li),blk(Lj),blk(Lk) + +def a2_rotation(roots): + """a fixed-point-free order-3 element of W(E8): product of the Coxeter elements of 4 orthogonal A2's.""" + n8=lambda r: r@r + def refl(a): + return np.eye(8,dtype=np.int64)*8 - 2*np.outer(a,a) # 8*s_a (norm 8): s_a = I - 2aa^T/8 + R=[tuple(r) for r in roots] + chosen=[] + def ok(a,b): + return all(np.dot(a,x)==0 and np.dot(b,x)==0 for p in chosen for x in p) + for a in roots: + if len(chosen)==4: break + if any(np.dot(a,x)!=0 for p in chosen for x in p): continue + for b in roots: + if np.dot(a,b)==-4 and ok(a,b): + chosen.append((a,b)); break + assert len(chosen)==4 + g=np.eye(8,dtype=object) + for a,b in chosen: + sa=np.eye(8,dtype=object)*8-2*np.outer(a,a).astype(object); sb=np.eye(8,dtype=object)*8-2*np.outer(b,b).astype(object) + g=g.dot(sa).dot(sb) + D=8**8//4 + assert all(x%D==0 for x in g.flatten()); g=(g//D).astype(np.int64) # g = 4 * (order-3 isometry) + I=np.eye(8,dtype=np.int64) + assert (g@g@g==64*I).all() and (g@g+4*g+16*I==0).all() + S=set(map(tuple,roots)); assert all(x%4==0 for r in roots for x in g@r) and all(tuple(g@r//4) in S for r in roots) + return g + +def lines_over(roots, basis): + """basis: list of integer matrices (incl. identity) spanning the algebra. Returns representative roots, the + real span dimension d = len(basis), and the A-nonorthogonality matrix.""" + seen={}; reps=[] + d=len(basis) + for r in roots: + key=None + span=[list(B@r) for B in basis] + assert rank_q(span)==d + # canonical key: set of roots in the span + found=False + for i,rep in enumerate(reps): + if rank_q([list(B@rep) for B in basis]+[list(r)])==d: + found=True; break + if not found: reps.append(r) + reps=np.array(reps) + v=len(reps) + NO=np.zeros((v,v),bool) + for B in basis: + NO|=(reps@B@reps.T)!=0 + np.fill_diagonal(NO,False) + return reps,NO + +def clique_number(NO): + v=len(NO); g=nx.Graph(); g.add_nodes_from(range(v)) + g.add_edges_from((i,j) for i in range(v) for j in range(i+1,v) if NO[i,j]) + K=max(nx.find_cliques(g),key=len); return len(K),sorted(K) + +def arank(reps,basis,idx): + rows=[list(B@reps[i]) for i in idx for B in basis] + r=rank_q(rows) if rows else 0 + assert r%len(basis)==0; return r//len(basis) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_e16.log b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_e16.log new file mode 100644 index 0000000000..636042ceec --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_e16.log @@ -0,0 +1,3 @@ +Eisen v 40 w 4 max flat sizes {1: 1, 2: 4, 3: 12} +Eisen cmax 16 sym 1 FEASIBLE obj 40.0 bound 9.0 x_K lb 0.225 900.7 +totals (rank of D_j, |A_j|): [(3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (2, 2), (2, 2), (2, 2), (2, 2), (2, 2), (2, 2), (2, 2), (2, 2)] diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_g20.log b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_g20.log new file mode 100644 index 0000000000..acf23a8502 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/ce_g20.log @@ -0,0 +1,3 @@ +Gauss v 60 w 4 max flat sizes {1: 1, 2: 6, 3: 15} +Gauss cmax 20 sym 1 FEASIBLE obj 60.0 bound 9.0 x_K lb 0.15 906.0 +totals (rank of D_j, |A_j|): [(3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3), (3, 3)] diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costexact.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costexact.py new file mode 100644 index 0000000000..7de27f4ca6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costexact.py @@ -0,0 +1,73 @@ +"""CP-SAT: minimise a valid lower bound of sum_j beta_j / d over decoders B = W U^T on the A-orthogonality pattern. +cost_j >= f(D_j), D_j = A_j cap R_j: f = 0 (empty), 1 (one line), 2 (2..L2 lines), 3 (in a hyperplane flat), w otherwise, +using the exact flat census. usage: costlb.py """ +import sys, time, pickle; sys.path.insert(0,sys.path[0]) +from algebra_lines import * +from ortools.sat.python import cp_model +name, cmax, sym, secs = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), float(sys.argv[4]) +roots=e8_roots(); I8=np.eye(8,dtype=np.int64); Li,Lj,Lk=left_quat() +basis={'Gauss':[I8,Li],'Eisen':[I8,a2_rotation(roots)],'Hurwitz':[I8,Li,Lj,Lk]}[name] +reps,NO=lines_over(roots,basis); v=len(reps); w=arank(reps,basis,range(v)) +# flats: hyperplanes = closures of rank w-1; rank-2 flats; computed exactly +def closure(idx): + r=arank(reps,basis,idx); return frozenset(j for j in range(v) if j in idx or arank(reps,basis,list(idx)+[j])==r), r +cache='flats_%s.pkl'%name +try: fl=pickle.load(open(cache,'rb')) +except Exception: + fl={1:{frozenset([i]) for i in range(v)}} + for r in range(2,w): + fl[r]=set() + for F in fl[r-1]: + bas=[] + for i in sorted(F): + if arank(reps,basis,bas+[i])>len(bas): bas.append(i) + if len(bas)==r-1: break + for j in range(v): + if j not in F: fl[r].add(closure(bas+[j])[0]) + pickle.dump(fl,open(cache,'wb')) +L={r:max(len(F) for F in fl[r]) for r in fl} +print(name,'v',v,'w',w,'max flat sizes',L,flush=True) +m=cp_model.CpModel() +u=[[m.NewBoolVar('') for j in range(cmax)] for T in range(v)] +W=u if sym else [[m.NewBoolVar('') for j in range(cmax)] for T in range(v)] +def AND(a,b): + if a is b: return a + z=m.NewBoolVar(''); m.AddBoolAnd([a,b]).OnlyEnforceIf(z); m.AddBoolOr([a.Not(),b.Not()]).OnlyEnforceIf(z.Not()); return z +d=[[AND(W[T][j],u[T][j]) for j in range(cmax)] for T in range(v)] +for T in range(v): m.AddBoolXOr(d[T]) # B_TT = 1 +for T in range(v): + for S in range(v): + if NO[T,S] and (not sym or T= sum_{l} lev_{j,l}; lev_1 = nonempty, lev_2 = >=2 lines, lev_r (3..w-1) = not inside a rank-(r-1) flat +obj=[] +# precompute: for levels r>=3 use flats of rank r-1 with more than L[r-2] lines (smaller ones are covered by count) +for j in range(cmax): + n=sum(d[T][j] for T in range(v)) + l1=m.NewBoolVar(''); m.Add(n>=1).OnlyEnforceIf(l1); m.Add(n==0).OnlyEnforceIf(l1.Not()); obj.append(l1) + if w>=2: + l2=m.NewBoolVar(''); m.Add(n>=2).OnlyEnforceIf(l2); m.Add(n<=1).OnlyEnforceIf(l2.Not()); obj.append(l2) + prevL=1 + for r in range(3,w+1): + # lev_r = 1 unless D_j inside some rank-(r-1) flat; sets of size <= L[r-2] are allowed free at this level (relaxation) + lr=m.NewBoolVar(''); obj.append(lr) + small=m.NewConstant(0) + big=list(fl[r-1]) + sel=[m.NewBoolVar('') for F in big] + for s_,F in zip(sel,big): + for T in range(v): + if T not in F: m.AddImplication(s_, d[T][j].Not()) + m.AddBoolOr(sel+[small,lr]) +obj_expr=sum(obj) +# symmetry: totals ordered by count +cnt=[sum(d[T][j] for T in range(v)) for j in range(cmax)] +for j in range(cmax-1): m.Add(cnt[j]>=cnt[j+1]) +m.Minimize(obj_expr) +sv=cp_model.CpSolver(); sv.parameters.max_time_in_seconds=secs; sv.parameters.num_workers=8 +t=time.time(); st=sv.Solve(m) +print(name,'cmax',cmax,'sym',sym,sv.StatusName(st),'obj',sv.ObjectiveValue() if st in (cp_model.OPTIMAL,cp_model.FEASIBLE) else None,'bound',sv.BestObjectiveBound(),'x_K lb',sv.BestObjectiveBound()/v,round(time.time()-t,1),flush=True) +if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + U=np.array([[sv.Value(u[T][j]) for j in range(cmax)] for T in range(v)]); Wv=np.array([[sv.Value(W[T][j]) for j in range(cmax)] for T in range(v)]) + tot=[ (arank(reps,basis,[T for T in range(v) if Wv[T,j] and U[T,j]]), int(U[:,j].sum())) for j in range(cmax)] + print('totals (rank of D_j, |A_j|):',tot) + np.savez('costlb_%s_%d_%d.npz'%(name,cmax,sym),U=U,W=Wv) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costlb.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costlb.py new file mode 100644 index 0000000000..da68b13dad --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/costlb.py @@ -0,0 +1,73 @@ +"""CP-SAT: minimise a valid lower bound of sum_j beta_j / d over decoders B = W U^T on the A-orthogonality pattern. +cost_j >= f(D_j), D_j = A_j cap R_j: f = 0 (empty), 1 (one line), 2 (2..L2 lines), 3 (in a hyperplane flat), w otherwise, +using the exact flat census. usage: costlb.py """ +import sys, time, pickle; sys.path.insert(0,sys.path[0]) +from algebra_lines import * +from ortools.sat.python import cp_model +name, cmax, sym, secs = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), float(sys.argv[4]) +roots=e8_roots(); I8=np.eye(8,dtype=np.int64); Li,Lj,Lk=left_quat() +basis={'Gauss':[I8,Li],'Eisen':[I8,a2_rotation(roots)],'Hurwitz':[I8,Li,Lj,Lk]}[name] +reps,NO=lines_over(roots,basis); v=len(reps); w=arank(reps,basis,range(v)) +# flats: hyperplanes = closures of rank w-1; rank-2 flats; computed exactly +def closure(idx): + r=arank(reps,basis,idx); return frozenset(j for j in range(v) if j in idx or arank(reps,basis,list(idx)+[j])==r), r +cache='flats_%s.pkl'%name +try: fl=pickle.load(open(cache,'rb')) +except Exception: + fl={1:{frozenset([i]) for i in range(v)}} + for r in range(2,w): + fl[r]=set() + for F in fl[r-1]: + bas=[] + for i in sorted(F): + if arank(reps,basis,bas+[i])>len(bas): bas.append(i) + if len(bas)==r-1: break + for j in range(v): + if j not in F: fl[r].add(closure(bas+[j])[0]) + pickle.dump(fl,open(cache,'wb')) +L={r:max(len(F) for F in fl[r]) for r in fl} +print(name,'v',v,'w',w,'max flat sizes',L,flush=True) +m=cp_model.CpModel() +u=[[m.NewBoolVar('') for j in range(cmax)] for T in range(v)] +W=u if sym else [[m.NewBoolVar('') for j in range(cmax)] for T in range(v)] +def AND(a,b): + if a is b: return a + z=m.NewBoolVar(''); m.AddBoolAnd([a,b]).OnlyEnforceIf(z); m.AddBoolOr([a.Not(),b.Not()]).OnlyEnforceIf(z.Not()); return z +d=[[AND(W[T][j],u[T][j]) for j in range(cmax)] for T in range(v)] +for T in range(v): m.AddBoolXOr(d[T]) # B_TT = 1 +for T in range(v): + for S in range(v): + if NO[T,S] and (not sym or T= sum_{l} lev_{j,l}; lev_1 = nonempty, lev_2 = >=2 lines, lev_r (3..w-1) = not inside a rank-(r-1) flat +obj=[] +# precompute: for levels r>=3 use flats of rank r-1 with more than L[r-2] lines (smaller ones are covered by count) +for j in range(cmax): + n=sum(d[T][j] for T in range(v)) + l1=m.NewBoolVar(''); m.Add(n>=1).OnlyEnforceIf(l1); m.Add(n==0).OnlyEnforceIf(l1.Not()); obj.append(l1) + if w>=2: + l2=m.NewBoolVar(''); m.Add(n>=2).OnlyEnforceIf(l2); m.Add(n<=1).OnlyEnforceIf(l2.Not()); obj.append(l2) + prevL=1 + for r in range(3,w+1): + # lev_r = 1 unless D_j inside some rank-(r-1) flat; sets of size <= L[r-2] are allowed free at this level (relaxation) + lr=m.NewBoolVar(''); obj.append(lr) + small=m.NewBoolVar(''); m.Add(n<=L[r-2]).OnlyEnforceIf(small); m.Add(n>L[r-2]).OnlyEnforceIf(small.Not()) + big=[F for F in fl[r-1] if len(F)>L[r-2]] + sel=[m.NewBoolVar('') for F in big] + for s_,F in zip(sel,big): + for T in range(v): + if T not in F: m.AddImplication(s_, d[T][j].Not()) + m.AddBoolOr(sel+[small,lr]) +obj_expr=sum(obj) +# symmetry: totals ordered by count +cnt=[sum(d[T][j] for T in range(v)) for j in range(cmax)] +for j in range(cmax-1): m.Add(cnt[j]>=cnt[j+1]) +m.Minimize(obj_expr) +sv=cp_model.CpSolver(); sv.parameters.max_time_in_seconds=secs; sv.parameters.num_workers=8 +t=time.time(); st=sv.Solve(m) +print(name,'cmax',cmax,'sym',sym,sv.StatusName(st),'obj',sv.ObjectiveValue() if st in (cp_model.OPTIMAL,cp_model.FEASIBLE) else None,'bound',sv.BestObjectiveBound(),'x_K lb',sv.BestObjectiveBound()/v,round(time.time()-t,1),flush=True) +if st in (cp_model.OPTIMAL,cp_model.FEASIBLE): + U=np.array([[sv.Value(u[T][j]) for j in range(cmax)] for T in range(v)]); Wv=np.array([[sv.Value(W[T][j]) for j in range(cmax)] for T in range(v)]) + tot=[ (arank(reps,basis,[T for T in range(v) if Wv[T,j] and U[T,j]]), int(U[:,j].sum())) for j in range(cmax)] + print('totals (rank of D_j, |A_j|):',tot) + np.savez('costlb_%s_%d_%d.npz'%(name,cmax,sym),U=U,W=Wv) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore1.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore1.py new file mode 100644 index 0000000000..d14a9f1a44 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore1.py @@ -0,0 +1,40 @@ +import sys, itertools, numpy as np +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from e8units import standard_e8 +import networkx as nx +L,G=standard_e8() # doubled coords, norm 8 +roots=np.vstack([L,-L]) +J=np.zeros((8,8),int) +for a in range(4): + J[2*a+1,2*a]=1; J[2*a,2*a+1]=-1 # J e_{2a}=e_{2a+1}, J e_{2a+1}=-e_{2a} +assert (J@J==-np.eye(8)).all() +S=set(map(tuple,roots)) +assert all(tuple(J@r) in S for r in roots) +# complex lines: orbits under +seen=set(); lines=[] +for r in roots: + t=tuple(r) + if t in seen: continue + orb=[r, J@r, -r, -(J@r)] + for o in orb: seen.add(tuple(o)) + lines.append(r) +lines=np.array(lines); v=len(lines); print('complex lines',v) +# Hermitian non-orth: !=0 or !=0 +R=lines@lines.T; I=lines@(J@lines.T) +nonorth=(R!=0)|(I!=0) +np.fill_diagonal(nonorth,False) +print('deg nonorth',set(nonorth.sum(1))) +g=nx.Graph(); g.add_nodes_from(range(v)) +for i in range(v): + for j in range(i+1,v): + if nonorth[i,j]: g.add_edge(i,j) +w=max(len(c) for c in nx.find_cliques(g)) +print('clique number of nonorth graph',w) +from minrank_sat import solve_minrank +K=max(nx.find_cliques(g),key=len) +pairs=[(i,j) for i in range(v) for j in range(i+1,v) if nonorth[i,j]] +for c in range(12,17): + for sym in (True,False): + r=solve_minrank(v,pairs,c,sym,K if len(K)==c else None) + print('c',c,'sym',sym,r,flush=True) + if r[0]=='SAT': break diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore2.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore2.py new file mode 100644 index 0000000000..0d6321fe9e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore2.py @@ -0,0 +1,9 @@ +import sys, numpy as np +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +exec(open(sys.argv[0].replace('explore2','explore1')).read().split('from minrank_sat')[0]) +from e8units import f2_rank +B0=(~nonorth).astype(int); +print('rank of [orth or diag] matrix', f2_rank(B0)) +from collections import Counter +Hv=Counter((int(a),int(b)) for a,b in zip(R.flatten(),I.flatten())) +print(Hv) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore3.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore3.py new file mode 100644 index 0000000000..d74229a58b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore3.py @@ -0,0 +1,9 @@ +import sys,time; sys.path.insert(0,sys.path[0]) +from gauss_geom import * +lines,J=gaussian_e8() +t=time.time() +F=flats(lines,J,3) +from collections import Counter +for r in F: print(r,len(F[r]),Counter(len(f) for f in F[r])) +print(time.time()-t) +import pickle; pickle.dump(F,open('gflats.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore4.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore4.py new file mode 100644 index 0000000000..0ca29b147b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/explore4.py @@ -0,0 +1,11 @@ +import sys; sys.path.insert(0,sys.path[0]) +from algebra_lines import * +roots=e8_roots(); I8=np.eye(8,dtype=np.int64) +Li,Lj,Lk=left_quat() +S=set(map(tuple,roots)) +for nm,M in (('i',Li),('j',Lj)): assert all(tuple(M@r) in S for r in roots), nm +g=a2_rotation(roots) +for name,basis in (('Gauss',[I8,Li]),('Eisen',[I8,g]),('Hurwitz',[I8,Li,Lj,Lk])): + reps,NO=lines_over(roots,basis) + w,K=clique_number(NO) + print(name,'v',len(reps),'deg nonorth',sorted(set(NO.sum(1))),'clique',w, 'rank of clique',arank(reps,basis,K)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/fx.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/fx.py new file mode 100644 index 0000000000..396859bc46 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/fx.py @@ -0,0 +1,25 @@ +"""float screen of skel.X (same profile), for exploration only""" +import sys, math +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from skel import profile +from fractions import Fraction as Fr +def Xf(s,h,k,x): + prof,m,D=profile(s,h,k,Fr(x)) + if D<=0: return None + m=float(m); D=float(D) + F=lambda q: sum(float(z)*float(r)*((m/float(r))**q-1) for r,z in prof.items()) + lo,hi=0.0,0.5 + for _ in range(100): + mid=(lo+hi)/2 + if F(mid)>=D: hi=mid + else: lo=mid + return hi +def Xstar(h,k,x,S=range(3,40,2)): + best=(None,0.0) + for s in S: + r=Xf(s,h,k,x) + if r and r>best[1]: best=(s,r) + return best +if __name__=='__main__': + for h in (3,4,5,6,7,8,9): + print(h,[ (str(x),Xstar(h,1,x)) for x in (Fr(2,5),Fr(1,2),Fr(3,5),Fr(2,3),Fr(3,4),Fr(4,5),Fr(9,10))]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gauss_geom.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gauss_geom.py new file mode 100644 index 0000000000..7caaaeddc5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gauss_geom.py @@ -0,0 +1,53 @@ +"""Gaussian E8: the 60 Q(i)-lines (J-orbits of E8 roots). Exact integer arithmetic. +A Q(i)-line through root r is the real plane ; complex rank of a line set = real rank / 2.""" +import sys, itertools, numpy as np +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from e8units import standard_e8, rank_q + +def gaussian_e8(): + L,G=standard_e8() + roots=np.vstack([L,-L]) + J=np.zeros((8,8),dtype=np.int64) + for a in range(4): + J[2*a+1,2*a]=1; J[2*a,2*a+1]=-1 + assert (J@J==-np.eye(8,dtype=np.int64)).all() + S=set(map(tuple,roots)); assert all(tuple(J@r) in S for r in roots) + seen=set(); lines=[] + for r in roots: + if tuple(r) in seen: continue + for o in (r,J@r,-r,-(J@r)): seen.add(tuple(o)) + lines.append(r) + lines=np.array(lines) + assert len(lines)==60 + return lines,J + +def nonorth(lines,J): + R=lines@lines.T + I=lines@J@lines.T + M=(R!=0)|(I!=0); np.fill_diagonal(M,False); return M + +def crank(lines,J,idx): + rows=[] + for i in idx: rows+= [list(lines[i]), list(J@lines[i])] + r=rank_q(rows) if rows else 0 + assert r%2==0; return r//2 + +def flats(lines,J,maxrank=3): + v=len(lines); out={1:{frozenset([i]) for i in range(v)}} + def closure(idx): + r=crank(lines,J,idx) + return frozenset(j for j in range(v) if j in idx or crank(lines,J,list(idx)+[j])==r), r + for r in range(2,maxrank+1): + out[r]=set() + for Fl in out[r-1]: + base=list(Fl) + # pick an independent basis of size r-1 from Fl + bas=[] + for i in base: + if crank(lines,J,bas+[i])>len(bas): bas.append(i) + if len(bas)==r-1: break + for j in range(v): + if j in Fl: continue + C,rr=closure(bas+[j]); assert rr==r + out[r].add(C) + return out diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin.py new file mode 100644 index 0000000000..160132e868 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin.py @@ -0,0 +1,13 @@ +import sys,time; sys.path.insert(0,sys.path[0]) +from gauss_geom import * +import networkx as nx +from minrank_sat import solve_asym +lines,J=gaussian_e8(); NO=nonorth(lines,J); v=60 +g=nx.Graph([(i,j) for i in range(v) for j in range(i+1,v) if NO[i,j]]) +K=sorted(max(nx.find_cliques(g),key=len)) +pat={(i,j) for i in range(v) for j in range(v) if NO[i,j]} +c=int(sys.argv[1]) +r=solve_asym(v,pat,None,c,'none',K) +print(c,len(K),r['result'],r['sec'],flush=True) +if r['result']=='SAT': + import numpy as np; np.save('gauss_B_c%d.npy'%c,r['B']) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_13.log b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_13.log new file mode 100644 index 0000000000..9850f666db --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_13.log @@ -0,0 +1 @@ +13 12 UNSAT 0.1 diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_14.log b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_14.log new file mode 100644 index 0000000000..430a070f3b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_14.log @@ -0,0 +1 @@ +14 12 UNSAT 3.6 diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_15.log b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_15.log new file mode 100644 index 0000000000..f26cab13c1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/gmin_15.log @@ -0,0 +1 @@ +15 12 SAT 1.8 diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/hurwitz_exact.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/hurwitz_exact.py new file mode 100644 index 0000000000..887fee6982 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/hurwitz_exact.py @@ -0,0 +1,58 @@ +"""Hurwitz view of E8: 10 quaternionic lines in H^2 (w = 2), each orthogonal to exactly one other. +Exact: the least Theorem-C copy floor sum_j rank G[R_j, A_j] (H-rank units) over ALL F_2 decoders B = W U^T on the +pattern (B_TT = 1, B_TS = 0 on non-orthogonal pairs), asymmetric allowed, any number of totals. +For w = 2 the rank of the Gram block between line sets R, A is exactly 0 if R x A has no equal or non-orthogonal pair, +2 if |R| >= 2 and |A| >= 2, and 1 otherwise (checked below from exact ranks). Totals of rank 0 only touch free entries +and are dropped, so a decoder of cost <= C has at most C totals. SAT (CaDiCaL via pysat) with a cardinality bound: +cost <= C is UNSAT for C = 9 and SAT for C = 10.""" +import sys; sys.path.insert(0,sys.path[0]) +from minrank_any import systems +from minrank_sat import CNF +from pysat.card import CardEnc, EncType +from pysat.solvers import Cadical153 +import itertools +S=systems()['Hurwitz']; v=S.v; NO=S.NO +assert S.w==2 and set(NO.sum(1))=={8} +for R in itertools.chain.from_iterable(itertools.combinations(range(v),k) for k in (1,2,3)): + for A in itertools.chain.from_iterable(itertools.combinations(range(v),k) for k in (1,2,3)): + import numpy as np + rk=S.rank(R) and None +def gram_rank(R,A): + # exact rank of the real Gram block between the H-spans, / 4 + from e8units import rank_q + BR=[S.basis[b]@S.V[i] for i in R for b in range(4)]; BA=[S.basis[b]@S.V[i] for i in A for b in range(4)] + M=[[int(x@S.Q@y) for y in BA] for x in BR] + r=rank_q(M); assert r%4==0; return r//4 +def formula(R,A): + if not any(T==S_ or NO[T,S_] for T in R for S_ in A): return 0 + return 2 if len(R)>=2 and len(A)>=2 else 1 +import random +rnd=random.Random(1); checked=0 +for _ in range(400): + R=rnd.sample(range(v),rnd.randint(1,4)); A=rnd.sample(range(v),rnd.randint(1,4)) + assert gram_rank(R,A)==formula(R,A); checked+=1 +print('rank formula checked on',checked,'random pairs') +def feasible(C): + f=CNF(); c=C + U=[[f.new() for j in range(c)] for T in range(v)]; W=[[f.new() for j in range(c)] for T in range(v)] + P={(T,S_,j):f.AND(W[T][j],U[S_][j]) for T in range(v) for S_ in range(v) for j in range(c)} + for T in range(v): f.parity([P[T,T,j] for j in range(c)],1) + for T in range(v): + for S_ in range(v): + if NO[T,S_]: f.parity([P[T,S_,j] for j in range(c)],0) + costlits=[] + for j in range(c): + # r1 >= any relevant pair; r2 >= (|R|>=2 and |A|>=2) + r1=f.new(); r2=f.new() + for T in range(v): + for S_ in range(v): + if T==S_ or NO[T,S_]: f.cl.append([-P[T,S_,j],r1]) + for T1,T2 in itertools.combinations(range(v),2): + for S1,S2 in itertools.combinations(range(v),2): + f.cl.append([-W[T1][j],-W[T2][j],-U[S1][j],-U[S2][j],r2]) + costlits+=[r1,r2] + enc=CardEnc.atmost(lits=costlits,bound=C,top_id=f.n,encoding=EncType.seqcounter) + s=Cadical153(bootstrap_with=f.cl+enc.clauses) + return s.solve() +for C in (8,9,10): + print('cost <=',C,'SAT' if feasible(C) else 'UNSAT',flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/k12.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/k12.py new file mode 100644 index 0000000000..43f22a9e79 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/k12.py @@ -0,0 +1,35 @@ +"""Coxeter-Todd K12 = {x in Z[w]^6 : x_i = x_j mod theta, sum x_i = 0 mod 3}; 756 minimal vectors (norm 6) = 126 lines (G34). +Units of Z[w]: +-w^a. Exact enumeration of the minimal vectors.""" +import sys, itertools; sys.path.insert(0,sys.path[0]) +import numpy as np +from klines import * +UNITS=[(1,0),(0,1),(-1,-1),(-1,0),(0,-1),(1,1)] # 1, w, w^2, -1, -w, -w^2 +def mul(x,y): + a,b=x; c,d=y; return (a*c-b*d, a*d+b*c-b*d) +THETA=(1,2) # theta = w - w^2 = w - (-1-w) = 1 + 2w (norm 3) +def norm(x): a,b=x; return a*a-a*b+b*b +def k12_min(): + out=set() + # type 1: theta*u1 at i, theta*u2 at j with u1 = -u2 mod theta (opposite sign classes) + for i,j in itertools.combinations(range(6),2): + for a in range(6): + for b in range(6): + if (a<3)==(b<3): continue + x=[(0,0)]*6; x[i]=mul(THETA,UNITS[a]); x[j]=mul(THETA,UNITS[b]); out.add(tuple(x)) + # type 2: eps * w^{a_i}, sum w^{a_i} = 0 mod 3 + for eps in (1,-1): + for e in itertools.product(range(3),repeat=6): + n=[e.count(0),e.count(1),e.count(2)] + if not (n[0]%3==n[1]%3==n[2]%3): continue + out.add(tuple(UNITS[k if eps==1 else k+3] for k in e)) + out=sorted(out) + assert len(out)==756 and all(sum(norm(c) for c in x)==6 for x in out) + return out +def k12_system(): + vs=k12_min() + S=eis_system('K12', vs, 6) + reps=dedup_lines(S.V, S.basis) + return System('G34(K12)', reps, S.Q, S.basis) +if __name__=='__main__': + S=k12_system(); print(S.name,'v',S.v,'w',S.w,'nonorth deg',set(S.NO.sum(1))) + a,K=S.alpha(); print('alpha',a,'rank of clique',S.rank(K)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/klines.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/klines.py new file mode 100644 index 0000000000..ef7cb552a0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/klines.py @@ -0,0 +1,51 @@ +"""Generic line systems over an algebra A (imaginary quadratic field or quaternions) in a real model. +A system = (V, Q, basis): V integer vectors (one per line), Q integer symmetric positive-definite Gram (real model), +basis = integer matrices spanning A (identity first), acting on the left. A-lines L = A.v. Two lines are A-orthogonal iff +v^T Q B u = 0 for every B in the basis. A-rank of a set = real rank of {B v} / dim A. Everything exact.""" +import itertools, sys +import numpy as np +sys.path.insert(0,'/Users/joelpulikkan/imb-main/research/format-limits/scripts') +from e8units import rank_q +import networkx as nx + +class System: + def __init__(self, name, V, Q, basis): + self.name=name; self.V=np.array(V,dtype=np.int64); self.Q=np.array(Q,dtype=np.int64); self.basis=[np.array(B,dtype=np.int64) for B in basis] + self.d=len(self.basis); self.v=len(self.V) + NO=np.zeros((self.v,self.v),bool) + for B in self.basis: NO|=(self.V@self.Q@B@self.V.T)!=0 + np.fill_diagonal(NO,False); self.NO=NO + self.w=self.rank(range(self.v)) + def rank(self,idx): + rows=[list(B@self.V[i]) for i in idx for B in self.basis] + if not rows: return 0 + r=rank_q(rows); assert r%self.d==0; return r//self.d + def graph(self): + g=nx.Graph(); g.add_nodes_from(range(self.v)) + g.add_edges_from((i,j) for i in range(self.v) for j in range(i+1,self.v) if self.NO[i,j]); return g + def alpha(self): + """max pairwise A-nonorthogonal set (clique of the nonorth graph); exact (networkx max clique enumeration).""" + K=max(nx.find_cliques(self.graph()),key=len); return len(K),sorted(K) + +def dedup_lines(vecs, basis, Q=None): + """group vectors into A-lines: u ~ v iff rank{B u, B v} = d. Keeps the first vector of each line.""" + reps=[]; keys={} + d=len(basis) + for x in vecs: + x=np.array(x,dtype=np.int64) + orb=[] + # line membership test via exact rank + found=False + for r in reps: + if rank_q([list(B@r) for B in basis]+[list(x)])==d: found=True; break + if not found: reps.append(x) + return reps + +# --------------------------------------------------------------------------------------------------------------- +# Eisenstein integers in real coordinates (a, b) <-> a + b w, w^2 + w + 1 = 0. 2*Re(x conj y) form per coordinate. +EIS_Q=np.array([[2,-1],[-1,2]]); EIS_W=np.array([[0,-1],[1,-1]]) # multiplication by w: (a,b) -> (-b, a-b) +def eis_system(name, vecs_complex, n): + """vecs_complex: list of tuples of (a,b) pairs, length n.""" + V=[[c for ab in x for c in ab] for x in vecs_complex] + Q=np.kron(np.eye(n,dtype=np.int64),EIS_Q); W=np.kron(np.eye(n,dtype=np.int64),EIS_W) + return System(name, V, Q, [np.eye(2*n,dtype=np.int64), W]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/meetratio.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/meetratio.py new file mode 100644 index 0000000000..9eca6d40a6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/meetratio.py @@ -0,0 +1,17 @@ +import sys,pickle; sys.path.insert(0,sys.path[0]) +from algebra_lines import * +name=sys.argv[1] +roots=e8_roots(); I8=np.eye(8,dtype=np.int64); Li,Lj,Lk=left_quat() +basis={'Gauss':[I8,Li],'Eisen':[I8,a2_rotation(roots)]}[name] +reps,NO=lines_over(roots,basis); v=len(reps) +fl=pickle.load(open('flats_%s.pkl'%name,'rb')) +w=4; fl[4]={frozenset(range(v))} +big=[F for F in fl[3] if len(F)==max(len(G) for G in fl[3])] +print('hyperplane flats of max size',len(big),len(big[0])) +best=0 +for r in fl: + for F in fl[r]: + meet=sum(1 for X in big if F & X) + best=max(best,meet/r) + +print('max meet/rank',best) diff --git a/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/minrank_any.py b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/minrank_any.py new file mode 100644 index 0000000000..747319ad0d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s1-extension-rings/minrank_any.py @@ -0,0 +1,28 @@ +"""F2 minrank (asymmetric B = W U^T allowed) of the A-orthogonality decoder pattern, by SAT with the clique gauge.""" +import sys,time; sys.path.insert(0,sys.path[0]) +from klines import * +from minrank_sat import solve_asym +def systems(): + from algebra_lines import e8_roots, left_quat, a2_rotation, lines_over + roots=e8_roots(); I8=np.eye(8,dtype=np.int64); Li,Lj,Lk=left_quat() + out={} + for nm,basis in (('Gauss',[I8,Li]),('Eisen',[I8,a2_rotation(roots)]),('Hurwitz',[I8,Li,Lj,Lk]),('E8',[I8])): + reps,_=lines_over(roots,basis); out[nm]=System(nm,reps,I8,basis) + return out +def minrank(S, cmin=None, cmax=None): + a,K=S.alpha() + pat={(i,j) for i in range(S.v) for j in range(S.v) if S.NO[i,j]} + c=cmin or a; res=[] + while True: + r=solve_asym(S.v,pat,None,c,'none',K); res.append((c,r['result'],r['sec'])) + print(S.name,c,r['result'],r['sec'],flush=True) + if r['result']=='SAT': return c,res,r + c+=1 + if cmax and c>cmax: return None,res,None +if __name__=='__main__': + nm=sys.argv[1] + if nm=='K12': + from k12 import k12_system; S=k12_system() + else: S=systems()[nm] + print(nm,'v',S.v,'w',S.w,'alpha',S.alpha()[0]) + minrank(S, int(sys.argv[2]) if len(sys.argv)>2 else None, int(sys.argv[3]) if len(sys.argv)>3 else None) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/REPORT.md b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/REPORT.md new file mode 100644 index 0000000000..35b8a72eaa --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/REPORT.md @@ -0,0 +1,21 @@ +# S2 report: cheaper decoder on the E8 unit (stopped early, budget) + +Work dir: $SP/s2/work (scripts; python = $SP/venv/bin/python). No package/verify.sh written yet. No kappa claim. + +## Results (own runs, exact unless noted) +1. **P111 (shipped bit puncture 39,52,57,71,76,78,103,109,114): 9-total F2 decoder unique (= B0|live) and c=8 UNSAT** (CaDiCaL, no proof log). `uniq.py

` -> `uniq111.log`. So every 9-total decoder on P111 is a label decoder. +2. **Theorem C floor = star floor on the realised units (label-decoder family, any number of totals):** + - P120 bit: all 511 reader sets R(p) span Q^8 (`t1.py`), so every 9-total decoder costs exactly 72 under pairing rank (= star). + - P111 bit: 9 deficient functionals (the punctured labels); pair costs 6 (44 ordered orthogonal punctured pairs), 7, 8 (`p111c.py`). Exact SAT over all decompositions I_9 = sum p_j q_j^T (n<=10 suffices for cost<=62): **cost <= 62 infeasible** (all cost-62 type combos UNSAT; calibration at 63 SAT) (`run62.py

62|63`, `dec9sat.py`). Theorem C min = 63 = star; no slack. + - P112 complex, frame decoder: ThmC 64 = star 64 (`cxp.py`; 2025 frames enumerated). + - Slack mechanism identified: bit side only when a total's readers lie in a hyperplane (drop to g = , cost 6 for orthogonal punctured pairs); complex side also via degenerate source spans. Realizability caveats: reader frame chains, copy terminal frame g != 0 (restoration), (B2) common cut. + - Random search over other 9-line punctures for ThmC <= 62: 1 tested before stop, none found (`search62.py`). +3. **Deep punctures with label (bit) decoders:** SA over p=9..26 (star, `sa_bit.py`) never below cst 63 (x 63/111 best). Sub-root-system units (`subsys.py`): D8 57/56, E7+A1 50/64, A8 63/36 etc. — all far worse; label decoders degrade with depth. Not an exhaustive result. +4. **Mixed units (bit, non-root Q-label):** hyperplane census by type (`hyper.py`, exact W(E8) orbits: E7 120, D7 1080, E6+A1 3360, A7 8640; 8 types). Adding a non-root port needs >= 14 punctured lines (`addports.py`, sampled normals); D8-axis construction (`mixed_d8.py`) gives x >= 0.67. No gain. +5. **Asymmetric sinks (complex P112):** no free sink relaxation (0 of 28,560 (t,w) pairs, float rank; `asym.py`). Rank-1 shared extra total reproduces C.6 (2 sinks / copy unit, `asym2.py`); per-target two-hyperplane covers abundant (target 75: 263,014 covers, 146 sinks, `covers.py`). Local search for >2 targets per rank-1 total (`asym_ls.py`) stopped without improvement over 2. Pricing: break-even needs (sinks/unit) x (helpers/sink) >= ~20 at 0.14% b; with 2 sinks/unit and <= 4.4 shipped helpers per target (496/112), max ~1.2% b vs 2.82% cost -> net negative (argued, optimistic bound). +6. SAT for 10-total P120 decoders with >=9 E7-hyperplane-cheap totals (`sat_cheap.py 10 1|0`): unresolved after ~40 min, killed. + +## Next steps to resume +- Run `search62.py` over many 9-line (and 10-12 line) punctures; if any ThmC<=62 found, build explicit drop-to- copy tree and check chain/restoration. +- Finish asym rank-1/rank-r group search (vectorise node enumeration) to bound sinks per copy unit. +- Package: README table + verify.sh from the scripts above. diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/addports.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/addports.py new file mode 100644 index 0000000000..35ba9572d7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/addports.py @@ -0,0 +1,37 @@ +"""Mixed units: can a non-root port be added? Bit side: Q-label lam needs an odd F2 label z with =0 for every +live line S with != 0. Enumerate hyperplane flats (rank-7 sets of roots) -> normals lam.""" +import numpy as np, itertools +from geo import * +from collections import Counter +from sympy import Matrix +# hyperplane flats: all subsets of lines spanning a rank-7 space, maximal: generate from 7 independent lines +rng = np.random.default_rng(1) +seen = {} +Lf = L.astype(float) +for trial in range(200000): + idx = rng.choice(NL, 7, replace=False) + if np.linalg.matrix_rank(Lf[idx]) < 7: continue + # normal vector + M = Matrix(L[idx].tolist()); ns = M.nullspace()[0] + lam = np.array([int(x) for x in (ns * np.lcm.reduce([int(t.q) for t in ns]))]) + g = np.gcd.reduce(np.abs(lam)); lam //= g + if lam[np.nonzero(lam)[0][0]] < 0: lam = -lam + key = tuple(lam) + if key in seen: continue + O = [i for i in range(NL) if L[i] @ lam == 0] + seen[key] = O +print('hyperplane normals found', len(seen)) +odd = [z for z in range(1, 512) if bin(z).count('1') % 2 == 1] +types = Counter() +best = {} +for key, O in seen.items(): + lam = np.array(key) + N = [i for i in range(NL) if L[i] @ lam != 0] + # min over odd z of |{S in N : =1}| (lines to puncture); if lam is itself a root line, exclude it + selfi = LIDX.get(tuple(lam * (2 // max(1, int(np.max(np.abs(lam)))))), None) if False else None + rootline = [i for i in N if abs(G[i, i]) and np.linalg.matrix_rank(np.vstack([L[i], lam]).astype(float)) == 1] + Nn = [i for i in N if i not in rootline] + m = min((sum(lab_dot(z, uint[S]) for S in Nn), z) for z in odd) + t = (len(O), len(Nn), bool(rootline), m[0]) + types[t] += 1 +print(sorted(types.items())) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_1.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_1.log new file mode 100644 index 0000000000..5c40386a30 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_1.log @@ -0,0 +1 @@ +start 2 [75, 76] diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_2.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_2.log new file mode 100644 index 0000000000..5c40386a30 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/als_75_0_2.log @@ -0,0 +1 @@ +start 2 [75, 76] diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/an75.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/an75.log new file mode 100644 index 0000000000..e69de29bb2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym.py new file mode 100644 index 0000000000..74a50702a6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym.py @@ -0,0 +1,46 @@ +"""Asymmetric sinks on the complex P112 unit (frame decoder: 8 coordinate totals, cost 64). +A relaxed target t takes a sink w != u_t (w.u_t = 1); its decoder row is n_t = L c + alpha * beta with an extra total +beta_S = on sources Q. Feasible iff (L_t, beta_t) is not in span{(L_S, beta_S) : S in Z(w) \\ {t}}, Z(w) the +live lines with = 1. Cost of the extra total = rank_F2 of the readers' rows ()_{S in Q}.""" +import numpy as np, itertools, sys +from collections import defaultdict +from geo import * +LIVE = list(range(8, 120)) +Lf = L.astype(float) +W_ALL = list(range(1, 512)) +def Zset(w, t): + return [S for S in LIVE if S != t and lab_dot(w, uint[S])] +def free_relax(): + out = [] + for t in LIVE: + for w in W_ALL: + if w == uint[t] or not lab_dot(w, uint[t]): continue + Z = Zset(w, t) + r1 = np.linalg.matrix_rank(Lf[Z]); r2 = np.linalg.matrix_rank(np.vstack([Lf[Z], Lf[t]])) + if r2 > r1: out.append((t, w)) + return out +def feasible_pairs(Q, g): + Qs = set(Q) + beta = {S: float(L[S] @ g) if S in Qs else 0.0 for S in LIVE} + ext = {S: np.append(Lf[S], beta[S]) for S in LIVE} + res = [] + for t in LIVE: + for w in W_ALL: + if w == uint[t] or not lab_dot(w, uint[t]): continue + Z = Zset(w, t) + M = np.array([ext[S] for S in Z]) + r1 = np.linalg.matrix_rank(M); r2 = np.linalg.matrix_rank(np.vstack([M, ext[t]])) + if r2 > r1: + row = tuple(lab_dot(w, uint[S]) for S in Q) + res.append((t, w, row)) + return res +def best_rank1(res): + groups = defaultdict(set) + for t, w, row in res: + if any(row): groups[row].add(t) + if not groups: return 0, None + row, ts = max(groups.items(), key=lambda kv: len(kv[1])) + return len(ts), sorted(ts) +if __name__ == '__main__': + fr = free_relax() + print('free relaxations (no extra total):', len(fr), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym2.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym2.py new file mode 100644 index 0000000000..b09304ba55 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym2.py @@ -0,0 +1,9 @@ +from asym import * +import time +for e, d0 in [(0, 75), (0, 76), (0, 77), (0, 8), (0, 20), (0, 50), (0, 100)]: + g = L[d0] + Q = [S for S in LIVE if L[S] @ L[e] == 0 and L[S] @ g != 0] + t0 = time.time() + res = feasible_pairs(Q, g) + k, ts = best_rank1(res) + print('e', e, 'd0', d0, '|Q|', len(Q), 'feasible (t,w)', len(res), 'targets', len({r[0] for r in res}), 'best rank-1 group', k, ts, round(time.time()-t0,1), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_ls.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_ls.py new file mode 100644 index 0000000000..c41fd19037 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_ls.py @@ -0,0 +1,45 @@ +"""Local search over Q (support of the extra total, beta = on Q) maximising the number of relaxed targets that can +share ONE rank-1 extra total (identical sink rows on Q). Complex P112 unit, frame decoder.""" +import numpy as np, random, sys, time +from collections import defaultdict +from geo import * +LIVE = list(range(8, 120)); LIVEMASK = sum(1 << i for i in LIVE) +Lf = L.astype(float) +TW = [(t, w) for t in LIVE for w in range(1, 512) if w != uint[t] and lab_dot(w, uint[t])] +ZL = {tw: np.array(members(Rmask[tw[1]] & LIVEMASK & ~(1 << tw[0]))) for tw in TW} +PINV = {tw: np.linalg.pinv(Lf[ZL[tw]]) for tw in TW} +def evaluate(Q, g): + Qa = np.zeros(NL); Qa[list(Q)] = 1 + beta = (Lf @ g) * Qa + groups = defaultdict(set) + for (t, w) in TW: + Z = ZL[(t, w)] + h = PINV[(t, w)] @ beta[Z] + if np.abs(Lf[Z] @ h - beta[Z]).max() > 1e-7: continue + if abs(beta[t] - Lf[t] @ h) < 1e-7: continue + row = tuple(sorted(S for S in Q if (Rmask[w] >> S) & 1)) + if not row: continue + groups[row].add(t) + if not groups: return 0, None + row, ts = max(groups.items(), key=lambda kv: len(kv[1])) + return len(ts), (row, sorted(ts)) +if __name__ == '__main__': + d0, e, seed, iters = int(sys.argv[1]), int(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4]) + g = Lf[d0]; rng = random.Random(seed) + Q = set(S for S in LIVE if L[S] @ L[e] == 0 and L[S] @ L[d0] != 0) + cand = [S for S in LIVE if L[S] @ L[d0] != 0] + best = evaluate(Q, g); cur = best[0] + print('start', best[0], best[1][1] if best[1] else None, flush=True) + t0 = time.time() + for it in range(iters): + Q2 = set(Q) + S = rng.choice(cand) + if S in Q2: Q2.remove(S) + else: Q2.add(S) + if not Q2: continue + r = evaluate(Q2, g) + if r[0] >= cur: + Q, cur = Q2, r[0] + if r[0] > best[0]: + best = r; print(it, 'best', r[0], r[1][1], 'Q', sorted(Q), round(time.time() - t0), flush=True) + print('final', best[0], best[1]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_nodes.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_nodes.py new file mode 100644 index 0000000000..ef8a141f19 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/asym_nodes.py @@ -0,0 +1,72 @@ +"""Nodes for rank-1 shared extra totals with beta = on Q (complex P112, frame decoder). +node = (t, w, n1, case, mu) with required-in set I and required-out set O for Q.""" +import pickle, numpy as np, sys +from fractions import Fraction as Fr +from geo import * +H = pickle.load(open('hyper.pkl', 'rb')) +NORMALS = [np.array(n, dtype=np.int64) for sz in H for (n, m) in H[sz]] +NMASK = [m for sz in H for (n, m) in H[sz]] +LIVE = list(range(8, 120)); LIVEMASK = sum(1 << i for i in LIVE) +def nodes_for_g(g, targets=LIVE): + g = np.array(g, dtype=np.int64) + pg = L @ g + out = [] + for t in targets: + for w in range(1, 512): + if w == uint[t] or not lab_dot(w, uint[t]): continue + Z = Rmask[w] & LIVEMASK & ~(1 << t) + Zl = members(Z) + for k, n1 in enumerate(NORMALS): + pn = L @ n1 + inH = [S for S in Zl if pn[S] == 0] + rest = [S for S in Zl if pn[S] != 0] + mus = {Fr(int(pg[S]), int(pn[S])) for S in rest} + if len(mus) != 1: continue + mu = mus.pop() + for case in ('A', 'B'): + # h = mu*n1 (A) or h = g - mu*n1 (B) + if case == 'A': + I = [S for S in rest] + Ob = [S for S in inH if pg[S] != 0] + if any(pg[S] == 0 for S in I): continue + hdot_t = mu * int(pn[t]) + else: + I = [S for S in inH if pg[S] != 0] + Ob = [S for S in rest if pg[S] != 0] + hdot_t = int(pg[t]) - mu * int(pn[t]) + if mu == 0 and case == 'A': continue + out.append(dict(t=t, w=w, k=k, case=case, mu=mu, I=frozenset(I), O=frozenset(Ob), hdot_t=hdot_t, + gt=int(pg[t]), Z=Z)) + return out +def compatible(a, b): + if a['t'] == b['t']: return False + if a['I'] & b['O'] or b['I'] & a['O']: return False + Q = a['I'] | b['I'] + for S in Q: + ra = 1 if (a['Z'] >> S & 1) or S == a['t'] else 0 + rb = 1 if (b['Z'] >> S & 1) or S == b['t'] else 0 + if ra != rb: return False + # diag: beta_t (= g.t if t in Q else 0) != h.t + for x in (a, b): + bt = x['gt'] if x['t'] in Q else 0 + if bt == x['hdot_t']: return False + return True +def node_ok_alone(a): + bt = a['gt'] if a['t'] in a['I'] else 0 + if bt == a['hdot_t']: return False + if not a['I']: return False # no extra total needed would be a free relaxation (none exist) + return True +if __name__ == '__main__': + import networkx as nx, time + d0 = int(sys.argv[1]) + t0 = time.time() + nodes = [a for a in nodes_for_g(L[d0]) if node_ok_alone(a)] + print('g = line', d0, 'nodes', len(nodes), 'targets', len({a['t'] for a in nodes}), round(time.time() - t0, 1), flush=True) + # dedupe by (t, I, O, row-relevant Z, diag) + Gx = nx.Graph(); Gx.add_nodes_from(range(len(nodes))) + for i in range(len(nodes)): + for j in range(i + 1, len(nodes)): + if compatible(nodes[i], nodes[j]): Gx.add_edge(i, j) + print('edges', Gx.number_of_edges(), flush=True) + cl, sz = nx.max_weight_clique(Gx, weight=None) + print('max rank-1 group', sz, sorted({nodes[i]['t'] for i in cl}), [sorted(nodes[i]['I']) for i in cl][:3], flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/binv.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/binv.py new file mode 100644 index 0000000000..17d70d2b14 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/binv.py @@ -0,0 +1,11 @@ +import numpy as np, sympy +from geo import * +def binv(P): + BD = sympy.Matrix(B0[np.ix_(P, P)].tolist()) + if BD.det() % 2 == 0: return None + return np.array(BD.inv_mod(2).tolist(), dtype=int) +if __name__ == '__main__': + P = [39, 52, 57, 71, 76, 78, 103, 109, 114] + Bi = binv(P); print(Bi) + orth = (G[np.ix_(P, P)] == 0).astype(int) + print('column weights', Bi.sum(0)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/bitp.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/bitp.py new file mode 100644 index 0000000000..d7e3394396 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/bitp.py @@ -0,0 +1,35 @@ +"""Bit-side punctured units with label decoders (M = B0 on the live lines): star cost = min-weight basis of F2^9 under +s(q) = dim span L[R(q) \\ P] (matroid greedy, exact); ThmC cost needs reader spans.""" +import numpy as np +from geo import * +RLIST = {p: np.array(members(Rmask[p])) for p in range(1, 512)} +def f2_indep_add(basis, x): + for b in basis: + x = min(x, x ^ b) + return x +def s_all(Pset): + live = np.ones(NL, bool); live[list(Pset)] = False + s = {} + for q in range(1, 512): + idx = RLIST[q][live[RLIST[q]]] + s[q] = rankQ_lines(list(idx)) + return s +def greedy_basis(w): + order = sorted(range(1, 512), key=lambda q: (w[q], q)) + basis, chosen = [], [] + for q in order: + x = q + for b in sorted(basis, reverse=True): + x = min(x, x ^ b) + if x: + basis.append(x); chosen.append(q) + if len(chosen) == 9: break + return chosen, sum(w[q] for q in chosen) +def star_cost(Pset): + s = s_all(Pset) + return greedy_basis(s) +if __name__ == '__main__': + print(star_cost(set())) + P111 = [39, 52, 57, 71, 76, 78, 103, 109, 114] + print(star_cost(set(P111))) + print(star_cost(set(range(8)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cert.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cert.py new file mode 100644 index 0000000000..351fbd107c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cert.py @@ -0,0 +1,133 @@ +"""Certified (rational interval) pressure-root tools. +g_q(r) = r*((m/r)^q - 1). Root of sum_i z_i g_q(r_i) = D (increasing in q). +All transcendental values are enclosed by rational intervals (Fraction endpoints, outward rounding).""" +from fractions import Fraction as Fr +import math + +DEN = 10**45 # outward rounding grid + + +def rd(x): # round down to grid + return Fr(math.floor(x * DEN), DEN) + + +def ru(x): + return Fr(math.ceil(x * DEN), DEN) + + +def ln_iv(x): + """enclosure of ln(x), x = Fraction > 0, via ln x = 2 atanh(z), z=(x-1)/(x+1); reduce by powers of 2.""" + x = Fr(x) + assert x > 0 + if x == 1: + return (Fr(0), Fr(0)) + # reduce: x = 2^e * y, y in [1/sqrt2, sqrt2] (use [2/3, 4/3]-ish via loop) + e = 0 + while x > Fr(4, 3): + x /= 2; e += 1 + while x < Fr(2, 3): + x *= 2; e -= 1 + lo2, hi2 = LN2 + lo, hi = _atanh2(x) + return (rd(lo + e * (lo2 if e >= 0 else hi2)), ru(hi + e * (hi2 if e >= 0 else lo2))) + + +def _atanh2(x): + z = (x - 1) / (x + 1) + az = abs(z) + s = Fr(0); t = z; k = 0 + N = 40 + for k in range(N): + s += t / (2 * k + 1) + t *= z * z + # tail bound: |sum_{k>=N} z^(2k+1)/(2k+1)| <= |z|^(2N+1)/((2N+1)(1-z^2)) + tail = az ** (2 * N + 1) / ((2 * N + 1) * (1 - z * z)) + s = 2 * s; tail = 2 * tail + return (rd(s - tail), ru(s + tail)) + + +def _ln2(): + # ln 2 = 2 atanh(1/3) + z = Fr(1, 3); s = Fr(0); t = z + N = 60 + for k in range(N): + s += t / (2 * k + 1); t *= z * z + tail = z ** (2 * N + 1) / ((2 * N + 1) * (1 - z * z)) + return (rd(2 * s - 2 * tail), ru(2 * s + 2 * tail)) + + +LN2 = _ln2() + + +def expm1_iv(lo, hi): + """enclosure of e^y - 1 for y in [lo,hi], 0 <= lo <= hi <= 1.""" + assert 0 <= lo <= hi <= 2 + N = 30 + + def ser(y): + s = Fr(0); t = Fr(1) + for i in range(1, N + 1): + t = t * y / i; s += t + return s + slo = ser(lo) # all terms >= 0 -> lower bound + shi = ser(hi) + 8 * hi ** (N + 1) / math.factorial(N + 1) # tail <= y^(N+1)/(N+1)! * e^y, e^y<=8 (y<=2) + return (rd(slo), ru(shi)) + + +def g_iv(r, m, q): + """enclosure of r*((m/r)^q - 1), q Fraction >= 0.""" + if r >= m: + return (Fr(0), Fr(0)) if r == m else None + l_lo, l_hi = ln_iv(Fr(m) / r) + lo, hi = expm1_iv(q * l_lo, q * l_hi) + return (r * lo, r * hi) + + +def F_iv(prof, m, q): + """prof: dict rank -> count (Fractions/ints allowed), returns enclosure of sum z g_q(r).""" + lo = Fr(0); hi = Fr(0) + for r, z in prof.items(): + if z == 0: + continue + a, b = g_iv(Fr(r), Fr(m), Fr(q)) + lo += z * a; hi += z * b + return (rd(lo), ru(hi)) + + +def root_bracket(prof, m, D, lo=Fr(0), hi=Fr(1, 10), iters=60): + """returns (qlo, qhi) with F(qlo) < D (certified via upper enclosure) and F(qhi) >= D (via lower enclosure): + the true root lies in [qlo, qhi]. qhi is a certified UPPER bound of the root.""" + D = Fr(D) + assert F_iv(prof, m, hi)[0] >= D, 'hi too small' + for _ in range(iters): + mid = rd((lo + hi) / 2) + flo, fhi = F_iv(prof, m, mid) + if flo >= D: + hi = mid + elif fhi < D: + lo = mid + else: + break + return lo, hi + + +def cost_iv(prof, m): + """enclosure of Cost = sum z r ln(m/r).""" + lo = Fr(0); hi = Fr(0) + for r, z in prof.items(): + if z == 0 or r >= m: + continue + a, b = ln_iv(Fr(m) / r) + lo += z * r * a; hi += z * r * b + return lo, hi + + +if __name__ == '__main__': + # self-test against floats + for x in (Fr(64, 7), Fr(73, 8), Fr(100, 3), Fr(3, 2), Fr(1, 5)): + lo, hi = ln_iv(x) + assert lo <= Fr(math.log(x)) + Fr(1, 10**12) and hi >= Fr(math.log(x)) - Fr(1, 10**12), x + assert hi - lo < Fr(1, 10**40), (x, float(hi - lo)) + print('ln ok', float(LN2[1] - LN2[0])) + lo, hi = expm1_iv(Fr(1, 100), Fr(1, 100)) + print('expm1(0.01)', float(lo), math.expm1(0.01), float(hi - lo)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/covers.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/covers.py new file mode 100644 index 0000000000..e5dce758dd --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/covers.py @@ -0,0 +1,25 @@ +import pickle, numpy as np, sys +from collections import Counter +from geo import * +H = pickle.load(open('hyper.pkl', 'rb')) +HYP = [(sz, n, m) for sz in H for (n, m) in H[sz]] +LIVE = list(range(8, 120)); LIVEMASK = sum(1 << i for i in LIVE) +Lf = L.astype(float) +def covers_for(t): + out = [] + for w in range(1, 512): + if w == uint[t] or not lab_dot(w, uint[t]): continue + Z = Rmask[w] & LIVEMASK & ~(1 << t) + for sz, n, m in HYP: + rest = Z & ~m + ri = members(rest) + if len(ri) == 0: continue + if np.linalg.matrix_rank(Lf[ri]) <= 7: + out.append((w, n, sz, bin(Z & m).count('1'), len(ri))) + return out +if __name__ == '__main__': + t = int(sys.argv[1]) + cv = covers_for(t) + print('target', t, 'covers', len(cv), 'distinct w', len({c[0] for c in cv})) + print(Counter((c[2], c[3], c[4]) for c in cv).most_common(20)) + pickle.dump(cv, open('covers_%d.pkl' % t, 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cxp.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cxp.py new file mode 100644 index 0000000000..cf54991de6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/cxp.py @@ -0,0 +1,35 @@ +"""Complex-side punctured units with frame decoders. For a frame K (8 pairwise orthogonal lines) the totals are +A_a = R_a = live lines non-orthogonal to f_a (plus f_a if live); ThmC cost rank_F2 B0[A,A], star cost dim span u_A.""" +import numpy as np, itertools +from geo import * +from e8units import f2_rank_rows +def f2rank_ints(vs): return f2_rank_rows(list(vs)) +def gram_rank_f2(idx): + # rank over F2 of B0[idx, idx] + rows = [sum(int(B0[i, j]) << k for k, j in enumerate(idx)) for i in idx] + return f2_rank_rows(rows) +# all frames +def frames(): + out = set() + adj0 = [[G[i, j] == 0 and i != j for j in range(NL)] for i in range(NL)] + def rec(cl, cand): + if len(cl) == 8: out.add(tuple(sorted(cl))); return + for c in cand: + if c > cl[-1]: + rec(cl + [c], [d for d in cand if d > c and G[c, d] == 0]) + for i in range(NL): + rec([i], [d for d in range(NL) if d > i and G[i, d] == 0]) + return sorted(out) +FRAMES = frames() +def cost(Pset, K): + th = st = 0 + for f in K: + A = [S for S in range(NL) if S not in Pset and (G[S, f] != 0)] + th += gram_rank_f2(A); st += f2rank_ints(uint[S] for S in A) + return th, st +def best(Pset): + return min((cost(Pset, K), K) for K in FRAMES) +if __name__ == '__main__': + print('frames', len(FRAMES)) + K0 = tuple(range(8)) + print('P120', cost(set(), K0), 'P112', cost(set(K0), K0)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9.py new file mode 100644 index 0000000000..653a13e7eb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9.py @@ -0,0 +1,75 @@ +"""Generalised label decoders on a 9-line puncture P whose labels are a basis D: Binv = B_D^{-1} = sum_j x_j y_j^T over F2 +(p_j = D x_j readers, q_j = D y_j sources... in D-coordinates). Term cost (pairing rank, Theorem C): + 8 generic; 7 if x_j or y_j is a unit vector; 6 if x_j = e_a, y_j = e_b, a != b, y_a orth y_b (lines). +Star cost: 7 if y_j unit else 8. Modes: 'thmc' (free), 'chain' (all x-unit terms share one a), 'star'.""" +import sys, numpy as np, sympy, itertools +from ortools.sat.python import cp_model +from geo import * +def binv(P): + BD = sympy.Matrix(B0[np.ix_(P, P)].tolist()) + if BD.det() % 2 == 0: return None + return np.array(BD.inv_mod(2).tolist(), dtype=int) +def solve(P, n=9, mode='thmc', tl=60): + Bi = binv(P); k = len(P) + orth = [[G[P[a], P[b]] == 0 and a != b for b in range(k)] for a in range(k)] + m = cp_model.CpModel() + X = [[m.NewBoolVar('') for j in range(n)] for r in range(k)] + Y = [[m.NewBoolVar('') for j in range(n)] for r in range(k)] + A = {} + for r in range(k): + for c in range(k): + ands = [] + for j in range(n): + a = m.NewBoolVar(''); m.AddMultiplicationEquality(a, [X[r][j], Y[c][j]]); ands.append(a) + if Bi[r, c]: m.AddBoolXOr(ands) + else: + t = m.NewBoolVar(''); m.AddBoolXOr(ands + [t]) # xor(ands) xor t = 1 -> xor(ands) = not t ; force t = 1 + m.Add(t == 1) + cost = [] + unitx_a = {} + for j in range(n): + sx = sum(X[r][j] for r in range(k)); sy = sum(Y[r][j] for r in range(k)) + used = m.NewBoolVar(''); m.Add(sx >= 1).OnlyEnforceIf(used); m.Add(sx == 0).OnlyEnforceIf(used.Not()) + m.Add(sy >= 1).OnlyEnforceIf(used); m.Add(sy == 0).OnlyEnforceIf(used.Not()) + ux = m.NewBoolVar(''); m.Add(sx == 1).OnlyEnforceIf(ux); m.Add(sx != 1).OnlyEnforceIf(ux.Not()) + uy = m.NewBoolVar(''); m.Add(sy == 1).OnlyEnforceIf(uy); m.Add(sy != 1).OnlyEnforceIf(uy.Not()) + disc = m.NewBoolVar('') + pairs = [] + for a in range(k): + for b in range(k): + if orth[a][b]: + pv = m.NewBoolVar(''); m.AddBoolAnd([X[a][j], Y[b][j], ux, uy]).OnlyEnforceIf(pv) + m.AddBoolOr([X[a][j].Not(), Y[b][j].Not(), ux.Not(), uy.Not()]).OnlyEnforceIf(pv.Not()) + pairs.append(pv) + m.AddMaxEquality(disc, pairs) if pairs else m.Add(disc == 0) + if mode == 'star': + # star cost: 7 if y unit else 8 + c = m.NewIntVar(0, 8, ''); m.Add(c == 8 * used - uy) + else: + one = m.NewBoolVar(''); m.AddMaxEquality(one, [ux, uy]) + c = m.NewIntVar(0, 8, ''); m.Add(c == 8 * used - one - disc) + cost.append(c) + for a in range(k): + v = m.NewBoolVar(''); m.AddBoolAnd([ux, X[a][j]]).OnlyEnforceIf(v) + m.AddBoolOr([ux.Not(), X[a][j].Not()]).OnlyEnforceIf(v.Not()) + unitx_a.setdefault(a, []).append((v, uy)) + if mode == 'chain': + # x-unit terms are read at unless they are also y-unit with cost 7 read at 0 (column terms, star). Require: + # non-star x-unit terms (not y-unit) and discounted terms all share one a. + sel = [m.NewBoolVar('') for a in range(k)] + m.Add(sum(sel) <= 1) + for a in range(k): + for (v, uy) in unitx_a[a]: + # v and not(uy) -> sel[a] ; discount also implies uy and v -> require sel[a] when the term is discounted + m.AddImplication(v, sel[a]).OnlyEnforceIf(uy.Not()) + # discounted terms: force their a selected + # (approximation: any x-unit & y-unit term with cost 6 needs sel; enforce via cost bookkeeping below) + m.Minimize(sum(cost)) + s = cp_model.CpSolver(); s.parameters.max_time_in_seconds = tl; s.parameters.num_workers = 4 + r = s.Solve(m) + return s.StatusName(r), s.ObjectiveValue(), s.BestObjectiveBound() +if __name__ == '__main__': + P = [39, 52, 57, 71, 76, 78, 103, 109, 114] + for mode in ('star', 'thmc'): + for n in (9, 10, 11): + print(mode, n, solve(P, n, mode, tl=60), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9sat.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9sat.py new file mode 100644 index 0000000000..478562cf59 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/dec9sat.py @@ -0,0 +1,80 @@ +"""Exact: on a 9-line puncture P whose labels form a basis D, is there a generalised label decoder +(I_9 = sum_j p_j q_j^T, i.e. Binv = sum_j x_j y_j^T in D-coordinates) with pairing-rank (Theorem C) cost <= budget? +Term types: '6' x=e_a, y=e_b (a != b, y_a orth y_b); '7r' x=e_a, y free; '7c' y=e_b, x free; '8' free. +For cost <= 62 only n=9 (a6 >= 1 + a8) and n=10 (a6 - a8 >= 8) are possible.""" +import sys, itertools, numpy as np, sympy, time +from pysat.solvers import Cadical153 +from geo import * +class F: + def __init__(s): s.n = 0; s.cl = [] + def new(s): s.n += 1; return s.n + def AND(s, a, b): + z = s.new(); s.cl += [[-z, a], [-z, b], [z, -a, -b]]; return z + def XOR(s, a, b): + z = s.new(); s.cl += [[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]; return z + def onehot(s, lits): + s.cl.append(list(lits)) + for i in range(len(lits)): + for j in range(i + 1, len(lits)): s.cl.append([-lits[i], -lits[j]]) +def binv(P): + BD = sympy.Matrix(B0[np.ix_(P, P)].tolist()) + if BD.det() % 2 == 0: return None + return np.array(BD.inv_mod(2).tolist(), dtype=int) +def feasible(Bi, orth, types): + k = len(Bi); f = F(); X = []; Y = [] + for t in types: + if t == '6': + sel = {(a, b): f.new() for a in range(k) for b in range(k) if orth[a][b]} + f.onehot(list(sel.values())) + x = [f.new() for _ in range(k)]; y = [f.new() for _ in range(k)] + for (a, b), s_ in sel.items(): + for r in range(k): + f.cl.append([-s_, x[r]] if r == a else [-s_, -x[r]]) + f.cl.append([-s_, y[r]] if r == b else [-s_, -y[r]]) + elif t in ('7r', '7c'): + u = [f.new() for _ in range(k)]; f.onehot(u) + v = [f.new() for _ in range(k)]; f.cl.append(v) + x, y = (u, v) if t == '7r' else (v, u) + else: + x = [f.new() for _ in range(k)]; y = [f.new() for _ in range(k)] + f.cl.append(x); f.cl.append(y) + X.append(x); Y.append(y) + for r in range(k): + for c in range(k): + ands = [f.AND(X[j][r], Y[j][c]) for j in range(len(types))] + acc = ands[0] + for a in ands[1:]: acc = f.XOR(acc, a) + f.cl.append([acc] if Bi[r, c] else [-acc]) + s = Cadical153(bootstrap_with=f.cl); r = s.solve() + return r +def combos(budget): + out = [] + for n in (9, 10): + for a6 in range(n + 1): + for a8 in range(n + 1 - a6): + a7 = n - a6 - a8 + if 6 * a6 + 7 * a7 + 8 * a8 <= budget: + for a7r in range(a7 + 1): + out.append(['6'] * a6 + ['7r'] * a7r + ['7c'] * (a7 - a7r) + ['8'] * a8) + return out +def min_cost(P, budget=62): + Bi = binv(P) + if Bi is None: return None + k = len(P); orth = [[G[P[a], P[b]] == 0 and a != b for b in range(k)] for a in range(k)] + for ty in combos(budget): + if feasible(Bi, orth, ty): return ty + return 'none<=%d' % budget +if __name__ == '__main__': + P = [int(x) for x in sys.argv[1].split(',')] + t = time.time(); print(min_cost(P), round(time.time() - t, 1)) +def combos_exact(budget): + cost = {'6': 6, '7r': 7, '7c': 7, '8': 8} + return [t for t in combos(budget) if sum(cost[x] for x in t) == budget] +def check_budget(P, budget=62, verbose=True): + Bi = binv(P) + k = len(P); orth = [[G[P[a], P[b]] == 0 and a != b for b in range(k)] for a in range(k)] + for ty in combos_exact(budget): + t = time.time(); r = feasible(Bi, orth, ty) + if verbose: print(''.join(x[0] + (x[1] if len(x) > 1 else '') + ' ' for x in ty), r, round(time.time() - t, 1), flush=True) + if r: return ty + return None diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/e8units.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/e8units.py new file mode 100644 index 0000000000..c4c1053bd1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/e8units.py @@ -0,0 +1,275 @@ +"""Geometry shared by the E8 / paired-cube checks of research/format-limits: exact linear algebra over F_2 and Q, +the 120 root lines of E8, and the 9-bit labels of a complex or bit unit. Everything is generated here; nothing is +read from a file. + +Conventions. A 'unit' on v ports has an integer Gram matrix G (diagonal 2 for E8) and F_2 labels u_T in F_2^c with +U U^T = (J + G) mod 2, i.e. = 1 + G_ST (mod 2). Vectors of F_2^n are Python integers (bit j = coordinate j) +or 0/1 numpy rows; both representations are used below and the function names say which. +Two coordinate systems of the same 120 E8 root lines are provided. standard_e8(): doubled standard coordinates +(roots +-2e_i +-2e_j and the half-integer (+-1)^8 with an even number of minus signs, G = L L^T/4); the totals +'coordinate a' used for the complex decoder are the 78 lines with a nonzero a-th coordinate. hamming_e8(): lines +numbered as in the Hamming-code construction: + lines 0..7 : the coordinate lines 2 e_a, + lines 8..119 : the 112 'code lines', for each of the 14 weight-4 words of the extended Hamming code [8,4,4] + (support i0 integer with bit j = row[j].""" + return sum(int(x) << j for j, x in enumerate(row)) + + +# --------------------------------------------------------------------------------------------------------------- +# exact rank over Q +def _reduce_int(r, pc, rr): + """r - (r[pc]/rr[pc]) rr, scaled to stay integral and divided by the content.""" + f, g = r[pc], rr[pc] + r = [g * x - f * y for x, y in zip(r, rr)] + c = math.gcd(*r) + return [x // c for x in r] if c > 1 else r + + +def rank_q(rows): + """exact rank over Q of a list of integer rows (fraction-free elimination).""" + red = [] # (pivot column, row) + for r in rows: + r = [int(x) for x in r] + for pc, rr in red: + if r[pc] != 0: + r = _reduce_int(r, pc, rr) + pc = next((j for j, x in enumerate(r) if x != 0), None) + if pc is not None: + red.append((pc, r)) + return len(red) + + +def row_basis_idx_q(rows): + """indices of a maximal independent set of rows over Q (greedy, in order).""" + chosen, red = [], [] + for i, r in enumerate(rows): + r = [int(x) for x in r] + for pc, rr in red: + if r[pc] != 0: + r = _reduce_int(r, pc, rr) + pc = next((j for j, x in enumerate(r) if x != 0), None) + if pc is not None: + red.append((pc, r)) + chosen.append(i) + return chosen + + +def gram_rank_q(L, Q, S): + """rank over Q of the Gram block L_S Q L_S^T (L integer v x h, Q integer h x h, S index list), computed on a + row basis B of L_S: rank(L_S Q L_S^T) = rank(B Q B^T) because L_S = C B with C of full column rank.""" + LS = [[int(x) for x in L[s]] for s in S] + idx = row_basis_idx_q(LS) + if not idx: + return 0 + B = [LS[i] for i in idx] + Qm = [[int(x) for x in row] for row in Q] + BQ = [[sum(b[k] * Qm[k][j] for k in range(len(b))) for j in range(len(Qm[0]))] for b in B] + BQBt = [[sum(x * y for x, y in zip(bq, b2)) for b2 in B] for bq in BQ] + return rank_q(BQBt) + + +# --------------------------------------------------------------------------------------------------------------- +# E8 +def hamming_e8(): + """(L, G): L = 120 x 8 integer root lines (norm 4), G = L L^T / 2 (diagonal 2, off-diagonal 0, +-1).""" + gen = np.array([[1, 1, 1, 1, 0, 0, 0, 0], [0, 0, 1, 1, 1, 1, 0, 0], [0, 0, 0, 0, 1, 1, 1, 1], + [0, 1, 0, 1, 0, 1, 0, 1]]) + code = {tuple((np.array(c) @ gen) % 2) for c in itertools.product([0, 1], repeat=4)} + w4 = sorted(c for c in code if sum(c) == 4) + assert len(w4) == 14 + L = [tuple(2 * (np.arange(8) == a).astype(int)) for a in range(8)] + for c in w4: + idx = [i for i in range(8) if c[i]] + for sg in itertools.product([1, -1], repeat=3): + x = np.zeros(8, dtype=int) + x[idx[0]] = 1 + for k in range(3): + x[idx[k + 1]] = sg[k] + L.append(tuple(x)) + L = np.array(L, dtype=np.int64) + G = L @ L.T + assert (G % 2 == 0).all() + G = G // 2 + assert len(L) == 120 and set(np.diag(G)) == {2} and set(G[~np.eye(120, dtype=bool)].tolist()) == {-1, 0, 1} + return L, G + + +def standard_e8(): + """(L, G): the 120 root lines of E8 in doubled standard coordinates (norm 8), G = L L^T / 4. Lines 0..55 are + +-2e_i +-2e_j (i> l & 1: continue + m, rr = closure_mask(basis + [l]) + new.add(m) + flats[r] = new + print('rank', r, len(new), round(time.time() - t, 1), flush=True) + return flats +def row_basis(mem): + b = [] + for i in mem: + if np.linalg.matrix_rank(Lf[b + [i]]) > len(b): b.append(i) + return b +if __name__ == '__main__': + fl = enumerate_flats(int(sys.argv[1]) if len(sys.argv) > 1 else 7) + pickle.dump(fl, open('flats.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/geo.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/geo.py new file mode 100644 index 0000000000..550452ffe9 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/geo.py @@ -0,0 +1,44 @@ +"""E8 geometry for S2: lines, Gram, F2 labels, W(E8) line permutations, exact ranks.""" +import numpy as np, itertools +from functools import lru_cache +from e8units import hamming_e8, e8_labels, f2_rank, rank_q +L, G = hamming_e8() +U9 = e8_labels(G).astype(np.int64) # 120x9, U U^T = (J+G) mod 2 +B0 = ((1 + G) % 2).astype(np.int64) +NL = 120 +uint = [int(''.join(map(str, r)), 2) for r in U9] # label as int (bit 8-j = coord j) +def lab_dot(a, b): return bin(a & b).count('1') & 1 +ALLP = list(range(1, 512)) +# R(p) as python int bitmask over 120 lines +Rmask = {p: sum(1 << T for T in range(NL) if lab_dot(uint[T], p)) for p in range(512)} +def members(mask): + out=[];i=0 + while mask: + if mask & 1: out.append(i) + mask >>= 1; i += 1 + return out +def rankQ_lines(idx): + if not idx: return 0 + return int(np.linalg.matrix_rank(L[idx].astype(float))) +def rankG(R, A): + """exact-enough rank over Q of G[R,A] = L_R L_A^T/2 (entries small ints; rank <= 8; verified exactly elsewhere)""" + if not R or not A: return 0 + return int(np.linalg.matrix_rank((L[R] @ L[A].T).astype(float))) +def rankG_exact(R, A): + if not R or not A: return 0 + return rank_q((L[R] @ L[A].T).tolist()) +# W(E8) generators: reflections in the 120 roots acting on lines +def line_index(): + d = {} + for i, r in enumerate(L): + d[tuple(r)] = i; d[tuple(-r)] = i + return d +LIDX = line_index() +def reflection_perm(r): + r = L[r]; nr = r @ r + perm = [] + for x in L: + y = x - (2 * (x @ r) // nr) * r if (2 * (x @ r)) % nr == 0 else None + perm.append(LIDX[tuple(y)]) + return perm +REFL = [reflection_perm(i) for i in range(NL)] diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/hyper.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/hyper.py new file mode 100644 index 0000000000..8bb6520124 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/hyper.py @@ -0,0 +1,50 @@ +"""All root hyperplanes of E8 by type, as W(E8)-orbits of one normal each (exact integer vectors, hamming coords).""" +import numpy as np, pickle, random +from fractions import Fraction +from math import gcd +from functools import reduce +from geo import * +from sympy import Matrix +def prim(v): + v = [int(x) for x in v]; g = reduce(gcd, [abs(x) for x in v if x] or [1]); v = [x // g for x in v] + i = next(i for i, x in enumerate(v) if x) + return tuple(-x for x in v) if v[i] < 0 else tuple(v) +def refl_vec(x, r): + # s_r(x) = x - 2 (x.r)/(r.r) r ; roots norm 4 in hamming coords + x = np.array(x, dtype=object); r = np.array(r, dtype=object) + num = 2 * int(x @ r); den = int(r @ r) + y = x * den - num * r + return prim(y) +def orth_mask(n): + n = np.array(n, dtype=np.int64) + return sum(1 << i for i in range(NL) if int(L[i] @ n) == 0) +def find_types(seed=1, trials=60000): + rng = random.Random(seed); reps = {} + for _ in range(trials): + idx = rng.sample(range(NL), 7) + M = Matrix(L[idx].tolist()) + if M.rank() < 7: continue + n = prim([x * reduce(lambda a, b: a * b // gcd(a, b), [t.q for t in M.nullspace()[0]]) for x in M.nullspace()[0]]) + sz = bin(orth_mask(n)).count('1') + if sz not in reps: reps[sz] = n + if len(reps) == 8: break + return reps +def orbit(n0): + seen = {n0}; frontier = [n0] + while frontier: + nf = [] + for n in frontier: + for r in range(NL): + m = refl_vec(n, L[r]) + if m not in seen: seen.add(m); nf.append(m) + frontier = nf + return sorted(seen) +if __name__ == '__main__': + reps = find_types() + print({k: v for k, v in sorted(reps.items())}) + H = {} + for sz in sorted(reps, reverse=True): + if sz < 28: continue + orb = orbit(reps[sz]); H[sz] = [(n, orth_mask(n)) for n in orb] + print(sz, len(orb), flush=True) + pickle.dump(H, open('hyper.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/minrank_sat.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/minrank_sat.py new file mode 100644 index 0000000000..99a4a8c833 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/minrank_sat.py @@ -0,0 +1,133 @@ +"""SAT encodings for the F_2 pairing of a decoder pattern. + +A pattern is a set of unordered 'off-cap' pairs {S, T} of ports and the question is whether there are words +u_T (and, in the asymmetric case, v_T) in F_2^c with + = 1 (unit diagonal) + = = 0 for every off-cap pair {S, T} +(symmetric case v = u). Such words exist iff the pairing matrix B = V U^T (B_TS = ) has unit diagonal, +vanishes on the pattern, and has F_2 rank at most c; the smallest c is the F_2 'minrank' of the pattern. + +Symmetry breaking. A set K of pairwise off-cap ports has B[K,K] = I forced, so U_K and V_K are invertible; the action +(U, V) -> (U P, V P^-T) of GL(c, F_2) leaves B fixed and can bring U_K to [I_|K| | 0] and V_K to [I_|K| | 0] +(for |K| = c this is U_K = V_K = I; for |K| < c the last columns of V_K are cleared with the remaining freedom of P). +In the symmetric case O(c, F_2) is transitive on orthonormal bases, so U_K = V_K = [I | 0] as well. + +solve_minrank() is the plain minrank question (gauge only when |K| = c). solve_asym() adds the gauge for any +|K| <= c and the extra constraints 'B is not symmetric' (some free pair has B_TS != B_ST) or 'B differs from B0 = +(J + G) mod 2' (some free pair), and optionally fixes U to given labels. The solver is Cadical 1.5.3 through pysat. +""" +import time + +import numpy as np +from pysat.solvers import Cadical153 + + +class CNF: + def __init__(self): + self.n = 0 + self.cl = [] + + def new(self): + self.n += 1 + return self.n + + def AND(self, a, b): + z = self.new() + self.cl += [[-z, a], [-z, b], [z, -a, -b]] + return z + + def XOR(self, a, b): + z = self.new() + self.cl += [[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]] + return z + + def xorlist(self, lits): + acc = lits[0] + for l in lits[1:]: + acc = self.XOR(acc, l) + return acc + + def parity(self, lits, val): + """add the constraint xor(lits) = val.""" + if not lits: + if val: + self.cl.append([]) + return + acc = self.xorlist(lits) + self.cl.append([acc] if val else [-acc]) + + +def build(v, offpairs, c, sym, clique): + f = CNF() + U = [[f.new() for _ in range(c)] for _ in range(v)] + V = U if sym else [[f.new() for _ in range(c)] for _ in range(v)] + for T in range(v): + f.parity([f.AND(U[T][j], V[T][j]) for j in range(c)], 1) + for (S, T) in offpairs: + f.parity([f.AND(U[S][j], V[T][j]) for j in range(c)], 0) + if not sym: + f.parity([f.AND(U[T][j], V[S][j]) for j in range(c)], 0) + if clique and len(clique) == c: # U_K V_K^T = I => WLOG U_K = V_K = I + for i, T in enumerate(clique): + for j in range(c): + f.cl.append([U[T][j]] if j == i else [-U[T][j]]) + if not sym: + f.cl.append([V[T][j]] if j == i else [-V[T][j]]) + return f + + +def solve_minrank(v, offpairs, c, sym, clique): + """('SAT'|'UNSAT', seconds, variables, clauses) for words in F_2^c on the pattern.""" + f = build(v, offpairs, c, sym, clique) + s = Cadical153(bootstrap_with=f.cl) + t = time.time() + r = s.solve() + return ('SAT' if r else 'UNSAT'), round(time.time() - t, 1), f.n, len(f.cl) + + +def solve_asym(v, pattern, B0, c, mode, K, U_fixed=None): + """Existence of source labels U and sink labels W (v x c over F_2), B = W U^T, with B_TT = 1, B_TS = 0 on the + ordered pairs of `pattern`, gauge U_K = W_K = [I | 0] on the pairwise off-cap set K, and + mode 'none' : no further condition, + mode 'nonsym' : B_TS != B_ST for some free pair (T, S), + mode 'neB0' : B_TS != B0_TS for some free pair. + U_fixed (v x c 0/1 array) additionally fixes U (then K's gauge is dropped). Returns a dict with the result and, + when satisfiable, the pairing B = W U^T and its F_2 rank.""" + f = CNF() + U = [[f.new() for _ in range(c)] for _ in range(v)] + W = [[f.new() for _ in range(c)] for _ in range(v)] + Bvar = {} + + def B(T, S): + if (T, S) not in Bvar: + Bvar[(T, S)] = f.xorlist([f.AND(W[T][j], U[S][j]) for j in range(c)]) + return Bvar[(T, S)] + for T in range(v): + f.cl.append([B(T, T)]) + for (T, S) in pattern: + f.cl.append([-B(T, S)]) + if U_fixed is not None: + for T in range(v): + for j in range(c): + f.cl.append([U[T][j]] if U_fixed[T][j] else [-U[T][j]]) + else: + for i, T in enumerate(K): + for j in range(c): + f.cl.append([U[T][j]] if j == i else [-U[T][j]]) + f.cl.append([W[T][j]] if j == i else [-W[T][j]]) + free = [(T, S) for T in range(v) for S in range(v) if T != S and (T, S) not in pattern] + if mode == 'nonsym': + f.cl.append([f.XOR(B(T, S), B(S, T)) for (T, S) in free if T < S]) + elif mode == 'neB0': + f.cl.append([B(T, S) if B0[T][S] == 0 else -B(T, S) for (T, S) in free]) + s = Cadical153(bootstrap_with=f.cl) + t = time.time() + r = s.solve() + out = dict(c=c, mode=mode, result='SAT' if r else 'UNSAT', sec=round(time.time() - t, 1), vars=f.n, clauses=len(f.cl)) + if r: + m = set(x for x in s.get_model() if x > 0) + Uv = np.array([[1 if U[T][j] in m else 0 for j in range(c)] for T in range(v)], dtype=np.int64) + Wv = np.array([[1 if W[T][j] in m else 0 for j in range(c)] for T in range(v)], dtype=np.int64) + out['B'] = (Wv @ Uv.T) % 2 + out['U'], out['W'] = Uv, Wv + return out diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed.py new file mode 100644 index 0000000000..c29e71d091 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed.py @@ -0,0 +1,41 @@ +"""General bit-unit evaluator: ports with Q-vectors lam (rows), F2 labels u (ints, c bits). Checks the label decoder +pattern (=0 when !=0, =1) and returns the min star cost over label decoders +(min-weight basis of the label space under s(q) = dim span{lam_S : =1}), and window h.""" +import numpy as np +def dot(a, b): return bin(a & b).count('1') & 1 +def check_pattern(lam, u): + Gm = lam @ lam.T + n = len(u) + for i in range(n): + if dot(u[i], u[i]) != 1: return False, ('diag', i) + for j in range(i + 1, n): + if Gm[i, j] != 0 and dot(u[i], u[j]) != 0: return False, ('pair', i, j) + return True, None +def star_cost(lam, u, cbits=9): + lamf = lam.astype(float) + s = {} + for q in range(1, 1 << cbits): + idx = [i for i in range(len(u)) if dot(u[i], q)] + s[q] = int(np.linalg.matrix_rank(lamf[idx])) if idx else 0 + # min weight basis of the span of the labels' dual... decoder: I_c = sum p_j q_j^T requires q's to span F2^c; + # but only the label span matters: restrict to q modulo span(u)^perp -> use greedy on all q, need rank = dim span(u) + span_dim = len(basis_of(u)) + order = sorted(s, key=lambda q: (s[q], q)) + # the decoder needs q_1..q_r whose restrictions to span(u) are a basis of its dual: i.e. the matrix [] + # has F2 rank r = dim span(u). Greedy on the vectors ()_T. + basis, chosen = [], [] + for q in order: + vec = sum(dot(u[i], q) << i for i in range(len(u))) + x = vec + for b in sorted(basis, reverse=True): + x = min(x, x ^ b) + if x: + basis.append(x); chosen.append(q) + if len(chosen) == span_dim: break + return sum(s[q] for q in chosen), chosen, span_dim +def basis_of(vs): + b = [] + for x in vs: + for y in sorted(b, reverse=True): x = min(x, x ^ y) + if x: b.append(x) + return b diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed_d8.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed_d8.py new file mode 100644 index 0000000000..108396ac90 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/mixed_d8.py @@ -0,0 +1,26 @@ +import numpy as np, itertools +from e8units import standard_e8, e8_labels +from mixed import * +L, G = standard_e8() +U = e8_labels(G); u = [int(''.join(map(str, r)), 2) for r in U] +Lq = L.astype(np.int64) +ok, why = check_pattern(Lq, u); print('E8 std ok', ok, star_cost(Lq, u)[0]) +half = list(range(56, 120)) +# punctured set: D8 roots involving e_0 .. ; add unit vectors +def try_unit(add_axes): + P = set(i for i in range(56) if any(L[i][a] != 0 for a in add_axes)) + live = [i for i in range(120) if i not in P] + lam = [Lq[i] for i in live]; lab = [u[i] for i in live] + for a in add_axes: + e = np.zeros(8, dtype=np.int64); e[a] = 2 + N = [k for k, i in enumerate(live) if e @ Lq[i] != 0] + # odd z orthogonal to labels of N + zs = [z for z in range(1, 512) if bin(z).count('1') % 2 and all(not dot(z, lab[k]) for k in N)] + if not zs: return None + lam.append(e); lab.append(zs[0]) + lam = np.array(lam); okp, why = check_pattern(lam, lab) + if not okp: return ('pattern fail', why) + c, ch, sd = star_cost(lam, lab) + return dict(v=len(lab), punct=len(P), cst=c, x=c / len(lab), labdim=sd) +for k in range(1, 9): + print(k, try_unit(list(range(k))), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/p111c.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/p111c.py new file mode 100644 index 0000000000..16e8403efe --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/p111c.py @@ -0,0 +1,18 @@ +import numpy as np +from bitp import * +P = [39, 52, 57, 71, 76, 78, 103, 109, 114] +Ps = set(P) +live = np.ones(NL, bool); live[P] = False +R = {p: [int(i) for i in RLIST[p][live[RLIST[p]]]] for p in range(1, 512)} +D = [p for p in range(1, 512) if rankQ_lines(R[p]) < 8] +print('deficient', D, 'labels of punctured', [uint[y] for y in P]) +print('G among punctured:'); print(G[np.ix_(P, P)]) +# cost table for pairs with a deficient side +from collections import Counter +cnt = Counter() +for d in D: + for q in range(1, 512): + cnt[('p=d', rankG(R[d], R[q]))] += 1 + cnt[('q=d', rankG(R[q], R[d]))] += 1 +print(cnt) +print('d-d costs'); print(np.array([[rankG(R[a], R[b]) for b in D] for a in D])) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/report.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/report.py new file mode 100644 index 0000000000..5593062527 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/report.py @@ -0,0 +1,60 @@ +"""Shared paths and PASS/FAIL reporting for the checks of research/format-limits. + +Every check prints one line per claim: 'PASS ' or 'FAIL ', and finish() exits +with status 1 if any claim failed. Paths: PKG is this package, DATA its data/ directory, REPO the repository root +(the package lives at REPO/research/format-limits; files of main are read from REPO and pinned in DEPENDS.json). +""" +import gzip +import json +import os +import pathlib +import sys + +HERE = pathlib.Path(__file__).resolve().parent +PKG = HERE.parent +DATA = PKG / 'data' +VENDOR = DATA / 'vendor' +REPO = pathlib.Path(os.environ.get('FORMAT_LIMITS_REPO', PKG.parent.parent)).resolve() + +_failed = [] + + +def check(cid, ok, detail=''): + print(('PASS ' if ok else 'FAIL ') + cid + (' ' + str(detail) if detail != '' else ''), flush=True) + if not ok: + _failed.append(cid) + return ok + + +def near(x, expected, places): + """x agrees with a value printed to `places` decimals: |x - expected| <= 0.6 * 10^-places (half a unit of the last + printed digit plus slack, so a float computed with a different library build cannot flip a rounding boundary).""" + return abs(float(x) - float(expected)) <= 0.6 * 10.0 ** (-places) + + +def near_all(xs, expected, places): + xs, expected = list(xs), list(expected) + return len(xs) == len(expected) and all( + (x == e) if isinstance(e, int) and not isinstance(e, bool) and p is None else near(x, e, p) + for x, e, p in zip(xs, expected, places)) + + +def finish(): + if _failed: + print('FAILED CLAIMS: ' + ' '.join(_failed), flush=True) + sys.exit(1) + + +def load_json(path): + path = pathlib.Path(path) + if path.suffix == '.gz': + return json.loads(gzip.decompress(path.read_bytes())) + return json.loads(path.read_text()) + + +def main_file(rel): + """a file of the repository's main branch (pinned by sha256 in DEPENDS.json).""" + p = REPO / rel + if not p.exists(): + sys.exit('missing file of main: %s (run from a checkout of the whole repository)' % rel) + return p diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62.py new file mode 100644 index 0000000000..7971fb2347 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62.py @@ -0,0 +1,4 @@ +import sys +from dec9sat import * +P = [int(x) for x in sys.argv[1].split(',')] +print('result', check_budget(P, int(sys.argv[2]))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62_111.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62_111.log new file mode 100644 index 0000000000..ff3d37a193 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/run62_111.log @@ -0,0 +1,30 @@ +6 7c 7c 7c 7c 7c 7c 7c 7c False 1.0 +6 7r 7c 7c 7c 7c 7c 7c 7c False 0.8 +6 7r 7r 7c 7c 7c 7c 7c 7c False 0.8 +6 7r 7r 7r 7c 7c 7c 7c 7c False 0.6 +6 7r 7r 7r 7r 7c 7c 7c 7c False 0.8 +6 7r 7r 7r 7r 7r 7c 7c 7c False 0.8 +6 7r 7r 7r 7r 7r 7r 7c 7c False 0.5 +6 7r 7r 7r 7r 7r 7r 7r 7c False 0.4 +6 7r 7r 7r 7r 7r 7r 7r 7r False 1.1 +6 6 7c 7c 7c 7c 7c 7c 8 False 1.3 +6 6 7r 7c 7c 7c 7c 7c 8 False 1.0 +6 6 7r 7r 7c 7c 7c 7c 8 False 1.0 +6 6 7r 7r 7r 7c 7c 7c 8 False 0.7 +6 6 7r 7r 7r 7r 7c 7c 8 False 1.1 +6 6 7r 7r 7r 7r 7r 7c 8 False 1.1 +6 6 7r 7r 7r 7r 7r 7r 8 False 1.3 +6 6 6 7c 7c 7c 7c 8 8 False 1.2 +6 6 6 7r 7c 7c 7c 8 8 False 1.2 +6 6 6 7r 7r 7c 7c 8 8 False 1.2 +6 6 6 7r 7r 7r 7c 8 8 False 1.2 +6 6 6 7r 7r 7r 7r 8 8 False 1.2 +6 6 6 6 7c 7c 8 8 8 False 27.0 +6 6 6 6 7r 7c 8 8 8 False 25.0 +6 6 6 6 7r 7r 8 8 8 False 26.0 +6 6 6 6 6 8 8 8 8 False 41.7 +6 6 6 6 6 6 6 6 7c 7c False 0.0 +6 6 6 6 6 6 6 6 7r 7c False 0.0 +6 6 6 6 6 6 6 6 7r 7r False 0.1 +6 6 6 6 6 6 6 6 6 8 False 0.0 +result None diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_1.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_1.log new file mode 100644 index 0000000000..e69de29bb2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_2.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_2.log new file mode 100644 index 0000000000..a459c94c8c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_2.log @@ -0,0 +1 @@ +0 [7, 9, 38, 73, 78, 88, 101, 110, 112] orth pairs 14 no diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_3.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/s62_3.log new file mode 100644 index 0000000000..e69de29bb2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sa_bit.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sa_bit.py new file mode 100644 index 0000000000..d55fb6a034 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sa_bit.py @@ -0,0 +1,27 @@ +import sys, random, math +from fractions import Fraction as Fr +from bitp import * +def ev(Pset): + live = [i for i in range(NL) if i not in Pset] + if rankQ_lines(live) < 8: return None + ch, c = greedy_basis(s_all(Pset)) + return c +def sa(p, iters, seed): + rng = random.Random(seed) + P = set(rng.sample(range(NL), p)) + c = ev(P); best = (Fr(c, NL - p), sorted(P), c) + T = 2.0 + for it in range(iters): + out = rng.choice(sorted(P)); inn = rng.choice([i for i in range(NL) if i not in P]) + Q = set(P); Q.remove(out); Q.add(inn) + cq = ev(Q) + if cq is None: continue + if cq <= c or rng.random() < math.exp(-(cq - c) / T): + P, c = Q, cq + if Fr(c, NL - p) < best[0]: best = (Fr(c, NL - p), sorted(P), c) + T = max(0.05, T * 0.999) + return best +if __name__ == '__main__': + p = int(sys.argv[1]); seed = int(sys.argv[2]); iters = int(sys.argv[3]) + b = sa(p, iters, seed) + print(p, seed, 'cst', b[2], 'v', NL - p, 'x', b[0], float(b[0]), b[1], flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_0.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_0.log new file mode 100644 index 0000000000..a2f2f7e38c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_0.log @@ -0,0 +1 @@ +vars 134531 clauses 581478 diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_1.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_1.log new file mode 100644 index 0000000000..e76400742c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_10_1.log @@ -0,0 +1 @@ +vars 134540 clauses 581494 diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_cheap.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_cheap.py new file mode 100644 index 0000000000..fe40ee28be --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/sat_cheap.py @@ -0,0 +1,58 @@ +"""SAT: is there an F2 decoder of the bit E8 pattern on the live lines with n totals, of which at least n - nfull have +their sources (or readers) inside an E7 hyperplane ell^perp (pairing rank <= 7)?""" +import sys, time, itertools +import numpy as np +from pysat.solvers import Cadical153 +from pysat.card import CardEnc, EncType +from geo import * +class F: + def __init__(s): s.n = 0; s.cl = [] + def new(s): s.n += 1; return s.n + def AND(s, a, b): + z = s.new(); s.cl += [[-z, a], [-z, b], [z, -a, -b]]; return z + def XOR(s, a, b): + z = s.new(); s.cl += [[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]; return z + def xorlist(s, l): + acc = l[0] + for x in l[1:]: acc = s.XOR(acc, x) + return acc +def build(live, n, nfull, hyper='E7', fix0=True): + v = len(live); f = F() + al = [[f.new() for _ in range(v)] for _ in range(n)] + be = [[f.new() for _ in range(v)] for _ in range(n)] + for T in range(v): + for S in range(v): + if T == S or G[live[T], live[S]] != 0: + acc = f.xorlist([f.AND(al[j][T], be[j][S]) for j in range(n)]) + f.cl.append([acc] if T == S else [-acc]) + # hyperplanes: E7 type, normal = root line ell; members = live lines orthogonal to ell + Hs = [[k for k in range(v) if G[live[k], l] == 0] for l in range(NL)] + cheap = [] + for j in range(n): + sels = [] + for h, mem in enumerate(Hs): + memset = set(mem) + for side, vec in (('A', be[j]), ('R', al[j])): + if fix0 and j == 0 and not (side == 'A' and h == 0): continue + x = f.new(); sels.append(x) + for k in range(v): + if k not in memset: f.cl.append([-x, -vec[k]]) + c = f.new(); cheap.append(c) + f.cl.append([-c] + sels) + if fix0: f.cl.append([cheap[0]]) + enc = CardEnc.atleast(lits=cheap, bound=n - nfull, top_id=f.n, encoding=EncType.seqcounter) + f.n = max(f.n, enc.nv); f.cl += enc.clauses + return f, al, be +if __name__ == '__main__': + n, nfull = int(sys.argv[1]), int(sys.argv[2]) + P = [int(x) for x in sys.argv[3].split(',')] if len(sys.argv) > 3 and sys.argv[3] else [] + live = [i for i in range(NL) if i not in P] + f, al, be = build(live, n, nfull) + print('vars', f.n, 'clauses', len(f.cl), flush=True) + s = Cadical153(bootstrap_with=f.cl); t = time.time(); r = s.solve() + print('n', n, 'nfull', nfull, 'live', len(live), 'SAT' if r else 'UNSAT', round(time.time() - t, 1), flush=True) + if r: + m = set(x for x in s.get_model() if x > 0) + for j in range(n): + R = [live[T] for T in range(len(live)) if al[j][T] in m]; A = [live[S] for S in range(len(live)) if be[j][S] in m] + print(j, len(R), len(A), rankG_exact(R, A)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/search62.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/search62.py new file mode 100644 index 0000000000..43649a3e85 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/search62.py @@ -0,0 +1,17 @@ +import sys, random, time +from dec9sat import * +from e8units import f2_rank_rows +seed = int(sys.argv[1]); N = int(sys.argv[2]); rng = random.Random(seed) +found = 0 +for it in range(N): + # biased: grow a set preferring lines orthogonal to many chosen ones + P = [rng.randrange(NL)] + while len(P) < 9: + cand = [l for l in range(NL) if l not in P and f2_rank_rows([uint[x] for x in P + [l]]) == len(P) + 1] + w = [1 + 3 * sum(G[l, x] == 0 for x in P) for l in cand] + P.append(rng.choices(cand, weights=w)[0]) + northo = sum(G[a, b] == 0 for a in P for b in P if a < b) + r = check_budget(P, 62, verbose=False) + print(it, sorted(P), 'orth pairs', northo, 'cost<=62' if r else 'no', r if r else '', flush=True) + if r: found += 1 +print('found', found) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/skel.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/skel.py new file mode 100644 index 0000000000..e3c6c815b6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/skel.py @@ -0,0 +1,85 @@ +"""R-free skeleton bound for the B_K compressed-window format (per port, v=1; v cancels). +Slots (per replica / v): + active : s stages x 2 wires (X,Y banks of the active pair) x one climb of a = h-k, child cap a + copies : s stages x cst/v = x (mass), child cap beta_max <= h (use h; E8: beta = 8 = h exactly) + idle : 2 copies-per-port x idle stretches of the B_K schedule, slot rank r, child cap max(r, h) +D/v = (s-1) k - s x ; m = k + s (h-k). +Variants switch individual scope assumptions off (see NOTES).""" +from fractions import Fraction as Fr +from cert import root_bracket, cost_iv, F_iv +import sys + + +def idle_slots(K, a, split=True): + sl = [] + if K >= 1 and 2 * K - 2 > 0: + sl.append(a * (2 * K - 2)) # pair 0 tail + for p in range(1, K): + if split: + sl += [2 * p * a, a] # pre-stretch split by the bridge + else: + sl += [(2 * p + 1) * a] + if 2 * K - 2 * p - 2 > 0: + sl.append(a * (2 * K - 2 * p - 2)) + return sl + + +def profile(s, h, k, x, variant='thm'): + """returns (prof, m, D) per port. prof: rank -> (fractional) count; mass at cap c entered as count mass/c at rank c.""" + assert s % 2 == 1 and s >= 3 + K = (s - 1) // 2 + a = h - k + m = k + s * a + x = Fr(x) + prof = {} + + def add(cap, mass): + if mass == 0: + return + cap = Fr(cap) + if cap >= m: # child of rank m costs nothing + return + prof[cap] = prof.get(cap, 0) + Fr(mass) / cap + loc_cap = h if variant in ('loc_h', 'idlefree_loch') else a + add(loc_cap, 2 * s * a) # active data climbs + xx = Fr(0) if variant == 'copyfree' else x + add(h, s * xx) # copies + if variant not in ('idlefree', 'idlefree_loch'): + for r in idle_slots(K, a, split=(variant != 'nosplit')): + if variant == 'idle_onechild': + continue + cap = r if variant == 'nohostcap' else max(r, h) + add(cap, 2 * r) + if variant == 'idle_onechild': # all idle of a copy in one child (cross-stretch batching) + for tot in idle_per_pair(K, a): + add(max(tot, h), 2 * tot) + D = (s - 1) * k - s * xx + return prof, m, D + + +def idle_per_pair(K, a): + out = [] + if 2 * K - 2 > 0: + out.append(a * (2 * K - 2)) + for p in range(1, K): + out.append(a * (2 * K - 1)) + return out + + +def X(s, h, k, x, variant='thm'): + prof, m, D = profile(s, h, k, x, variant) + if D <= 0: + return None + lo, hi = root_bracket(prof, m, D, hi=Fr(1, 2)) + clo, chi = cost_iv(prof, m) + return dict(s=s, m=m, D=D, lo=lo, hi=hi, DoverCost=D / clo, cost=clo) + + +if __name__ == '__main__': + h, k, x = int(sys.argv[1]), int(sys.argv[2]), Fr(sys.argv[3]) + variant = sys.argv[4] if len(sys.argv) > 4 else 'thm' + for s in range(3, 32, 2): + r = X(s, h, k, x, variant) + if r is None: + print(s, 'D<=0'); continue + print('s=%2d m=%3d D/v=%6.3f root in [%.9e, %.9e] D/Cost=%.9e' % (s, r['m'], float(r['D']), r['lo'], r['hi'], float(r['DoverCost'])), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/subsys.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/subsys.py new file mode 100644 index 0000000000..983673376d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/subsys.py @@ -0,0 +1,52 @@ +import itertools +from bitp import * +def diagram(name): + """edges of a Dynkin diagram (disjoint union), returns (n, edges)""" + parts = name.split('+'); n = 0; E = [] + for part in parts: + mult = 1 + if '*' in part: mult, part = int(part.split('*')[0]), part.split('*')[1] + for _ in range(mult): + t, r = part[0], int(part[1:]) + if t == 'A': e = [(i, i + 1) for i in range(r - 1)] + elif t == 'D': e = [(i, i + 1) for i in range(r - 2)] + [(r - 3, r - 1)] + elif t == 'E': e = [(0, 2), (2, 3), (3, 4), (1, 3)] + [(i, i + 1) for i in range(4, r - 1)] + E += [(a + n, b + n) for a, b in e]; n += r + return n, E +def embed(name, start=0): + n, E = diagram(name) + adj = {i: set() for i in range(n)} + for a, b in E: adj[a].add(b); adj[b].add(a) + sol = [] + def rec(k): + if k == n: return True + for c in range(NL): + if c in sol: continue + ok = all((abs(G[c, sol[j]]) == 1) == (j in adj[k]) for j in range(k)) + if ok: + sol.append(c) + if rec(k + 1): return True + sol.pop() + return False + assert rec(0) + return sol +def closure(S): + S = set(S) + while True: + new = set() + for a in S: + for b in S: + new.add(REFL[a][b]) + if new <= S: return sorted(S) + S |= new +def report(name): + X = closure(embed(name)) + P = set(range(NL)) - set(X) + live = sorted(X) + h = rankQ_lines(live) + s = s_all(P); ch, c = greedy_basis(s) + return len(X), h, c, ch +if __name__ == '__main__': + import sys + for nm in sys.argv[1:]: + print(nm, report(nm), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/t1.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/t1.py new file mode 100644 index 0000000000..887aaccdd7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/t1.py @@ -0,0 +1,10 @@ +from geo import * +from collections import Counter +c = Counter() +for p in ALLP: + m = members(Rmask[p]); c[(len(m), rankQ_lines(m))] += 1 +print(sorted(c.items())) +# check reflections preserve G +import random +P = REFL[5]; assert all(abs(G[P[i],P[j]])==abs(G[i,j]) for i in range(120) for j in range(120)) +print('ok') diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/thmc111.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/thmc111.py new file mode 100644 index 0000000000..f59448396d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/thmc111.py @@ -0,0 +1,8 @@ +from bitp import * +P = {39, 52, 57, 71, 76, 78, 103, 109, 114} +live = np.ones(NL, bool); live[list(P)] = False +defic = [p for p in range(1, 512) if rankQ_lines(list(RLIST[p][live[RLIST[p]]])) < 8] +print('P111: reader sets R(p)\\P not spanning Q^8:', len(defic), defic[:20]) +s = s_all(P); ch, c = greedy_basis(s) +from collections import Counter +print('star greedy', c, Counter(s.values())) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq.py new file mode 100644 index 0000000000..083a8e708a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq.py @@ -0,0 +1,22 @@ +"""Is the 9-total F2 decoder of a punctured bit pattern unique (= B0 restricted)? And is c=8 UNSAT?""" +import sys, numpy as np +from geo import * +from minrank_sat import solve_asym, solve_minrank +from e8units import first_clique +def run(live): + v = len(live) + pat = {(i, j) for i in range(v) for j in range(v) if i != j and G[live[i], live[j]] != 0} + adj = [[i != j and G[live[i], live[j]] != 0 for j in range(v)] for i in range(v)] + K = first_clique(adj, 8) + B0l = B0[np.ix_(live, live)] + out = {} + if K is None: + out['clique'] = None; return out + for c, mode in ((9, 'neB0'), (8, 'none')): + r = solve_asym(v, pat, B0l, c, mode, K) + out[(c, mode)] = r['result'] + return out +if __name__ == '__main__': + P = [int(x) for x in sys.argv[1].split(',')] if len(sys.argv) > 1 and sys.argv[1] else [] + live = [i for i in range(NL) if i not in P] + print(len(live), run(live), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq111.log b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq111.log new file mode 100644 index 0000000000..16c894740e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/uniq111.log @@ -0,0 +1 @@ +111 {(9, 'neB0'): 'UNSAT', (8, 'none'): 'UNSAT'} diff --git a/research/e8-decoder-floors-and-cold14-notes/s2-decoder/xtab.py b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/xtab.py new file mode 100644 index 0000000000..f2a4162925 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s2-decoder/xtab.py @@ -0,0 +1,11 @@ +from fractions import Fraction as Fr +from skel import X +import sys +def best(h, cst, v, S=(3,5,7,9,11)): + out={} + for s in S: + r=X(s,h,1,Fr(cst,v)) + out[s]=float(r['hi']) if r else None + return out +for h,cst,v in [(8,63,111),(9,64,112),(8,72,120),(8,62,111),(8,54,102),(9,56,104),(8,7,63*1)]: + print(h,cst,v, {k:('%.4e'%x if x else None) for k,x in best(h,cst,v).items()}) diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/README.md b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/README.md new file mode 100644 index 0000000000..72ac4ba772 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/README.md @@ -0,0 +1,288 @@ +# Beyond E8 (no κ claim): ADE screen erratum, certified triple-family and Lorentzian ceilings, and a helper-adjusted ranking + +These notes follow up #620 (GamingPuzzled). They make **no κ claim** and ship no word. + +Status labels: + +- **checked**: `verify.sh` recomputes the item; it is exact or certified unless marked *model*. +- **argued**: a proof written in this README. +- **model**: an estimate. It is not a bound. + +```sh +sh verify.sh /path/to/integer-mult-bounds-main /fresh/out # PY=python3.11 ...; about 40 min, steps in parallel +``` + +Requirements: Python ≥ 3.11 with numpy, networkx and python-sat. The scripts import #595's `skel.py`, `cert.py`, `minrank_sat.py` and +`e8units.py` from `research/format-limits/scripts` of main. `check.py` compares every recomputed field with +`expected.json`: + +- integers, fractions and certified interval ends must be equal; +- model floats must agree to a relative 10⁻⁹. + +SAT answers come from CaDiCaL through python-sat, **without proof logs**. + +Conventions are those of #595 (§A): + +- a unit is v lines with a Gram matrix G and window h (k = 1); +- decoder pattern over F2: B_TT = 1, and B_TS = 0 wherever G_TS ≠ 0; +- c is the number of totals; +- star cost cst = Σ_j β_j, where β_j is the dimension of the gathered span (the copy drops f = V → 0); +- x = cst/v; +- X* is the certified root of skel.X, maximised over odd s ∈ [3, 15]. + +## 1. Erratum to #620's ADE screen (`ade_screen_fixed.py`, checked) + +#630 (vectorsss) is right: #620's `E6()` took the complement of two **orthogonal** roots. That gives 30 lines of type D6. +E6 is the complement of an A2 (a·b = 1), which has 36 lines. Recomputing the whole table found two more problems with #620's +screen. + +- #620's column "c" was rank_F2(J + G) of one decoder. That is only an **upper** bound on the minrank. The column is now + the exact F2 minrank, with asymmetric decoders allowed: SAT from the clique bound ω upwards, under the GL(c) gauge on the clique. +- #620 charged every total the full window (x = c·h/v). The table now also gives the star cost that label decoders on + the B0 labels actually attain, cst_lab: a matroid-greedy minimum over bases of F2^c of Σ dim span(gathered set). It is + an upper bound on cst_min. + +| unit | v | h | rank_F2(J+G) | ω | **minrank** | x_proxy = c·h/v | cst_lab (x_lab) | X* (proxy / lab) | +|---|---|---|---|---|---|---|---|---| +| A4 | 10 | 4 | 5 | 4 | 5 | 2 | 15 (3/2) | – / – | +| A5 | 15 | 5 | 5 | 5 | 5 | 5/3 | 20 (4/3) | – / – | +| A6 | 21 | 6 | 7 | 6 | 7 | 2 | 35 (5/3) | – / – | +| A7 | 28 | 7 | 7 | 7 | 7 | 7/4 | 42 (3/2) | – / – | +| A8 | 36 | 8 | 9 | 8 | 8..9 (c = 8 timed out after 600 s) | ≥ 16/9 | 63 (7/4) | – / – | +| D4 | 12 | 4 | 3 | 3 | 3 | 1 | 12 (1) | – / – | +| D5 | 20 | 5 | 5 | 4 | 5 | 5/4 | 20 (1) | – / – | +| D6 | 30 | 6 | 5 | 5 | 5 | 1 | 30 (1) | – / – | +| D7 | 42 | 7 | 7 | 6 | 7 | 7/6 | 42 (1) | – / – | +| D8 | 56 | 8 | 7 | 7 | 7 | 1 | 56 (1) | – / – | +| D9 | 72 | 9 | 9 | 8 | 8..9 (timeout) | ≥ 1 (c ≥ 8: 8·9/72) | 72 (1) | – / – | +| D10 | 90 | 10 | 9 | 9 | 9 | 1 | 90 (1) | – / – | +| D12 | 132 | 12 | 11 | 11 | 11 | 1 | 132 (1) | – / – | +| D6 (#620's "E6") | 30 | 6 | 5 | 5 | 5 | 1 | 30 (1) | – / – | +| **E6 (corrected)** | 36 | 6 | 7 | 5 | **7** (5 and 6 UNSAT) | 7/6 | **35 (35/36)** | – / ≤ 5.69·10⁻⁵ (see below) | +| E7 | 63 | 7 | 7 | 7 | 7 | 7/9 | 49 (7/9) | 1.8468·10⁻³ (s = 11) | +| E8 | 120 | 8 | 9 | 8 | 9 | 3/5 | 72 (3/5) | 5.1598·10⁻³ (s = 5) | + +("–" means D_s = (s − 1) − s·x ≤ 0 for every s ≤ 15.) + +Findings: + +- **E6 (corrected).** + - The minrank is 7. ω = 5 (c = 5 and 6 are UNSAT), and #630's rank_F2(J+G) = 7 is attained. + - The proxy deficit is negative, as #630 says. + - The attainable label-decoder cost is 35 < 36, though: one total spans only 5 dimensions. That makes x = 35/36 < 1, so D_s > 0 for + s > 36. + - The ceiling there is negligible. max over odd s ≤ 801 of D_s/Cost_s is 5.69·10⁻⁵ (at s = 81), and D_s/Cost_s is an upper + bound on X (#595 §A.2). + - So the conclusion "E6 gives no useful saving" stands, but the reason is the tiny D_s/Cost_s, not D ≤ 0 at every s. +- **A_n and D_n.** x ≥ 1 under both the proxy (at the lower end of the minrank) and the attained label cost, so there is no saving + in this screen. The D_n label decoders attain exactly x = 1 (D = −1). +- **E7 and E8.** These rows are unchanged from #620: X* = 1.8468·10⁻³ (E7) and 5.1598·10⁻³ (E8 120/72). +- **Scope.** The cst_lab column is an attained upper bound for label decoders on the B0 labels. A lower bound over + all decoders is proved only for E8 (#595 C.6, S2). The "no saving" verdicts for A_n, D_n and E6 rest on #620's + full-rank-totals premise; this table only recomputes them. + +## 2. The triple family T_h (`triple.py`, checked) + +This is the upstream Alon-type unit (`upstream/build/sections/03-motifs.tex`). + +**Bit side.** Lines t_T, T ∈ C([h], 3), with the form I − J/9, so G_ST = |S∩T| − 1. + +- The pattern requires B_TS = 0 where |S∩T| ∈ {0, 2}. +- The paper's centre decoder C_i is B0 = M Mᵀ mod 2, with c = h totals. + +**Complex side.** Labels are t_T ∈ F2^h with the dot product. + +- The pattern requires B_TS = 0 where |S∩T| = 1. +- The decoder is the h + 1 centres C_i and C_*, with B_ST = (|S∩T| − 1)/2 over ℚ(i). + +**Copy cost (derived, not a proxy).** Total C_i gathers the C(h−1, 2) lines T ∋ i, and its readers are the same set. + +| | bit | complex | +|---|---|---| +| span of a centre C_i | dimension h − 1 over ℚ (the functional 2x_i − Σ_{j≠i} x_j vanishes on it) | V_i = {x : Σ_{j≠i} x_j = 0}, dimension h − 1 over F2 | +| #446 Theorem C pairing rank of C_i | h − 1 (computed) | h − 1 for even h; h − 2 for odd h, because 1 + e_i ∈ V_i ∩ V_i^⊥ | +| C_* | – | costs h | +| **cst, attained** | **h(h − 1)** | **h²** | +| cst, Theorem C floor of the centre decoder | h(h − 1) | h² for even h; h(h − 1) for odd h | + +**Minrank (bit).** + +- For 8 ≤ h ≤ 11 the minrank is h (SAT). +- For h ≡ 0 (mod 4) it is h by the clique argument: ⌊h/4⌋ disjoint 4-sets give 4⌊h/4⌋ pairwise even-intersection triples. +- h = 6 and 7 give 6. +- Label decoders on the B0 labels have every gathered span of dimension ≥ h − 1 (`lab_min`, checked for h ≤ 11). So the centre decoder is + optimal among them. + +Certified ceilings (selected; every h from 6 to 32 is in `triple.json`): + +| h | v | bit x = 6/(h−2) | **bit X*** | complex x = h²/C(h,3) | **complex X*** | +|---|---|---|---|---|---| +| 9 | 84 | 6/7 (window 8: the form is degenerate at h = 9) | 7.2457·10⁻⁴ (s = 15) | 27/28 | – (Theorem C floor: 6.358·10⁻⁴) | +| 12 | 220 | 3/5 | 3.3282·10⁻³ | 36/55 | 2.4187·10⁻³ | +| 15 | 455 | 6/13 | 6.4061·10⁻³ | 0.4945 | 5.3738·10⁻³ | +| 16 | 560 | 3/7 | 6.9611·10⁻³ | 0.4571 | 6.1233·10⁻³ | +| 18 | 816 | 3/8 | 7.5613·10⁻³ | 0.3971 | 6.9874·10⁻³ | +| 20 | 1140 | 1/3 | 7.7612·10⁻³ | 0.3509 | 7.3512·10⁻³ | +| **21** | 1330 | 6/19 | **7.7735·10⁻³ (max)** | 0.3316 | 7.4224·10⁻³ | +| **22** | 1540 | 3/10 | 7.7474·10⁻³ | 0.3143 | **7.4444·10⁻³ (max)** | +| 24 | 2024 | 3/11 | 7.6184·10⁻³ | 0.2846 | 7.3882·10⁻³ | +| 30 | 4060 | 3/14 | 6.9743·10⁻³ | 0.2217 | 6.8600·10⁻³ | + +All maxima are at s = 3. + +- **Bit side.** The triple family's ceiling exceeds E8's realised bit unit (6.0005·10⁻³) for every h from 15 to 32. +- **Complex side.** It exceeds E8's realised complex unit (5.1863·10⁻³) for every h from 15 to 32. At the + attained cost the odd-h rows are slightly lower; at their Theorem C floor they equal the bit values. +- **S1's float screen.** It gave 7.4 / 7.2·10⁻³ at h = 24 with the proxy cst = h² / h(h+1). The derived cst is h(h−1) / h², and + the certified values are 7.6184 / 7.3882·10⁻³. + +## 3. Lorentzian I_{n,1} (`lorentz.py`, `lorentz_minrank.py`, checked) + +The units are #634's triples e_A + e_{n+1} and pairs e_i − e_j, n odd. + +- The binary labels (the 3-set, or the complement of the pair) fit the #595 pattern exactly: ⟨u_S, u_T⟩ = 1 + G_ST mod 2 (checked). +- The window is the Lorentz Gram rank n. #634 note 3 proves that n − 1 is impossible for n = 11, 13. + +| unit | v | h | c (minrank) | ω | readers z per line | cst_lab (per total) | x | **X*** | +|---|---|---|---|---|---|---|---|---| +| I_{9,1} = E8 (control, quotient by the isotropic w) | 120 | 8 | 9 | 8 | 63 | 72 (8 each) | 3/5 | 5.1598·10⁻³ | +| I_{11,1} | 220 | 11 | **11** (10 UNSAT) | 10 | 115 (triples), 129 (pairs) | **110 (10 each)** | 1/2 | **7.1715·10⁻³** (s = 3) | +| I_{13,1} | 364 | 13 | ≤ 13 (ω = 12) | 12 | 183, 231 | **156 (12 each)** | 3/7 | **8.6247·10⁻³** (s = 3) | + +Unlike E8, each total spans only h − 1 dimensions. That is where x < 3/5 comes from. Both ceilings exceed E8's. + +## 4. Helper floors: the #595 §D method (`cover.py`, checked; one bound argued) + +The method computes cover numbers at the source end: the fewest proper hyperplane sections of a source's readers T(s) that cover T(s). + +| unit | readers | rank | cover number | evidence | +|---|---|---|---|---| +| E8 bit (control) | 63 | 7 | 3 | no 2-cover; best pair union 52; 3-cover found (= #595 D1) | +| T_h, h = 8 … 24 | 3·C(h−3, 2) | h − 1 | **2** | exact certificate: two rank-(h−2) subsets with union T(s) | +| I_{11,1} / I_{13,1}, pair-type sources | 129 / 231 | h − 1 | 2 | certificate | +| I_{11,1} / I_{13,1}, triple-type sources | 115 / 183 | h − 1 | ≤ 3 (no 2-cover found in sampling) | 3-cover certificate | + +**Triple family, max section (argued; matches the search for h = 10, 12, 16, 20, 24).** + +- **Setup.** For S = {1, 2, 3} and O = [h] \ S (n′ = h − 3), a section is {(a, {x, y}) : c_a + c_x + c_y = 0} for a functional c. +- **Constant c on O.** At most two of the three a-classes are covered, so the section has ≤ 2C(n′, 2) readers. If all three were + covered, c would vanish on T(s). +- **Non-constant c on O.** Each a-class meets a "sum graph" with at most C(n′−1, 2) edges (n′ ≥ 6). That gives ≤ 3C(h−4, 2) readers, + and c = e_z attains it. +- **Result.** For h ≥ 9 the max section is 3C(h−4, 2), the leftover is 3(h − 4), and the #595 floors are: + - sink-only: R ≥ v − ⌊v/(3(h−4))⌋, about 0.98–0.99 helpers per port; + - strict: 0. +- **Comparison.** E8 bit has 212 / 138, which is 1.91 / 1.24 per port. + +**Conclusion of §4.** The design-free floors do **not** penalise these families; they are lower per port than E8's. +The §D method cannot rank them. Their helper cost is in the work per port: 630 readers per source for T_24 bit, against 63 +for E8. + +## 5. Helper-adjusted ranking (`est_model.py`, `rank.py`; model) + +**Model.** + +- **Base.** #595's per-port slot profile at window h. +- **Data runs.** Cut with the shipped run histograms: #555 x/y for bit, #569 four-cold for complex (#595 a4). +- **Helpers.** + - ρ = R/v helpers per port, each climbing h per stage. + - Cut with the shipped helper-run histograms (#555; #568 implied). + - Hosted into the largest B_K idle slots first. A hosted slot of rank r leaves a remainder of rank r/a, as in #555's 42/28/14 → 6/4/2. + - Mass beyond the hostable idle mass stays as extra children. +- **Root.** The same pressure root, at m = 1 + s(h−1) and D = (s−1) − s·x, with the best s. +- **Calibration.** + - Bit: the model reproduces #555 to 0.3% (factor 1.0032). + - Complex: the factor is 1.129, because #568's hosting is better than the rule above. +- **Piece-size scaling to larger h.** + - abs: shipped absolute ranks (pessimistic). + - rel: ranks scale with the window (optimistic). +- **Loss-chain factors.** #634's a ≈ (1/m)(D/W)/L_ent is reported per candidate in `rank.json`. It matches the root to < 1% at E8. + +**Columns.** + +- **ρ\*** is the break-even helpers per port: the ρ at which a candidate's estimate equals the E8 word of the same side in the same + model (bit 1.8058·10⁻³ at ρ = 439/111 = 3.95; complex 1.6108·10⁻³ at ρ = 496/112 = 4.43). +- **ρ_dim** transfers E8's shipped intensity per reader per window dimension: ρ_E8 · (z/(h−1)) / (z_E8/(h_E8−1)). + +| candidate | h | v | x | readers z | X* (certified) | **ρ\*** abs / rel | ρ_dim | a_est at ρ = ρ_E8, abs / rel | **a_est at ρ_dim**, abs / rel | +|---|---|---|---|---|---|---|---|---|---| +| E8 bit 111/63 (#555) | 8 | 111 | 0.5676 | 63 | 6.0005e-3 | 3.95 (ref) | 3.95 | 1.806e-3 | **1.806e-3** | +| E8 complex 112/64 (#568) | 9 | 112 | 0.5714 | 52 | 5.1863e-3 | 4.43 (ref) | 4.43 | 1.611e-3 | **1.611e-3** | +| **I_{11,1} bit** | 11 | 220 | 0.5000 | 118.5 | 7.1715e-3 | 2.66 / 3.34 | 5.21 | 1.480e-3 / 1.620e-3 | **1.225e-3 / 1.339e-3** | +| I_{13,1} bit | 13 | 364 | 0.4286 | 193.3 | 8.6247e-3 | 2.64 / 3.43 | 7.08 | 1.438e-3 / 1.647e-3 | 0.942e-3 / 1.079e-3 | +| T_13 bit | 13 | 286 | 0.5455 | 135 | 4.3818e-3 | 1.03 / 1.75 | 4.94 | 1.046e-3 / 1.198e-3 | 0.904e-3 / 1.026e-3 | +| T_12 bit | 12 | 220 | 0.6000 | 108 | 3.3282e-3 | 0.55 / 0.98 | 4.31 | 0.954e-3 / 1.070e-3 | 0.900e-3 / 1.009e-3 | +| T_14 bit | 14 | 364 | 0.5000 | 165 | 5.5852e-3 | 1.27 / 2.08 | 5.58 | 1.092e-3 / 1.275e-3 | 0.866e-3 / 1.001e-3 | +| T_16 bit | 16 | 560 | 0.4286 | 234 | 6.9611e-3 | 1.46 / 2.41 | 6.86 | 1.109e-3 / 1.340e-3 | 0.745e-3 / 0.898e-3 | +| T_20 bit | 20 | 1140 | 0.3333 | 408 | 7.7612e-3 | 1.20 / 2.43 | 9.44 | 1.029e-3 / 1.326e-3 | 0.533e-3 / 0.679e-3 | +| T_24 bit | 24 | 2024 | 0.2727 | 630 | 7.6184e-3 | 0.80 / 2.19 | 12.04 | 0.918e-3 / 1.248e-3 | 0.387e-3 / 0.517e-3 | +| T_16 complex | 16 | 560 | 0.4571 | 325 | 6.1233e-3 | 2.14 / 2.92 | 14.76 | 1.087e-3 / 1.270e-3 | 0.424e-3 / 0.490e-3 | +| T_22 complex | 22 | 1540 | 0.3143 | 1026 | 7.4444e-3 | 1.92 / 3.07 | 33.29 | 0.995e-3 / 1.265e-3 | 0.182e-3 / 0.229e-3 | +| T_24 complex | 24 | 2024 | 0.2846 | 1393 | 7.3882e-3 | 1.67 / 2.93 | 41.26 | 0.943e-3 / 1.230e-3 | 0.141e-3 / 0.181e-3 | + +The full list (T_10 … T_30, both sides) is in `rank.json`. + +**Answer to "is there a family that beats E8 on both the ceiling and the helper-adjusted estimate?" — no candidate in +this screen does.** + +- **The ceiling.** Many candidates beat E8's ceiling: T_h for h ≥ 15 on both sides, I_{11,1} and I_{13,1}. +- **The helper-adjusted estimate.** Even at E8's own helper intensity per port (3.95), every candidate stays below E8: + - the best is I_{13,1} rel at 1.647·10⁻³, against 1.806·10⁻³; + - the break-even ρ\* is 2.6–3.4 helpers per port for I_{n,1} and ≤ 2.5 for the bit triples. +- **Why the break-even is implausible.** These units have 1.3–1.9× (I_{n,1}) and up to 3× (T_24) more readers per window + dimension than E8, which ships 3.95. +- **Loss-chain view (#634).** + - 1/m falls from 1/50 (E8, s = 7) to 1/91 (I_{11,1}, s = 9) and 1/162 (T_24 bit, s = 7). + - L_ent rises from 3.19 to 3.79–4.46. + - The smaller copy share x improves D/W by at most about 1.5×. +- **Ranking by a_est at ρ_dim.** + 1. I_{11,1}: 1.23–1.34·10⁻³ + 2. I_{13,1}: 0.94–1.08·10⁻³ + 3. T_12–T_14 bit: about 0.9–1.0·10⁻³ + 4. larger triples, falling as 1/h + 5. complex triples: their readers grow like h³ +- **Closest lead.** I_{11,1} beats E8 only if a design reaches ≤ 2.7–3.3 helpers per port on 118 readers per line. If + its minrank had been 10, cst would have been 100 and X* 9.14·10⁻³. The minrank is 11 (c = 10 UNSAT). + +**Model caveats.** + +- The helper piece statistics and hosting rule are transferred from the E8 words. +- ρ for a new family is unknown; ρ\* is the honest output. +- s is free on both sides. +- None of this bounds κ. + +## 6. Not done + +- **Q3: a rigorous lower bound on the copy cost for the Gaussian and Eisenstein E8 views.** Not attempted beyond observing that the + counting bounds available here are weak: + - covering lines by flats gives 4; + - the minrank gives 15 ≤ 0.25v; + - the clique-matroid argument gives 12. + S1's question (Σ_j dim span A_j ≥ 0.71v?) stays open. +- The exact minrank of A8 and D9 (SAT timeouts at c = 8). Not needed: x ≥ 1 either way. +- Exact max section and cover number for the triple-type sources of I_{n,1}. +- Punctures of I_{n,1}, which might help as 120 → 111 did for E8. + +## Files + +- `scripts/units.py`: exact line systems and ranks, the clique bound, and the time-limited gauged minrank SAT. +- `scripts/ade_screen_fixed.py`: §1; it replaces #620's `ade_screen.py`. +- `scripts/triple.py`: §2. +- `scripts/lorentz.py`, `scripts/lorentz_minrank.py`: §3. +- `scripts/cover.py`: §4. +- `scripts/est_model.py`, `scripts/rank.py`: §5. +- `scripts/check.py`, `expected.json`, `verify.sh`: recomputation. + +## Credits + +Builds on: + +- DreamingOfClouds: #595 format limits (skel/cert/minrank_sat, §A, §D, a4 histograms) and #446 Theorems A–C; +- vectorsss: #630, the E6 correction; +- ShikeChen01: #634, the Lorentzian family and the loss chain; +- the upstream paper (03-motifs); +- jeremykothe2: #568; +- #555 and #569, whose shipped statistics are used; +- #620 (GamingPuzzled). + +Prepared by Joel Pulikkan (GamingPuzzled) with Claude Opus 5.5 (Anthropic) via Claude Code. Apache-2.0. diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/expected.json b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/expected.json new file mode 100644 index 0000000000..aa07c6745b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/expected.json @@ -0,0 +1,1916 @@ +{ + "ade": { + "A4": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 5, + "cst_lab": 15, + "h": 4, + "minrank": 5, + "omega": 4, + "v": 10, + "x_lab": "3/2" + }, + "A5": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 5, + "cst_lab": 20, + "h": 5, + "minrank": 5, + "omega": 5, + "v": 15, + "x_lab": "4/3" + }, + "A6": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 7, + "cst_lab": 35, + "h": 6, + "minrank": 7, + "omega": 6, + "v": 21, + "x_lab": "5/3" + }, + "A7": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 7, + "cst_lab": 42, + "h": 7, + "minrank": 7, + "omega": 7, + "v": 28, + "x_lab": "3/2" + }, + "A8": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 9, + "cst_lab": 63, + "h": 8, + "minrank": "8..9", + "omega": 8, + "v": 36, + "x_lab": "7/4" + }, + "D10": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 9, + "cst_lab": 90, + "h": 10, + "minrank": 9, + "omega": 9, + "v": 90, + "x_lab": "1" + }, + "D12": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 11, + "cst_lab": 132, + "h": 12, + "minrank": 11, + "omega": 11, + "v": 132, + "x_lab": "1" + }, + "D4": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 3, + "cst_lab": 12, + "h": 4, + "minrank": 3, + "omega": 3, + "v": 12, + "x_lab": "1" + }, + "D5": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 5, + "cst_lab": 20, + "h": 5, + "minrank": 5, + "omega": 4, + "v": 20, + "x_lab": "1" + }, + "D6": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 5, + "cst_lab": 30, + "h": 6, + "minrank": 5, + "omega": 5, + "v": 30, + "x_lab": "1" + }, + "D6(archived \"E6\")": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 5, + "cst_lab": 30, + "h": 6, + "minrank": 5, + "omega": 5, + "v": 30, + "x_lab": "1" + }, + "D7": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 7, + "cst_lab": 42, + "h": 7, + "minrank": 7, + "omega": 6, + "v": 42, + "x_lab": "1" + }, + "D8": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 7, + "cst_lab": 56, + "h": 8, + "minrank": 7, + "omega": 7, + "v": 56, + "x_lab": "1" + }, + "D9": { + "DoverCost_lab": null, + "Xstar_lab": null, + "c0": 9, + "cst_lab": 72, + "h": 9, + "minrank": "8..9", + "omega": 8, + "v": 72, + "x_lab": "1" + }, + "E6": { + "DoverCost_lab": { + "hi": "1000000000000000000000000000000000000000000000/17583496298980754515651120025120377029171980162981", + "s": 81 + }, + "Xstar_lab": null, + "c0": 7, + "cst_lab": 35, + "h": 6, + "minrank": 7, + "omega": 5, + "v": 36, + "x_lab": "35/36" + }, + "E7": { + "DoverCost_lab": null, + "Xstar_lab": { + "hi": "923423603180445516994123700271757115842773/500000000000000000000000000000000000000000000", + "s": 11 + }, + "c0": 7, + "cst_lab": 49, + "h": 7, + "minrank": 7, + "omega": 7, + "v": 63, + "x_lab": "7/9" + }, + "E8": { + "DoverCost_lab": null, + "Xstar_lab": { + "hi": "5159798631791548059744023646544519579038021/1000000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": 9, + "cst_lab": 72, + "h": 8, + "minrank": 9, + "omega": 8, + "v": 120, + "x_lab": "3/5" + } + }, + "cover": { + "E8|0": { + "best_section_found": 36, + "cover_lb": 2, + "cover_ub": 3, + "h": 8, + "rank_readers": 7, + "readers": 63 + }, + "L11|0": { + "best_section_found": 87, + "cover_lb": 2, + "cover_ub": 3, + "h": 11, + "rank_readers": 10, + "readers": 115 + }, + "L11|165": { + "best_section_found": 120, + "cover_lb": 2, + "cover_ub": 2, + "h": 11, + "rank_readers": 10, + "readers": 129 + }, + "L13|0": { + "best_section_found": 147, + "cover_lb": 2, + "cover_ub": 3, + "h": 13, + "rank_readers": 12, + "readers": 183 + }, + "L13|286": { + "best_section_found": 220, + "cover_lb": 2, + "cover_ub": 2, + "h": 13, + "rank_readers": 12, + "readers": 231 + }, + "T10|0": { + "best_section_found": 45, + "cover_lb": 2, + "cover_ub": 2, + "h": 10, + "rank_readers": 9, + "readers": 63 + }, + "T12|0": { + "best_section_found": 84, + "cover_lb": 2, + "cover_ub": 2, + "h": 12, + "rank_readers": 11, + "readers": 108 + }, + "T16|0": { + "best_section_found": 198, + "cover_lb": 2, + "cover_ub": 2, + "h": 16, + "rank_readers": 15, + "readers": 234 + }, + "T20|0": { + "best_section_found": 360, + "cover_lb": 2, + "cover_ub": 2, + "h": 20, + "rank_readers": 19, + "readers": 408 + }, + "T24|0": { + "best_section_found": 570, + "cover_lb": 2, + "cover_ub": 2, + "h": 24, + "rank_readers": 23, + "readers": 630 + }, + "T8|0": { + "best_section_found": 20, + "cover_lb": 2, + "cover_ub": 2, + "h": 8, + "rank_readers": 7, + "readers": 30 + } + }, + "lorentz": { + "11": { + "Xstar": { + "hi": "1792880663455379886862817162551664296188391/250000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 11, + "cst_lab": 110, + "omega": 10, + "per_total": [ + 10, + 10, + 10, + 10, + 10, + 10, + 10, + 10, + 10, + 10, + 10 + ], + "v": 220, + "window": 11, + "x": "1/2", + "z": [ + 115, + 129 + ] + }, + "13": { + "Xstar": { + "hi": "8624695507724970504337402843475501867942509/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 13, + "cst_lab": 156, + "omega": 12, + "per_total": [ + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12, + 12 + ], + "v": 364, + "window": 13, + "x": "3/7", + "z": [ + 183, + 231 + ] + }, + "9": { + "Xstar": { + "hi": "5159798631791548059744023646544519579038021/1000000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": 9, + "cst_lab": 72, + "omega": 8, + "per_total": [ + 8, + 8, + 8, + 8, + 8, + 8, + 8, + 8, + 8 + ], + "v": 120, + "window": 8, + "x": "3/5", + "z": [ + 63 + ] + } + }, + "lorentz_minrank": { + "11": { + "c0": 11, + "hi": 11, + "lo": 11, + "log": { + "10": "UNSAT" + }, + "omega": 10 + } + }, + "rank": { + "E8 bit 111/63 (#555)": { + "Xs": 5, + "Xstar": 0.00600049473312529, + "a_dim_abs": 0.0018058295009581778, + "a_dim_rel": 0.0018058295009581778, + "a_read_abs": 0.0018058295009581778, + "a_read_rel": 0.0018058295009581778, + "a_same_abs": 0.0018058295009581778, + "a_same_rel": 0.0018058295009581778, + "cst": 63, + "h": 8, + "rho_star_abs": 3.954954954679124, + "rho_star_rel": 3.954954954679124, + "v": 111, + "x": 0.5675675675675675, + "z": 63.0 + }, + "E8 bit 120/72": { + "Xs": 5, + "Xstar": 0.005159798631791548, + "a_dim_abs": 0.0016034717449358912, + "a_dim_rel": 0.0016034717449358912, + "a_read_abs": 0.0016034717449358912, + "a_read_rel": 0.0016034717449358912, + "a_same_abs": 0.0016034717449358912, + "a_same_rel": 0.0016034717449358912, + "cst": 72, + "h": 8, + "rho_star_abs": 3.2809337786602555, + "rho_star_rel": 3.2809337786602555, + "v": 120, + "x": 0.6, + "z": 63.0 + }, + "E8 complex 112/64 (#568)": { + "Xs": 5, + "Xstar": 0.005186252252886545, + "a_dim_abs": 0.0016107569812721682, + "a_dim_rel": 0.0016107569812721682, + "a_read_abs": 0.0016107569812721682, + "a_read_rel": 0.0016107569812721682, + "a_same_abs": 0.0016107569812721682, + "a_same_rel": 0.0016107569812721682, + "cst": 64, + "h": 9, + "rho_star_abs": 4.428571428434225, + "rho_star_rel": 4.428571428434225, + "v": 112, + "x": 0.5714285714285714, + "z": 52.0 + }, + "I_{11,1} bit": { + "Xs": 3, + "Xstar": 0.007171522653821519, + "a_dim_abs": 0.0012246258053555746, + "a_dim_rel": 0.0013387673129528718, + "a_read_abs": 0.0009392956976872076, + "a_read_rel": 0.0010222574941126607, + "a_same_abs": 0.0014795179684754338, + "a_same_rel": 0.001620305554277553, + "cst": 110, + "h": 11, + "rho_star_abs": 2.661677615469671, + "rho_star_rel": 3.343132941881777, + "v": 220, + "x": 0.5, + "z": 118.5 + }, + "I_{13,1} bit": { + "Xs": 3, + "Xstar": 0.00862469550772497, + "a_dim_abs": 0.0009421714307318571, + "a_dim_rel": 0.0010785736492618507, + "a_read_abs": 0.000605362942552331, + "a_read_rel": 0.0006919920072509971, + "a_same_abs": 0.00143769279819993, + "a_same_rel": 0.0016469528589961638, + "cst": 156, + "h": 13, + "rho_star_abs": 2.6371141389972763, + "rho_star_rel": 3.433550539557473, + "v": 364, + "x": 0.42857142857142855, + "z": 193.28571428571428 + }, + "T_10 bit": { + "Xs": 9, + "Xstar": 0.0015532374018115726, + "a_dim_abs": 0.0006366782737705969, + "a_dim_rel": 0.0006670208710166347, + "a_read_abs": 0.0005648600231426096, + "a_read_rel": 0.0005916051335760184, + "a_same_abs": 0.0005648600231426096, + "a_same_rel": 0.0005916051335760184, + "cst": 90, + "h": 10, + "rho_star_abs": 0.0, + "rho_star_rel": 0.0, + "v": 120, + "x": 0.75, + "z": 63.0 + }, + "T_11 bit": { + "Xs": 5, + "Xstar": 0.0024323916899852945, + "a_dim_abs": 0.0008255146093040117, + "a_dim_rel": 0.0009039827913448415, + "a_read_abs": 0.0006934655057453908, + "a_read_rel": 0.0007535652717616766, + "a_same_abs": 0.0007902736647929714, + "a_same_rel": 0.0008639705203513044, + "cst": 110, + "h": 11, + "rho_star_abs": 0.0, + "rho_star_rel": 0.14618176319345366, + "v": 165, + "x": 0.6666666666666666, + "z": 84.0 + }, + "T_12 bit": { + "Xs": 5, + "Xstar": 0.003328195617920844, + "a_dim_abs": 0.0008997028737231361, + "a_dim_rel": 0.0010087127548971155, + "a_read_abs": 0.0006771271392320587, + "a_read_rel": 0.000747282541655889, + "a_same_abs": 0.0009540840873135277, + "a_same_rel": 0.0010697568404938681, + "cst": 132, + "h": 12, + "rho_star_abs": 0.5452559926197864, + "rho_star_rel": 0.976033717233804, + "v": 220, + "x": 0.6, + "z": 108.0 + }, + "T_12 complex": { + "Xs": 5, + "Xstar": 0.0024186870696829414, + "a_dim_abs": 0.0006281113047948817, + "a_dim_rel": 0.0006734081685599144, + "a_read_abs": 0.0004934908042433177, + "a_read_rel": 0.0005275648784861581, + "a_same_abs": 0.0008059951535310397, + "a_same_rel": 0.0008648149595573777, + "cst": 144, + "h": 12, + "rho_star_abs": 0.48710020128055476, + "rho_star_rel": 0.6897237952216528, + "v": 220, + "x": 0.6545454545454545, + "z": 111.0 + }, + "T_13 bit": { + "Xs": 3, + "Xstar": 0.004381759860146903, + "a_dim_abs": 0.0009036809025035332, + "a_dim_rel": 0.0010264161026451776, + "a_read_abs": 0.0006080025550507143, + "a_read_rel": 0.0006887166295234427, + "a_same_abs": 0.00104557348670486, + "a_same_rel": 0.0011976316369566441, + "cst": 156, + "h": 13, + "rho_star_abs": 1.02635551666026, + "rho_star_rel": 1.7492625691374997, + "v": 286, + "x": 0.5454545454545454, + "z": 135.0 + }, + "T_14 bit": { + "Xs": 3, + "Xstar": 0.005585241543794565, + "a_dim_abs": 0.0008655227738654691, + "a_dim_rel": 0.0010005620795299343, + "a_read_abs": 0.0005329883240333378, + "a_read_rel": 0.0006147499988792014, + "a_same_abs": 0.001091738055930747, + "a_same_rel": 0.001275022866536123, + "cst": 182, + "h": 14, + "rho_star_abs": 1.2655638638534583, + "rho_star_rel": 2.084736299002543, + "v": 364, + "x": 0.5, + "z": 165.0 + }, + "T_14 complex": { + "Xs": 3, + "Xstar": 0.004293631356651573, + "a_dim_abs": 0.0005483091957474773, + "a_dim_rel": 0.0006129532971435715, + "a_read_abs": 0.00036255701361225895, + "a_read_rel": 0.00040443259957845306, + "a_same_abs": 0.0010234719625590752, + "a_same_rel": 0.0011551969513878845, + "cst": 196, + "h": 14, + "rho_star_abs": 1.7584143653948558, + "rho_star_rel": 2.303974145979737, + "v": 364, + "x": 0.5384615384615384, + "z": 198.0 + }, + "T_16 bit": { + "Xs": 3, + "Xstar": 0.006961145145311995, + "a_dim_abs": 0.0007452933227853547, + "a_dim_rel": 0.0008983077741330437, + "a_read_abs": 0.0003966910615280629, + "a_read_rel": 0.00047508518050712675, + "a_same_abs": 0.0011091266842283764, + "a_same_rel": 0.0013403348998162832, + "cst": 240, + "h": 16, + "rho_star_abs": 1.4634474871854763, + "rho_star_rel": 2.407953140209429, + "v": 560, + "x": 0.42857142857142855, + "z": 234.0 + }, + "T_16 complex": { + "Xs": 3, + "Xstar": 0.006123263792854414, + "a_dim_abs": 0.00042377794371244247, + "a_dim_rel": 0.0004898786258478099, + "a_read_abs": 0.00024031529461724212, + "a_read_rel": 0.00027781609507599884, + "a_same_abs": 0.0010870303652881837, + "a_same_rel": 0.0012695930715339167, + "cst": 256, + "h": 16, + "rho_star_abs": 2.142810977966292, + "rho_star_rel": 2.924762014299631, + "v": 560, + "x": 0.45714285714285713, + "z": 325.0 + }, + "T_18 bit": { + "Xs": 3, + "Xstar": 0.007561304679511506, + "a_dim_abs": 0.0006309954656744363, + "a_dim_rel": 0.0007835103298075171, + "a_read_abs": 0.00029383722494590747, + "a_read_rel": 0.00036345237440677647, + "a_same_abs": 0.0010783101326397855, + "a_same_rel": 0.0013418062817776315, + "cst": 306, + "h": 18, + "rho_star_abs": 1.368834213280934, + "rho_star_rel": 2.4796230376523454, + "v": 816, + "x": 0.375, + "z": 315.0 + }, + "T_18 complex": { + "Xs": 3, + "Xstar": 0.006987429549461068, + "a_dim_abs": 0.0003181181271738347, + "a_dim_rel": 0.0003808654851440009, + "a_read_abs": 0.00015774463573908155, + "a_read_rel": 0.00018848056558206242, + "a_same_abs": 0.0010800778491285182, + "a_same_rel": 0.0012989589307333856, + "cst": 324, + "h": 18, + "rho_star_abs": 2.1875754409848014, + "rho_star_rel": 3.1392038152262103, + "v": 816, + "x": 0.39705882352941174, + "z": 500.0 + }, + "T_20 bit": { + "Xs": 3, + "Xstar": 0.0077611939844754885, + "a_dim_abs": 0.0005332995281648943, + "a_dim_rel": 0.0006787099642924436, + "a_read_abs": 0.00022024900488504246, + "a_read_rel": 0.0002797644201233015, + "a_same_abs": 0.0010288039557595971, + "a_same_rel": 0.001325574314045184, + "cst": 380, + "h": 20, + "rho_star_abs": 1.1988456732069608, + "rho_star_rel": 2.429817240772536, + "v": 1140, + "x": 0.3333333333333333, + "z": 408.0 + }, + "T_20 complex": { + "Xs": 3, + "Xstar": 0.007351183769407273, + "a_dim_abs": 0.00023908863409453362, + "a_dim_rel": 0.00029449766676582914, + "a_read_abs": 0.00010522230157651156, + "a_read_rel": 0.00012939200383095016, + "a_same_abs": 0.0010433886289098093, + "a_same_rel": 0.0012872933425278842, + "cst": 400, + "h": 20, + "rho_star_abs": 2.0844154812948545, + "rho_star_rel": 3.1535716298094485, + "v": 1140, + "x": 0.3508771929824561, + "z": 731.0 + }, + "T_22 bit": { + "Xs": 3, + "Xstar": 0.007747442341172933, + "a_dim_abs": 0.0004526916133157033, + "a_dim_rel": 0.0005912195595351862, + "a_read_abs": 0.00016803400556938305, + "a_read_rel": 0.0002180636537321941, + "a_same_abs": 0.0009734733602727194, + "a_same_rel": 0.001290865418594146, + "cst": 462, + "h": 22, + "rho_star_abs": 0.9919938849634491, + "rho_star_rel": 2.3212470892758574, + "v": 1540, + "x": 0.3, + "z": 513.0 + }, + "T_22 complex": { + "Xs": 3, + "Xstar": 0.007444442957394497, + "a_dim_abs": 0.00018201365840515342, + "a_dim_rel": 0.00022938261676260345, + "a_read_abs": 7.189964043107868e-05, + "a_read_rel": 9.059494947906635e-05, + "a_same_abs": 0.0009950241720641595, + "a_same_rel": 0.0012650966137252916, + "cst": 484, + "h": 22, + "rho_star_abs": 1.9187685724318726, + "rho_star_rel": 3.0672522520035272, + "v": 1540, + "x": 0.3142857142857143, + "z": 1026.0 + }, + "T_24 bit": { + "Xs": 3, + "Xstar": 0.007618412475718811, + "a_dim_abs": 0.00038697123547943907, + "a_dim_rel": 0.0005173002697658811, + "a_read_abs": 0.00013032317121934275, + "a_read_rel": 0.00017285017348586576, + "a_same_abs": 0.0009180763062131824, + "a_same_rel": 0.0012476664622146996, + "cst": 552, + "h": 24, + "rho_star_abs": 0.7990600730408914, + "rho_star_rel": 2.185761201326386, + "v": 2024, + "x": 0.2727272727272727, + "z": 630.0 + }, + "T_24 complex": { + "Xs": 3, + "Xstar": 0.007388222422275112, + "a_dim_abs": 0.00014064176962379717, + "a_dim_rel": 0.00018076803466452233, + "a_read_abs": 5.0458669755174115e-05, + "a_read_rel": 6.48413396110639e-05, + "a_same_abs": 0.0009433375657405344, + "a_same_rel": 0.0012302261372722726, + "cst": 576, + "h": 24, + "rho_star_abs": 1.6675037080858601, + "rho_star_rel": 2.930999599993811, + "v": 2024, + "x": 0.2845849802371542, + "z": 1393.0 + }, + "T_26 bit": { + "Xs": 3, + "Xstar": 0.007428231018537684, + "a_dim_abs": 0.0003333773974749822, + "a_dim_rel": 0.00045483199889902295, + "a_read_abs": 0.0001026889042022319, + "a_read_rel": 0.00013905065899780917, + "a_same_abs": 0.0008651482519676222, + "a_same_rel": 0.0012012355585059723, + "cst": 650, + "h": 26, + "rho_star_abs": 0.6150653007352958, + "rho_star_rel": 2.040078298159642, + "v": 2600, + "x": 0.25, + "z": 759.0 + }, + "T_26 complex": { + "Xs": 3, + "Xstar": 0.007249283806134319, + "a_dim_abs": 0.00011029490561668976, + "a_dim_rel": 0.0001447741189583302, + "a_read_abs": 3.624882766282049e-05, + "a_read_rel": 4.751118139800312e-05, + "a_same_abs": 0.0008922422561732215, + "a_same_rel": 0.0011895040631954342, + "cst": 676, + "h": 26, + "rho_star_abs": 1.3607824246719247, + "rho_star_rel": 2.7717935470718658, + "v": 2600, + "x": 0.26, + "z": 1840.0 + }, + "T_28 bit": { + "Xs": 3, + "Xstar": 0.0072076119088542445, + "a_dim_abs": 0.00028944394817983555, + "a_dim_rel": 0.000402115662153537, + "a_read_abs": 8.211037410852214e-05, + "a_read_rel": 0.00011325612669387032, + "a_same_abs": 0.0008157057167079721, + "a_same_rel": 0.0011543833755217059, + "cst": 756, + "h": 28, + "rho_star_abs": 0.49403276352677494, + "rho_star_rel": 1.8931100377812982, + "v": 3276, + "x": 0.23076923076923078, + "z": 900.0 + }, + "T_28 complex": { + "Xs": 3, + "Xstar": 0.007065766446158325, + "a_dim_abs": 8.774277335209955e-05, + "a_dim_rel": 0.00011741486045123343, + "a_read_abs": 2.6606929081828828e-05, + "a_read_rel": 3.5559762278825425e-05, + "a_same_abs": 0.0008435227641570476, + "a_same_rel": 0.0011466627081791326, + "cst": 784, + "h": 28, + "rho_star_abs": 1.1542831762199057, + "rho_star_rel": 2.604334763236693, + "v": 3276, + "x": 0.23931623931623933, + "z": 2375.0 + }, + "T_30 bit": { + "Xs": 3, + "Xstar": 0.006974330753293903, + "a_dim_abs": 0.00025316737186053704, + "a_dim_rel": 0.00035750594180493134, + "a_read_abs": 6.654007199535274e-05, + "a_read_rel": 9.331882840159467e-05, + "a_same_abs": 0.000770036303954073, + "a_same_rel": 0.001108605043605795, + "cst": 870, + "h": 30, + "rho_star_abs": 0.3769432929402683, + "rho_star_rel": 1.6355415471480228, + "v": 4060, + "x": 0.21428571428571427, + "z": 1053.0 + }, + "T_30 complex": { + "Xs": 3, + "Xstar": 0.0068600041595340226, + "a_dim_abs": 7.07404153900592e-05, + "a_dim_rel": 9.631573881733644e-05, + "a_read_abs": 1.991384474990046e-05, + "a_read_rel": 2.708365352329143e-05, + "a_same_abs": 0.0007979115595605061, + "a_same_rel": 0.0011037594235652427, + "cst": 900, + "h": 30, + "rho_star_abs": 0.9186809969833121, + "rho_star_rel": 2.43670740055677, + "v": 4060, + "x": 0.22167487684729065, + "z": 3006.0 + } + }, + "triple": { + "bit|10": { + "X_att": { + "hi": "776618700905786288335441724939300911501049/500000000000000000000000000000000000000000000", + "s": 9 + }, + "X_thmC": { + "hi": "776618700905786288335441724939300911501049/500000000000000000000000000000000000000000000", + "s": 9 + }, + "c0": 10, + "c_centre": 10, + "clique_lb": 8, + "cst_att": 90, + "cst_thmC": 90, + "lab_min": 9, + "minrank": 10, + "span": 10, + "v": 120, + "window": 10, + "z": 63 + }, + "bit|11": { + "X_att": { + "hi": "9729566759941177950254953543662850279361/4000000000000000000000000000000000000000000", + "s": 5 + }, + "X_thmC": { + "hi": "9729566759941177950254953543662850279361/4000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": 11, + "c_centre": 11, + "clique_lb": 8, + "cst_att": 110, + "cst_thmC": 110, + "lab_min": 10, + "minrank": 11, + "span": 11, + "v": 165, + "window": 11, + "z": 84 + }, + "bit|12": { + "X_att": { + "hi": "1664097808960422102478315942164499574573709/500000000000000000000000000000000000000000000", + "s": 5 + }, + "X_thmC": { + "hi": "1664097808960422102478315942164499574573709/500000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": 12, + "c_centre": 12, + "clique_lb": 12, + "cst_att": 132, + "cst_thmC": 132, + "lab_min": null, + "span": 12, + "v": 220, + "window": 12, + "z": 108 + }, + "bit|13": { + "X_att": { + "hi": "876351972029380636708406626667056116275489/200000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "876351972029380636708406626667056116275489/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 13, + "c_centre": 13, + "clique_lb": 12, + "cst_att": 156, + "cst_thmC": 156, + "lab_min": null, + "span": 13, + "v": 286, + "window": 13, + "z": 135 + }, + "bit|14": { + "X_att": { + "hi": "2792620771897282852189170476719937141751869/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "2792620771897282852189170476719937141751869/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 14, + "c_centre": 14, + "clique_lb": 12, + "cst_att": 182, + "cst_thmC": 182, + "lab_min": null, + "span": 14, + "v": 364, + "window": 14, + "z": 165 + }, + "bit|15": { + "X_att": { + "hi": "6406069772985404330023495589330195798538623/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "6406069772985404330023495589330195798538623/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 15, + "c_centre": 15, + "clique_lb": 12, + "cst_att": 210, + "cst_thmC": 210, + "lab_min": null, + "span": 15, + "v": 455, + "window": 15, + "z": 198 + }, + "bit|16": { + "X_att": { + "hi": "1392229029062399051075438194402522640302777/200000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1392229029062399051075438194402522640302777/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 16, + "c_centre": 16, + "clique_lb": 16, + "cst_att": 240, + "cst_thmC": 240, + "lab_min": null, + "span": 16, + "v": 560, + "window": 16, + "z": 234 + }, + "bit|17": { + "X_att": { + "hi": "1465693281711998724459178333745512645691633/200000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1465693281711998724459178333745512645691633/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 17, + "c_centre": 17, + "clique_lb": 16, + "cst_att": 272, + "cst_thmC": 272, + "lab_min": null, + "span": 17, + "v": 680, + "window": 17, + "z": 273 + }, + "bit|18": { + "X_att": { + "hi": "7561304679511505685840511148398945806547997/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7561304679511505685840511148398945806547997/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 18, + "c_centre": 18, + "clique_lb": 16, + "cst_att": 306, + "cst_thmC": 306, + "lab_min": null, + "span": 18, + "v": 816, + "window": 18, + "z": 315 + }, + "bit|19": { + "X_att": { + "hi": "3848388541781380504935849984349260921590029/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3848388541781380504935849984349260921590029/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 19, + "c_centre": 19, + "clique_lb": 16, + "cst_att": 342, + "cst_thmC": 342, + "lab_min": null, + "span": 19, + "v": 969, + "window": 19, + "z": 360 + }, + "bit|20": { + "X_att": { + "hi": "7761193984475488541474597070646268548443911/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7761193984475488541474597070646268548443911/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 20, + "c_centre": 20, + "clique_lb": 20, + "cst_att": 380, + "cst_thmC": 380, + "lab_min": null, + "span": 20, + "v": 1140, + "window": 20, + "z": 408 + }, + "bit|21": { + "X_att": { + "hi": "3886742179208512283986043378547492466168477/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3886742179208512283986043378547492466168477/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 21, + "c_centre": 21, + "clique_lb": 20, + "cst_att": 420, + "cst_thmC": 420, + "lab_min": null, + "span": 21, + "v": 1330, + "window": 21, + "z": 459 + }, + "bit|22": { + "X_att": { + "hi": "968430292646616679760190882220172170491423/125000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "968430292646616679760190882220172170491423/125000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 22, + "c_centre": 22, + "clique_lb": 20, + "cst_att": 462, + "cst_thmC": 462, + "lab_min": null, + "span": 22, + "v": 1540, + "window": 22, + "z": 513 + }, + "bit|23": { + "X_att": { + "hi": "7693231728662683170660852027822329546324907/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7693231728662683170660852027822329546324907/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 23, + "c_centre": 23, + "clique_lb": 20, + "cst_att": 506, + "cst_thmC": 506, + "lab_min": null, + "span": 23, + "v": 1771, + "window": 23, + "z": 570 + }, + "bit|24": { + "X_att": { + "hi": "76184124757188113191697098613985872361809/10000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "76184124757188113191697098613985872361809/10000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 24, + "c_centre": 24, + "clique_lb": 24, + "cst_att": 552, + "cst_thmC": 552, + "lab_min": null, + "span": 24, + "v": 2024, + "window": 24, + "z": 630 + }, + "bit|25": { + "X_att": { + "hi": "376432642736300557446960102225830269162543/50000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "376432642736300557446960102225830269162543/50000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 25, + "c_centre": 25, + "clique_lb": 24, + "cst_att": 600, + "cst_thmC": 600, + "lab_min": null, + "span": 25, + "v": 2300, + "window": 25, + "z": 693 + }, + "bit|26": { + "X_att": { + "hi": "7428231018537683957858597239010123303160069/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7428231018537683957858597239010123303160069/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 26, + "c_centre": 26, + "clique_lb": 24, + "cst_att": 650, + "cst_thmC": 650, + "lab_min": null, + "span": 26, + "v": 2600, + "window": 26, + "z": 759 + }, + "bit|27": { + "X_att": { + "hi": "3660196538785299904400494241940577921923249/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3660196538785299904400494241940577921923249/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 27, + "c_centre": 27, + "clique_lb": 24, + "cst_att": 702, + "cst_thmC": 702, + "lab_min": null, + "span": 27, + "v": 2925, + "window": 27, + "z": 828 + }, + "bit|28": { + "X_att": { + "hi": "7207611908854244520128418116655666381120681/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7207611908854244520128418116655666381120681/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 28, + "c_centre": 28, + "clique_lb": 28, + "cst_att": 756, + "cst_thmC": 756, + "lab_min": null, + "span": 28, + "v": 3276, + "window": 28, + "z": 900 + }, + "bit|29": { + "X_att": { + "hi": "1772944080991738203222085612509317797957919/250000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1772944080991738203222085612509317797957919/250000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 29, + "c_centre": 29, + "clique_lb": 28, + "cst_att": 812, + "cst_thmC": 812, + "lab_min": null, + "span": 29, + "v": 3654, + "window": 29, + "z": 975 + }, + "bit|30": { + "X_att": { + "hi": "3487165376646951661307460845762307144468649/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3487165376646951661307460845762307144468649/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 30, + "c_centre": 30, + "clique_lb": 28, + "cst_att": 870, + "cst_thmC": 870, + "lab_min": null, + "span": 30, + "v": 4060, + "window": 30, + "z": 1053 + }, + "bit|31": { + "X_att": { + "hi": "6856379355605141806551117156232066918164489/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "6856379355605141806551117156232066918164489/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 31, + "c_centre": 31, + "clique_lb": 28, + "cst_att": 930, + "cst_thmC": 930, + "lab_min": null, + "span": 31, + "v": 4495, + "window": 31, + "z": 1134 + }, + "bit|32": { + "X_att": { + "hi": "1684691068896542322057274798652315439539961/250000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1684691068896542322057274798652315439539961/250000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": 32, + "c_centre": 32, + "clique_lb": 32, + "cst_att": 992, + "cst_thmC": 992, + "lab_min": null, + "span": 32, + "v": 4960, + "window": 32, + "z": 1218 + }, + "bit|6": { + "X_att": null, + "X_thmC": null, + "c0": 6, + "c_centre": 6, + "clique_lb": 4, + "cst_att": 30, + "cst_thmC": 30, + "lab_min": 5, + "minrank": 6, + "span": 6, + "v": 20, + "window": 6, + "z": 9 + }, + "bit|7": { + "X_att": null, + "X_thmC": null, + "c0": 7, + "c_centre": 7, + "clique_lb": 4, + "cst_att": 42, + "cst_thmC": 42, + "lab_min": 6, + "minrank": 6, + "span": 7, + "v": 35, + "window": 7, + "z": 18 + }, + "bit|8": { + "X_att": null, + "X_thmC": null, + "c0": 8, + "c_centre": 8, + "clique_lb": 8, + "cst_att": 56, + "cst_thmC": 56, + "lab_min": 7, + "minrank": 8, + "span": 8, + "v": 56, + "window": 8, + "z": 30 + }, + "bit|9": { + "X_att": { + "hi": "90571224493574240162196975489905526046641/125000000000000000000000000000000000000000000", + "s": 15 + }, + "X_thmC": { + "hi": "90571224493574240162196975489905526046641/125000000000000000000000000000000000000000000", + "s": 15 + }, + "c0": 9, + "c_centre": 9, + "clique_lb": 8, + "cst_att": 72, + "cst_thmC": 72, + "lab_min": 8, + "minrank": 9, + "span": 9, + "v": 84, + "window": 8, + "z": 45 + }, + "complex|10": { + "X_att": { + "hi": "46529923804317346175407621444719552528113/62500000000000000000000000000000000000000000", + "s": 13 + }, + "X_thmC": { + "hi": "46529923804317346175407621444719552528113/62500000000000000000000000000000000000000000", + "s": 13 + }, + "c0": null, + "c_centre": 11, + "clique_lb": null, + "cst_att": 100, + "cst_thmC": 100, + "lab_min": null, + "span": 10, + "v": 120, + "window": 10, + "z": 56 + }, + "complex|11": { + "X_att": { + "hi": "1569351408934127525135471259432051738258449/1000000000000000000000000000000000000000000000", + "s": 9 + }, + "X_thmC": { + "hi": "9729566759941177950254953543662850279361/4000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": null, + "c_centre": 12, + "clique_lb": null, + "cst_att": 121, + "cst_thmC": 110, + "lab_min": null, + "span": 11, + "v": 165, + "window": 11, + "z": 80 + }, + "complex|12": { + "X_att": { + "hi": "1209343534841470684318420758529555314453317/500000000000000000000000000000000000000000000", + "s": 5 + }, + "X_thmC": { + "hi": "1209343534841470684318420758529555314453317/500000000000000000000000000000000000000000000", + "s": 5 + }, + "c0": null, + "c_centre": 13, + "clique_lb": null, + "cst_att": 144, + "cst_thmC": 144, + "lab_min": null, + "span": 12, + "v": 220, + "window": 12, + "z": 111 + }, + "complex|13": { + "X_att": { + "hi": "199769727269112531541778166044309728022199/62500000000000000000000000000000000000000000", + "s": 5 + }, + "X_thmC": { + "hi": "876351972029380636708406626667056116275489/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 14, + "clique_lb": null, + "cst_att": 169, + "cst_thmC": 156, + "lab_min": null, + "span": 13, + "v": 286, + "window": 13, + "z": 150 + }, + "complex|14": { + "X_att": { + "hi": "858726271330314573637654262938667670823633/200000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "858726271330314573637654262938667670823633/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 15, + "clique_lb": null, + "cst_att": 196, + "cst_thmC": 196, + "lab_min": null, + "span": 14, + "v": 364, + "window": 14, + "z": 198 + }, + "complex|15": { + "X_att": { + "hi": "2686895853961696186903984617799778789049013/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "6406069772985404330023495589330195798538623/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 16, + "clique_lb": null, + "cst_att": 225, + "cst_thmC": 210, + "lab_min": null, + "span": 15, + "v": 455, + "window": 15, + "z": 256 + }, + "complex|16": { + "X_att": { + "hi": "612326379285441438349302245569560909643769/100000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "612326379285441438349302245569560909643769/100000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 17, + "clique_lb": null, + "cst_att": 256, + "cst_thmC": 256, + "lab_min": null, + "span": 16, + "v": 560, + "window": 16, + "z": 325 + }, + "complex|17": { + "X_att": { + "hi": "1659784821813878644657103134107956066145561/250000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1465693281711998724459178333745512645691633/200000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 18, + "clique_lb": null, + "cst_att": 289, + "cst_thmC": 272, + "lab_min": null, + "span": 17, + "v": 680, + "window": 17, + "z": 406 + }, + "complex|18": { + "X_att": { + "hi": "3493714774730533826480050052509795932564883/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3493714774730533826480050052509795932564883/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 19, + "clique_lb": null, + "cst_att": 324, + "cst_thmC": 324, + "lab_min": null, + "span": 18, + "v": 816, + "window": 18, + "z": 500 + }, + "complex|19": { + "X_att": { + "hi": "7213972525897024631880150025153852766379711/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3848388541781380504935849984349260921590029/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 20, + "clique_lb": null, + "cst_att": 361, + "cst_thmC": 342, + "lab_min": null, + "span": 19, + "v": 969, + "window": 19, + "z": 608 + }, + "complex|20": { + "X_att": { + "hi": "7351183769407273419643766843023513501975683/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7351183769407273419643766843023513501975683/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 21, + "clique_lb": null, + "cst_att": 400, + "cst_thmC": 400, + "lab_min": null, + "span": 20, + "v": 1140, + "window": 20, + "z": 731 + }, + "complex|21": { + "X_att": { + "hi": "371117624979200571438964217918510257732123/50000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3886742179208512283986043378547492466168477/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 22, + "clique_lb": null, + "cst_att": 441, + "cst_thmC": 420, + "lab_min": null, + "span": 21, + "v": 1330, + "window": 21, + "z": 870 + }, + "complex|22": { + "X_att": { + "hi": "3722221478697248456951607309406426793429999/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3722221478697248456951607309406426793429999/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 23, + "clique_lb": null, + "cst_att": 484, + "cst_thmC": 484, + "lab_min": null, + "span": 22, + "v": 1540, + "window": 22, + "z": 1026 + }, + "complex|23": { + "X_att": { + "hi": "1857490263160947621505492133309189739520661/250000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7693231728662683170660852027822329546324907/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 24, + "clique_lb": null, + "cst_att": 529, + "cst_thmC": 506, + "lab_min": null, + "span": 23, + "v": 1771, + "window": 23, + "z": 1200 + }, + "complex|24": { + "X_att": { + "hi": "3694111211137556145811533525602499139495193/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3694111211137556145811533525602499139495193/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 25, + "clique_lb": null, + "cst_att": 576, + "cst_thmC": 576, + "lab_min": null, + "span": 24, + "v": 2024, + "window": 24, + "z": 1393 + }, + "complex|25": { + "X_att": { + "hi": "7326228893872904681522362224654898454900829/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "376432642736300557446960102225830269162543/50000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 26, + "clique_lb": null, + "cst_att": 625, + "cst_thmC": 600, + "lab_min": null, + "span": 25, + "v": 2300, + "window": 25, + "z": 1606 + }, + "complex|26": { + "X_att": { + "hi": "7249283806134318615399525498332877759821711/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "7249283806134318615399525498332877759821711/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 27, + "clique_lb": null, + "cst_att": 676, + "cst_thmC": 676, + "lab_min": null, + "span": 26, + "v": 2600, + "window": 26, + "z": 1840 + }, + "complex|27": { + "X_att": { + "hi": "7161430130975224035011494905234030738938597/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3660196538785299904400494241940577921923249/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 28, + "clique_lb": null, + "cst_att": 729, + "cst_thmC": 702, + "lab_min": null, + "span": 27, + "v": 2925, + "window": 27, + "z": 2096 + }, + "complex|28": { + "X_att": { + "hi": "3532883223079162464397295906337603810243307/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3532883223079162464397295906337603810243307/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 29, + "clique_lb": null, + "cst_att": 784, + "cst_thmC": 784, + "lab_min": null, + "span": 28, + "v": 3276, + "window": 28, + "z": 2375 + }, + "complex|29": { + "X_att": { + "hi": "6964677485548873524245916399877387448213993/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "1772944080991738203222085612509317797957919/250000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 30, + "clique_lb": null, + "cst_att": 841, + "cst_thmC": 812, + "lab_min": null, + "span": 29, + "v": 3654, + "window": 29, + "z": 2678 + }, + "complex|30": { + "X_att": { + "hi": "6860004159534022552535681427343661198392509/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "6860004159534022552535681427343661198392509/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 31, + "clique_lb": null, + "cst_att": 900, + "cst_thmC": 900, + "lab_min": null, + "span": 30, + "v": 4060, + "window": 30, + "z": 3006 + }, + "complex|31": { + "X_att": { + "hi": "6753170162846582752491642054337717127054927/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "6856379355605141806551117156232066918164489/1000000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 32, + "clique_lb": null, + "cst_att": 961, + "cst_thmC": 930, + "lab_min": null, + "span": 31, + "v": 4495, + "window": 31, + "z": 3360 + }, + "complex|32": { + "X_att": { + "hi": "3322638549104997085850699001241537189343943/500000000000000000000000000000000000000000000", + "s": 3 + }, + "X_thmC": { + "hi": "3322638549104997085850699001241537189343943/500000000000000000000000000000000000000000000", + "s": 3 + }, + "c0": null, + "c_centre": 33, + "clique_lb": null, + "cst_att": 1024, + "cst_thmC": 1024, + "lab_min": null, + "span": 32, + "v": 4960, + "window": 32, + "z": 3741 + }, + "complex|6": { + "X_att": null, + "X_thmC": null, + "c0": null, + "c_centre": 7, + "clique_lb": null, + "cst_att": 36, + "cst_thmC": 36, + "lab_min": null, + "span": 6, + "v": 20, + "window": 6, + "z": 10 + }, + "complex|7": { + "X_att": null, + "X_thmC": null, + "c0": null, + "c_centre": 8, + "clique_lb": null, + "cst_att": 49, + "cst_thmC": 42, + "lab_min": null, + "span": 7, + "v": 35, + "window": 7, + "z": 16 + }, + "complex|8": { + "X_att": null, + "X_thmC": null, + "c0": null, + "c_centre": 9, + "clique_lb": null, + "cst_att": 64, + "cst_thmC": 64, + "lab_min": null, + "span": 8, + "v": 56, + "window": 8, + "z": 25 + }, + "complex|9": { + "X_att": null, + "X_thmC": { + "hi": "317912890061411348291420253531214257236569/500000000000000000000000000000000000000000000", + "s": 15 + }, + "c0": null, + "c_centre": 10, + "clique_lb": null, + "cst_att": 81, + "cst_thmC": 72, + "lab_min": null, + "span": 9, + "v": 84, + "window": 9, + "z": 38 + } + } +} \ No newline at end of file diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/ade_screen_fixed.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/ade_screen_fixed.py new file mode 100644 index 0000000000..f9a9bc338b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/ade_screen_fixed.py @@ -0,0 +1,131 @@ +"""Corrected ADE unit screen (replaces #620's ade_screen.py; erratum found by #630). + +Per unit (real root lines, window h = Gram rank, k = 1): + v, h exact + c0 = rank_F2(J + G) the label decoder B0 used by #620 (an UPPER bound on the F2 minrank of the pattern) + omega largest pairwise non-orthogonal set (identity block of the pattern) -> minrank >= omega + c = minrank exact F2 minrank of the pattern (asymmetric decoders allowed, SAT; c-1 UNSAT, c SAT) + x_proxy = c*h/v #620's full-rank-totals premise (each total charged the window), now with the exact minrank + cst_lab star cost attainable by label decoders on the B0 labels: min over bases {p_j} of F2^c0 of + sum_j dim_Q span{lines T : u_T . p_j = 1} (matroid greedy, exact); an UPPER bound on cst_min + X* certified ceiling of #595 (skel.X, rational intervals) maximised over odd s in 3..15 + +What changed against #620: + * E6 was built as the complement of two ORTHOGONAL roots (= D6, 30 lines); it is now the complement of an A2 (36 lines). + * the column 'c' was rank_F2(J+G) of one decoder; it is now the exact minrank (both are printed). + * an attainable label-decoder star cost is added next to the proxy. +Usage: python ade_screen_fixed.py REPO_ROOT [--json out.json] +""" +import json +import sys +import time +from fractions import Fraction as Fr + +import numpy as np + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X, profile # noqa: E402 (#595, certified ceiling) +from cert import cost_iv # noqa: E402 +from minrank_sat import solve_asym # noqa: E402 (#595 SAT encoding, GL(c) gauge on a clique) +from e8units import sym_factor_f2 # noqa: E402 + +import units as U # noqa: E402 + +S_RANGE = range(3, 17, 2) +TIMEOUT = 600 # seconds per SAT query (A8, D9 hit it: minrank left as a range) + + +def xstar(h, x): + best = None + for s in S_RANGE: + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi'], r['lo']) + return best + + +def minrank(G, omega_set, c0): + lo, hi, log = U.minrank_f2(G, omega_set, c0, solve_asym, timeout=TIMEOUT) + return (lo if lo == hi else '%d..%d' % (lo, hi)), log, lo + + +def label_cost(L, G): + B0 = (1 + G) % 2 + Uf = sym_factor_f2(B0).astype(np.int64) + c0 = Uf.shape[1] + costs = [] + for p in range(1, 1 << c0): + pv = np.array([(p >> j) & 1 for j in range(c0)]) + R = np.nonzero((Uf @ pv) % 2)[0] + costs.append((U.rank_q(L[R].tolist()), p)) + costs.sort() + basis, tot = {}, 0 + for cst, p in costs: + x = p + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + if x: + tot += cst + if len(basis) == c0: + break + return tot, c0 + + +def unit(name, L, G=None): + if G is None: + G = U.gram_doubled(L) + v = len(L) + h = U.rank_q(G.tolist()) + c0 = U.f2_rank((1 + G) % 2) + om = U.nonorth_clique(G) + c, log, clo = minrank(G, om, c0) + cst_lab, _ = label_cost(L, G) + xp = Fr(clo * h, v) # proxy at the certified lower end of the minrank + xl = Fr(cst_lab, v) + row = dict(unit=name, v=v, h=h, c0=c0, omega=len(om), minrank=c, sat_log={k: v_ for k, v_ in log.items()}, + x_proxy=str(xp), cst_lab=cst_lab, x_lab=str(xl)) + for key, x in (('proxy', xp), ('lab', xl)): + b = xstar(h, x) + row['Xstar_' + key] = None if b is None else dict(s=b[0], hi=str(b[1]), hi_float=float(b[1])) + if row['Xstar_lab'] is None and xl < 1: + # positive deficit only at s > 1/(1 - x): bound the ceiling by D_s / Cost_s (#595 A.2) for odd s up to 801 + best = None + for s_ in range(3, 802, 2): + prof, m, D = profile(s_, h, 1, xl) + if D <= 0: + continue + b = D / cost_iv(prof, m)[0] + if best is None or b > best[1]: + best = (s_, b) + row['DoverCost_lab'] = dict(s=best[0], hi=str(best[1]), hi_float=float(best[1])) + print(' %s: x_lab < 1, deficit > 0 only for s > %s; max over odd s <= 801 of D_s/Cost_s = %.4e at s = %d' + % (name, 1 / (1 - xl), float(best[1]), best[0])) + return row + + +def main(): + rows = [] + cases = [('A%d' % n, U.roots_A(n)) for n in (4, 5, 6, 7, 8)] + cases += [('D%d' % n, U.roots_D(n)) for n in (4, 5, 6, 7, 8, 9, 10, 12)] + cases += [('D6(archived "E6")', U.roots_D6_archived()), ('E6', U.roots_E6()), ('E7', U.roots_E7()), ('E8', U.roots_E8())] + for name, L in cases: + t = time.time() + r = unit(name, L) + rows.append(r) + f = lambda k: ('%.4e@s%d' % (r[k]['hi_float'], r[k]['s'])) if r[k] else 'D<=0' + print('%-18s v=%4d h=%2d c0=%2d omega=%2d minrank=%-5s x_proxy=%-7s cst_lab=%4d x_lab=%-8s X*proxy=%-14s X*lab=%-14s (%.0fs)' + % (name, r['v'], r['h'], r['c0'], r['omega'], r['minrank'], r['x_proxy'], r['cst_lab'], r['x_lab'], + f('Xstar_proxy'), f('Xstar_lab'), time.time() - t), flush=True) + if '--json' in sys.argv: + json.dump(rows, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/check.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/check.py new file mode 100644 index 0000000000..ee0706f08d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/check.py @@ -0,0 +1,104 @@ +"""Compare a fresh run (OUT/*.json) with the shipped expected values (expected.json); one PASS/FAIL line per claim. + +Exact fields (integers, fractions, certified interval ends) must be equal. Model estimates (rank.json) are compared +with relative tolerance 1e-9. A minrank that the shipped run left as a range 'lo..hi' (SAT timeout) is accepted when +the fresh value lies in that range. +Usage: python check.py OUT expected.json (python check.py --freeze OUT expected.json writes it) +""" +import json +import pathlib +import sys + +FIELDS = { + 'ade': (['unit'], ['v', 'h', 'c0', 'omega', 'cst_lab', 'x_lab', 'Xstar_lab', 'DoverCost_lab']), + 'triple': (['side', 'h'], ['v', 'window', 'span', 'c_centre', 'c0', 'clique_lb', 'z', 'cst_att', 'cst_thmC', + 'X_att', 'X_thmC', 'lab_min']), + 'lorentz': (['n'], ['v', 'window', 'c0', 'omega', 'z', 'cst_lab', 'per_total', 'x', 'Xstar']), + 'cover': (['unit', 'source'], ['readers', 'rank_readers', 'h', 'cover_lb', 'cover_ub', 'best_section_found']), + 'lorentz_minrank': (['n'], ['omega', 'c0', 'lo', 'hi', 'log']), + 'rank': (['name'], ['h', 'v', 'cst', 'x', 'z', 'Xstar', 'Xs', 'rho_star_abs', 'rho_star_rel', 'a_same_abs', + 'a_same_rel', 'a_dim_abs', 'a_dim_rel', 'a_read_abs', 'a_read_rel']), +} +TOL = {'rank'} + + +def strip(row, fields): + out = {} + for f in fields: + v = row.get(f) + if isinstance(v, dict) and 'hi' in v: + v = {k: v[k] for k in ('s', 'hi') if k in v} + out[f] = v + return out + + +def load(out): + d = {} + for name in FIELDS: + p = pathlib.Path(out) / (name + '.json') + if p.exists(): + d[name] = json.loads(p.read_text()) + return d + + +def key(row, kf): + return '|'.join(str(row.get(k)) for k in kf) + + +def freeze(out, exp): + d = load(out) + E = {} + for name, rows in d.items(): + kf, ff = FIELDS[name] + E[name] = {key(r, kf): dict(strip(r, ff), **({'minrank': r['minrank']} if 'minrank' in r else {})) for r in rows} + pathlib.Path(exp).write_text(json.dumps(E, indent=1, sort_keys=True, default=str)) + + +def close(a, b, tol): + if isinstance(a, float) and isinstance(b, (int, float)) and tol: + return abs(a - b) <= 1e-9 * max(1.0, abs(b)) if abs(b) < 1 else abs(a / b - 1) <= 1e-9 + if isinstance(a, dict) and isinstance(b, dict): + return a.keys() == b.keys() and all(close(a[k], b[k], tol) for k in a) + if isinstance(a, list) and isinstance(b, list): + return len(a) == len(b) and all(close(x, y, tol) for x, y in zip(a, b)) + return a == b + + +def minrank_ok(fresh, exp): + if isinstance(exp, str) and '..' in exp: + lo, hi = map(int, exp.split('..')) + if isinstance(fresh, str): + flo, fhi = map(int, fresh.split('..')) + return lo <= flo and fhi <= hi + return lo <= int(fresh) <= hi + return fresh == exp + + +def check(out, exp): + E = json.loads(pathlib.Path(exp).read_text()) + d = load(out) + bad = 0 + for name, rows in E.items(): + if name not in d: + print('FAIL %s missing' % name); bad += 1; continue + kf, ff = FIELDS[name] + fresh = {key(r, kf): r for r in d[name]} + for k, ev in rows.items(): + r = fresh.get(k) + ok = r is not None + if ok: + sr = strip(r, ff) + ok = all(close(sr[f], ev[f], name in TOL) for f in ff) + if 'minrank' in ev: + ok = ok and minrank_ok(r.get('minrank'), ev['minrank']) + print(('PASS ' if ok else 'FAIL ') + '%s %s' % (name, k), flush=True) + bad += not ok + print('%d FAIL' % bad) + return bad + + +if __name__ == '__main__': + if sys.argv[1] == '--freeze': + freeze(sys.argv[2], sys.argv[3]) + else: + sys.exit(1 if check(sys.argv[1], sys.argv[2]) else 0) diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/cover.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/cover.py new file mode 100644 index 0000000000..041baf1e42 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/cover.py @@ -0,0 +1,147 @@ +"""#595 section D cover numbers for candidate units (source end; the terminal end is the same geometry for these +self-dual line systems). + +For a source s, the readers T(s) are the lines orthogonal to s. A register crossing from into serves the +readers in w^perp, a 'hyperplane section' of T(s). The cover number is the least number of proper sections covering T(s). + * lower bound 2 <=> rank_Q T(s) = h - 1 (then no single proper section covers T(s)); exact + * upper bound 2 <=> two subsets of T(s), each of rank <= h - 2, with union T(s) (certificate); exact check + (the witnesses are found by sampling hyperplanes through h - 2 random readers plus structured functionals; + a found pair is re-checked exactly) + * the best single section found (a LOWER bound on the largest section; sampled, not exhaustive) +#595's single-cut floors (d_geometry.single_cut_bounds): sink-only R >= (c - 1) v - floor(v / leftover), strict +R >= max(c v - 2 v, 0). With cover number 2 the strict floor is 0 and the sink-only floor is at most v. +Control: E8 bit (cover number 3, best pair union 52 of 63: #595 D1). +Usage: python cover.py REPO_ROOT unit[,unit...] [samples] [--json out] unit in {E8, T, L} +""" +import itertools +import sys + +import numpy as np + +sys.dont_write_bytecode = True +import units as U # noqa: E402 + + +def load(name): + if name == 'E8': + L = U.roots_E8(); G = U.gram_doubled(L); return L, G, [0] + if name.startswith('T'): + L, G, _ = U.triple_family(int(name[1:])); return L, G, [0] + if name.startswith('L'): + n = int(name[1:]) + L, G, _ = U.lorentz_family(n) + from math import comb + return L, G, [0, comb(n, 3)] # one triple-type and one pair-type source + raise ValueError(name) + + +def sections_for(L, G, s, h, samples, rng): + T = np.nonzero(G[s] == 0)[0] + T = T[T != s] + X = L[T].astype(float) + rT = U.rank_q(L[T].tolist()) + secs = set() + # structured functionals: coordinate patterns c with c . t = 0 + n = L.shape[1] + cands = [] + for i in range(n): + e = np.zeros(n); e[i] = 1; cands.append(e) + for i, j in itertools.combinations(range(n), 2): + for sg in (1, -1): + e = np.zeros(n); e[i] = 1; e[j] = sg; cands.append(e) + for i, j, k in itertools.combinations(range(n), 3): + for sg in itertools.product((1, -1), repeat=2): + e = np.zeros(n); e[i] = 1; e[j] = sg[0]; e[k] = sg[1]; cands.append(e) + for i, j in itertools.combinations(range(n), 2): # triple family: 1 off {i, j}, -2 on {i, j} (two-class sections) + e = np.ones(n); e[i] = e[j] = -2; cands.append(e) + for i in range(n - 1): + for wgt in (1, 2, 3): + e = np.ones(n); e[-1] = wgt; e[i] -= 1; cands.append(e) + e = np.ones(n); e[-1] = wgt; e[i] += 1; cands.append(e) + for c in cands: + m = np.abs(X @ c) < 1e-9 + if 0 < m.sum() < len(T): + secs.add(tuple(np.nonzero(m)[0])) + # random hyperplanes of span(T) through h - 2 readers + for _ in range(samples): + idx = rng.choice(len(T), rT - 1, replace=False) + M = X[idx] + if np.linalg.matrix_rank(M) < rT - 1: + continue + # normal of the hyperplane inside span(T): project onto span(T) via SVD of X + _, _, Vt = np.linalg.svd(np.vstack([M, np.zeros((0, n))])) + null = Vt[rT - 1:] # functionals vanishing on M + # choose a functional that does not vanish on all of T + best = None + for c in null: + m = np.abs(X @ c) < 1e-7 + if m.sum() < len(T): + best = m; break + if best is not None: + secs.add(tuple(np.nonzero(best)[0])) + return T, rT, secs + + +def run(name, samples=20000, seed=1): + L, G, srcs = load(name) + h = U.rank_q(G.tolist()) + rng = np.random.default_rng(seed) + out = [] + for s in srcs: + T, rT, secs = sections_for(L, G, s, h, samples, rng) + # exact check of every kept section: rank <= h - 2 (a proper section) + secs = [S for S in secs if U.rank_q(L[T[list(S)]].tolist()) <= h - 2] + masks = [sum(1 << int(k) for k in S) for S in secs] + full = (1 << len(T)) - 1 + best1 = max(len(S) for S in secs) + order = sorted(range(len(masks)), key=lambda i: -len(secs[i])) + bestpair, cover2 = 0, None + for a in range(len(order)): + ma = masks[order[a]] + for b in range(a, len(order)): + u = bin(ma | masks[order[b]]).count('1') + if u > bestpair: + bestpair = u + if u == len(T): + cover2 = (secs[order[a]], secs[order[b]]) + break + if cover2: + break + lb = 2 if rT == h - 1 else 1 + res = dict(unit=name, source=int(s), readers=len(T), rank_readers=rT, h=h, cover_lb=lb, + cover_ub=2 if cover2 else None, best_section_found=best1, best_pair_union=bestpair, + n_sections=len(secs)) + if not cover2: # look for a 3-cover among the largest sections + top = order[:400] + found = None + for a in range(len(top)): + for b in range(a, len(top)): + rest = full & ~(masks[top[a]] | masks[top[b]]) + for c in range(len(masks)): + if rest & ~masks[c] == 0: + found = (secs[top[a]], secs[top[b]], secs[c]); break + if found: + break + if found: + break + if found: + assert set().union(*found) == set(range(len(T))) + res['cover_ub'] = 3 + res['cover3_sizes'] = tuple(len(x) for x in found) + if cover2: + A, B = cover2 + assert U.rank_q(L[T[list(A)]].tolist()) <= h - 2 and U.rank_q(L[T[list(B)]].tolist()) <= h - 2 + assert set(A) | set(B) == set(range(len(T))) + res['cover2_sizes'] = (len(A), len(B)) + print(res, flush=True) + out.append(res) + return out + + +if __name__ == '__main__': + import json + allres = [] + for u in sys.argv[2].split(','): + allres += run(u, int(sys.argv[3]) if len(sys.argv) > 3 else 20000) + if '--json' in sys.argv: + json.dump(allres, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/est_model.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/est_model.py new file mode 100644 index 0000000000..2ba54d9815 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/est_model.py @@ -0,0 +1,135 @@ +"""Helper-adjusted estimate of the achievable saving a for a unit family (MODEL, not a bound). + +The #595 ceiling X(s, h, x) is helper-free. On the realised E8 words the gap to the ceiling is 87-93% helper cost +(#595 A.5). This model puts the helpers back with the piece statistics of the shipped words and solves the same +pressure root, so that unit families can be compared at the same design maturity: + + per port and replica (v = 1), stage count s = 2K + 1, a = h - 1, m = 1 + s a, D = (s - 1) - s x (#595 A.1, k = 1) + data 2 s a mass, cut into pieces with the per-port run histogram of the shipped word + (bit: #555 x/y runs; complex: #569 four-cold data runs, as replayed by #595 a4_gap.py) + copies s x mass in children of rank h - 1 (#555: 9 x 7, #568: 8 x 8) + helpers rho = R / v helpers per port, each climbing h per stage, cut with the shipped helper run histogram + (bit: #555, complex: #568's implied helper histogram, both from #595 a4_gap.py) + idle the B_K idle slots (skel.idle_slots, 2 per port), hosting: the helper mass is hosted into the largest idle + slots first; a hosted slot of rank r leaves a remainder child of rank r / a (the #555 pattern: 42, 28, 14 -> + 6, 4, 2); helper mass beyond the hostable mass stays as extra paid children (#595 step D semantics) +Piece-size scaling to another window h: + 'abs' pieces keep the shipped absolute ranks (their number grows like h) -- pessimistic for large h + 'rel' pieces scale with the window (rank x a/a_ref; same number of stops per register) -- optimistic +Calibration: the model is run on the E8 words themselves (bit h 8, x 63/111, rho 439/111; complex h 9, x 64/112, +rho 496/112) and the ratio to the published value (1.7331e-3 / 1.5987e-3, #595 A.5) is reported; the same factor is +applied to every candidate of that side (a_cal). +Also printed: #634's loss-chain factors 1/m, D/W (W = 2K v, whole hosting), L_ent = ln m - E ln r on the model profile. +""" +import math +import sys +from fractions import Fraction as Fr + +REPO = sys.argv[1] if len(sys.argv) > 1 else None +if REPO: + sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import idle_slots # noqa: E402 + +# shipped piece statistics (checked by #595 a4_gap.py: A4.runs.#555, A4.data.#568, A4.inputs.#568) +BIT = dict(h=8, v=111, R=439, + data={1: 10 + 317, 2: 209 + 92, 3: 10, 4: 3, 5: 92, 6: 10, 7: 9}, # x + y runs per stage + helpers={1: 994, 2: 1041, 3: 70, 4: 44, 5: 10}, published=Fr(17331, 10 ** 7), x=Fr(63, 111), s=9) +CX = dict(h=9, v=112, R=496, + data={1: 718, 2: 74, 3: 60, 4: 92, 5: 42, 6: 28}, # per call, v = 112 + helpers={1: 115952, 2: 26320, 3: 28064, 4: 7072, 5: 1008, 6: 2016}, published=Fr(15987, 10 ** 7), + x=Fr(64, 112), s=9) + + +def _norm(hist, mass): + tot = sum(r * z for r, z in hist.items()) + return {Fr(r): Fr(z) * mass / tot for r, z in hist.items()} + + +def profile(side, h, x, rho, s, mode): + ref = BIT if side == 'bit' else CX + a, K = h - 1, (s - 1) // 2 + m = 1 + s * a + D = (s - 1) - s * Fr(x) + scale = Fr(a, ref['h'] - 1) if mode == 'rel' else Fr(1) + prof = {} + + def add(r, n): + r = Fr(r) + if n <= 0 or r >= m: + return + prof[r] = prof.get(r, 0) + n + # data: 2 s a mass with the reference run shape + for r, n in _norm(ref['data'], 2 * s * a).items(): + rr = min(r * scale, a) + add(rr, n * r / rr) # keep the mass of each class + # copies + add(a, Fr(s) * Fr(x) / a) + # helpers + hmass = Fr(rho) * s * h + hsc = Fr(h, ref['h']) if mode == 'rel' else Fr(1) + for r, n in _norm(ref['helpers'], hmass).items(): + rr = min(r * hsc, h) + add(rr, n * r / rr) + # idle with hosting + slots = sorted([r for r in idle_slots(K, a) for _ in range(2)], reverse=True) + left = hmass + for r in slots: + rem = Fr(r, a) # remainder piece of a hosted slot + cap = r - rem + if left >= cap: + left -= cap + add(rem, 1) + elif left > 0: + add(r - left, 1) # partially hosted slot: one paid child of the rest + left = 0 + else: + add(r, 1) + return prof, m, D + + +def root(prof, m, D, iters=200): + m = float(m); D = float(D) + F = lambda q: sum(float(z) * float(r) * ((m / float(r)) ** q - 1) for r, z in prof.items()) + lo, hi = 0.0, 0.5 + for _ in range(iters): + mid = (lo + hi) / 2 + if F(mid) >= D: + hi = mid + else: + lo = mid + return hi + + +def chain(prof, m, D, s): + """#634 loss chain on the model profile: a ~ (1/m)(D/W)/L_ent with W = (mass + D)/m data-equivalent wires per port + (= 2K when every helper is hosted) and L_ent = ln m - E ln r (rank-weighted).""" + mass = sum(float(r * z) for r, z in prof.items()) + W = (mass + float(D)) / float(m) + Eln = sum(float(r * z) * math.log(float(r)) for r, z in prof.items()) / mass + Lent = math.log(float(m)) - Eln + return dict(inv_m=1 / float(m), W=W, DW=float(D) / W, Lent=Lent, first=float(D) / W / float(m) / Lent) + + +def estimate(side, h, x, rho, mode, S=range(3, 26, 2)): + best = None + for s in S: + prof, m, D = profile(side, h, x, rho, s, mode) + if D <= 0: + continue + a = root(prof, m, D) + if best is None or a > best[1]: + best = (s, a, chain(prof, m, D, s)) + return best + + +def calibration(side, mode): + ref = BIT if side == 'bit' else CX + prof, m, D = profile(side, ref['h'], ref['x'], Fr(ref['R'], ref['v']), ref['s'], mode) + return float(ref['published']) / root(prof, m, D) + + +if __name__ == '__main__': + for side in ('bit', 'complex'): + for mode in ('abs', 'rel'): + print(side, mode, 'calibration factor (published / model at s = 9): %.4f' % calibration(side, mode)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz.py new file mode 100644 index 0000000000..0e78709fa0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz.py @@ -0,0 +1,87 @@ +"""Lorentzian I_{n,1} units (#634 note 3: triples e_A + e_{n+1} and pairs e_i - e_j, n odd) as bit units. + +Per n, exactly: v; window = rank of the Lorentz Gram (#634 proves the fitting minrank over char != 2 is n for n = 11, 13, +i.e. no rank-(n-1) labelling as for E8 = I_{9,1}); the binary labels (3-set / complement of a pair) satisfy + = 1 + G_ST mod 2 (checked), so B0 = U U^T is a decoder of the #595 pattern with c0 = rank_F2(U) totals; +omega (pairwise non-orthogonal clique, exact) is a lower bound on the F_2 minrank; z = orthogonal neighbours per line; +cst_lab = attainable star cost of the label decoders on these labels (matroid-greedy over the 2^n - 1 functionals); +X* certified (skel.X) at x = cst_lab / v. n = 9 is the E8 control (window 8 after the isotropic quotient). +Usage: python lorentz.py REPO_ROOT [n ...] [--json out] +""" +import sys +from fractions import Fraction as Fr + +import numpy as np + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X # noqa: E402 + +import units as U # noqa: E402 + + +def greedy_cost(L, lab, rad=None): + """rad: isotropic radical vector (n = 9); spans are then measured in the quotient by .""" + n = lab.shape[1] + costs = [] + for p in range(1, 1 << n): + pv = np.array([(p >> j) & 1 for j in range(n)]) + R = np.nonzero((lab @ pv) % 2)[0] + costs.append(((U.rank_q(L[R].tolist() + ([rad] if rad is not None else [])) - (1 if rad is not None else 0)) if len(R) else 0, p)) + costs.sort() + basis, tot, used = {}, 0, [] + for cst, p in costs: + x = p + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + if x: + tot += cst; used.append(cst) + if len(basis) == n: + break + return tot, sorted(used) + + +def main(): + args = sys.argv[2:] + jout = None + if '--json' in args: + jout = args[args.index('--json') + 1] + args = args[:args.index('--json')] + ns = [int(a) for a in args] or [9, 11, 13] + rows = [] + for n in ns: + L, G, lab = U.lorentz_family(n) + v = len(L) + assert (((lab @ lab.T) % 2) == ((1 + G) % 2)).all(), 'labels do not fit the pattern' + h = U.rank_q(G.tolist()) + c0 = U.f2_rank(lab) + om = U.nonorth_clique(G) + z = sorted(set(int((G[i] == 0).sum()) for i in range(v))) + rad = [1] * n + [3] if n == 9 else None + cst, used = greedy_cost(L, lab, rad) + # for n = 9 the label space is degenerate (w = (1,..,1,3) is isotropic): ranks of spans include the radical, + # the quotient (E8) costs dim span - 1 per total that contains the radical; report the Gram ranks too + x = Fr(cst, v) + best = None + for s in range(3, 17, 2): + r = X(s, h, 1, x) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi']) + print('I_{%d,1}: v=%d window(Gram rank)=%d c0=rank_F2(labels)=%d omega=%d z=%s cst_lab=%d (per total %s) x=%s=%.4f X*=%s' + % (n, v, h, c0, len(om), z, cst, used, x, float(x), + ('%.5e@s%d' % (float(best[1]), best[0])) if best else 'D<=0'), flush=True) + rows.append(dict(n=n, v=v, window=h, c0=c0, omega=len(om), z=z, cst_lab=cst, per_total=used, x=str(x), + Xstar=None if best is None else dict(s=best[0], hi=str(best[1])))) + if jout: + import json + json.dump(rows, open(jout, 'w'), indent=1) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz_minrank.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz_minrank.py new file mode 100644 index 0000000000..500d459c1c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/lorentz_minrank.py @@ -0,0 +1,20 @@ +"""F_2 minrank of the #595 pattern on I_{n,1} (n = 11: is it 10 = omega or 11 = c0?). SAT with the clique gauge, +time-limited; prints the outcome per c. Usage: python lorentz_minrank.py REPO_ROOT n timeout_s [--json out]""" +import sys +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from minrank_sat import solve_asym # noqa: E402 +import units as U # noqa: E402 + +n, T = int(sys.argv[2]), int(sys.argv[3]) +L, G, lab = U.lorentz_family(n) +om = U.nonorth_clique(G) +c0 = U.f2_rank(lab) +print('I_{%d,1}: omega=%d c0=%d' % (n, len(om), c0), flush=True) +lo, hi, log = U.minrank_f2(G, om, c0, solve_asym, timeout=T) +print(lo, hi, log, flush=True) +if '--json' in sys.argv: + import json + json.dump([dict(n=n, omega=len(om), c0=c0, lo=lo, hi=hi, log={k: v[0] for k, v in log.items()})], + open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1) diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/rank.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/rank.py new file mode 100644 index 0000000000..500b2b215f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/rank.py @@ -0,0 +1,115 @@ +"""Rank candidate unit families by a helper-adjusted estimate of the achievable saving (MODEL; see est_model.py). + +Per candidate (side, h, v, cst, z): + X* certified #595 ceiling (skel.X, best odd s in 3..15) at x = cst / v [exact interval upper end] + rho* break-even helpers per port: the rho at which the calibrated model estimate (best s) equals the calibrated + model estimate of the E8 word of the same side at its shipped intensity and best s (bit #555 at + rho = 439/111, complex #568 at rho = 496/112): same model, same s freedom on both sides + a_est calibrated model estimate at three helper-intensity hypotheses (E8's shipped intensity transferred): + rho_same = rho_E8 (helpers per port unchanged) + rho_dim = rho_E8 * (z/(h-1)) / (z_E8/(h_E8-1)) (readers per window dimension) <- central + rho_read = rho_E8 * z / z_E8 (readers per port) + in both piece-scaling modes ('abs' pessimistic, 'rel' optimistic). +Candidates: E8 references; triple family T_h (bit cst = h(h-1), complex cst = h^2: centre decoders, checked by +triple.py); Lorentzian I_{11,1}, I_{13,1} (label decoders, cst 110 / 156, checked by lorentz.py). +Usage: python rank.py REPO_ROOT [--json out] +""" +import json +import sys +from fractions import Fraction as Fr +from math import comb + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X # noqa: E402 + +import est_model as E # noqa: E402 + +REF = {'bit': dict(h=8, rho=Fr(439, 111), z=63, a=1.7331e-3), + 'complex': dict(h=9, rho=Fr(496, 112), z=52, a=1.5987e-3)} + + +def xstar(h, x): + best = None + for s in range(3, 17, 2): + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi']) + return best + + +def candidates(): + C = [dict(name='E8 bit 111/63 (#555)', side='bit', h=8, v=111, cst=63, z=63), + dict(name='E8 bit 120/72', side='bit', h=8, v=120, cst=72, z=63), + dict(name='E8 complex 112/64 (#568)', side='complex', h=9, v=112, cst=64, z=52), + dict(name='I_{11,1} bit', side='bit', h=11, v=220, cst=110, z=Fr(165 * 115 + 55 * 129, 220)), + dict(name='I_{13,1} bit', side='bit', h=13, v=364, cst=156, z=Fr(286 * 183 + 78 * 231, 364))] + for h in (10, 11, 12, 13, 14, 16, 18, 20, 22, 24, 26, 28, 30): + C.append(dict(name='T_%d bit' % h, side='bit', h=h, v=comb(h, 3), cst=h * (h - 1), z=3 * comb(h - 3, 2))) + for h in (12, 14, 16, 18, 20, 22, 24, 26, 28, 30): + C.append(dict(name='T_%d complex' % h, side='complex', h=h, v=comb(h, 3), cst=h * h, + z=comb(h - 3, 3) + 3 * (h - 3))) + return C + + +def cal_est(side, h, x, rho, mode, cal): + b = E.estimate(side, h, x, rho, mode) + return (None, None, None) if b is None else (b[0], b[1] * cal, b[2]) + + +def breakeven(side, h, x, mode, cal, target): + lo, hi = Fr(0), Fr(400) + f = lambda r: (cal_est(side, h, x, r, mode, cal)[1] or 0.0) + if f(lo) < target: + return 0.0 + if f(hi) >= target: + return float(hi) + for _ in range(40): + mid = (lo + hi) / 2 + if f(mid) >= target: + lo = mid + else: + hi = mid + return float(lo) + + +def main(): + rows = [] + cals = {(sd, md): E.calibration(sd, md) for sd in ('bit', 'complex') for md in ('abs', 'rel')} + print('calibration factors', {k: round(v, 4) for k, v in cals.items()}) + target = {sd: cal_est(sd, REF[sd]['h'], Fr(63, 111) if sd == 'bit' else Fr(64, 112), REF[sd]['rho'], 'abs', + cals[(sd, 'abs')])[1] for sd in ('bit', 'complex')} + print('E8 targets (calibrated model, best s):', {k: '%.4e' % v for k, v in target.items()}) + for c in candidates(): + sd, h = c['side'], c['h'] + x = Fr(c['cst'], c['v']) + ref = REF[sd] + xs = xstar(h, x) + rh = {'same': ref['rho'], + 'dim': ref['rho'] * Fr(c['z']) / (h - 1) / Fr(ref['z'], ref['h'] - 1), + 'read': ref['rho'] * Fr(c['z']) / ref['z']} + row = dict(name=c['name'], side=sd, h=h, v=c['v'], cst=c['cst'], x=float(x), z=float(c['z']), + Xstar=float(xs[1]) if xs else None, Xs=xs[0] if xs else None, + rho={k: float(v) for k, v in rh.items()}) + for md in ('abs', 'rel'): + row['rho_star_' + md] = breakeven(sd, h, x, md, cals[(sd, md)], target[sd]) + for k, r in rh.items(): + s_, a_, ch = cal_est(sd, h, x, r, md, cals[(sd, md)]) + row['a_%s_%s' % (k, md)] = a_ + row['s_%s_%s' % (k, md)] = s_ + if k == 'dim' and md == 'abs' and ch: + row['chain_dim_abs'] = {kk: round(vv, 5) for kk, vv in ch.items()} + rows.append(row) + f = lambda v: '%.3e' % v if v else ' - ' + print('%-26s h=%2d v=%5d x=%.4f z=%7.1f X*=%s | rho*: abs %6.2f rel %6.2f | rho same/dim/read %5.2f/%6.2f/%7.2f | ' + 'a_dim abs %s rel %s | a_same abs %s | a_read abs %s' + % (row['name'], h, c['v'], row['x'], row['z'], f(row['Xstar']), row['rho_star_abs'], row['rho_star_rel'], + row['rho']['same'], row['rho']['dim'], row['rho']['read'], f(row['a_dim_abs']), f(row['a_dim_rel']), + f(row['a_same_abs']), f(row['a_read_abs'])), flush=True) + if '--json' in sys.argv: + json.dump(rows, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/triple.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/triple.py new file mode 100644 index 0000000000..da28fda4a3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/triple.py @@ -0,0 +1,173 @@ +"""Triple family T_h (upstream 03-motifs.tex, the Alon-type unit): decoder, copy cost and certified ceiling. + +Bit side. Lines t_T (T a 3-subset of [h]) in Q^h with the form I - J/9: G_ST = |S n T| - 1. Pattern: B_TS = 0 where +|S n T| in {0, 2}, free where |S n T| = 1. Label decoder B0 = M M^T mod 2 (M = incidence; the h centres C_i of the +paper): B0_ST = |S n T| mod 2. +Complex side. Lines t_T in F_2^h with the dot product (t_T . t_T = 1). Pattern: B_TS = 0 where |S n T| = 1 (odd), +free where |S n T| in {0, 2}. Decoder over a field of characteristic 0: the paper's h + 1 centres C_i, C_*, with +B_ST = (|S n T| - 1)/2. + +Per h the script computes, exactly: + window rank of the form on span{t_T} (Q for bit, F_2 for complex) + c totals of the centre decoder; for bit also rank_F2(B0) and the clique lower bound 4*floor(h/4) + (4 triples inside a 4-set are pairwise at even intersection; odd-weight pairwise-orthogonal vectors of + F_2^h are independent, so the clique number is <= h) + cst_att star cost of the centre decoder = sum_i dim span(gathered lines) (attained: copy f = V -> 0) + cst_thmC #446 Theorem C floor of the centre decoder = sum_i rank of the pairing span(gathered) x span(readers) + X*(att) certified #595 ceiling skel.X at x = cst_att / v, best odd s in 3..15 +For small h (bit, h <= MAXEXACT) also: + minrank exact F_2 minrank of the pattern (SAT, #595's encoder), and + lab_min min over nonzero p in F_2^h of dim_Q span{t_T : |T n supp p| odd} (every label decoder on the B0 labels + has each total gathering such a set, so cst >= c * lab_min for those decoders) +Usage: python triple.py REPO_ROOT HMIN HMAX [--exact MAXEXACT] [--json out] +""" +import itertools +import json +import sys +from fractions import Fraction as Fr +from math import comb + +import numpy as np + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X # noqa: E402 +from minrank_sat import solve_asym # noqa: E402 + +import units as U # noqa: E402 + +S_RANGE = range(3, 17, 2) + + +def xstar(h, x): + best = None + for s in S_RANGE: + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi']) + return best + + +def q_pair_rank(rows, Qform): + """rank over Q of the Gram matrix of span(rows) under Qform (integer).""" + idx = [] + red = [] + import math + for i, r in enumerate(rows): + rr0 = [int(x) for x in r] + rr = rr0 + for pc, b in red: + if rr[pc]: + f, g = rr[pc], b[pc] + rr = [g * x - f * y for x, y in zip(rr, b)] + c = math.gcd(*rr) + if c > 1: + rr = [x // c for x in rr] + pc = next((j for j, x in enumerate(rr) if x), None) + if pc is not None: + red.append((pc, rr)); idx.append(i) + B = np.array([rows[i] for i in idx], dtype=object) + Gm = B.dot(np.array(Qform, dtype=object)).dot(B.T) + return len(idx), U.rank_q(Gm.tolist()) + + +def f2_dim(rows): + return U.f2_rank(np.array(rows)) + + +def f2_pair_rank(rows): + """dim and dot-product rank of span(rows) over F_2.""" + rows = np.array(rows, dtype=np.int64) % 2 + # basis + basis = [] + for r in rows: + x = int(''.join(str(int(b)) for b in r[::-1]), 2) + for b in basis: + x = min(x, x ^ b) + if x: + basis.append(x) + n = rows.shape[1] + Bm = np.array([[(b >> j) & 1 for j in range(n)] for b in basis], dtype=np.int64) + return len(basis), U.f2_rank((Bm @ Bm.T) % 2) + + +def bit_row(h, exact): + M, G, T = U.triple_family(h) + v = len(T) + Qf = 9 * np.eye(h, dtype=np.int64) - 1 # 9 (I - J/9); ranks are unchanged by the factor 9 + dimall, window = q_pair_rank(M.tolist(), Qf.tolist()) + c0 = U.f2_rank(M @ M.T % 2) + att, thm = 0, 0 + per = [] + for i in range(h): + R = [k for k, t in enumerate(T) if i in t] + d, pr = q_pair_rank(M[R].tolist(), Qf.tolist()) + att += d; thm += pr; per.append((d, pr)) + row = dict(side='bit', h=h, v=v, window=window, span=dimall, c_centre=h, c0=c0, clique_lb=4 * (h // 4), + z=3 * comb(h - 3, 2), cst_att=att, cst_thmC=thm, per_total=sorted(set(per))) + if exact and h <= exact: + off = [(i, j) for i in range(v) for j in range(i + 1, v) if G[i, j] != 0] + # explicit clique: disjoint 4-sets' triples + cl = [] + for b in range(h // 4): + pts = range(4 * b, 4 * b + 4) + cl += [T.index(tuple(t)) for t in itertools.combinations(pts, 3)] + lo, hi, log = U.minrank_f2(G, cl, c0, solve_asym, timeout=600) + c = lo if lo == hi else '%d..%d' % (lo, hi) + row['minrank'] = c + row['minrank_log'] = log + lab = None + for p in range(1, 1 << h): + P = [j for j in range(h) if p >> j & 1] + R = [k for k, t in enumerate(T) if len(set(t) & set(P)) % 2] + d = U.rank_q(M[R].tolist()) + lab = d if lab is None else min(lab, d) + row['lab_min'] = lab + return row + + +def cx_row(h): + M, G, T = U.triple_family(h) + v = len(T) + dimall, window = f2_pair_rank(M.tolist()) + # centre decoder rank over Q of (M M^T - J)/2 = rank of [M | 1] (Gram-type), exact + Bc = (M @ M.T - 1) + cdec = U.rank_q(np.hstack([M, np.ones((v, 1), dtype=np.int64)]).tolist()) + att, thm = 0, 0 + per = [] + for i in range(h): + R = [k for k, t in enumerate(T) if i in t] + d, pr = f2_pair_rank(M[R].tolist()) + att += d; thm += pr; per.append((d, pr)) + d, pr = f2_pair_rank(M.tolist()) # C_* gathers every line + att += d; thm += pr; per.append((d, pr)) + return dict(side='complex', h=h, v=v, window=window, span=dimall, c_centre=h + 1, rank_centre_matrix=cdec, + z=comb(h - 3, 3) + 3 * (h - 3), cst_att=att, cst_thmC=thm, per_total=sorted(set(per))) + + +def main(): + hmin, hmax = int(sys.argv[2]), int(sys.argv[3]) + exact = int(sys.argv[sys.argv.index('--exact') + 1]) if '--exact' in sys.argv else 0 + rows = [] + for h in range(hmin, hmax + 1): + for r in (bit_row(h, exact), cx_row(h)): + x_att, x_thm = Fr(r['cst_att'], r['v']), Fr(r['cst_thmC'], r['v']) + wnd = r['window'] + # the window entering skel.X is the rank of the label space (Theorem B uses the window form) + for key, x in (('att', x_att), ('thmC', x_thm)): + b = xstar(wnd, x) + r['X_' + key] = None if b is None else dict(s=b[0], hi=str(b[1]), f=float(b[1])) + r['x_att'], r['x_thmC'] = str(x_att), str(x_thm) + rows.append(r) + fx = lambda k: ('%.5e@s%d' % (r[k]['f'], r[k]['s'])) if r[k] else 'D<=0' + extra = (' minrank=%s lab_min=%s' % (r.get('minrank'), r.get('lab_min'))) if 'minrank' in r else '' + print('%-7s h=%2d v=%5d window=%2d c=%2d cst_att=%4d cst_thmC=%4d x_att=%.4f X*att=%s X*thmC=%s z=%d per=%s%s' + % (r['side'], h, r['v'], wnd, r['c_centre'], r['cst_att'], r['cst_thmC'], float(x_att), fx('X_att'), + fx('X_thmC'), r['z'], r['per_total'], extra), flush=True) + if '--json' in sys.argv: + json.dump(rows, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/units.py b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/units.py new file mode 100644 index 0000000000..358c72b4b5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/scripts/units.py @@ -0,0 +1,210 @@ +"""Exact line systems used by the S3 screens (integer coordinates, exact Gram matrices). + +Every unit is returned as (L, G, name) with L an integer v x n matrix of line representatives and G the exact integer +Gram matrix of the unit's bilinear form (diagonal 2). The F2 decoder pattern of #595 is: B_TT = 1 and B_TS = 0 whenever +G_TS != 0 (non-orthogonal lines); the entries at orthogonal pairs are free. + +Families + * ADE root systems A_n, D_n, E6, E7, E8 (doubled coordinates, G = L L^T / 4). + E6 is the root lines orthogonal to an A2 = , a = (1/2)^8, b = e1 + e2 (a.b = 1), as in #630; + the archived #620 screen used b = e1 - e2 (a.b = 0), which gives D6. + * Triple family T_h (upstream 03-motifs.tex, bit construction): the C(h,3) indicator vectors t_T of 3-subsets of [h] + with the form I - J/9, so G_ST = |S n T| - 1 (diagonal 2). + * Lorentzian I_{n,1} (n odd, #634 note 3): e_A + e_{n+1} (|A| = 3) and e_i - e_j, Lorentz form diag(1^n, -1). +""" +import itertools + +import numpy as np + + +def roots_A(n): + R = [] + for i in range(n + 1): + for j in range(i + 1, n + 1): + r = [0] * (n + 1); r[i] = 2; r[j] = -2; R.append(r) + return np.array(R, dtype=np.int64) + + +def roots_D(n): + R = [] + for i in range(n): + for j in range(i + 1, n): + for s in (1, -1): + r = [0] * n; r[i] = 2; r[j] = 2 * s; R.append(r) + return np.array(R, dtype=np.int64) + + +def roots_E8(): + """120 lines, doubled coordinates: +-2e_i +-2e_j (one sign class per line) and (+-1)^8 with an even number of + minus signs and first coordinate +1.""" + R = [list(r) for r in roots_D(8)] + for sg in itertools.product((1, -1), repeat=8): + if sg[0] == 1 and sg.count(-1) % 2 == 0: + R.append(list(sg)) + return np.array(R, dtype=np.int64) + + +def perp(R, w): + return R[(R @ np.asarray(w, dtype=np.int64)) == 0] + + +A_VEC = np.ones(8, dtype=np.int64) # (1/2)^8 doubled +B_E6 = np.array([2, 2, 0, 0, 0, 0, 0, 0]) # e1 + e2 doubled; a.b = 4/4 = 1 -> A2 with a +B_D6 = np.array([2, -2, 0, 0, 0, 0, 0, 0]) # e1 - e2 doubled; a.b = 0 -> A1 + A1 (archived "E6") + + +def roots_E7(): + return perp(roots_E8(), A_VEC) + + +def roots_E6(): + return perp(perp(roots_E8(), A_VEC), B_E6) + + +def roots_D6_archived(): + return perp(perp(roots_E8(), A_VEC), B_D6) + + +def gram_doubled(L): + G4 = L @ L.T + assert (G4 % 4 == 0).all() + G = G4 // 4 + assert (np.diag(G) == 2).all() + return G + + +def triple_family(h): + """lines t_T, T in C([h],3); G_ST = |S n T| - 1 (form I - J/9).""" + T = list(itertools.combinations(range(h), 3)) + M = np.zeros((len(T), h), dtype=np.int64) + for i, t in enumerate(T): + M[i, list(t)] = 1 + G = M @ M.T - 1 + return M, G, T + + +def lorentz_family(n): + """I_{n,1}: e_A + e_{n+1} (|A| = 3) and e_i - e_j; Lorentz Gram; binary labels: A, resp. [n] minus {i, j}.""" + vecs, labels = [], [] + for A in itertools.combinations(range(n), 3): + v = [0] * (n + 1) + for a in A: + v[a] = 1 + v[n] = 1 + vecs.append(v); labels.append([1 if i in A else 0 for i in range(n)]) + for i, j in itertools.combinations(range(n), 2): + v = [0] * (n + 1); v[i] = 1; v[j] = -1 + vecs.append(v); labels.append([0 if k in (i, j) else 1 for k in range(n)]) + L = np.array(vecs, dtype=np.int64) + Q = np.diag([1] * n + [-1]).astype(np.int64) + G = L @ Q @ L.T + return L, G, np.array(labels, dtype=np.int64) + + +# ------------------------------------------------------------------ exact linear algebra +def rank_q(rows): + """exact rank over Q (fraction-free Bareiss-style elimination on Python ints).""" + import math + red = [] + for r in rows: + r = [int(x) for x in r] + for pc, rr in red: + if r[pc]: + f, g = r[pc], rr[pc] + r = [g * x - f * y for x, y in zip(r, rr)] + c = math.gcd(*r) + if c > 1: + r = [x // c for x in r] + pc = next((j for j, x in enumerate(r) if x), None) + if pc is not None: + red.append((pc, r)) + return len(red) + + +def rank_mod_p(M, p): + """rank of an integer matrix over F_p (p prime).""" + A = (np.array(M, dtype=object) % p).tolist() + rk, rows, cols = 0, len(A), len(A[0]) if A else 0 + for c in range(cols): + piv = next((i for i in range(rk, rows) if A[i][c] % p), None) + if piv is None: + continue + A[rk], A[piv] = A[piv], A[rk] + inv = pow(A[rk][c], p - 2, p) + A[rk] = [(x * inv) % p for x in A[rk]] + for i in range(rows): + if i != rk and A[i][c] % p: + f = A[i][c] + A[i] = [(x - f * y) % p for x, y in zip(A[i], A[rk])] + rk += 1 + return rk + + +def f2_rank(M): + rows = [int(''.join('1' if x & 1 else '0' for x in row[::-1]) or '0', 2) for row in np.asarray(M)] + basis = {} + for x in rows: + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + return len(basis) + + +def gram_rank(G, idx): + """rank over Q of the principal block G[idx, idx] (= pairing rank of span(idx) with itself).""" + if len(idx) == 0: + return 0 + return rank_q(G[np.ix_(idx, idx)].tolist()) + + +def nonorth_clique(G): + """exact maximum set of pairwise non-orthogonal lines (identity block of the pattern => minrank lower bound).""" + import networkx as nx + v = len(G) + H = nx.Graph() + H.add_nodes_from(range(v)) + H.add_edges_from((i, j) for i in range(v) for j in range(i + 1, v) if G[i, j] != 0) + best, _ = nx.max_weight_clique(H, weight=None) + return sorted(best) + + +def _query(args): + solver, v, pattern, c, clique = args + out = solver(v, pattern, None, c, 'none', clique) + if out['result'] == 'SAT': + B = out['B'] + assert all(B[i, i] == 1 for i in range(v)) and all(B[i, j] == 0 for i, j in pattern) + return out['result'], out['sec'] + + +def minrank_f2(G, clique, cmax, solver=None, timeout=300): + """exact F_2 minrank of the pattern of G (B_TT = 1, B_TS = 0 where G_TS != 0), asymmetric decoders allowed. + Asks SAT for c = |clique|, |clique| + 1, ... < cmax; each query fixes the GL(c, F_2) gauge U_K = W_K = [I | 0] on + the clique K (solve_asym of #595's minrank_sat.py; a SAT answer is re-checked on the returned B). A query that does + not finish within `timeout` seconds is recorded as 'TIMEOUT' and the search stops there. + Returns (lo, hi, log): the minrank lies in [lo, hi]; lo == hi when it is determined (hi = cmax is attained by a + known decoder).""" + import multiprocessing as mp + v = len(G) + pattern = [(i, j) for i in range(v) for j in range(v) if i != j and G[i, j] != 0] + log = {} + lo = len(clique) + ctx = mp.get_context('fork') + for c in range(len(clique), cmax): + with ctx.Pool(1) as pool: + res = pool.apply_async(_query, ((solver, v, pattern, c, list(clique)),)) + try: + r, sec = res.get(timeout) + except mp.TimeoutError: + log[c] = ('TIMEOUT', timeout) + pool.terminate() + return lo, cmax, log + log[c] = (r, sec) + if r == 'SAT': + return c, c, log + lo = c + 1 + return cmax, cmax, log diff --git a/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/verify.sh new file mode 100644 index 0000000000..eebeb5ee68 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s3-unit-families/verify.sh @@ -0,0 +1,21 @@ +#!/bin/sh +# Recompute every number of this package and compare with expected.json. +# Usage: sh verify.sh REPO_ROOT OUT_DIR +# REPO_ROOT: a checkout of CrocSwap/integer-mult-bounds main (reads research/format-limits/scripts: skel.py, cert.py, +# minrank_sat.py, e8units.py of #595); OUT_DIR: fresh output directory. +# Python >= 3.11 with numpy, networkx, python-sat on PATH as `python3` (or set PY). About 40 min on 10 cores +# (the steps run in parallel; the ADE step spends up to 600 s in each of its two SAT timeouts). +set -eu +REPO=$(cd "$1" && pwd); OUT=$2 +PY=${PY:-python3} +HERE=$(cd "$(dirname "$0")" && pwd) +mkdir -p "$OUT" +cd "$HERE/scripts" +$PY -B ade_screen_fixed.py "$REPO" --json "$OUT/ade.json" > "$OUT/ade.log" 2>&1 & +$PY -B triple.py "$REPO" 6 32 --exact 11 --json "$OUT/triple.json" > "$OUT/triple.log" 2>&1 & +$PY -B lorentz.py "$REPO" 9 11 13 --json "$OUT/lorentz.json" > "$OUT/lorentz.log" 2>&1 & +$PY -B lorentz_minrank.py "$REPO" 11 600 --json "$OUT/lorentz_minrank.json" > "$OUT/lorentz_minrank.log" 2>&1 & +$PY -B cover.py "$REPO" E8,T8,T10,T12,T16,T20,T24,L11,L13 20000 --json "$OUT/cover.json" > "$OUT/cover.log" 2>&1 & +$PY -B rank.py "$REPO" --json "$OUT/rank.json" > "$OUT/rank.log" 2>&1 & +wait +$PY -B check.py "$OUT" "$HERE/expected.json" diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/README.md b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/README.md new file mode 100644 index 0000000000..39d4367cbe --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/README.md @@ -0,0 +1,277 @@ +# Lorentzian unit I_{11,1}: punctures, E8 skeleton features, and the helper budget (no κ claim) + +This note follows up S3 (`e8-decoder-floors-and-cold14-notes/s3-unit-families`). That package found I_{11,1} to be the best non-E8 unit lead: + +- v 220, window h 11, minrank 11; +- label decoder cost 110, so x = 1/2; +- certified ceiling X* 7.1715·10⁻³; +- but the helper-adjusted model puts it below E8 unless it reaches ≤ 2.66 (abs) / 3.34 (rel) helpers per port. E8 ships 3.95. + +This package asks three questions: + +1. Can punctures (the analogue of E8's 120 → 111) lower x or raise X*? +2. Which E8 geometry sets E8's helper count, and does I_{11,1} have it? +3. What does the calibrated model say for the best punctured unit? + +**Answers, in short:** + +1. **No improving puncture.** + - For the centre (label) decoder, no puncture of any size improves x. This is an exact certificate for I_{11,1} and I_{13,1}. + - For every label decoder, the same holds up to 8 (I_{11,1}) or 10 (I_{13,1}) punctures. + - No puncture of ≤ 19 lines lowers the minrank to 10 (SAT). + - The E8 mechanism exists here: puncturing a line makes its own label functional one unit cheaper. But that functional starts at h, not h − 1, so it only ties the centres. +2. **The E8 skeleton features carry over, but the count lands at or above break-even.** + - Carried over: cell cover number 3, orthogonal 4-frames with F₂ parity, and a "design" obstruction for triple sources. + - Not carried over: a partition into such frames is impossible (55 pairs is odd), and each needer must sum 56–70 lines against E8's 30. + - The transferred skeleton counts 3.33–3.58 helpers per port. + - Within the cell architecture, carriers alone are exactly 2 per port. Reaching ≤ 3 would need a middle network that costs less than ⅔ of E8's per cell while carrying about twice E8's demand per needer (argued; not a theorem over all designs). +3. **The model gives no gain over E8.** The best unit is the unpunctured one. At the transferred-skeleton intensity the model gives 1.58·10⁻³ (abs) / 1.73·10⁻³ (rel), against E8's 1.806·10⁻³. Only the optimistic variant (pair sources on two non-cell branches, rel scaling) touches E8: 1.811·10⁻³, which is +0.3% and inside model error. + +Status labels: + +- **checked**: `verify.sh` recomputes it. Exact rational arithmetic, or a CP-SAT/SAT optimum or UNSAT as stated. +- **argued**: a proof written here. +- **model**: an estimate, not a bound. + +SAT answers come from CaDiCaL 1.5.3 (python-sat) without proof logs. CP-SAT optima are reported only when the status is OPTIMAL. + +```sh +PY=/path/to/python3.11 sh verify.sh /path/to/integer-mult-bounds-main /fresh/out # ~30-40 min, 10 cores +``` + +Requirements: numpy, sympy, networkx, python-sat, ortools. The scripts import #595's `skel.py`/`cert.py` (main, `research/format-limits/scripts`) and S3's `units.py`, `est_model.py` and `rank.py`, which are copied unchanged into `scripts/`. `check.py` compares every log with `expected/`; it ignores run times and allows a relative difference of 1e-9 on model floats. + +Conventions follow #595 §A and S3: + +- **Pattern.** The decoder pattern is B_TT = 1, with B_TS = 0 where G_TS ≠ 0. +- **Labels.** The labels are S3's I_{n,1} labels (a 3-set, or the complement of a pair). +- **Reader set.** R(p) = {t : ⟨u_t, p⟩ = 1}. +- **Star cost.** A label total p costs dim_Q span(R(p) ∩ live). +- **Optimal label decoder.** It is the minimum-weight basis of the live label space (matroid greedy, exact). + +## 1. Reader census (`census.py`, checked) + +| n | functional weight k | readers | span | Gram rank | +|---|---|---|---|---| +| 11 | 1 (the 11 centres e_i) | 90 (45 triples ∋ i, 45 pairs ∌ i) | **10** | 10 | +| 11 | 2 … 11 | 90 … 220 | **11** | 11 | +| 13 | 1 (centres) | 132 | **12** | 12 | +| 13 | 2 … 13 | 132 … 364 | **13** | 13 | + +Only the centres span h − 1, so the unpunctured label decoder is forced to be the centre decoder: 11 × 10 = 110 and 13 × 12 = 156. The Theorem C pairing floor (#446) of every centre equals its span, so on the unpunctured unit Theorem C = star. + +## 2. Punctures (Q1) + +### 2.1 The centre decoder never improves (`partition.py`, checked certificate + argued) + +**Theorem P1.** For every live set L of I_{11,1}: Σ_i rank(R(e_i) ∩ L) ≥ |L|/2. For I_{13,1}: Σ_i rank(R(e_i) ∩ L) ≥ (3/7)|L|. + +**Proof (I_{11,1}).** + +- `partition.py` partitions the 220 lines into 22 sets I_{i,1}, I_{i,2}, each linearly independent and contained in R(e_i). Exact ranks are verified; every set has size 10. +- Then 2·rank(R(e_i) ∩ L) ≥ |L ∩ I_{i,1}| + |L ∩ I_{i,2}|. Summing over i gives the claim. + +**Proof (I_{13,1}).** Take three copies of each of the 364 lines. They split into 91 independent sets, 7 inside each R(e_i), each of size 12. ∎ + +**Consequence.** For the centre decoder, x(P) ≥ 1/2 (resp. ≥ 3/7) for every puncture P, so the unpunctured ratio is never beaten. + +**Scope.** + +- Costs are span ranks. This is the right measure for live sets with a nondegenerate span. +- A degenerate live set (a radical appears, e.g. the E8 sub-configuration) is dominated by its quotient unit (#595 Lemma Q). It is a different unit with a smaller window; see §2.4. + +### 2.2 All label decoders, small punctures (`packbases.py`, checked + argued) + +`packbases.py` treats each functional orbit (Hamming weight k, under S_n). It strips coloops (lines outside the span of the rest of R(p)) and packs q disjoint bases into the rest. The packings are verified exactly. + +**The E8 mechanism, here.** For k = 3 (triple labels) and k = n − 2 (pair labels), the coloop is the line itself: + +- R(u_t) = {t} ∪ (readers of t); +- the readers of t span only h − 1. + +So puncturing t makes its own label functional drop from h to h − 1. That is exactly E8's 8 → 7. Here, however, h − 1 is what the centres already cost. + +**Theorem P2.** Every label decoder has x ≥ 1/2 on I_{11,1} for |P| ≤ 8, and x ≥ 3/7 on I_{13,1} for |P| ≤ 10. + +**Proof.** + +1. **Disjoint bases protect rank.** If R(p) minus its coloops holds q disjoint bases, deleting P costs that rest at most ⌊|P|/q⌋ rank. +2. **Every non-centre functional keeps cost ≥ h − 1.** The binding orbits are: + - n = 11: k = 9, rest rank 10, q = 9, so |P| ≤ 8; + - n = 13: k = 11, rest rank 12, q = 11, so |P| ≤ 10. + + The full table is in `packbases*.log`; for n = 11 the other orbits allow 10–39 deletions. +3. **The live space is unchanged.** The weight-n functional holds 20 (resp. 28) disjoint bases, so the live span and window stay h. +4. **Combine with P1.** Let S be the set of centres in an optimal basis. Then W ≥ Σ_{i∈S} r_i + (h − 1)(n − |S|) ≥ Σ_i r_i ≥ |L|/2 (resp. (3/7)|L|), because r_i ≤ h − 1. ∎ + +**First cheap non-centre total (`example_pairlabel.py`, checked).** + +- **Puncture.** Delete the pair {0,1} and the 9 triples {0,1,x}, 10 lines in all. The pair-label functional then drops to 9: star 109/210 = 0.519, X* 6.3495·10⁻³ (s = 3). +- **Ratio.** One cost unit costs 10 punctures, while improving x needs more than 1 unit per 2 punctures. +- **Theorem C.** The floor here is 108 (X* at 108/210 = 6.5549·10⁻³), still below the unpunctured unit. + +### 2.3 Fewer totals after puncturing? (`clique_orbits.py`, `punct_sat.py`, checked by SAT) + +The maximum pairwise non-orthogonal cliques of I_{11,1} (the #595 pattern's identity blocks) are 38,786 cliques of size 10, in 5 S_11-orbits. The orbit sizes are: + +| orbit size | composition | +|---|---| +| 27,720 | 8 triples + 2 pairs | +| 9,240 | 7 triples + 3 pairs | +| 495 | 8 triples + 2 pairs | +| 1,320 | 3 triples + 7 pairs | +| 11 | 10 pairs | + +(#634's 10,395 sets Cl(a, M) are pairwise *orthogonal*. They belong to #634's knowing graph, not to this pattern.) + +**Gauge argument.** + +- A line lies in at most 1,784 of these cliques. So any ≤ 21 punctures leave a maximum clique alive (`survival_bound`, checked). +- The clique gauge U_K = W_K = [I | 0] on one representative per orbit, forced live, is therefore without loss of generality for k ≤ 21. + +**Result: the minrank stays 11 for every puncture of ≤ 19 lines.** With c = 10 and k = 19, the SAT instance is UNSAT for all 5 orbits (`sat19_o*.log`). Runs at k = 0, 4 and 8 (orbit 0) were UNSAT in under 6 s. + +**Consequence.** A 10-total decoder needs ≥ 20 punctures. If its totals cost 10 each, that gives x ≥ 100/200 = 1/2. + +**Scope.** General (non-label) 11-total decoders with totals cheaper than 10 are not excluded. The same open item, the non-star decoder, holds for E8 (#595 F.2, #631). + +### 2.4 Window-lowering sub-units (`punct.py`, checked) + +These keep the lines avoiding a set of coordinates. Labels are inherited from I_{11,1}; Theorem C is the greedy Gram-rank floor. + +| live lines | v | h | star (x) | X* | Theorem C (X*) | +|---|---|---|---|---|---| +| all | 220 | 11 | 110 (1/2) | **7.1715·10⁻³** (s 3) | 110 | +| avoid coordinate 10 (I_{10,1}-type) | 165 | 10 | 99 (3/5) | 4.0462·10⁻³ (s 5) | 88 (6.3329·10⁻³) | +| avoid 9, 10 (the E8 lines; radical 1) | 120 | 8 | 80 (2/3) | 3.4352·10⁻³ | 80 | +| avoid 8, 9, 10 | 84 | 8 | 63 (3/4) | 1.9850·10⁻³ | 63 | + +The inherited labels are not minimal on sub-units. E8's own labels give 72, not 80 (#595 C.6). So these rows are upper bounds on those units' cost. The E8 rows are the known E8 unit, and the I_{10,1}-type unit loses even at its Theorem C floor. + +### 2.5 Heuristic search (`anneal.py`, exploratory; logs in `data/`) + +The search used two objectives: + +- x, 3 seeds × 4000 moves; +- an X* proxy (2 − 3x)/h that also rewards lowering h, 1 seed. + +Each run used float ranks and exact greedy bases. Every run ended at the unpunctured unit. This is evidence only. + +## 3. Helpers (Q2) + +### 3.1 What sets E8's 3.95 helpers per port + +The shipped word is #555 (= `research/e8-bit-p111`, R = 439 at v = 111). It sits on #543's L3 skeleton (`research/e8-bit-r420`, itself on #493's `e8-bit-carrier-skeleton` partition): + +| component | count | per port (v = 111) | geometric reason | +|---|---|---|---| +| carriers | 240 | 2.16 | 3 branches per source, at the three cells {s, e} of its 4-frame. 3 is the minimum: s⊥ = E7 is a spherical 4-design, so no 2 hyperplane sections cover it (#463). Checked here as "no nonzero quadratic form vanishes on the 63 readers" (`twocover.py 8`). | +| empty cell registers | 180 | 1.62 | 30 orthogonal 4-frames × 6 cells; **3 registers per cell** (#493/#543 family bound): 2 line branches + 1 register that delivers the cell sum to the two quads (D4 frames of 6 cells from 6 blocks) and keeps a line | +| totals | 9 | 0.08 | F₂ minrank 9 | +| extra carriers (puncture) | 10 | 0.09 | #545 pseudo-target mini-nodes | + +**E8 features this rests on:** + +- **(a) Good 4-frames.** The 120 lines partition into 30 orthogonal 4-frames whose half-sum is a lattice vector (F₂ parity). +- **(b) Free inclusion–exclusion on the target side.** By (a), every reader s of t meets 1 or 3 of t's frame-mates. + - So t's three needers at the D6 frames {t, e}⊥ count every reader an odd number of times. + - The exceptions are t's 3 frame-mates, which are fixed by single "d-line" reads. +- **(c) Small, well-shared needer sums.** {t, e}⊥ holds 30 lines. Each is assembled from two quad outputs plus one Q-delivery, and each quad output serves 6 needers. +- **(d) Block nodes.** D4 blocks in complementary pairs supply the Q-deliveries. + +### 3.2 What I_{11,1} (and I_{13,1}) has (`cellcover.py`, `twocover.py`, `frames.py`, checked; parity argued) + +| feature | E8 | I_{11,1} | I_{13,1} | +|---|---|---|---| +| readers per source | 63 | 115 (triple), 129 (pair) | 183, 231 | +| cell cover number (branches into 2-dim cells {s, e}) | 3 | **3** (both types; CP-SAT OPTIMAL; attained by pairwise-orthogonal e's) | **3** (both types) | +| 2 branches of *any* frame dimension possible? | no (quadratic forms vanishing on readers: dim 0) | **triple: no (dim 0)**; pair: not excluded (dim 1; S3 has a 2-cover certificate) | triple: no; pair: not excluded | +| good 4-frames (orthogonal, F₂ parity) | 9,450 | **20,790**: 6,930 of type (pair, pair, triple, triple), 13,860 of type (triple)⁴ | not computed | +| partition into good 4-frames | yes (30) | **impossible**: every good frame holds 0 or 2 pairs, and 55 is odd | – | +| lines in a cell-perp (needer demand) | 30 | **56–70** | – | +| needers per port | 1.5 | 1.5 (3 per target, each shared by 2 targets) | 1.5 | + +**Parity (argued).** For example, {e₀−e₁, e₂−e₃, t_{401}, t_{423}} has half-sum e₀ + e₂ + e₄ + e₁₂ ∈ Z^{11,1} ∩ w⊥. For every integral line s, the sum of s's products with the frame is then even, so (b) holds verbatim, including the d-line correction. `frames.py` checks the parity condition directly on all 1,926,540 orthogonal 4-sets. + +### 3.3 A concrete skeleton sketch (argued count) and the ≤ 3 obstruction + +**Sketch: the E8 cell architecture transferred.** + +- Cover the lines by good 4-frames: 54 frames plus one frame reused for the odd pair, about 6 extra registers. +- Sources branch into their 3 cells: 2 carriers per source, 440. +- Each of the 330 cells holds 3 registers, adding 1 empty register per cell (330). +- Targets read 3 needers at {t, e}⊥ plus the d-lines. +- 11 totals. + +| variant | helpers | **per port** | +|---|---|---| +| all sources on cells | 440 + 330 + 11 + 6 = **787** | **3.58** | +| pair sources on their 2-cover (2 non-cell branches) | 385 + 330 + 11 + 6 = **732** | **3.33** | + +Not constructed: + +- the middle network (quad and block-node analogues); +- the d-line reads, cleanup, hosting. + +**The count assumes a middle as cheap per cell as E8's, while each needer must collect 56–70 lines instead of 30.** + +**Obstruction to ≤ 3 helpers per port (argued, conditional on the architecture).** + +- **Cell-branch designs.** Sources branch only into 2-dim cells and carriers are pure, as in #493/#543/#555. Then carriers are **exactly ≥ 2 per port**, because the cell cover number is 3 for both line types (checked). +- **Remaining budget.** ≤ 3 per port leaves ≤ 1 − 11/220 = 0.95 per port for the whole middle, i.e. ≤ 0.63 empty registers per cell. +- **Comparison with E8.** E8 needs 1 per cell (#493's family bound of 3 per cell) for half the demand per needer. +- **Any design with pure carriers.** Triple sources need ≥ 3 branches whatever the frame dimension (exact: no quadratic form vanishes on their 115 readers, the analogue of E8's design-strength carrier bound). So carriers are ≥ (2·165 + 55)/220 = **1.75 per port**, leaving ≤ 1.2 per port for the middle and totals. +- **Design-free floors.** S3 §4's floors (cover numbers 2–3, about 1 per port) remain the only bounds that hold for every design, so this is **not** a theorem excluding ≤ 3. +- **What it says.** A ≤ 3 skeleton must abandon E8's "3 registers per cell" middle, or share registers across sources (mixing carriers) far more than #555 does. + +## 4. Calibrated model on the best unit (Q3; `model3.py`, model) + +The best punctured unit is the unpunctured 220/110 (§2). S3's calibrated helper model (`est_model.py`, unchanged; bit calibration 1.0032 against #555) gives E8's target a = 1.8058·10⁻³ (#555 at ρ = 439/111): + +| unit | ρ* abs / rel | a_est at ρ = 3.577 (cells) | at 3.327 (pairs on 2 branches) | at 3.955 (E8's) | at 3.0 | at ρ_dim 5.207 | +|---|---|---|---|---|---|---| +| I_{11,1} 220/110 | 2.66 / 3.34 | 1.580 / 1.730 ·10⁻³ | 1.654 / **1.811**·10⁻³ | 1.480 / 1.620 | 1.716 / 1.875 | 1.225 / 1.339 | +| I_{11,1} 210/109 | 2.38 / 2.84 | 1.495 / 1.638 | 1.566 / 1.715 | 1.401 / 1.534 | 1.625 / 1.775 | 1.165 / 1.267 | + +(Pairs are abs / rel; the unit is 10⁻³.) + +Reading: + +- The transferred-skeleton intensity is 3.33–3.58 against break-even 2.66 / 3.34. +- Only the optimistic corner (rel scaling, pair 2-branch variant, a middle as cheap as E8's) reaches E8, at +0.3%. That is inside model error and rests on the unbuilt middle. +- Clearing E8 robustly (rel) needs ρ ≈ 3.0, i.e. the ≤ 0.63-per-cell middle of §3.3. + +## 5. Not done / open + +- **Exact rank-loss thresholds.** δ_d for the non-centre functionals: the MIP in a first draft was too slow and was dropped. These would extend P2 beyond 8 / 10 punctures (the 10-line example shows the first sub-10 total appears at exactly 10 for pair labels, and it is not an improvement). +- **Non-label decoders.** Non-label 11-total decoders on punctured I_{11,1}; Theorem C floors are listed for comparison. +- **The middle network.** Constructing a middle network for I_{11,1} with ≤ 0.63 empty registers per cell, or proving a family bound for it. D4 blocks and complementary pairs (E8 feature (d)) were not searched. +- **I_{13,1} frames.** Good-frame and frame-partition data for I_{13,1}. +- **Overlap check.** #920 (complex shared-parity frame dimensions), #912 (Hurwitz adaptive routing) and #907 (abelian Cayley metric scope) do not touch Lorentzian units, punctures or bit carrier skeletons. + +## Files + +- `scripts/census.py`: §1. +- `scripts/partition.py`: P1. +- `scripts/packbases.py`: P2. +- `scripts/punct.py`: exact star and Theorem C evaluation, §2.4. +- `scripts/example_pairlabel.py`: the 10-line puncture example of §2.2. +- `scripts/clique_orbits.py`, `scripts/punct_sat.py`: §2.3. +- `scripts/anneal.py`: §2.5, not in `verify.sh`. +- `scripts/cellcover.py`, `scripts/twocover.py`, `scripts/frames.py`: §3. +- `scripts/model3.py`: §4. +- Copied unchanged from S3: `units.py`, `est_model.py`, `rank.py`, `cover.py`. +- `scripts/check.py`, `expected/`, `verify.sh`; `data/` holds the anneal logs. + +## Credits + +This note builds on: + +- DreamingOfClouds: #595 (skel/cert, §A/§D/§F, Lemma Q), #446 Theorem C, #463's carrier bound, #545's punctured star; +- ShikeChen01: #634, the Lorentzian family; +- Chi Zhao: #631, the puncture-threshold method; +- eumemic: #493, #543 and the E8 bit skeletons; +- #555; +- S3 (GamingPuzzled). + +Prepared by Joel Pulikkan (GamingPuzzled) with Claude Opus 5.5 (Anthropic) via Claude Code. Apache-2.0. diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_X_4.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_X_4.log new file mode 100644 index 0000000000..0e654a524a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_X_4.log @@ -0,0 +1 @@ +final best -0.045454545454545456 (np.int64(110), 220, np.int64(11)) dead [] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_1.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_1.log new file mode 100644 index 0000000000..576e385301 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_1.log @@ -0,0 +1 @@ +final best 0.5 (np.int64(110), 220, np.int64(11)) dead [] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_2.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_2.log new file mode 100644 index 0000000000..576e385301 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_2.log @@ -0,0 +1 @@ +final best 0.5 (np.int64(110), 220, np.int64(11)) dead [] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_3.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_3.log new file mode 100644 index 0000000000..576e385301 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/data/anneal11_x_3.log @@ -0,0 +1 @@ +final best 0.5 (np.int64(110), 220, np.int64(11)) dead [] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover11.log new file mode 100644 index 0000000000..4b7c8cce2a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover11.log @@ -0,0 +1,4 @@ +n=11 triple source (readers 115) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=11 triple source (readers 115) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) +n=11 pair source (readers 129) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=11 pair source (readers 129) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover13.log new file mode 100644 index 0000000000..ac261138d2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover13.log @@ -0,0 +1,4 @@ +n=13 triple source (readers 183) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=13 triple source (readers 183) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) +n=13 pair source (readers 231) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=13 pair source (readers 231) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover8.log new file mode 100644 index 0000000000..7e133838b5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/cellcover8.log @@ -0,0 +1,2 @@ +n=8 E8 root source (readers 63) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=8 E8 root source (readers 63) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/census.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/census.log new file mode 100644 index 0000000000..f2667108c6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/census.log @@ -0,0 +1,24 @@ +11 {'k': 1, 'readers': 90, 'triples': 45, 'pairs': 45, 'span': 10, 'gram': 10} +11 {'k': 2, 'readers': 90, 'triples': 72, 'pairs': 18, 'span': 11, 'gram': 11} +11 {'k': 3, 'readers': 116, 'triples': 85, 'pairs': 31, 'span': 11, 'gram': 11} +11 {'k': 4, 'readers': 116, 'triples': 88, 'pairs': 28, 'span': 11, 'gram': 11} +11 {'k': 5, 'readers': 110, 'triples': 85, 'pairs': 25, 'span': 11, 'gram': 11} +11 {'k': 6, 'readers': 110, 'triples': 80, 'pairs': 30, 'span': 11, 'gram': 11} +11 {'k': 7, 'readers': 104, 'triples': 77, 'pairs': 27, 'span': 11, 'gram': 11} +11 {'k': 8, 'readers': 104, 'triples': 80, 'pairs': 24, 'span': 11, 'gram': 11} +11 {'k': 9, 'readers': 130, 'triples': 93, 'pairs': 37, 'span': 11, 'gram': 11} +11 {'k': 10, 'readers': 130, 'triples': 120, 'pairs': 10, 'span': 11, 'gram': 11} +11 {'k': 11, 'readers': 220, 'triples': 165, 'pairs': 55, 'span': 11, 'gram': 11} +13 {'k': 1, 'readers': 132, 'triples': 66, 'pairs': 66, 'span': 12, 'gram': 12} +13 {'k': 2, 'readers': 132, 'triples': 110, 'pairs': 22, 'span': 13, 'gram': 13} +13 {'k': 3, 'readers': 184, 'triples': 136, 'pairs': 48, 'span': 13, 'gram': 13} +13 {'k': 4, 'readers': 184, 'triples': 148, 'pairs': 36, 'span': 13, 'gram': 13} +13 {'k': 5, 'readers': 188, 'triples': 150, 'pairs': 38, 'span': 13, 'gram': 13} +13 {'k': 6, 'readers': 188, 'triples': 146, 'pairs': 42, 'span': 13, 'gram': 13} +13 {'k': 7, 'readers': 176, 'triples': 140, 'pairs': 36, 'span': 13, 'gram': 13} +13 {'k': 8, 'readers': 176, 'triples': 136, 'pairs': 40, 'span': 13, 'gram': 13} +13 {'k': 9, 'readers': 180, 'triples': 138, 'pairs': 42, 'span': 13, 'gram': 13} +13 {'k': 10, 'readers': 180, 'triples': 150, 'pairs': 30, 'span': 13, 'gram': 13} +13 {'k': 11, 'readers': 232, 'triples': 176, 'pairs': 56, 'span': 13, 'gram': 13} +13 {'k': 12, 'readers': 232, 'triples': 220, 'pairs': 12, 'span': 13, 'gram': 13} +13 {'k': 13, 'readers': 364, 'triples': 286, 'pairs': 78, 'span': 13, 'gram': 13} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/example.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/example.log new file mode 100644 index 0000000000..6478d610c2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/example.log @@ -0,0 +1 @@ +punctured 10: v=210 h=11 star=109 per total [9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10] thmC=108 x=109/210=0.5190 X*=6.34946e-03@s3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames11.log new file mode 100644 index 0000000000..6e0f546a89 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames11.log @@ -0,0 +1,6 @@ +n=11 v=220 4-frames=1926540 good=20790 types(line sizes)={(2, 2, 3, 3): 6930, (3, 3, 3, 3): 13860} + example [(0, 1, 2), (0, 3, 4), (1, 2), (3, 4)] + example [(0, 1, 2), (0, 3, 4), (1, 3, 5), (2, 4, 5)] +cell-perp {t,e} contains 56 lines; t-perp 115 +pairs per good frame: [0, 2]; pairs in total: 55 +NO partition into good 4-frames: every good frame holds an even number of pairs, 55 is odd diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames8.log new file mode 100644 index 0000000000..b9d67679f0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/frames8.log @@ -0,0 +1,4 @@ +n=8 v=120 4-frames=122850 good=9450 types(line sizes)={(0, 0, 0, 0): 9450} +cell-perp {t,e} contains 30 lines; t-perp 63 +partition into good 4-frames: OPTIMAL + verified partition of 120 lines into 30 good 4-frames; frame types {(0, 0, 0, 0): 30} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/model3.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/model3.log new file mode 100644 index 0000000000..58a3cef187 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/model3.log @@ -0,0 +1,13 @@ +E8 #555 calibrated model target a = 1.8058e-03 +I_{11,1} 220/110 (best; no improving puncture): x=1/2 X*=7.1715e-03 | rho* abs 2.66 rel 3.34 + rho skel_cells = 3.577: a_est abs 1.5796e-03 rel 1.7300e-03 (E8 1.8058e-03) + rho skel_pairs2 = 3.327: a_est abs 1.6537e-03 rel 1.8112e-03 (E8 1.8058e-03) + rho E8_shipped = 3.955: a_est abs 1.4795e-03 rel 1.6203e-03 (E8 1.8058e-03) + rho rho_dim = 5.207: a_est abs 1.2246e-03 rel 1.3388e-03 (E8 1.8058e-03) + rho three = 3.000: a_est abs 1.7162e-03 rel 1.8751e-03 (E8 1.8058e-03) +I_{11,1} punctured 210/109 (pair-label total at 9): x=109/210 X*=6.3495e-03 | rho* abs 2.38 rel 2.84 + rho skel_cells = 3.577: a_est abs 1.4954e-03 rel 1.6377e-03 (E8 1.8058e-03) + rho skel_pairs2 = 3.327: a_est abs 1.5655e-03 rel 1.7146e-03 (E8 1.8058e-03) + rho E8_shipped = 3.955: a_est abs 1.4006e-03 rel 1.5339e-03 (E8 1.8058e-03) + rho rho_dim = 5.207: a_est abs 1.1646e-03 rel 1.2673e-03 (E8 1.8058e-03) + rho three = 3.000: a_est abs 1.6246e-03 rel 1.7751e-03 (E8 1.8058e-03) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/orbits11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/orbits11.log new file mode 100644 index 0000000000..e8bcc374a6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/orbits11.log @@ -0,0 +1,6 @@ +n=11 omega=10 maximum cliques=38786 orbits=5 + orbit size 27720: 8 triples + 2 pairs, rep [(0, 1, 2), (0, 2, 4), (0, 2, 5), (0, 2, 6), (0, 2, 7), (3, 8, 10), (3, 9, 10), (8, 9, 10), (0, 10), (2, 10)] + orbit size 9240: 7 triples + 3 pairs, rep [(0, 1, 2), (3, 4, 10), (3, 5, 10), (3, 6, 10), (3, 7, 10), (3, 8, 10), (3, 9, 10), (0, 3), (1, 3), (2, 3)] + orbit size 495: 8 triples + 2 pairs, rep [(0, 1, 2), (0, 1, 3), (0, 1, 4), (0, 1, 5), (0, 1, 6), (0, 1, 7), (0, 1, 9), (0, 1, 10), (0, 8), (1, 8)] + orbit size 1320: 3 triples + 7 pairs, rep [(0, 1, 2), (0, 1, 3), (0, 2, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (0, 10)] + orbit size 11: 0 triples + 10 pairs, rep [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (0, 10)] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases11.log new file mode 100644 index 0000000000..889734de46 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases11.log @@ -0,0 +1,11 @@ +n=11 k=1: |R|=90 rank=10 coloops=0 []; rest rank 10 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/9) +n=11 k=2: |R|=90 rank=11 coloops=0 []; rest rank 11 holds 8 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/8) +n=11 k=3: |R|=116 rank=11 coloops=1 [0]; rest rank 10 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/11) +n=11 k=4: |R|=116 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=5: |R|=110 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=6: |R|=110 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=7: |R|=104 rank=11 coloops=0 []; rest rank 11 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/9) +n=11 k=8: |R|=104 rank=11 coloops=0 []; rest rank 11 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/9) +n=11 k=9: |R|=130 rank=11 coloops=1 [219]; rest rank 10 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/9) +n=11 k=10: |R|=130 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=11: |R|=220 rank=11 coloops=0 []; rest rank 11 holds 20 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/20) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases13.log new file mode 100644 index 0000000000..4f26d87979 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/packbases13.log @@ -0,0 +1,13 @@ +n=13 k=1: |R|=132 rank=12 coloops=0 []; rest rank 12 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/11) +n=13 k=2: |R|=132 rank=13 coloops=0 []; rest rank 13 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/10) +n=13 k=3: |R|=184 rank=13 coloops=1 [0]; rest rank 12 holds 15 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/15) +n=13 k=4: |R|=184 rank=13 coloops=0 []; rest rank 13 holds 14 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/14) +n=13 k=5: |R|=188 rank=13 coloops=0 []; rest rank 13 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/10) +n=13 k=6: |R|=188 rank=13 coloops=0 []; rest rank 13 holds 14 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/14) +n=13 k=7: |R|=176 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=8: |R|=176 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=9: |R|=180 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=10: |R|=180 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=11: |R|=232 rank=13 coloops=1 [363]; rest rank 12 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/11) +n=13 k=12: |R|=232 rank=13 coloops=0 []; rest rank 13 holds 12 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/12) +n=13 k=13: |R|=364 rank=13 coloops=0 []; rest rank 13 holds 28 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/28) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition11.log new file mode 100644 index 0000000000..58924687b1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition11.log @@ -0,0 +1 @@ +n=11: 1 copies of each of the 220 lines partitioned into 22 independent slots (2 per centre) VERIFIED exactly => sum_i rank(R(e_i) & L) >= (1/2)|L| for every live set L; slot sizes [10] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition13.log new file mode 100644 index 0000000000..d26df56c43 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/partition13.log @@ -0,0 +1 @@ +n=13: 3 copies of each of the 364 lines partitioned into 91 independent slots (7 per centre) VERIFIED exactly => sum_i rank(R(e_i) & L) >= (3/7)|L| for every live set L; slot sizes [12] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/punct11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/punct11.log new file mode 100644 index 0000000000..d1bca15dc7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/punct11.log @@ -0,0 +1,4 @@ +I_{11,1} minus coords []: v=220 h=11 rad=0 c0=11 star=110 [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10] thmC=110 x=1/2=0.5000 X*=7.17152e-03@s3 +I_{11,1} minus coords [10]: v=165 h=10 rad=0 c0=11 star=99 [9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9] thmC=88 x=3/5=0.6000 X*=4.04623e-03@s5 +I_{11,1} minus coords [9, 10]: v=120 h=8 rad=1 c0=10 star=80 [8, 8, 8, 8, 8, 8, 8, 8, 8, 8] thmC=80 x=2/3=0.6667 X*=3.43519e-03@s5 +I_{11,1} minus coords [8, 9, 10]: v=84 h=8 rad=0 c0=9 star=63 [7, 7, 7, 7, 7, 7, 7, 7, 7] thmC=63 x=3/4=0.7500 X*=1.98499e-03@s9 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o0.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o0.log new file mode 100644 index 0000000000..f69211c82f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o0.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 10, 11, 12, 13, 128, 129, 164, 174, 191] +result UNSAT sec 7484.3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o1.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o1.log new file mode 100644 index 0000000000..ebd5aa496d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o1.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 114, 119, 123, 126, 128, 129, 167, 176, 184] +result UNSAT sec 272.5 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o2.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o2.log new file mode 100644 index 0000000000..9df2abbbb7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o2.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 1, 2, 3, 4, 5, 7, 8, 172, 181] +result UNSAT sec 102.0 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o3.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o3.log new file mode 100644 index 0000000000..11ff895be8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o3.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 1, 9, 168, 169, 170, 171, 172, 173, 174] +result UNSAT sec 2413.2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o4.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o4.log new file mode 100644 index 0000000000..6374b814d8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/sat19_o4.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[165, 166, 167, 168, 169, 170, 171, 172, 173, 174] +result UNSAT sec 72.2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/survival11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/survival11.log new file mode 100644 index 0000000000..80cab23a7b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/survival11.log @@ -0,0 +1 @@ +{'count': 38786, 'max_per_line': 1784, 'survive_upto': 21, 'orbits': 5} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover11.log new file mode 100644 index 0000000000..077769d929 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover11.log @@ -0,0 +1,2 @@ +n=11 triple source: 115 readers spanning 10 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) +n=11 pair source: 129 readers spanning 10 dims; quadratic forms vanishing on all readers: dim 1 -> 2-cover not excluded diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover13.log new file mode 100644 index 0000000000..8ff142c645 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover13.log @@ -0,0 +1,2 @@ +n=13 triple source: 183 readers spanning 12 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) +n=13 pair source: 231 readers spanning 12 dims; quadratic forms vanishing on all readers: dim 1 -> 2-cover not excluded diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover8.log new file mode 100644 index 0000000000..4fcaaa148f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/expected/twocover8.log @@ -0,0 +1 @@ +n=8 E8 root source: 63 readers spanning 7 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover11.log new file mode 100644 index 0000000000..4b7c8cce2a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover11.log @@ -0,0 +1,4 @@ +n=11 triple source (readers 115) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=11 triple source (readers 115) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) +n=11 pair source (readers 129) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=11 pair source (readers 129) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover13.log new file mode 100644 index 0000000000..ac261138d2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover13.log @@ -0,0 +1,4 @@ +n=13 triple source (readers 183) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=13 triple source (readers 183) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) +n=13 pair source (readers 231) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=13 pair source (readers 231) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover8.log new file mode 100644 index 0000000000..7e133838b5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/cellcover8.log @@ -0,0 +1,2 @@ +n=8 E8 root source (readers 63) pairwise-orthogonal=False: cell cover number 3 (OPTIMAL) +n=8 E8 root source (readers 63) pairwise-orthogonal=True: cell cover number 3 (OPTIMAL) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/census.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/census.log new file mode 100644 index 0000000000..f2667108c6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/census.log @@ -0,0 +1,24 @@ +11 {'k': 1, 'readers': 90, 'triples': 45, 'pairs': 45, 'span': 10, 'gram': 10} +11 {'k': 2, 'readers': 90, 'triples': 72, 'pairs': 18, 'span': 11, 'gram': 11} +11 {'k': 3, 'readers': 116, 'triples': 85, 'pairs': 31, 'span': 11, 'gram': 11} +11 {'k': 4, 'readers': 116, 'triples': 88, 'pairs': 28, 'span': 11, 'gram': 11} +11 {'k': 5, 'readers': 110, 'triples': 85, 'pairs': 25, 'span': 11, 'gram': 11} +11 {'k': 6, 'readers': 110, 'triples': 80, 'pairs': 30, 'span': 11, 'gram': 11} +11 {'k': 7, 'readers': 104, 'triples': 77, 'pairs': 27, 'span': 11, 'gram': 11} +11 {'k': 8, 'readers': 104, 'triples': 80, 'pairs': 24, 'span': 11, 'gram': 11} +11 {'k': 9, 'readers': 130, 'triples': 93, 'pairs': 37, 'span': 11, 'gram': 11} +11 {'k': 10, 'readers': 130, 'triples': 120, 'pairs': 10, 'span': 11, 'gram': 11} +11 {'k': 11, 'readers': 220, 'triples': 165, 'pairs': 55, 'span': 11, 'gram': 11} +13 {'k': 1, 'readers': 132, 'triples': 66, 'pairs': 66, 'span': 12, 'gram': 12} +13 {'k': 2, 'readers': 132, 'triples': 110, 'pairs': 22, 'span': 13, 'gram': 13} +13 {'k': 3, 'readers': 184, 'triples': 136, 'pairs': 48, 'span': 13, 'gram': 13} +13 {'k': 4, 'readers': 184, 'triples': 148, 'pairs': 36, 'span': 13, 'gram': 13} +13 {'k': 5, 'readers': 188, 'triples': 150, 'pairs': 38, 'span': 13, 'gram': 13} +13 {'k': 6, 'readers': 188, 'triples': 146, 'pairs': 42, 'span': 13, 'gram': 13} +13 {'k': 7, 'readers': 176, 'triples': 140, 'pairs': 36, 'span': 13, 'gram': 13} +13 {'k': 8, 'readers': 176, 'triples': 136, 'pairs': 40, 'span': 13, 'gram': 13} +13 {'k': 9, 'readers': 180, 'triples': 138, 'pairs': 42, 'span': 13, 'gram': 13} +13 {'k': 10, 'readers': 180, 'triples': 150, 'pairs': 30, 'span': 13, 'gram': 13} +13 {'k': 11, 'readers': 232, 'triples': 176, 'pairs': 56, 'span': 13, 'gram': 13} +13 {'k': 12, 'readers': 232, 'triples': 220, 'pairs': 12, 'span': 13, 'gram': 13} +13 {'k': 13, 'readers': 364, 'triples': 286, 'pairs': 78, 'span': 13, 'gram': 13} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/clique_orbits11.json b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/clique_orbits11.json new file mode 100644 index 0000000000..1c2cd3f198 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/clique_orbits11.json @@ -0,0 +1,92 @@ +{ + "n": 11, + "omega": 10, + "count": 38786, + "orbits": [ + { + "size": 27720, + "rep": [ + 0, + 10, + 11, + 12, + 13, + 128, + 129, + 164, + 174, + 191 + ], + "triples": 8, + "pairs": 2 + }, + { + "size": 9240, + "rep": [ + 0, + 114, + 119, + 123, + 126, + 128, + 129, + 167, + 176, + 184 + ], + "triples": 7, + "pairs": 3 + }, + { + "size": 495, + "rep": [ + 0, + 1, + 2, + 3, + 4, + 5, + 7, + 8, + 172, + 181 + ], + "triples": 8, + "pairs": 2 + }, + { + "size": 1320, + "rep": [ + 0, + 1, + 9, + 168, + 169, + 170, + 171, + 172, + 173, + 174 + ], + "triples": 3, + "pairs": 7 + }, + { + "size": 11, + "rep": [ + 165, + 166, + 167, + 168, + 169, + 170, + 171, + 172, + 173, + 174 + ], + "triples": 0, + "pairs": 10 + } + ] +} \ No newline at end of file diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/example.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/example.log new file mode 100644 index 0000000000..6478d610c2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/example.log @@ -0,0 +1 @@ +punctured 10: v=210 h=11 star=109 per total [9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10] thmC=108 x=109/210=0.5190 X*=6.34946e-03@s3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames11.log new file mode 100644 index 0000000000..6e0f546a89 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames11.log @@ -0,0 +1,6 @@ +n=11 v=220 4-frames=1926540 good=20790 types(line sizes)={(2, 2, 3, 3): 6930, (3, 3, 3, 3): 13860} + example [(0, 1, 2), (0, 3, 4), (1, 2), (3, 4)] + example [(0, 1, 2), (0, 3, 4), (1, 3, 5), (2, 4, 5)] +cell-perp {t,e} contains 56 lines; t-perp 115 +pairs per good frame: [0, 2]; pairs in total: 55 +NO partition into good 4-frames: every good frame holds an even number of pairs, 55 is odd diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames8.log new file mode 100644 index 0000000000..b9d67679f0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/frames8.log @@ -0,0 +1,4 @@ +n=8 v=120 4-frames=122850 good=9450 types(line sizes)={(0, 0, 0, 0): 9450} +cell-perp {t,e} contains 30 lines; t-perp 63 +partition into good 4-frames: OPTIMAL + verified partition of 120 lines into 30 good 4-frames; frame types {(0, 0, 0, 0): 30} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.json b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.json new file mode 100644 index 0000000000..fccb6524e2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.json @@ -0,0 +1,48 @@ +{ + "target": 0.0018058295009581778, + "rhos": { + "skel_cells": 3.577272727272727, + "skel_pairs2": 3.327272727272727, + "E8_shipped": 3.954954954954955, + "rho_dim": 5.207357357357357, + "three": 3.0 + }, + "rows": [ + { + "name": "I_{11,1} 220/110 (best; no improving puncture)", + "x": "1/2", + "Xstar": 0.007171522653821519, + "Xs": 3, + "rho_star_abs": 2.661677615469671, + "a_skel_cells_abs": 0.001579623287924217, + "a_skel_pairs2_abs": 0.0016536874570675746, + "a_E8_shipped_abs": 0.0014795179684754338, + "a_rho_dim_abs": 0.0012246258053555746, + "a_three_abs": 0.0017161978484465659, + "rho_star_rel": 3.343132941881777, + "a_skel_cells_rel": 0.0017300228085654601, + "a_skel_pairs2_rel": 0.0018112054999365138, + "a_E8_shipped_rel": 0.001620305554277553, + "a_rho_dim_rel": 0.0013387673129528718, + "a_three_rel": 0.0018751364738724314 + }, + { + "name": "I_{11,1} punctured 210/109 (pair-label total at 9)", + "x": "109/210", + "Xstar": 0.00634946069331934, + "Xs": 3, + "rho_star_abs": 2.375657844095258, + "a_skel_cells_abs": 0.0014953581884038232, + "a_skel_pairs2_abs": 0.0015654701238274383, + "a_E8_shipped_abs": 0.0014005944896649703, + "a_rho_dim_abs": 0.0011646251714212523, + "a_three_abs": 0.0016246403189826247, + "rho_star_rel": 2.8384077548253117, + "a_skel_cells_rel": 0.0016377099752249154, + "a_skel_pairs2_rel": 0.001714558212979752, + "a_E8_shipped_rel": 0.0015338502528991858, + "a_rho_dim_rel": 0.0012673405048824879, + "a_three_rel": 0.0017750707046173396 + } + ] +} \ No newline at end of file diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.log new file mode 100644 index 0000000000..58a3cef187 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/model3.log @@ -0,0 +1,13 @@ +E8 #555 calibrated model target a = 1.8058e-03 +I_{11,1} 220/110 (best; no improving puncture): x=1/2 X*=7.1715e-03 | rho* abs 2.66 rel 3.34 + rho skel_cells = 3.577: a_est abs 1.5796e-03 rel 1.7300e-03 (E8 1.8058e-03) + rho skel_pairs2 = 3.327: a_est abs 1.6537e-03 rel 1.8112e-03 (E8 1.8058e-03) + rho E8_shipped = 3.955: a_est abs 1.4795e-03 rel 1.6203e-03 (E8 1.8058e-03) + rho rho_dim = 5.207: a_est abs 1.2246e-03 rel 1.3388e-03 (E8 1.8058e-03) + rho three = 3.000: a_est abs 1.7162e-03 rel 1.8751e-03 (E8 1.8058e-03) +I_{11,1} punctured 210/109 (pair-label total at 9): x=109/210 X*=6.3495e-03 | rho* abs 2.38 rel 2.84 + rho skel_cells = 3.577: a_est abs 1.4954e-03 rel 1.6377e-03 (E8 1.8058e-03) + rho skel_pairs2 = 3.327: a_est abs 1.5655e-03 rel 1.7146e-03 (E8 1.8058e-03) + rho E8_shipped = 3.955: a_est abs 1.4006e-03 rel 1.5339e-03 (E8 1.8058e-03) + rho rho_dim = 5.207: a_est abs 1.1646e-03 rel 1.2673e-03 (E8 1.8058e-03) + rho three = 3.000: a_est abs 1.6246e-03 rel 1.7751e-03 (E8 1.8058e-03) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/orbits11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/orbits11.log new file mode 100644 index 0000000000..e8bcc374a6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/orbits11.log @@ -0,0 +1,6 @@ +n=11 omega=10 maximum cliques=38786 orbits=5 + orbit size 27720: 8 triples + 2 pairs, rep [(0, 1, 2), (0, 2, 4), (0, 2, 5), (0, 2, 6), (0, 2, 7), (3, 8, 10), (3, 9, 10), (8, 9, 10), (0, 10), (2, 10)] + orbit size 9240: 7 triples + 3 pairs, rep [(0, 1, 2), (3, 4, 10), (3, 5, 10), (3, 6, 10), (3, 7, 10), (3, 8, 10), (3, 9, 10), (0, 3), (1, 3), (2, 3)] + orbit size 495: 8 triples + 2 pairs, rep [(0, 1, 2), (0, 1, 3), (0, 1, 4), (0, 1, 5), (0, 1, 6), (0, 1, 7), (0, 1, 9), (0, 1, 10), (0, 8), (1, 8)] + orbit size 1320: 3 triples + 7 pairs, rep [(0, 1, 2), (0, 1, 3), (0, 2, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (0, 10)] + orbit size 11: 0 triples + 10 pairs, rep [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (0, 10)] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases11.log new file mode 100644 index 0000000000..889734de46 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases11.log @@ -0,0 +1,11 @@ +n=11 k=1: |R|=90 rank=10 coloops=0 []; rest rank 10 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/9) +n=11 k=2: |R|=90 rank=11 coloops=0 []; rest rank 11 holds 8 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/8) +n=11 k=3: |R|=116 rank=11 coloops=1 [0]; rest rank 10 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/11) +n=11 k=4: |R|=116 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=5: |R|=110 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=6: |R|=110 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=7: |R|=104 rank=11 coloops=0 []; rest rank 11 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/9) +n=11 k=8: |R|=104 rank=11 coloops=0 []; rest rank 11 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/9) +n=11 k=9: |R|=130 rank=11 coloops=1 [219]; rest rank 10 holds 9 disjoint bases (verified exactly) -> rank(R - P) >= 10 - floor(|P|/9) +n=11 k=10: |R|=130 rank=11 coloops=0 []; rest rank 11 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/10) +n=11 k=11: |R|=220 rank=11 coloops=0 []; rest rank 11 holds 20 disjoint bases (verified exactly) -> rank(R - P) >= 11 - floor(|P|/20) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases13.log new file mode 100644 index 0000000000..4f26d87979 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/packbases13.log @@ -0,0 +1,13 @@ +n=13 k=1: |R|=132 rank=12 coloops=0 []; rest rank 12 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/11) +n=13 k=2: |R|=132 rank=13 coloops=0 []; rest rank 13 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/10) +n=13 k=3: |R|=184 rank=13 coloops=1 [0]; rest rank 12 holds 15 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/15) +n=13 k=4: |R|=184 rank=13 coloops=0 []; rest rank 13 holds 14 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/14) +n=13 k=5: |R|=188 rank=13 coloops=0 []; rest rank 13 holds 10 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/10) +n=13 k=6: |R|=188 rank=13 coloops=0 []; rest rank 13 holds 14 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/14) +n=13 k=7: |R|=176 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=8: |R|=176 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=9: |R|=180 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=10: |R|=180 rank=13 coloops=0 []; rest rank 13 holds 13 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/13) +n=13 k=11: |R|=232 rank=13 coloops=1 [363]; rest rank 12 holds 11 disjoint bases (verified exactly) -> rank(R - P) >= 12 - floor(|P|/11) +n=13 k=12: |R|=232 rank=13 coloops=0 []; rest rank 13 holds 12 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/12) +n=13 k=13: |R|=364 rank=13 coloops=0 []; rest rank 13 holds 28 disjoint bases (verified exactly) -> rank(R - P) >= 13 - floor(|P|/28) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition11.log new file mode 100644 index 0000000000..58924687b1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition11.log @@ -0,0 +1 @@ +n=11: 1 copies of each of the 220 lines partitioned into 22 independent slots (2 per centre) VERIFIED exactly => sum_i rank(R(e_i) & L) >= (1/2)|L| for every live set L; slot sizes [10] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition13.log new file mode 100644 index 0000000000..d26df56c43 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/partition13.log @@ -0,0 +1 @@ +n=13: 3 copies of each of the 364 lines partitioned into 91 independent slots (7 per centre) VERIFIED exactly => sum_i rank(R(e_i) & L) >= (3/7)|L| for every live set L; slot sizes [12] diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/punct11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/punct11.log new file mode 100644 index 0000000000..d1bca15dc7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/punct11.log @@ -0,0 +1,4 @@ +I_{11,1} minus coords []: v=220 h=11 rad=0 c0=11 star=110 [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10] thmC=110 x=1/2=0.5000 X*=7.17152e-03@s3 +I_{11,1} minus coords [10]: v=165 h=10 rad=0 c0=11 star=99 [9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9] thmC=88 x=3/5=0.6000 X*=4.04623e-03@s5 +I_{11,1} minus coords [9, 10]: v=120 h=8 rad=1 c0=10 star=80 [8, 8, 8, 8, 8, 8, 8, 8, 8, 8] thmC=80 x=2/3=0.6667 X*=3.43519e-03@s5 +I_{11,1} minus coords [8, 9, 10]: v=84 h=8 rad=0 c0=9 star=63 [7, 7, 7, 7, 7, 7, 7, 7, 7] thmC=63 x=3/4=0.7500 X*=1.98499e-03@s9 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o0.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o0.log new file mode 100644 index 0000000000..f69211c82f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o0.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 10, 11, 12, 13, 128, 129, 164, 174, 191] +result UNSAT sec 7484.3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o1.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o1.log new file mode 100644 index 0000000000..ebd5aa496d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o1.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 114, 119, 123, 126, 128, 129, 167, 176, 184] +result UNSAT sec 272.5 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o2.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o2.log new file mode 100644 index 0000000000..9df2abbbb7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o2.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 1, 2, 3, 4, 5, 7, 8, 172, 181] +result UNSAT sec 102.0 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o3.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o3.log new file mode 100644 index 0000000000..11ff895be8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o3.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[0, 1, 9, 168, 169, 170, 171, 172, 173, 174] +result UNSAT sec 2413.2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o4.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o4.log new file mode 100644 index 0000000000..6374b814d8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/sat19_o4.log @@ -0,0 +1,2 @@ +n=11 c=10 k=19 clauses=1504140 clique=[165, 166, 167, 168, 169, 170, 171, 172, 173, 174] +result UNSAT sec 72.2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/survival11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/survival11.log new file mode 100644 index 0000000000..80cab23a7b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/survival11.log @@ -0,0 +1 @@ +{'count': 38786, 'max_per_line': 1784, 'survive_upto': 21, 'orbits': 5} diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover11.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover11.log new file mode 100644 index 0000000000..077769d929 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover11.log @@ -0,0 +1,2 @@ +n=11 triple source: 115 readers spanning 10 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) +n=11 pair source: 129 readers spanning 10 dims; quadratic forms vanishing on all readers: dim 1 -> 2-cover not excluded diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover13.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover13.log new file mode 100644 index 0000000000..8ff142c645 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover13.log @@ -0,0 +1,2 @@ +n=13 triple source: 183 readers spanning 12 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) +n=13 pair source: 231 readers spanning 12 dims; quadratic forms vanishing on all readers: dim 1 -> 2-cover not excluded diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover8.log b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover8.log new file mode 100644 index 0000000000..4fcaaa148f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/run1-logs/twocover8.log @@ -0,0 +1 @@ +n=8 E8 root source: 63 readers spanning 7 dims; quadratic forms vanishing on all readers: dim 0 -> NO 2-cover (branch cover number >= 3) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/anneal.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/anneal.py new file mode 100644 index 0000000000..2e3731a979 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/anneal.py @@ -0,0 +1,80 @@ +"""Heuristic (model-free, exploratory) search for punctures of I_{n,1} that lower the star label-decoder ratio +x = C_star(P)/v (window kept) or raise X*(h, x). Float ranks; every reported point is re-evaluated exactly by punct.py. +Usage: python anneal.py n seed steps [objective: x|X]""" +import sys, math, random, itertools +import numpy as np +sys.dont_write_bytecode = True +import units as U + +n, seed, steps = int(sys.argv[1]), int(sys.argv[2]), int(sys.argv[3]) +obj = sys.argv[4] if len(sys.argv) > 4 else 'x' +rng = random.Random(seed) +L, G, lab = U.lorentz_family(n) +v = len(L) +Q = np.diag([1.0] * n + [-1.0]) +Lf = L.astype(float) +F = [np.array([(p >> j) & 1 for j in range(n)]) for p in range(1, 1 << n)] +RM = np.array([(lab @ f) % 2 for f in F], dtype=bool) # functional x line + + +def cost(live): + lv = np.nonzero(live)[0] + Ll = Lf[lv] + h = np.linalg.matrix_rank(Ll @ Q @ Ll.T) + rad = np.linalg.matrix_rank(Ll) - h + items = [] + for i in range(len(F)): + R = lv[RM[i, lv]] + if len(R) == 0: + continue + sp = np.linalg.matrix_rank(Lf[R] @ Q @ Ll.T) if rad else np.linalg.matrix_rank(Lf[R]) + w = 0 + for t in R: + w |= 1 << int(t) + items.append((sp, w)) + items.sort(key=lambda t: t[0]) + basis, tot, seen = {}, 0, set() + for sp, w in items: + if w in seen: + continue + seen.add(w) + x = w + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x; break + if x: + tot += sp + return tot, h + + +def score(live): + c, h = cost(live) + x = c / live.sum() + if obj == 'x': + return x - 0.0 * h, (c, int(live.sum()), h) + # proxy for X*: X ~ (2 - 3x)/... use D/Cost-like proxy at s = 3: (2 - 3x)/(h) as monotone stand-in + return -(2 - 3 * x) / h, (c, int(live.sum()), h) + + +live = np.ones(v, dtype=bool) +cur, info = score(live) +best = (cur, info, live.copy()) +T0 = 0.01 +for it in range(steps): + T = T0 * (1 - it / steps) + 1e-5 + nl = live.copy() + for _ in range(rng.choice([1, 1, 1, 2, 3])): + t = rng.randrange(v); nl[t] = not nl[t] + if nl.sum() < v // 3: + continue + s, inf = score(nl) + if s <= cur or rng.random() < math.exp(-(s - cur) / T): + live, cur, info = nl, s, inf + if s < best[0]: + best = (s, inf, live.copy()) + print('it %d best %.5f (C=%d v=%d h=%d) dead=%s' % (it, s, inf[0], inf[1], inf[2], + list(np.nonzero(~live)[0]) if (~live).sum() < 40 else '%d lines' % (~live).sum()), flush=True) +print('final best', best[0], best[1], 'dead', list(np.nonzero(~best[2])[0])) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cellcover.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cellcover.py new file mode 100644 index 0000000000..4bd4fbb1ee --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cellcover.py @@ -0,0 +1,50 @@ +"""Cell cover numbers (E8 skeleton analogue of the source-branch count). +A source s branches into cells {s, e} (e a line orthogonal to s); a branch at frame span{s, e} can be read by exactly +the targets t orthogonal to both s and e. The cell cover number b(s) is the least number of such e covering every +reader t of s (t orthogonal to s, t != s), where t is covered by e when t is orthogonal to e and t != e. +E8 control: b = 3 (the 4-frame, #463's carrier bound). CP-SAT, exact optimum (OPTIMAL status reported). +Also reports whether an optimal cover can be chosen pairwise orthogonal (a frame), and the target-side analogue is +the same number because the reader relation is symmetric. Usage: python cellcover.py n""" +import sys, itertools +import numpy as np +from ortools.sat.python import cp_model +sys.dont_write_bytecode = True +import units as U + + +def lines(n): + return list(itertools.combinations(range(n), 3)) + list(itertools.combinations(range(n), 2)) + + +def cover(G, s, frame=False, tl=600): + v = len(G) + R = [t for t in range(v) if t != s and G[s, t] == 0] + m = cp_model.CpModel() + x = {e: m.NewBoolVar('x%d' % e) for e in R} + for t in R: + m.AddBoolOr([x[e] for e in R if e != t and G[t, e] == 0]) + if frame: + for e, f in itertools.combinations(R, 2): + if G[e, f] != 0: + m.AddBoolOr([x[e].Not(), x[f].Not()]) + m.Minimize(sum(x.values())) + so = cp_model.CpSolver(); so.parameters.max_time_in_seconds = tl; so.parameters.num_workers = 4 + st = so.Solve(m) + return so.StatusName(st), int(so.ObjectiveValue()), int(so.BestObjectiveBound()), [e for e in R if so.Value(x[e])], len(R) + + +if __name__ == '__main__': + n = int(sys.argv[1]) + if n == 8: + sys.path.insert(0, '.') + L = U.roots_E8(); G = U.gram_doubled(L); reps = [('E8 root', 0)] + else: + L, G, lab = U.lorentz_family(n) + LN = lines(n) + reps = [('triple', 0), ('pair', LN.index((0, 1)))] + for name, s in reps: + for fr in (False, True): + st, val, bd, sol, z = cover(G, s, fr) + assert st == 'OPTIMAL' and val == bd + print('n=%s %s source (readers %d) pairwise-orthogonal=%s: cell cover number %d (OPTIMAL)' % ( + n, name, z, fr, val), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/census.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/census.py new file mode 100644 index 0000000000..fd83dabc27 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/census.py @@ -0,0 +1,25 @@ +"""Reader-set census of the label functionals of I_{n,1} (S_n orbit = Hamming weight of the functional). +For each weight k: |R(p)|, its type split (triples / pairs), span rank over Q, Gram (pairing) rank of G[R,R].""" +import sys, json +import numpy as np +sys.dont_write_bytecode = True +import units as U + +def census(n): + L, G, lab = U.lorentz_family(n) + out = [] + for k in range(1, n + 1): + p = np.array([1] * k + [0] * (n - k)) + R = np.nonzero((lab @ p) % 2)[0] + ntri = int((R < len(L) - n * (n - 1) // 2).sum()) + sp = U.rank_q(L[R].tolist()) if len(R) else 0 + gr = U.gram_rank(G, R) + out.append(dict(k=k, readers=len(R), triples=ntri, pairs=len(R) - ntri, span=sp, gram=gr)) + return out + +if __name__ == '__main__': + res = {} + for n in map(int, sys.argv[1:] or ['9', '11', '13']): + res[n] = census(n) + for r in res[n]: + print(n, r, flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/check.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/check.py new file mode 100644 index 0000000000..1bdc4bb6e6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/check.py @@ -0,0 +1,27 @@ +"""Compare every recomputed log with the frozen expected log, ignoring run times ('sec ...') and float noise in the +model lines (relative 1e-9). Usage: python check.py EXPECTED_DIR OUT_DIR""" +import os, re, sys +exp, out = sys.argv[1], sys.argv[2] +num = re.compile(r'-?\d+\.\d+(?:e[-+]?\d+)?') +norm = lambda s: re.sub(r'sec [\d.]+', 'sec *', re.sub(r'\d+\.\d+s user.*', '', s)).strip() +bad = 0 +for f in sorted(os.listdir(exp)): + if not f.endswith('.log'): + continue + a = [norm(l) for l in open(os.path.join(exp, f)) if l.strip()] + try: + b = [norm(l) for l in open(os.path.join(out, f)) if l.strip()] + except FileNotFoundError: + print('FAIL', f, 'missing'); bad += 1; continue + ok = len(a) == len(b) + if ok: + for x, y in zip(a, b): + if num.sub('#', x) != num.sub('#', y): + ok = False; break + for u, w in zip(num.findall(x), num.findall(y)): + if abs(float(u) - float(w)) > 1e-9 * max(1.0, abs(float(u))): + ok = False + print('PASS' if ok else 'FAIL', f) + bad += not ok +print('%d FAIL' % bad) +sys.exit(1 if bad else 0) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/clique_orbits.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/clique_orbits.py new file mode 100644 index 0000000000..3d345eb342 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/clique_orbits.py @@ -0,0 +1,76 @@ +"""Maximum pairwise non-orthogonal cliques of I_{n,1} (the #595 pattern graph, G_st != 0) and their S_n orbits. +Orbits are found by isomorphism of the coloured incidence graph (clique lines x points of [n]); each class is checked +with networkx's exact VF2 isomorphism against its representative.""" +import sys, itertools, json +from collections import Counter, defaultdict +import networkx as nx +import numpy as np +sys.dont_write_bytecode = True +import units as U + + +def lines(n): + return list(itertools.combinations(range(n), 3)) + list(itertools.combinations(range(n), 2)) + + +def inc_graph(n, cl, LN): + g = nx.Graph() + for p in range(n): + g.add_node(('p', p), col='p') + for t in cl: + g.add_node(('l', t), col='T' if len(LN[t]) == 3 else 'P') + for p in LN[t]: + g.add_edge(('l', t), ('p', p)) + return g + + +def orbits(n): + L, G, lab = U.lorentz_family(n) + LN = lines(n) + v = len(L) + H = nx.Graph(); H.add_nodes_from(range(v)) + H.add_edges_from((i, j) for i in range(v) for j in range(i + 1, v) if G[i, j] != 0) + cls = [sorted(c) for c in nx.find_cliques(H)] + om = max(map(len, cls)) + cls = [c for c in cls if len(c) == om] + by = defaultdict(list) + for c in cls: + g = inc_graph(n, c, LN) + by[nx.weisfeiler_lehman_graph_hash(g, node_attr='col', iterations=4)].append(c) + reps = [] + nm = lambda a, b: a['col'] == b['col'] + for hsh, cs in by.items(): + local = [] + for c in cs: + g = inc_graph(n, c, LN) + for r in local: + if nx.is_isomorphic(g, r[1], node_match=nm): + r[2].append(c); break + else: + local.append([c, g, [c]]) + reps += [(r[0], r[2]) for r in local] + return om, cls, reps, LN + + +if __name__ == '__main__': + n = int(sys.argv[1]) + om, cls, reps, LN = orbits(n) + print('n=%d omega=%d maximum cliques=%d orbits=%d' % (n, om, len(cls), len(reps))) + out = [] + for rep, mem in reps: + typ = Counter(len(LN[t]) for t in rep) + print(' orbit size %d: %d triples + %d pairs, rep %s' % (len(mem), typ[3], typ[2], [LN[t] for t in rep])) + out.append(dict(size=len(mem), rep=rep, triples=typ[3], pairs=typ[2])) + if len(sys.argv) > 2: + json.dump(dict(n=n, omega=om, count=len(cls), orbits=out), open(sys.argv[2], 'w'), indent=1) + + +def survival_bound(n): + """Union bound: k punctures kill at most k * max_t #{maximum cliques through t} cliques, so some maximum clique + survives every puncture of k < count / max_per_line lines (then the clique gauge of punct_sat.py is WLOG up to S_n, + run once per orbit).""" + om, cls, reps, LN = orbits(n) + per = Counter(t for c in cls for t in c) + mx = max(per.values()) + kmax = (len(cls) - 1) // mx + return dict(count=len(cls), max_per_line=mx, survive_upto=kmax, orbits=len(reps)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cover.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cover.py new file mode 100644 index 0000000000..041baf1e42 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/cover.py @@ -0,0 +1,147 @@ +"""#595 section D cover numbers for candidate units (source end; the terminal end is the same geometry for these +self-dual line systems). + +For a source s, the readers T(s) are the lines orthogonal to s. A register crossing from into serves the +readers in w^perp, a 'hyperplane section' of T(s). The cover number is the least number of proper sections covering T(s). + * lower bound 2 <=> rank_Q T(s) = h - 1 (then no single proper section covers T(s)); exact + * upper bound 2 <=> two subsets of T(s), each of rank <= h - 2, with union T(s) (certificate); exact check + (the witnesses are found by sampling hyperplanes through h - 2 random readers plus structured functionals; + a found pair is re-checked exactly) + * the best single section found (a LOWER bound on the largest section; sampled, not exhaustive) +#595's single-cut floors (d_geometry.single_cut_bounds): sink-only R >= (c - 1) v - floor(v / leftover), strict +R >= max(c v - 2 v, 0). With cover number 2 the strict floor is 0 and the sink-only floor is at most v. +Control: E8 bit (cover number 3, best pair union 52 of 63: #595 D1). +Usage: python cover.py REPO_ROOT unit[,unit...] [samples] [--json out] unit in {E8, T, L} +""" +import itertools +import sys + +import numpy as np + +sys.dont_write_bytecode = True +import units as U # noqa: E402 + + +def load(name): + if name == 'E8': + L = U.roots_E8(); G = U.gram_doubled(L); return L, G, [0] + if name.startswith('T'): + L, G, _ = U.triple_family(int(name[1:])); return L, G, [0] + if name.startswith('L'): + n = int(name[1:]) + L, G, _ = U.lorentz_family(n) + from math import comb + return L, G, [0, comb(n, 3)] # one triple-type and one pair-type source + raise ValueError(name) + + +def sections_for(L, G, s, h, samples, rng): + T = np.nonzero(G[s] == 0)[0] + T = T[T != s] + X = L[T].astype(float) + rT = U.rank_q(L[T].tolist()) + secs = set() + # structured functionals: coordinate patterns c with c . t = 0 + n = L.shape[1] + cands = [] + for i in range(n): + e = np.zeros(n); e[i] = 1; cands.append(e) + for i, j in itertools.combinations(range(n), 2): + for sg in (1, -1): + e = np.zeros(n); e[i] = 1; e[j] = sg; cands.append(e) + for i, j, k in itertools.combinations(range(n), 3): + for sg in itertools.product((1, -1), repeat=2): + e = np.zeros(n); e[i] = 1; e[j] = sg[0]; e[k] = sg[1]; cands.append(e) + for i, j in itertools.combinations(range(n), 2): # triple family: 1 off {i, j}, -2 on {i, j} (two-class sections) + e = np.ones(n); e[i] = e[j] = -2; cands.append(e) + for i in range(n - 1): + for wgt in (1, 2, 3): + e = np.ones(n); e[-1] = wgt; e[i] -= 1; cands.append(e) + e = np.ones(n); e[-1] = wgt; e[i] += 1; cands.append(e) + for c in cands: + m = np.abs(X @ c) < 1e-9 + if 0 < m.sum() < len(T): + secs.add(tuple(np.nonzero(m)[0])) + # random hyperplanes of span(T) through h - 2 readers + for _ in range(samples): + idx = rng.choice(len(T), rT - 1, replace=False) + M = X[idx] + if np.linalg.matrix_rank(M) < rT - 1: + continue + # normal of the hyperplane inside span(T): project onto span(T) via SVD of X + _, _, Vt = np.linalg.svd(np.vstack([M, np.zeros((0, n))])) + null = Vt[rT - 1:] # functionals vanishing on M + # choose a functional that does not vanish on all of T + best = None + for c in null: + m = np.abs(X @ c) < 1e-7 + if m.sum() < len(T): + best = m; break + if best is not None: + secs.add(tuple(np.nonzero(best)[0])) + return T, rT, secs + + +def run(name, samples=20000, seed=1): + L, G, srcs = load(name) + h = U.rank_q(G.tolist()) + rng = np.random.default_rng(seed) + out = [] + for s in srcs: + T, rT, secs = sections_for(L, G, s, h, samples, rng) + # exact check of every kept section: rank <= h - 2 (a proper section) + secs = [S for S in secs if U.rank_q(L[T[list(S)]].tolist()) <= h - 2] + masks = [sum(1 << int(k) for k in S) for S in secs] + full = (1 << len(T)) - 1 + best1 = max(len(S) for S in secs) + order = sorted(range(len(masks)), key=lambda i: -len(secs[i])) + bestpair, cover2 = 0, None + for a in range(len(order)): + ma = masks[order[a]] + for b in range(a, len(order)): + u = bin(ma | masks[order[b]]).count('1') + if u > bestpair: + bestpair = u + if u == len(T): + cover2 = (secs[order[a]], secs[order[b]]) + break + if cover2: + break + lb = 2 if rT == h - 1 else 1 + res = dict(unit=name, source=int(s), readers=len(T), rank_readers=rT, h=h, cover_lb=lb, + cover_ub=2 if cover2 else None, best_section_found=best1, best_pair_union=bestpair, + n_sections=len(secs)) + if not cover2: # look for a 3-cover among the largest sections + top = order[:400] + found = None + for a in range(len(top)): + for b in range(a, len(top)): + rest = full & ~(masks[top[a]] | masks[top[b]]) + for c in range(len(masks)): + if rest & ~masks[c] == 0: + found = (secs[top[a]], secs[top[b]], secs[c]); break + if found: + break + if found: + break + if found: + assert set().union(*found) == set(range(len(T))) + res['cover_ub'] = 3 + res['cover3_sizes'] = tuple(len(x) for x in found) + if cover2: + A, B = cover2 + assert U.rank_q(L[T[list(A)]].tolist()) <= h - 2 and U.rank_q(L[T[list(B)]].tolist()) <= h - 2 + assert set(A) | set(B) == set(range(len(T))) + res['cover2_sizes'] = (len(A), len(B)) + print(res, flush=True) + out.append(res) + return out + + +if __name__ == '__main__': + import json + allres = [] + for u in sys.argv[2].split(','): + allres += run(u, int(sys.argv[3]) if len(sys.argv) > 3 else 20000) + if '--json' in sys.argv: + json.dump(allres, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/est_model.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/est_model.py new file mode 100644 index 0000000000..2ba54d9815 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/est_model.py @@ -0,0 +1,135 @@ +"""Helper-adjusted estimate of the achievable saving a for a unit family (MODEL, not a bound). + +The #595 ceiling X(s, h, x) is helper-free. On the realised E8 words the gap to the ceiling is 87-93% helper cost +(#595 A.5). This model puts the helpers back with the piece statistics of the shipped words and solves the same +pressure root, so that unit families can be compared at the same design maturity: + + per port and replica (v = 1), stage count s = 2K + 1, a = h - 1, m = 1 + s a, D = (s - 1) - s x (#595 A.1, k = 1) + data 2 s a mass, cut into pieces with the per-port run histogram of the shipped word + (bit: #555 x/y runs; complex: #569 four-cold data runs, as replayed by #595 a4_gap.py) + copies s x mass in children of rank h - 1 (#555: 9 x 7, #568: 8 x 8) + helpers rho = R / v helpers per port, each climbing h per stage, cut with the shipped helper run histogram + (bit: #555, complex: #568's implied helper histogram, both from #595 a4_gap.py) + idle the B_K idle slots (skel.idle_slots, 2 per port), hosting: the helper mass is hosted into the largest idle + slots first; a hosted slot of rank r leaves a remainder child of rank r / a (the #555 pattern: 42, 28, 14 -> + 6, 4, 2); helper mass beyond the hostable mass stays as extra paid children (#595 step D semantics) +Piece-size scaling to another window h: + 'abs' pieces keep the shipped absolute ranks (their number grows like h) -- pessimistic for large h + 'rel' pieces scale with the window (rank x a/a_ref; same number of stops per register) -- optimistic +Calibration: the model is run on the E8 words themselves (bit h 8, x 63/111, rho 439/111; complex h 9, x 64/112, +rho 496/112) and the ratio to the published value (1.7331e-3 / 1.5987e-3, #595 A.5) is reported; the same factor is +applied to every candidate of that side (a_cal). +Also printed: #634's loss-chain factors 1/m, D/W (W = 2K v, whole hosting), L_ent = ln m - E ln r on the model profile. +""" +import math +import sys +from fractions import Fraction as Fr + +REPO = sys.argv[1] if len(sys.argv) > 1 else None +if REPO: + sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import idle_slots # noqa: E402 + +# shipped piece statistics (checked by #595 a4_gap.py: A4.runs.#555, A4.data.#568, A4.inputs.#568) +BIT = dict(h=8, v=111, R=439, + data={1: 10 + 317, 2: 209 + 92, 3: 10, 4: 3, 5: 92, 6: 10, 7: 9}, # x + y runs per stage + helpers={1: 994, 2: 1041, 3: 70, 4: 44, 5: 10}, published=Fr(17331, 10 ** 7), x=Fr(63, 111), s=9) +CX = dict(h=9, v=112, R=496, + data={1: 718, 2: 74, 3: 60, 4: 92, 5: 42, 6: 28}, # per call, v = 112 + helpers={1: 115952, 2: 26320, 3: 28064, 4: 7072, 5: 1008, 6: 2016}, published=Fr(15987, 10 ** 7), + x=Fr(64, 112), s=9) + + +def _norm(hist, mass): + tot = sum(r * z for r, z in hist.items()) + return {Fr(r): Fr(z) * mass / tot for r, z in hist.items()} + + +def profile(side, h, x, rho, s, mode): + ref = BIT if side == 'bit' else CX + a, K = h - 1, (s - 1) // 2 + m = 1 + s * a + D = (s - 1) - s * Fr(x) + scale = Fr(a, ref['h'] - 1) if mode == 'rel' else Fr(1) + prof = {} + + def add(r, n): + r = Fr(r) + if n <= 0 or r >= m: + return + prof[r] = prof.get(r, 0) + n + # data: 2 s a mass with the reference run shape + for r, n in _norm(ref['data'], 2 * s * a).items(): + rr = min(r * scale, a) + add(rr, n * r / rr) # keep the mass of each class + # copies + add(a, Fr(s) * Fr(x) / a) + # helpers + hmass = Fr(rho) * s * h + hsc = Fr(h, ref['h']) if mode == 'rel' else Fr(1) + for r, n in _norm(ref['helpers'], hmass).items(): + rr = min(r * hsc, h) + add(rr, n * r / rr) + # idle with hosting + slots = sorted([r for r in idle_slots(K, a) for _ in range(2)], reverse=True) + left = hmass + for r in slots: + rem = Fr(r, a) # remainder piece of a hosted slot + cap = r - rem + if left >= cap: + left -= cap + add(rem, 1) + elif left > 0: + add(r - left, 1) # partially hosted slot: one paid child of the rest + left = 0 + else: + add(r, 1) + return prof, m, D + + +def root(prof, m, D, iters=200): + m = float(m); D = float(D) + F = lambda q: sum(float(z) * float(r) * ((m / float(r)) ** q - 1) for r, z in prof.items()) + lo, hi = 0.0, 0.5 + for _ in range(iters): + mid = (lo + hi) / 2 + if F(mid) >= D: + hi = mid + else: + lo = mid + return hi + + +def chain(prof, m, D, s): + """#634 loss chain on the model profile: a ~ (1/m)(D/W)/L_ent with W = (mass + D)/m data-equivalent wires per port + (= 2K when every helper is hosted) and L_ent = ln m - E ln r (rank-weighted).""" + mass = sum(float(r * z) for r, z in prof.items()) + W = (mass + float(D)) / float(m) + Eln = sum(float(r * z) * math.log(float(r)) for r, z in prof.items()) / mass + Lent = math.log(float(m)) - Eln + return dict(inv_m=1 / float(m), W=W, DW=float(D) / W, Lent=Lent, first=float(D) / W / float(m) / Lent) + + +def estimate(side, h, x, rho, mode, S=range(3, 26, 2)): + best = None + for s in S: + prof, m, D = profile(side, h, x, rho, s, mode) + if D <= 0: + continue + a = root(prof, m, D) + if best is None or a > best[1]: + best = (s, a, chain(prof, m, D, s)) + return best + + +def calibration(side, mode): + ref = BIT if side == 'bit' else CX + prof, m, D = profile(side, ref['h'], ref['x'], Fr(ref['R'], ref['v']), ref['s'], mode) + return float(ref['published']) / root(prof, m, D) + + +if __name__ == '__main__': + for side in ('bit', 'complex'): + for mode in ('abs', 'rel'): + print(side, mode, 'calibration factor (published / model at s = 9): %.4f' % calibration(side, mode)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/example_pairlabel.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/example_pairlabel.py new file mode 100644 index 0000000000..d012374848 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/example_pairlabel.py @@ -0,0 +1,13 @@ +"""First cheap non-centre functional: puncture the pair {0,1} and the 9 triples {0,1,x}. Exact star evaluation.""" +import sys, itertools +sys.dont_write_bytecode = True +import punct +punct._REPO = sys.argv[1] +LN = list(itertools.combinations(range(11), 3)) + list(itertools.combinations(range(11), 2)) +dead = {LN.index((0, 1))} | {LN.index(tuple(sorted((0, 1, x)))) for x in range(2, 11)} +live = [i for i in range(220) if i not in dead] +r = punct.evaluate(11, live) +xs = punct.Xstar(r['h'], r['x']) +print('punctured %d: v=%d h=%d star=%d per total %s thmC=%d x=%s=%.4f X*=%.5e@s%d' % ( + len(dead), r['v'], r['h'], r['star'], sorted(c for c, _ in r['star_totals']), r['thmC'], r['x'], float(r['x']), + float(xs[1]), xs[0])) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/frames.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/frames.py new file mode 100644 index 0000000000..d15654b518 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/frames.py @@ -0,0 +1,89 @@ +"""E8 skeleton features transferred to I_{n,1}: good 4-frames, F2 parity, partitions, cell-perp sizes. +A 4-frame is a set of 4 pairwise orthogonal lines. It is GOOD when for every member t and every line s orthogonal to t +(s not in the frame) the number of other members orthogonal to s is odd (E8: the half-sum of a 4-frame is a lattice +vector, so the count is 1 or 3). Then the three cells {t, e} of t cover t-perp, and over F2 the three cell-perp sums +count every non-frame reader of t an odd number of times (the inclusion-exclusion is free; only the 3 frame lines +are counted twice and need the single 'd-line' reads of #493/#543). +Checks: number of 4-frames, number of good 4-frames, orbit types, and whether the lines can be partitioned into good +4-frames (CP-SAT exact cover). Usage: python frames.py n (n = 8 is the E8 control)""" +import sys, itertools +from collections import Counter +import numpy as np +from ortools.sat.python import cp_model +sys.dont_write_bytecode = True +import units as U + + +def setup(n): + if n == 8: + L = U.roots_E8(); G = U.gram_doubled(L); LN = [tuple(r) for r in L] + else: + L, G, lab = U.lorentz_family(n) + LN = list(itertools.combinations(range(n), 3)) + list(itertools.combinations(range(n), 2)) + return L, G, LN + + +def frames4(G): + v = len(G) + O = [set(np.nonzero(G[i] == 0)[0]) - {i} for i in range(v)] + out = [] + for a in range(v): + for b in O[a]: + if b <= a: continue + for c in O[a] & O[b]: + if c <= b: continue + for d in O[a] & O[b] & O[c]: + if d <= c: continue + out.append((a, b, c, d)) + return out, O + + +def good(F, O): + for t in F: + others = [e for e in F if e != t] + for s in O[t]: + if s in F: continue + if sum(1 for e in others if s in O[e]) % 2 == 0: + return False + return True + + +if __name__ == '__main__': + n = int(sys.argv[1]) + L, G, LN = setup(n) + v = len(G) + Fs, O = frames4(G) + gd = [F for F in Fs if good(F, O)] + typ = Counter(tuple(sorted(len(LN[t]) if n != 8 else 0 for t in F)) for F in gd) + print('n=%d v=%d 4-frames=%d good=%d types(line sizes)=%s' % (n, v, len(Fs), len(gd), dict(typ)), flush=True) + if n != 8: + for ty in typ: + ex = next(F for F in gd if tuple(sorted(len(LN[t]) for t in F)) == ty) + print(' example', [LN[t] for t in ex]) + # cell-perp sizes + if gd: + F = gd[0]; t, e = F[0], F[1] + print('cell-perp {t,e} contains %d lines; t-perp %d' % (len(O[t] & O[e]), len(O[t]))) + if n != 8: + npairs = sum(1 for t in range(v) if len(LN[t]) == 2) + ppf = sorted(set(sum(1 for t in F if len(LN[t]) == 2) for F in gd)) + print('pairs per good frame: %s; pairs in total: %d' % (ppf, npairs)) + if all(c % 2 == 0 for c in ppf) and npairs % 2 == 1: + print('NO partition into good 4-frames: every good frame holds an even number of pairs, %d is odd' % npairs) + sys.exit(0) + if v % 4 == 0 and gd: + m = cp_model.CpModel() + x = [m.NewBoolVar('f%d' % i) for i in range(len(gd))] + inc = [[] for _ in range(v)] + for i, F in enumerate(gd): + for t in F: inc[t].append(x[i]) + for t in range(v): + m.AddExactlyOne(inc[t]) + so = cp_model.CpSolver(); so.parameters.max_time_in_seconds = 600; so.parameters.num_workers = 4 + st = so.Solve(m) + print('partition into good 4-frames:', so.StatusName(st)) + if st in (cp_model.OPTIMAL, cp_model.FEASIBLE): + P = [gd[i] for i in range(len(gd)) if so.Value(x[i])] + assert sorted(t for F in P for t in F) == list(range(v)) and all(good(F, O) for F in P) + print(' verified partition of %d lines into %d good 4-frames; frame types %s' % ( + v, len(P), dict(Counter(tuple(sorted(len(LN[t]) if n != 8 else 0 for t in F)) for F in P)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/model3.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/model3.py new file mode 100644 index 0000000000..0c7767062c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/model3.py @@ -0,0 +1,41 @@ +"""Q3 (MODEL, not a bound): S3's calibrated helper model (est_model.py, unchanged) on the best punctured I_{11,1} unit. +Q1 shows no puncture lowers x below 1/2 for centre decoders (certificate) and none was found for label decoders, +so the best unit is the unpunctured 220/110 (h = 11); the first sub-10 total (210/109) is shown for comparison. +Helper intensities rho (helpers per port): + skel_cells : E8 cell architecture transferred (2 carriers/source + 1 aux per cell, 3 cells/line/2 = 1.5/port, + + 11 totals + one extra 4-frame for the odd pair count: (440 + 330 + 11 + 6)/220) + skel_pairs2 : same, pair sources with 2 branches (pair 2-cover; frames not cells): (385 + 330 + 11 + 6)/220 + E8 shipped : 439/111 + rho_dim : S3's readers-per-window-dimension transfer +Usage: python model3.py REPO_ROOT [--json out]""" +import sys, json +from fractions import Fraction as Fr +sys.dont_write_bytecode = True +import rank as RK # noqa (imports skel via REPO in argv[1]) +E = RK.E + +units = [dict(name='I_{11,1} 220/110 (best; no improving puncture)', h=11, v=220, cst=110), + dict(name='I_{11,1} punctured 210/109 (pair-label total at 9)', h=11, v=210, cst=109)] +rhos = {'skel_cells': Fr(440 + 330 + 11 + 6, 220), 'skel_pairs2': Fr(385 + 330 + 11 + 6, 220), + 'E8_shipped': Fr(439, 111), 'rho_dim': Fr(439, 111) * Fr(165 * 115 + 55 * 129, 220) / 10 / Fr(63, 7), + 'three': Fr(3)} +cals = {md: E.calibration('bit', md) for md in ('abs', 'rel')} +target = RK.cal_est('bit', 8, Fr(63, 111), Fr(439, 111), 'abs', E.calibration('bit', 'abs'))[1] +out = [] +print('E8 #555 calibrated model target a = %.4e' % target) +for u in units: + x = Fr(u['cst'], u['v']) + xs = RK.xstar(u['h'], x) + row = dict(name=u['name'], x=str(x), Xstar=float(xs[1]), Xs=xs[0]) + for md in ('abs', 'rel'): + row['rho_star_' + md] = RK.breakeven('bit', u['h'], x, md, cals[md], target) + for k, r in rhos.items(): + s_, a_, _ = RK.cal_est('bit', u['h'], x, r, md, cals[md]) + row['a_%s_%s' % (k, md)] = a_ + out.append(row) + print('%s: x=%s X*=%.4e | rho* abs %.2f rel %.2f' % (u['name'], x, row['Xstar'], row['rho_star_abs'], row['rho_star_rel'])) + for k, r in rhos.items(): + print(' rho %-11s = %.3f: a_est abs %.4e rel %.4e (E8 %.4e)' % (k, float(r), row['a_%s_abs' % k], row['a_%s_rel' % k], target)) +if '--json' in sys.argv: + json.dump(dict(target=target, rhos={k: float(v) for k, v in rhos.items()}, rows=out), + open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/packbases.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/packbases.py new file mode 100644 index 0000000000..a3ef1601b8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/packbases.py @@ -0,0 +1,67 @@ +"""Disjoint spanning sets inside reader sets: a puncture-robustness certificate. +For a functional p (Hamming weight k, S_n orbit representative) we pack q pairwise disjoint bases of span R(p) inside +R(p) (matroid union by augmenting paths; every packed basis is re-verified with exact rational rank). +If R(p) holds q disjoint bases, deleting any P leaves rank(R(p) - P) >= rank R(p) - floor(|P| / q): some basis loses +at most floor(|P|/q) lines. Usage: python packbases.py n""" +import sys +from collections import deque +import numpy as np +sys.dont_write_bytecode = True +import units as U + + +def pack(Lsub, q, seed=0): + Lf = Lsub.astype(float) + m = len(Lsub) + S = [set() for _ in range(q)] + where = {} + ind = lambda ss: len(ss) == 0 or np.linalg.matrix_rank(Lf[sorted(ss)]) == len(ss) + for x in range(m): + prev = {x: None}; dq = deque([x]); found = None + while dq and found is None: + y = dq.popleft() + for si in range(q): + if where.get(y) == si: + continue + if ind(S[si] | {y}): + found = (y, si); break + for z in S[si]: + if z not in prev and ind((S[si] - {z}) | {y}): + prev[z] = (y, si); dq.append(z) + if found is None: + continue + y, si = found + while True: + old = where.get(y) + if old is not None: + S[old].discard(y) + S[si].add(y); where[y] = si + if prev[y] is None: + break + y, si = prev[y] + return S + + +if __name__ == '__main__': + n = int(sys.argv[1]) + L, G, lab = U.lorentz_family(n) + for k in range(1, n + 1): + p = np.array([1] * k + [0] * (n - k)) + R = np.nonzero((lab @ p) % 2)[0] + r0 = U.rank_q(L[R].tolist()) + # strip coloops (lines outside the span of the rest of R): deleting one drops the rank by one + col = [t for t in R if U.rank_q(L[[u for u in R if u != t]].tolist()) < r0] + R = np.array([t for t in R if t not in col]) + r = U.rank_q(L[R].tolist()) + assert r == r0 - len(col) + q = len(R) // r + while q > 0: + S = pack(L[R], q) + if all(len(s) == r for s in S): + break + q -= 1 + for s in S: + assert U.rank_q(L[R[sorted(s)]].tolist()) == r + assert len(set().union(*S)) == q * r + print('n=%d k=%d: |R|=%d rank=%d coloops=%d %s; rest rank %d holds %d disjoint bases (verified exactly)' + ' -> rank(R - P) >= %d - floor(|P|/%d)' % (n, k, len(R) + len(col), r0, len(col), [int(c) for c in col], r, q, r, q), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/partition.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/partition.py new file mode 100644 index 0000000000..f8c7370cf9 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/partition.py @@ -0,0 +1,86 @@ +"""Matroid-partition certificate for the centre decoder of I_{n,1} under every puncture. + +Claim: for every live set L of lines, sum_i rank(R(e_i) & L) >= |L| / 2. +Certificate: a partition of all v lines into 2n sets I_{i,1}, I_{i,2}, each linearly independent and contained in +R(e_i) (the readers of centre i). Then 2 rank(R(e_i) & L) >= |L & I_{i,1}| + |L & I_{i,2}| and summing gives the +claim. For n = 11, 13: v = 2 n (n - 1) / 2 ... (220 = 22 x 10, 364 = 26 x 14 ... see output). +Found by Edmonds/Knuth matroid partitioning (shortest augmenting paths, float rank oracle); the final partition is +verified with exact rational ranks. Usage: python partition.py n copies mult [--json out] +(copies of each line = mult, slots per centre = copies: proves sum_i rank >= (mult/copies)|L|)""" +import sys, json, random +from collections import deque +import numpy as np +sys.dont_write_bytecode = True +import units as U + + +def run(n, copies=2, seed=0, mult=1): + L0, G, lab0 = U.lorentz_family(n) + L = np.repeat(L0, mult, axis=0); lab = np.repeat(lab0, mult, axis=0) # each line mult times + v = len(L) + Lf = L.astype(float) + allowed = [[i for i in range(n) if lab[t, i]] for t in range(v)] # t in R(e_i) <=> label_t[i] = 1 + slots = [(i, c) for i in range(n) for c in range(copies)] + S = [set() for _ in slots] + where = {} + indep = lambda ss: len(ss) == 0 or np.linalg.matrix_rank(Lf[sorted(ss)]) == len(ss) + rng = random.Random(seed) + order = list(range(v)); rng.shuffle(order) + for x in order: + # BFS: nodes are elements; edge y -> slot s: y can enter s (allowed) and S_s + y indep => done + # else y -> z for z in S_s with S_s - z + y independent (z then must move) + prev = {x: None} + q = deque([x]); found = None + while q and found is None: + y = q.popleft() + for si, (i, c) in enumerate(slots): + if i not in allowed[y] or where.get(y) == si: + continue + if indep(S[si] | {y}): + found = (y, si); break + for z in S[si]: + if z not in prev and indep((S[si] - {z}) | {y}): + prev[z] = (y, si); q.append(z) + if found is None: + return None, x + y, si = found + # augment along the path: y goes into si; then walk back + while True: + old = where.get(y) + if old is not None: + S[old].discard(y) + S[si].add(y); where[y] = si + if prev[y] is None: + break + y, si = prev[y][0], prev[y][1] + # prev[z] = (y', si'): y' entered si' displacing z -- here z is the element we just moved; walk on + # sanity + return (S, slots, L, lab), None + + +def verify(S, slots, L, lab): + for (i, c), ss in zip(slots, S): + ss = sorted(ss) + assert all(lab[t, i] for t in ss) + assert U.rank_q(L[ss].tolist()) == len(ss), 'dependent slot' + allv = sorted(t for ss in S for t in ss) + assert allv == list(range(len(L))), 'not a partition' + return True + + +if __name__ == '__main__': + n = int(sys.argv[1]); copies = int(sys.argv[2]); mult = int(sys.argv[3]) + for seed in range(5): + res, fail = run(n, copies, seed, mult) + if res: + S, slots, L, lab = res + verify(S, slots, L, lab) + print('n=%d: %d copies of each of the %d lines partitioned into %d independent slots (%d per centre) VERIFIED exactly' + ' => sum_i rank(R(e_i) & L) >= (%d/%d)|L| for every live set L; slot sizes %s' % ( + n, mult, len(L) // mult, len(slots), copies, mult, copies, sorted(set(len(s) for s in S)))) + if '--json' in sys.argv: + json.dump(dict(n=n, copies=copies, slots=[dict(centre=i, copy=c, lines=sorted(int(t) for t in s)) + for (i, c), s in zip(slots, S)]), + open(sys.argv[sys.argv.index('--json') + 1], 'w')) + break + print('seed %d: no augmenting path for element %d (a set violating |S| <= copies*rank exists)' % (seed, fail)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct.py new file mode 100644 index 0000000000..13c2a25093 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct.py @@ -0,0 +1,107 @@ +"""Punctures of the Lorentzian units I_{n,1} (#634 note 3), star label decoders and Theorem C floors. + +A puncture keeps a set of live lines. For a live set: + h = Gram rank of the live lines (stage window, after the Lemma Q quotient by the radical), + c0 = F2 rank of the live labels, + star = min over bases of the live label space of sum_j dim_Q span(R(p_j) & live) (matroid greedy, exact) + thmC = the same with the Gram rank of the block G[R,R] (#446 Theorem C pairing floor of a label total) + x = star / v_live. +Spans are measured modulo the radical of the live Gram matrix (Lemma Q, #595 C.4), so that n = 9 gives E8 (8 per total).""" +import itertools, sys +from fractions import Fraction as Fr +import numpy as np +sys.dont_write_bytecode = True +import units as U + + +def family(n): + L, G, lab = U.lorentz_family(n) + return L, G, lab + + +def _rank_f(M): + if len(M) == 0: + return 0 + return int(np.linalg.matrix_rank(np.asarray(M, dtype=float))) + + +def evaluate(n, live, exact=True, fam=None): + L, G, lab = fam or family(n) + live = np.asarray(sorted(live)) + Ll, Gl, labl = L[live], G[np.ix_(live, live)], lab[live] + rk = U.rank_q if exact else _rank_f + span_all = rk(Ll.tolist()) + h = rk(Gl.tolist()) + rad = span_all - h # dimension of the radical of the live span + # radical basis: vectors in span(Ll) orthogonal (Lorentz) to every live line + Q = np.diag([1] * n + [-1]) + costs = [] + for pint in range(1, 1 << n): + pv = np.array([(pint >> j) & 1 for j in range(n)]) + R = np.nonzero((labl @ pv) % 2)[0] + if len(R) == 0: + continue + sp = rk(Ll[R].tolist()) + if rad: + # dim of image in span/radical: rank of Gram pairing of R against all live lines + sp = rk((Ll[R] @ Q @ Ll.T).tolist()) + gr = rk(Gl[np.ix_(R, R)].tolist()) + word = sum(1 << int(i) for i in R) # reader word over the live lines + costs.append((sp, gr, word, pint)) + star, used = _greedy(costs, 0, n) + thc, usedc = _greedy(costs, 1, n) + c0 = U.f2_rank(labl) + return dict(n=n, v=len(live), h=h, rad=rad, c0=c0, star=star, star_totals=used, thmC=thc, thmC_totals=usedc, + x=Fr(star, len(live))) + + +def _greedy(costs, key, n): + cs = sorted(costs, key=lambda t: (t[key], t[3])) + basis, tot, used = {}, 0, [] + seen = set() + for t in cs: + if t[2] in seen: + continue + seen.add(t[2]) + x = t[2] + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + if x: + tot += t[key]; used.append((t[key], t[3])) + return tot, used + + +def coord_puncture(n, S): + """live = lines not involving any coordinate in S (triples A with A & S empty, pairs disjoint from S).""" + T = list(itertools.combinations(range(n), 3)); P = list(itertools.combinations(range(n), 2)) + lines = T + P + return [i for i, l in enumerate(lines) if not set(l) & set(S)] + + +def Xstar(h, x, smax=15): + sys.path.insert(0, _REPO + '/research/format-limits/scripts') + from skel import X + best = None + for s in range(3, smax + 1, 2): + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi']) + return best + + +_REPO = None +if __name__ == '__main__': + _REPO = sys.argv[1] + for n in map(int, sys.argv[2:]): + for k in range(0, n - 7): + live = coord_puncture(n, range(n - k, n)) + r = evaluate(n, live) + xs = Xstar(r['h'], r['x']) + print('I_{%d,1} minus coords %s: v=%d h=%d rad=%d c0=%d star=%d %s thmC=%d x=%s=%.4f X*=%s' % ( + n, list(range(n - k, n)), r['v'], r['h'], r['rad'], r['c0'], r['star'], sorted(c for c, _ in r['star_totals']), + r['thmC'], r['x'], float(r['x']), ('%.5e@s%d' % (float(xs[1]), xs[0])) if xs else 'D<=0'), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct_sat.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct_sat.py new file mode 100644 index 0000000000..e0d7ade518 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/punct_sat.py @@ -0,0 +1,75 @@ +"""Can a puncture of at most k lines bring the F2 minrank of the I_{n,1} pattern down to n - 1? +Variables: live selectors y_t, sink/source labels W, U (v x c over F2), B = W U^T. Constraints: live t => B_tt = 1; +live s, t with G_st != 0 => B_ts = 0; at most k lines dead. Gauge U_K = W_K = [I|0] on one fixed maximum clique K that +is forced live (WLOG while k is below the clique hitting number, see clique_hit.py, and S_n is transitive on maximum +cliques). CaDiCaL 1.5.3 via python-sat, no proof log. +Usage: python punct_sat.py n c k [timeout_s [orbits.json orbit_index]]""" +import sys, time, itertools +import numpy as np +from pysat.formula import CNF, IDPool +from pysat.card import CardEnc, EncType +from pysat.solvers import Cadical153 +sys.dont_write_bytecode = True +import units as U + + +def build(n, c, k, K=None): + L, G, lab = U.lorentz_family(n) + v = len(L) + if K is None: + K = U.nonorth_clique(G) + pool = IDPool(); cl = [] + nv = lambda: pool.id() + y = [nv() for _ in range(v)] + Uv = [[nv() for _ in range(c)] for _ in range(v)] + Wv = [[nv() for _ in range(c)] for _ in range(v)] + + def AND(a, b): + z = nv(); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + + def XOR(a, b): + z = nv(); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + + def B(T, S): + lits = [AND(Wv[T][j], Uv[S][j]) for j in range(c)] + z = lits[0] + for l in lits[1:]: + z = XOR(z, l) + return z + for T in range(v): + cl.append([-y[T], B(T, T)]) + for T in range(v): + for S in range(v): + if S != T and G[T, S] != 0: + cl.append([-y[T], -y[S], -B(T, S)]) + for i, T in enumerate(K): + cl.append([y[T]]) + for j in range(c): + cl.append([Uv[T][j]] if j == i else [-Uv[T][j]]) + cl.append([Wv[T][j]] if j == i else [-Wv[T][j]]) + card = CardEnc.atmost(lits=[-x for x in y], bound=k, vpool=pool, encoding=EncType.seqcounter) + cl.extend(card.clauses) + return cl, y, Uv, Wv, L, G, lab, K + + +if __name__ == '__main__': + n, c, k = map(int, sys.argv[1:4]) + T = float(sys.argv[4]) if len(sys.argv) > 4 else 3600 + K = None + if len(sys.argv) > 6: # clique orbit json, orbit index + import json + K = json.load(open(sys.argv[5]))['orbits'][int(sys.argv[6])]['rep'] + cl, y, Uv, Wv, L, G, lab, K = build(n, c, k, K) + print('n=%d c=%d k=%d clauses=%d clique=%s' % (n, c, k, len(cl), K), flush=True) + s = Cadical153(bootstrap_with=cl) + import threading + tm = threading.Timer(T, lambda: s.interrupt()); tm.start() + t0 = time.time() + r = s.solve_limited(expect_interrupt=True) + tm.cancel() + res = {True: 'SAT', False: 'UNSAT', None: 'TIMEOUT'}[r] + print('result', res, 'sec %.1f' % (time.time() - t0), flush=True) + if r: + m = set(x for x in s.get_model() if x > 0) + dead = [t for t in range(len(L)) if y[t] not in m] + print('dead', dead, flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/rank.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/rank.py new file mode 100644 index 0000000000..500b2b215f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/rank.py @@ -0,0 +1,115 @@ +"""Rank candidate unit families by a helper-adjusted estimate of the achievable saving (MODEL; see est_model.py). + +Per candidate (side, h, v, cst, z): + X* certified #595 ceiling (skel.X, best odd s in 3..15) at x = cst / v [exact interval upper end] + rho* break-even helpers per port: the rho at which the calibrated model estimate (best s) equals the calibrated + model estimate of the E8 word of the same side at its shipped intensity and best s (bit #555 at + rho = 439/111, complex #568 at rho = 496/112): same model, same s freedom on both sides + a_est calibrated model estimate at three helper-intensity hypotheses (E8's shipped intensity transferred): + rho_same = rho_E8 (helpers per port unchanged) + rho_dim = rho_E8 * (z/(h-1)) / (z_E8/(h_E8-1)) (readers per window dimension) <- central + rho_read = rho_E8 * z / z_E8 (readers per port) + in both piece-scaling modes ('abs' pessimistic, 'rel' optimistic). +Candidates: E8 references; triple family T_h (bit cst = h(h-1), complex cst = h^2: centre decoders, checked by +triple.py); Lorentzian I_{11,1}, I_{13,1} (label decoders, cst 110 / 156, checked by lorentz.py). +Usage: python rank.py REPO_ROOT [--json out] +""" +import json +import sys +from fractions import Fraction as Fr +from math import comb + +REPO = sys.argv[1] +sys.path.insert(0, REPO + '/research/format-limits/scripts') +sys.dont_write_bytecode = True +from skel import X # noqa: E402 + +import est_model as E # noqa: E402 + +REF = {'bit': dict(h=8, rho=Fr(439, 111), z=63, a=1.7331e-3), + 'complex': dict(h=9, rho=Fr(496, 112), z=52, a=1.5987e-3)} + + +def xstar(h, x): + best = None + for s in range(3, 17, 2): + r = X(s, h, 1, Fr(x)) + if r and (best is None or r['hi'] > best[1]): + best = (s, r['hi']) + return best + + +def candidates(): + C = [dict(name='E8 bit 111/63 (#555)', side='bit', h=8, v=111, cst=63, z=63), + dict(name='E8 bit 120/72', side='bit', h=8, v=120, cst=72, z=63), + dict(name='E8 complex 112/64 (#568)', side='complex', h=9, v=112, cst=64, z=52), + dict(name='I_{11,1} bit', side='bit', h=11, v=220, cst=110, z=Fr(165 * 115 + 55 * 129, 220)), + dict(name='I_{13,1} bit', side='bit', h=13, v=364, cst=156, z=Fr(286 * 183 + 78 * 231, 364))] + for h in (10, 11, 12, 13, 14, 16, 18, 20, 22, 24, 26, 28, 30): + C.append(dict(name='T_%d bit' % h, side='bit', h=h, v=comb(h, 3), cst=h * (h - 1), z=3 * comb(h - 3, 2))) + for h in (12, 14, 16, 18, 20, 22, 24, 26, 28, 30): + C.append(dict(name='T_%d complex' % h, side='complex', h=h, v=comb(h, 3), cst=h * h, + z=comb(h - 3, 3) + 3 * (h - 3))) + return C + + +def cal_est(side, h, x, rho, mode, cal): + b = E.estimate(side, h, x, rho, mode) + return (None, None, None) if b is None else (b[0], b[1] * cal, b[2]) + + +def breakeven(side, h, x, mode, cal, target): + lo, hi = Fr(0), Fr(400) + f = lambda r: (cal_est(side, h, x, r, mode, cal)[1] or 0.0) + if f(lo) < target: + return 0.0 + if f(hi) >= target: + return float(hi) + for _ in range(40): + mid = (lo + hi) / 2 + if f(mid) >= target: + lo = mid + else: + hi = mid + return float(lo) + + +def main(): + rows = [] + cals = {(sd, md): E.calibration(sd, md) for sd in ('bit', 'complex') for md in ('abs', 'rel')} + print('calibration factors', {k: round(v, 4) for k, v in cals.items()}) + target = {sd: cal_est(sd, REF[sd]['h'], Fr(63, 111) if sd == 'bit' else Fr(64, 112), REF[sd]['rho'], 'abs', + cals[(sd, 'abs')])[1] for sd in ('bit', 'complex')} + print('E8 targets (calibrated model, best s):', {k: '%.4e' % v for k, v in target.items()}) + for c in candidates(): + sd, h = c['side'], c['h'] + x = Fr(c['cst'], c['v']) + ref = REF[sd] + xs = xstar(h, x) + rh = {'same': ref['rho'], + 'dim': ref['rho'] * Fr(c['z']) / (h - 1) / Fr(ref['z'], ref['h'] - 1), + 'read': ref['rho'] * Fr(c['z']) / ref['z']} + row = dict(name=c['name'], side=sd, h=h, v=c['v'], cst=c['cst'], x=float(x), z=float(c['z']), + Xstar=float(xs[1]) if xs else None, Xs=xs[0] if xs else None, + rho={k: float(v) for k, v in rh.items()}) + for md in ('abs', 'rel'): + row['rho_star_' + md] = breakeven(sd, h, x, md, cals[(sd, md)], target[sd]) + for k, r in rh.items(): + s_, a_, ch = cal_est(sd, h, x, r, md, cals[(sd, md)]) + row['a_%s_%s' % (k, md)] = a_ + row['s_%s_%s' % (k, md)] = s_ + if k == 'dim' and md == 'abs' and ch: + row['chain_dim_abs'] = {kk: round(vv, 5) for kk, vv in ch.items()} + rows.append(row) + f = lambda v: '%.3e' % v if v else ' - ' + print('%-26s h=%2d v=%5d x=%.4f z=%7.1f X*=%s | rho*: abs %6.2f rel %6.2f | rho same/dim/read %5.2f/%6.2f/%7.2f | ' + 'a_dim abs %s rel %s | a_same abs %s | a_read abs %s' + % (row['name'], h, c['v'], row['x'], row['z'], f(row['Xstar']), row['rho_star_abs'], row['rho_star_rel'], + row['rho']['same'], row['rho']['dim'], row['rho']['read'], f(row['a_dim_abs']), f(row['a_dim_rel']), + f(row['a_same_abs']), f(row['a_read_abs'])), flush=True) + if '--json' in sys.argv: + json.dump(rows, open(sys.argv[sys.argv.index('--json') + 1], 'w'), indent=1, default=str) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/twocover.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/twocover.py new file mode 100644 index 0000000000..f0d98c4c65 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/twocover.py @@ -0,0 +1,39 @@ +"""Exact test: can the readers of a source be covered by TWO hyperplane sections (i.e. two branch frames of any +dimension)? Readers T(s) = lines orthogonal to s. Two sections {t : n1.t = 0} and {t : n2.t = 0} (n1, n2 nonzero on +span T(s)) cover T(s) iff the quadratic form q(y) = (n1.y)(n2.y) vanishes on every reader, and q is then a nonzero +quadratic form on span T(s). So: if the only quadratic form on span T(s) vanishing on all readers is 0, the branch +cover number is >= 3 (no two branches suffice in any design). Computed exactly over Q (sympy nullspace). +If the space is nonzero we report its dimension (a 2-cover may or may not exist; S3 gives certificates for pairs). +Usage: python twocover.py n (n = 8: E8 control)""" +import sys, itertools +import numpy as np +import sympy as sp +sys.dont_write_bytecode = True +import units as U + + +def test(L, G, s, Q): + R = [t for t in range(len(L)) if t != s and G[s, t] == 0] + M = sp.Matrix(L[R].tolist()) + B = M.T.columnspace() # basis of span T(s) in ambient coordinates + Bm = sp.Matrix.hstack(*B) # ambient x r + Y = [Bm.solve_least_squares(sp.Matrix(L[t].tolist())) for t in R] # exact coordinates (consistent system) + r = Bm.shape[1] + idx = [(i, j) for i in range(r) for j in range(i, r)] + A = sp.Matrix([[y[i] * y[j] * (1 if i == j else 2) for (i, j) in idx] for y in Y]) + ns = A.nullspace() + return len(R), r, len(ns) + + +if __name__ == '__main__': + n = int(sys.argv[1]) + if n == 8: + L = U.roots_E8(); G = U.gram_doubled(L); reps = [('E8 root', 0)]; Q = None + else: + L, G, lab = U.lorentz_family(n) + LN = list(itertools.combinations(range(n), 3)) + list(itertools.combinations(range(n), 2)) + reps = [('triple', 0), ('pair', LN.index((0, 1)))] + for name, s in reps: + z, r, d = test(L, G, s, None) + print('n=%d %s source: %d readers spanning %d dims; quadratic forms vanishing on all readers: dim %d -> %s' % ( + n, name, z, r, d, 'NO 2-cover (branch cover number >= 3)' if d == 0 else '2-cover not excluded'), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/units.py b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/units.py new file mode 100644 index 0000000000..358c72b4b5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/scripts/units.py @@ -0,0 +1,210 @@ +"""Exact line systems used by the S3 screens (integer coordinates, exact Gram matrices). + +Every unit is returned as (L, G, name) with L an integer v x n matrix of line representatives and G the exact integer +Gram matrix of the unit's bilinear form (diagonal 2). The F2 decoder pattern of #595 is: B_TT = 1 and B_TS = 0 whenever +G_TS != 0 (non-orthogonal lines); the entries at orthogonal pairs are free. + +Families + * ADE root systems A_n, D_n, E6, E7, E8 (doubled coordinates, G = L L^T / 4). + E6 is the root lines orthogonal to an A2 = , a = (1/2)^8, b = e1 + e2 (a.b = 1), as in #630; + the archived #620 screen used b = e1 - e2 (a.b = 0), which gives D6. + * Triple family T_h (upstream 03-motifs.tex, bit construction): the C(h,3) indicator vectors t_T of 3-subsets of [h] + with the form I - J/9, so G_ST = |S n T| - 1 (diagonal 2). + * Lorentzian I_{n,1} (n odd, #634 note 3): e_A + e_{n+1} (|A| = 3) and e_i - e_j, Lorentz form diag(1^n, -1). +""" +import itertools + +import numpy as np + + +def roots_A(n): + R = [] + for i in range(n + 1): + for j in range(i + 1, n + 1): + r = [0] * (n + 1); r[i] = 2; r[j] = -2; R.append(r) + return np.array(R, dtype=np.int64) + + +def roots_D(n): + R = [] + for i in range(n): + for j in range(i + 1, n): + for s in (1, -1): + r = [0] * n; r[i] = 2; r[j] = 2 * s; R.append(r) + return np.array(R, dtype=np.int64) + + +def roots_E8(): + """120 lines, doubled coordinates: +-2e_i +-2e_j (one sign class per line) and (+-1)^8 with an even number of + minus signs and first coordinate +1.""" + R = [list(r) for r in roots_D(8)] + for sg in itertools.product((1, -1), repeat=8): + if sg[0] == 1 and sg.count(-1) % 2 == 0: + R.append(list(sg)) + return np.array(R, dtype=np.int64) + + +def perp(R, w): + return R[(R @ np.asarray(w, dtype=np.int64)) == 0] + + +A_VEC = np.ones(8, dtype=np.int64) # (1/2)^8 doubled +B_E6 = np.array([2, 2, 0, 0, 0, 0, 0, 0]) # e1 + e2 doubled; a.b = 4/4 = 1 -> A2 with a +B_D6 = np.array([2, -2, 0, 0, 0, 0, 0, 0]) # e1 - e2 doubled; a.b = 0 -> A1 + A1 (archived "E6") + + +def roots_E7(): + return perp(roots_E8(), A_VEC) + + +def roots_E6(): + return perp(perp(roots_E8(), A_VEC), B_E6) + + +def roots_D6_archived(): + return perp(perp(roots_E8(), A_VEC), B_D6) + + +def gram_doubled(L): + G4 = L @ L.T + assert (G4 % 4 == 0).all() + G = G4 // 4 + assert (np.diag(G) == 2).all() + return G + + +def triple_family(h): + """lines t_T, T in C([h],3); G_ST = |S n T| - 1 (form I - J/9).""" + T = list(itertools.combinations(range(h), 3)) + M = np.zeros((len(T), h), dtype=np.int64) + for i, t in enumerate(T): + M[i, list(t)] = 1 + G = M @ M.T - 1 + return M, G, T + + +def lorentz_family(n): + """I_{n,1}: e_A + e_{n+1} (|A| = 3) and e_i - e_j; Lorentz Gram; binary labels: A, resp. [n] minus {i, j}.""" + vecs, labels = [], [] + for A in itertools.combinations(range(n), 3): + v = [0] * (n + 1) + for a in A: + v[a] = 1 + v[n] = 1 + vecs.append(v); labels.append([1 if i in A else 0 for i in range(n)]) + for i, j in itertools.combinations(range(n), 2): + v = [0] * (n + 1); v[i] = 1; v[j] = -1 + vecs.append(v); labels.append([0 if k in (i, j) else 1 for k in range(n)]) + L = np.array(vecs, dtype=np.int64) + Q = np.diag([1] * n + [-1]).astype(np.int64) + G = L @ Q @ L.T + return L, G, np.array(labels, dtype=np.int64) + + +# ------------------------------------------------------------------ exact linear algebra +def rank_q(rows): + """exact rank over Q (fraction-free Bareiss-style elimination on Python ints).""" + import math + red = [] + for r in rows: + r = [int(x) for x in r] + for pc, rr in red: + if r[pc]: + f, g = r[pc], rr[pc] + r = [g * x - f * y for x, y in zip(r, rr)] + c = math.gcd(*r) + if c > 1: + r = [x // c for x in r] + pc = next((j for j, x in enumerate(r) if x), None) + if pc is not None: + red.append((pc, r)) + return len(red) + + +def rank_mod_p(M, p): + """rank of an integer matrix over F_p (p prime).""" + A = (np.array(M, dtype=object) % p).tolist() + rk, rows, cols = 0, len(A), len(A[0]) if A else 0 + for c in range(cols): + piv = next((i for i in range(rk, rows) if A[i][c] % p), None) + if piv is None: + continue + A[rk], A[piv] = A[piv], A[rk] + inv = pow(A[rk][c], p - 2, p) + A[rk] = [(x * inv) % p for x in A[rk]] + for i in range(rows): + if i != rk and A[i][c] % p: + f = A[i][c] + A[i] = [(x - f * y) % p for x, y in zip(A[i], A[rk])] + rk += 1 + return rk + + +def f2_rank(M): + rows = [int(''.join('1' if x & 1 else '0' for x in row[::-1]) or '0', 2) for row in np.asarray(M)] + basis = {} + for x in rows: + while x: + hb = x.bit_length() - 1 + if hb in basis: + x ^= basis[hb] + else: + basis[hb] = x + break + return len(basis) + + +def gram_rank(G, idx): + """rank over Q of the principal block G[idx, idx] (= pairing rank of span(idx) with itself).""" + if len(idx) == 0: + return 0 + return rank_q(G[np.ix_(idx, idx)].tolist()) + + +def nonorth_clique(G): + """exact maximum set of pairwise non-orthogonal lines (identity block of the pattern => minrank lower bound).""" + import networkx as nx + v = len(G) + H = nx.Graph() + H.add_nodes_from(range(v)) + H.add_edges_from((i, j) for i in range(v) for j in range(i + 1, v) if G[i, j] != 0) + best, _ = nx.max_weight_clique(H, weight=None) + return sorted(best) + + +def _query(args): + solver, v, pattern, c, clique = args + out = solver(v, pattern, None, c, 'none', clique) + if out['result'] == 'SAT': + B = out['B'] + assert all(B[i, i] == 1 for i in range(v)) and all(B[i, j] == 0 for i, j in pattern) + return out['result'], out['sec'] + + +def minrank_f2(G, clique, cmax, solver=None, timeout=300): + """exact F_2 minrank of the pattern of G (B_TT = 1, B_TS = 0 where G_TS != 0), asymmetric decoders allowed. + Asks SAT for c = |clique|, |clique| + 1, ... < cmax; each query fixes the GL(c, F_2) gauge U_K = W_K = [I | 0] on + the clique K (solve_asym of #595's minrank_sat.py; a SAT answer is re-checked on the returned B). A query that does + not finish within `timeout` seconds is recorded as 'TIMEOUT' and the search stops there. + Returns (lo, hi, log): the minrank lies in [lo, hi]; lo == hi when it is determined (hi = cmax is attained by a + known decoder).""" + import multiprocessing as mp + v = len(G) + pattern = [(i, j) for i in range(v) for j in range(v) if i != j and G[i, j] != 0] + log = {} + lo = len(clique) + ctx = mp.get_context('fork') + for c in range(len(clique), cmax): + with ctx.Pool(1) as pool: + res = pool.apply_async(_query, ((solver, v, pattern, c, list(clique)),)) + try: + r, sec = res.get(timeout) + except mp.TimeoutError: + log[c] = ('TIMEOUT', timeout) + pool.terminate() + return lo, cmax, log + log[c] = (r, sec) + if r == 'SAT': + return c, c, log + lo = c + 1 + return cmax, cmax, log diff --git a/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/verify.sh new file mode 100644 index 0000000000..9fa2b4b7ea --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s4-lorentzian-helpers/verify.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# S4 package: punctures of I_{11,1}/I_{13,1}, E8-skeleton features, calibrated model. No kappa claim. +# Usage: sh verify.sh /path/to/integer-mult-bounds-main /fresh/out [PY=python3.11] (~30-40 min on 10 cores) +set -eu +REPO=$1; OUT=$2; PY=${PY:-python3} +HERE=$(cd "$(dirname "$0")" && pwd) +mkdir -p "$OUT"; cd "$HERE/scripts" +run() { name=$1; shift; "$PY" -B "$@" > "$OUT/$name.log" 2>&1; } +# exact certificates and exact computations (fast) +run census census.py 11 13 +run partition11 partition.py 11 2 1 +run partition13 partition.py 13 7 3 +run twocover8 twocover.py 8 +run twocover11 twocover.py 11 +run twocover13 twocover.py 13 +run cellcover8 cellcover.py 8 +run cellcover11 cellcover.py 11 +run cellcover13 cellcover.py 13 +run orbits11 clique_orbits.py 11 "$OUT/clique_orbits11.json" +"$PY" -B -c "import clique_orbits as C; print(C.survival_bound(11))" > "$OUT/survival11.log" +# slower steps in parallel +run packbases11 packbases.py 11 & +run packbases13 packbases.py 13 & +run punct11 punct.py "$REPO" 11 & +run example example_pairlabel.py "$REPO" & +run frames8 frames.py 8 & +run frames11 frames.py 11 & +run model3 model3.py "$REPO" --json "$OUT/model3.json" & +for o in 0 1 2 3 4; do run sat19_o$o punct_sat.py 11 10 19 7200 "$OUT/clique_orbits11.json" $o & done +wait +"$PY" -B check.py "$HERE/expected" "$OUT" diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/README.md b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/README.md new file mode 100644 index 0000000000..6e6f70e552 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/README.md @@ -0,0 +1,113 @@ +# E8 over extension rings: copy-cost floors under the #595 format (no κ claim) + +This package follows up #620 §4 (S1). It asks whether E8, viewed as a unit over a larger ring, can beat real E8 under the #595 format: + +- over Z[i]: 60 Gaussian lines, w = 4; +- over Z[ω]: the 40 Witting lines, w = 4; +- over the Hurwitz quaternions: 10 lines, w = 2. + +The quantity that decides it is x_K = Σ_j β_j / v. Here β_j is the #446 Theorem C pairing-rank cost of total j, counted in K-ranks, and the minimum runs over **every** F2 decoder on the K-orthogonality pattern: any number of totals, asymmetric W ≠ U allowed, and no star premise. + +To beat the bit E8 ceiling X* = 6.0005e-3, a w = 4 view needs x_K < ~0.708, so Σβ ≤ 42 for the Gaussian view. A w = 2 view needs x_K < ~0.86. These thresholds come from S1's float screen `fx.py`, which uses skel.X's profile with float roots. They are not certified here. + +Labels used below: +- **checked**: `verify.sh` recomputes the claim with exact arithmetic, or with SAT where stated (CaDiCaL via python-sat, no proof logs). +- **argued**: the proof is in this README. +- **recorded**: a run whose log is shipped; `verify.sh` does not repeat it. + +## Results + +| view | v | w | floor proven here | best decoder known | excluded? | +|---|---|---|---|---|---| +| Eisenstein (Witting) | 40 | 4 | **Σβ ≥ 40 for every decoder** (x_K ≥ 1) | 40 (frames) | **yes** | +| Hurwitz | 10 | 2 | **Σβ ≥ 10 for every decoder** (x_K ≥ 1) | 10 (frames) | **yes** | +| Gaussian | 60 | 4 | **Σβ ≥ 40 for every decoder** (x_K ≥ 2/3); Σβ = 60 for 15-total decoders (pending; see G2) | 60 (frames) | **no**: the window 40–42 is open | + +**Theorem (Eisenstein and Hurwitz).** Viewed over Z[ω] or over the Hurwitz order, E8 cannot beat real E8 in the #595 format. Under #446 Theorem C, every decoder of the K-orthogonality pattern pays Σ_j β_j ≥ v. This covers any number of totals, asymmetric pairings and non-star copy trees. So x_K ≥ 1, D_s/k = (s − 1)·v − s·x_K·v < 0 at every s, and there is no saving at all. **Gaussian E8 is not settled.** The floor proven here is 40 and exclusion needs 43. + +## Model (as in #595 C.6 and #446 Theorem C) + +- **Decoder.** B = Σ_j w_j u_j^T over F2, with readers R_j = supp w_j and sources A_j = supp u_j. +- **Pattern.** B_TT = 1, and B_TS = 0 whenever the lines T and S are K-non-orthogonal. B_TS is free on orthogonal pairs. +- **Cost.** By Theorem C, β_j/k ≥ rk(span A_j × span R_j), the rank of the real pairing divided by k = dim_Q K. For K-subspaces this is the K-rank of the Gram block G[R_j, A_j]. Since the form is positive definite, rk(X × Y) = dim Y − dim(Y ∩ X^⊥). +- **Cost-0 totals.** A total of cost 0 has R_j × A_j made only of orthogonal pairs. It touches only free entries and can be dropped. Every total that is kept costs ≥ 1. +- **Construction (checked).** `lines.py` builds the three views exactly from the 240 roots in doubled coordinates. The Gaussian view uses J = left multiplication by i on H². The Eisenstein view uses a fixed-point-free order-3 element ρ of W(E8), the product of the Coxeter elements of 4 orthogonal A2's. The Hurwitz view uses left multiplication by i, j, k. + +## E. Eisenstein: Σβ ≥ 40 for every decoder (argued; every ingredient checked by `eisen_floor.py`) + +**Lemma (sum-rank).** Let M_j = P_{R_j} G P_{A_j}, padded to v × v. Then Σ_j β_j ≥ Σ_j rank M_j ≥ rank(Σ_j M_j) = rank(G ∘ N), where N_TS = #{j : T ∈ R_j, S ∈ A_j} is an integer matrix. Because N ≡ B (mod 2), N_TT is odd and N_TS is even on every non-orthogonal pair. + +**Lemma (odd line lattices).** For every line T, pick a Z-basis of span(T) ∩ E8. Suppose every such line lattice has **odd** determinant. Then the real Gram G̃ on these bases is integral, its diagonal blocks are invertible mod 2, and its off-diagonal blocks vanish on orthogonal pairs. So G̃ ∘ (N ⊗ J) ≡ diag(G̃_TT) (mod 2), which has full rank. Hence rank_Q ≥ Σ_T dim T, that is, Σ_j β_j ≥ v in K-units. + +- **Eisenstein satisfies it.** Each line lattice is ⟨r, ρr⟩ ≅ A2, with Gram [[2, −1], [−1, 2]] and det 3 (checked for all 40 lines). +- **Complex form of the same proof.** Let h_E(x, y) = (⟨x,y⟩ − ω⟨x, ρy⟩)/4. It is Z[ω]-valued and sesquilinear (checked), and its K-ranks equal the Theorem C ranks (checked on 200 random blocks). Its diagonal is 2 + ω, of norm 3, so a unit at the inert prime 2. Hence (h_E ∘ N) mod 2 is (2+ω)·I over F4, of rank 40 (checked on random admissible N as a sanity test). +- **Why real E8 itself escapes.** Its line lattices are √2·Z (det 2), so the lemma does not apply. This is consistent with the shipped 72 < 120. +- **The other two views.** The Gaussian line lattices are √2·Z[i] (det 4) and the Hurwitz ones are D4 (det 4), so neither is covered. + +## H. Hurwitz: min Σβ = 10 exactly (argued; checked by `hurwitz_floor.py`, plus a direct SAT) + +**Structure (checked).** There are 10 lines and w = 2. Each line is orthogonal to exactly one other, so the orthogonal pairs form a perfect matching. Any two distinct lines span H². + +**Cost formula.** Because of that structure, a total costs one of three amounts: +- 0 if R × A has no equal-or-non-orthogonal pair; +- 2 if |R| ≥ 2 and |A| ≥ 2; +- 1 otherwise. + +This follows from rk = dim Y − dim(Y ∩ X^⊥) and is checked exactly on all 3,025 pairs with |R|, |A| ≤ 2. Larger sets add nothing, because their span is already H². + +**Proof that every decoder costs ≥ 10.** +1. Drop the cost-0 totals. +2. Let X be the sum of the cost-2 totals, so they cost 2g ≥ 2·rank X. Let ρ be the rows of the single-row totals and γ the columns of the single-column totals; these cost ≥ |ρ| + |γ|. +3. B + X vanishes outside the rows ρ and the columns γ. So on ρ' × γ' (the complements), X agrees with the pattern. +4. Let C = ρ' ∩ γ'. The C × C part of X is block-diagonal, with blocks [1] or [[1, *], [*, 1]], because non-orthogonal entries are 0. So rank X ≥ |C|/2. +5. Hence cost ≥ |C| + (10 − |ρ'|) + (10 − |γ'|) ≥ 10, using |ρ'| + |γ'| ≤ 10 + |C|. + +The 5 orthogonal pairs, used as frames, attain 10. + +**Checks.** +- The reduced form (2g + |ρ| + |γ| ≤ 9, for g = 0..4) is UNSAT. +- This closes S1's open item 2. A direct SAT run with S1-style encoding (`hurwitz_direct.py C`: arbitrary totals, exact cost formula, cost cap C) gives: + - C = 10: SAT (calibration). + - C = 9: see `logs/hurwitz_direct.log` (**recorded** if present). + +## G. Gaussian (not settled) + +**G1. Floor of 40 for every decoder (checked, `gauss_bound40.py`).** +- **Counting step.** Take each of the 640 twelve-line sets C of pairwise K-non-orthogonal lines. The pattern forces B[C, C] = I₁₂, and every total contributes a rank-1 F2 matrix there. So ≥ 12 totals hit C on both sides. +- **Reduction to flats.** For fixed spans, hit(R, A) is largest at R = lines(X) and A = lines(Y). +- **The inequality.** The script checks hit(X, Y) ≤ 192·rk(X, Y) on all 2271² flat pairs. Hits are exact integers. Ranks are certified lower bounds: integer pairing matrices, rank mod 2³¹ − 1. +- **Conclusion.** Σβ ≥ 12·640/192 = **40**. +- **Binding pair.** X is a 6-line 2-flat through ℓ and Y is the 15-line hyperplane ℓ^⊥: pairing rank 1, 192 hits. +- **Trying to improve it.** An LP over uniform orbit weights added every maximal clique size (7, 8, 10, 11, 12), the singletons, the non-orthogonal pairs and the whole set (minrank 15). It stays at 40 (`gauss_lp3.py` in the work dir; float LP, **recorded**). + +**G2. Decoders with exactly minrank = 15 totals.** +- **Why every total then costs 4.** With n = rank B = 15, W and U have full column rank. If every nonzero vector of col W and of col U has a support that spans K⁴, then each total pairs K⁴ with K⁴ at rank 4, and Σβ = 60. +- **The SAT question.** `gauss_c15.py` asks whether a valid 15-total decoder exists together with x ≠ 0 such that supp(Wx) lies in one of the 1500 three-dimensional flats. The calibration without the flat constraint is SAT in 1 s. The main run is **pending** (see `logs/gauss_c15.log`). If it comes back UNSAT, every 15-total decoder costs exactly 60. That would be the Gaussian analogue of #595's "9 totals × 8" premise, but without the star assumption. + +**G3. What the 2-adic method can give (checked, `gauss_pi.py`).** +- The Gaussian diagonal is 2 = −i·π². The off-diagonal entries are units on 32 pairs per line and π·unit on 12 pairs per line. +- The mod-π reduction of G ∘ N/2 is I plus free entries on the unit pairs. +- The unit graph splits into 12 five-cliques, so this method can certify at most 12. +- No per-line 2-adic rescaling helps either. Write p_T, q_T for the 2-adic valuations of the determinants of the row and column rescalings of line T. One needs p_T + q_T = −2 on every line and p_T + q_S ≥ 0 on every unit pair (argued). Adding the two pair inequalities for a unit pair (T, S) gives 0 ≤ (p_T + q_T) + (p_S + q_S) = −4, a contradiction. + +**Open (Gaussian).** +- A decoder with Σβ ≤ 42 would need ≥ 16 totals, most of them with a support confined to a hyperplane flat. By counting, n_full ≤ 10 totals of cost 4. +- The obvious next step is an LP with new kinds of Z-sets: rectangles C₁ × C₂ with a known minrank, or sets that the cheap (6-line 2-flat, ℓ^⊥) totals rarely hit on both sides. +- Alternatively, an exact SAT with sub-spanning totals. + +## Files (`scripts/`) + +| file | what | +|---|---| +| `lines.py` | exact E8 roots, the three views, K-ranks, pairing ranks, Hermitian Gram matrices | +| `flats.py` | all flats of a view (Gaussian: 2271) | +| `eisen_floor.py` | §E checks | +| `hurwitz_floor.py`, `hurwitz_direct.py` | §H checks; the direct SAT with a cost cap | +| `gauss_bound40.py`, `gauss_pi.py`, `gauss_c15.py` | §G1, G3, G2 | +| `verify.sh`, `expected.txt` | runs the checked items and compares the PASS/RESULT lines with the frozen output | + +```sh +PY=/path/to/python3.11 sh verify.sh # numpy, scipy, networkx, python-sat; ~3-6 min +``` + +Credits: builds on S1 (#620 §4), #595 (DreamingOfClouds, format limits) and #446 (Theorems A–C). Prepared with Claude Opus 5.5 via Claude Code. diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/expected.txt b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/expected.txt new file mode 100644 index 0000000000..607dc2c94a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/expected.txt @@ -0,0 +1,19 @@ +Gauss v 60 w 4 nonorth degree [44] +Eisen v 40 w 4 nonorth degree [27] +Hurwitz v 10 w 2 nonorth degree [8] +PASS eisen.gram: hE sesquilinear, diag 2+w (norm 3), 27 non-orthogonal pairs per line all of norm 1 +PASS eisen.linelattice: every line lattice has Gram [[2,-1],[-1,2]] (det 3, odd) +PASS eisen.krank: rank_K hE[R,A] == pairing rank on 200 random (R,A) +PASS eisen.reduction: (hE o N) mod 2 has F4-rank 40 for 50 random admissible N +PASS eisen.sumrank: sum_j rank >= rank(hE o N) on 20 random term families +RESULT eisen: every decoder (any number of totals, asymmetric) has sum_j beta_j >= 40 = v => x_K >= 1 +PASS hurwitz.structure: v 10, w 2, 5 orthogonal pairs (perfect matching), any two lines span H^2 +PASS hurwitz.costformula: exact pairing rank == formula on all 3025 (R,A) with |R|,|A| <= 2 +PASS hurwitz.sat: 2g + |rho| + |gamma| <= 9 is UNSAT for every g = 0..4 (CaDiCaL, no proof log) +PASS hurwitz.attained: frame decoder (5 orthogonal pairs) is a valid decoder of cost 10 +RESULT hurwitz: min over all decoders of sum_j beta_j = 10 = v => x_K = 1 +PASS gauss.pi: unit graph has 6144 five-cliques and a partition into 12 of them -> mod-pi bound <= 12 +PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all 2271x2271 flat pairs, histogram [5450, 369780, 2631390, 2150820, 1] +PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks) +RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43) +calibration (no flat constraint) SAT diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/gauss_c15.log b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/gauss_c15.log new file mode 100644 index 0000000000..079c77110d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/gauss_c15.log @@ -0,0 +1,2 @@ +calibration (no flat constraint) SAT 1.1 s +main run (support in a 3-flat): still running at package time (>20 min CPU) diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/hurwitz_direct.log b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/hurwitz_direct.log new file mode 100644 index 0000000000..720f0f4710 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/logs/hurwitz_direct.log @@ -0,0 +1,2 @@ +C 10 SAT +C 9: still running at package time (>20 min CPU) diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/eisen_floor.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/eisen_floor.py new file mode 100644 index 0000000000..ce0cceac1b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/eisen_floor.py @@ -0,0 +1,107 @@ +"""Eisenstein E8 (Witting, 40 lines, w = 4): every decoder on the pattern costs >= 40 (Theorem C pairing rank). + +Checks (exact): + 1. hE(x,y) = ( - w)/4 is Z[w]-valued, sesquilinear (hE(x,rho y) = w hE, hE(rho x,y) = w^2 hE), + diag = 2+w on all 40 representatives (norm 3: a 2-adic unit), |hE|^2 = 1 on the 27 non-orthogonal pairs per line. + 2. For random line sets R, A: K-rank of hE[R,A] (realified, exact) == Theorem-C pairing rank (real rank / 2). + 3. Line lattices: Gram of {r, rho r} is [[2,-1],[-1,2]] (det 3, odd) for every line. + 4. Reduction: for random integer N with N_TT odd and N_TS even on non-orthogonal pairs (arbitrary on orthogonal + pairs), (hE o N) mod 2 over F4 = Z[w]/2 has rank 40, hence rank_K(hE o N) = 40. + 5. Sum-rank: on random term families, sum_j rank hE[R_j,A_j] >= rank (hE o N). +""" +import sys, random; sys.path.insert(0, sys.path[0]) +from lines import * + +def f4_rank(Ma, Mb): + """rank over F4 = F2[w]/(w^2+w+1) of the matrix (Ma + Mb w) mod 2. Elements as 2-bit ints a + 2b.""" + mul = [[0]*4 for _ in range(4)] + def m(x, y): # (a+bw)(c+dw) = ac + (ad+bc)w + bd w^2, w^2 = w+1 + a, b = x & 1, x >> 1; c, d = y & 1, y >> 1 + r0 = (a*c + b*d) & 1; r1 = (a*d + b*c + b*d) & 1 + return r0 | (r1 << 1) + for x in range(4): + for y in range(4): mul[x][y] = m(x, y) + inv = {1: 1, 2: 3, 3: 2} + M = [[int(a % 2) | (int(b % 2) << 1) for a, b in zip(ra, rb)] for ra, rb in zip(Ma, Mb)] + n, mm = len(M), len(M[0]); r = 0 + for c in range(mm): + p = next((i for i in range(r, n) if M[i][c]), None) + if p is None: continue + M[r], M[p] = M[p], M[r] + iv = inv[M[r][c]]; M[r] = [mul[iv][x] for x in M[r]] + for i in range(n): + if i != r and M[i][c]: + f = M[i][c]; M[i] = [x ^ mul[f][y] for x, y in zip(M[i], M[r])] + r += 1 + return r + +def realify(Ma, Mb): + """a + b w acting on Q^2 = Q(w) (basis 1, w): mult by w maps 1->w, w->w^2=-1-w.""" + n, m = Ma.shape + R = np.zeros((2*n, 2*m), dtype=object) + for i in range(n): + for j in range(m): + a, b = int(Ma[i, j]), int(Mb[i, j]) + R[2*i:2*i+2, 2*j:2*j+2] = [[a, -b], [b, a - b]] + return R + +def main(seed=1, trials=200): + W = views(); S = W['Eisen']; V = S.reps; rho4 = W['_rho']; v = S.v + assert v == 40 and S.w == 4 + a, b = eisen_gram(V, rho4) + # 1. sesquilinearity on all representative pairs and their rho-images + rV = (V @ rho4.T) // 4 + a2, b2 = eisen_gram_pair(V, rV, rho4) + # hE(x, rho y) = w hE(x,y): w(a + b w) = a w + b(-1 - w) = -b + (a - b) w + assert (a2 == -b).all() and (b2 == a - b).all() + a3, b3 = eisen_gram_pair(rV, V, rho4) + # hE(rho x, y) = w^2 hE = (-1-w)(a+bw) = -a - (a) w - b w - b w^2 = -a + b + (-a) w ... compute: w^2(a+bw)=a w^2 + b w^3 = a(-1-w) + b + assert (a3 == -a + b).all() and (b3 == -a).all() + assert set(zip(np.diag(a).tolist(), np.diag(b).tolist())) == {(2, 1)} + nrm = a*a - a*b + b*b + off = ~np.eye(v, dtype=bool) + assert ((nrm > 0) & off == S.NO).all() and set(nrm[S.NO].tolist()) == {1} and (S.NO.sum(1) == 27).all() + print('PASS eisen.gram: hE sesquilinear, diag 2+w (norm 3), 27 non-orthogonal pairs per line all of norm 1') + # 3. line lattices + for T in range(v): + r = V[T]; rr = rho4 @ r // 4 + Gm = [[r @ r // 4, r @ rr // 4], [rr @ r // 4, rr @ rr // 4]] + assert Gm == [[2, -1], [-1, 2]] + print('PASS eisen.linelattice: every line lattice has Gram [[2,-1],[-1,2]] (det 3, odd)') + rnd = random.Random(seed) + # 2. K-rank of hE block == Theorem C pairing rank + for _ in range(trials): + R = rnd.sample(range(v), rnd.randint(1, 8)); A = rnd.sample(range(v), rnd.randint(1, 8)) + Rk = rank_q(realify(a[np.ix_(R, A)], b[np.ix_(R, A)]).tolist()) + assert Rk % 2 == 0 and Rk // 2 == S.pair_rank(R, A) + print(f'PASS eisen.krank: rank_K hE[R,A] == pairing rank on {trials} random (R,A)') + # 4. reduction + for _ in range(trials // 4): + N = np.zeros((v, v), dtype=np.int64) + for T in range(v): + for U_ in range(v): + if T == U_: N[T, U_] = 2 * rnd.randint(0, 3) + 1 + elif S.NO[T, U_]: N[T, U_] = 2 * rnd.randint(0, 3) + else: N[T, U_] = rnd.randint(0, 7) + assert f4_rank(a * N, b * N) == 40 + print(f'PASS eisen.reduction: (hE o N) mod 2 has F4-rank 40 for {trials//4} random admissible N') + # 5. sum-rank on random term families (not necessarily decoders) + for _ in range(trials // 10): + terms = [(rnd.sample(range(v), rnd.randint(1, 10)), rnd.sample(range(v), rnd.randint(1, 10))) for _ in range(rnd.randint(1, 6))] + N = np.zeros((v, v), dtype=np.int64) + for R, A in terms: + for T in R: + for U_ in A: N[T, U_] += 1 + lhs = sum(S.pair_rank(R, A) for R, A in terms) + rhs = rank_q(realify(a * N, b * N).tolist()) // 2 + assert lhs >= rhs + print(f'PASS eisen.sumrank: sum_j rank >= rank(hE o N) on {trials//10} random term families') + print('RESULT eisen: every decoder (any number of totals, asymmetric) has sum_j beta_j >= 40 = v => x_K >= 1') + +def eisen_gram_pair(X, Y, rho4): + A = X @ Y.T; Bm = X @ rho4 @ Y.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/flats.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/flats.py new file mode 100644 index 0000000000..b064c81896 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/flats.py @@ -0,0 +1,26 @@ +"""All flats (line sets of K-subspaces spanned by configuration lines) of a view, with dims. Exact.""" +import sys, pickle; sys.path.insert(0,sys.path[0]) +from lines import * +def flats(S): + v=S.v; out={} + def close(idx): + d=S.arank(idx); return frozenset(T for T in range(v) if S.arank(list(idx)+[T])==d), d + level=[ (frozenset([T]),1) for T in range(v)] + for f,d in level: out[f]=d + frontier=list(out.items()) + while frontier: + new={} + for f,d in frontier: + if d>=S.w: continue + for T in range(v): + if T in f or any(T in g and f= 40 (x_K >= 2/3). + +Argument. C ranges over the 640 twelve-line sets of pairwise K-non-orthogonal lines. On C x C the pattern forces +B[C,C] = I_12 (diagonal 1, all off-diagonal pairs non-orthogonal), and each total contributes a rank-1 F2 matrix there, +so at least 12 totals hit C on both sides (R_j & C != {} and A_j & C != {}). Summing: sum_j hit(R_j, A_j) >= 12*640. +For fixed spans X = span R_j, Y = span A_j the hit count is largest for R = lines(X), A = lines(Y) (flats), and this +script shows hit(X, Y) <= 192 * rank(X, Y) for all 2271^2 flat pairs, rank = K-rank of the pairing (Theorem C cost). +Hence sum_j beta_j >= 12*640/192 = 40. Binding pair: X = 6-line 2-flat containing l, Y = the 15-line hyperplane l^perp +(pairing rank 1, 192 hits). Mixing all clique sizes, singletons, non-orthogonal pairs and the whole set in an LP over +uniform orbit weights does not improve 40 (gauss_lp3.py in the work dir). +Ranks: exact integer pairing matrices, rank mod p = 2^31-1 (a certified lower bound on the rank over Q).""" +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from flats import flats +import networkx as nx + +def main(): + W = views(); S = W['Gauss']; J = W['_J']; v = S.v + F = flats(S); fl = sorted(F.items(), key=lambda t: (t[1], len(t[0]), sorted(t[0]))) + n = len(fl); assert n == 2271 + g = nx.Graph([(i, j) for i in range(v) for j in range(i+1, v) if S.NO[i, j]]) + C = [c for c in nx.find_cliques(g) if len(c) == 12] + assert len(C) == 640 and all(S.NO[a, b] for c in C for a, b in itertools.combinations(c, 2)) + Lo = np.zeros((n, v), np.int64) + for i, (f, d) in enumerate(fl): Lo[i, sorted(f)] = 1 + inc = np.zeros((v, len(C)), np.int64) + for k, c in enumerate(C): inc[c, k] = 1 + meet = (Lo @ inc > 0).astype(np.int64); hits = meet @ meet.T # exact integers + # exact integer bases (vectors r_T, J r_T), pairing ranks by batched elimination mod p = 2^31-1 + # (rank mod p <= rank over Q, so these are certified LOWER bounds; the claim only needs lower bounds) + def ibasis(f): + rows = [] + for T in sorted(f): + for x in (S.reps[T], J @ S.reps[T]): + if rank_q(rows + [x]) > len(rows): rows.append(x) + return np.array(rows, np.int64) + Bs = [ibasis(f) for f, d in fl] + assert all(len(b) == 2 * d for b, (f, d) in zip(Bs, fl)) + P = 2147483647 + def modrank(A): + A = A % P; N, r, c = A.shape; used = np.zeros((N, r), bool); idx = np.arange(N) + for col in range(c): + cand = (A[:, :, col] != 0) & ~used; has = cand.any(1) + if not has.any(): continue + pr = cand.argmax(1); it = idx[has]; prow = A[it, pr[has]] + inv = np.array([pow(int(x), P - 2, P) for x in prow[:, col]], np.int64) if len(it) < 64 else modinv(prow[:, col]) + prow = (prow * inv[:, None]) % P + fac = A[it, :, col].copy(); fac[np.arange(len(it)), pr[has]] = 0 + A[it] = (A[it] - (fac[:, :, None] * prow[:, None, :]) % P) % P + used[it, pr[has]] = True + return used.sum(1) + def modinv(x): + res = np.ones_like(x); base = x % P; e = P - 2 + while e: + if e & 1: res = (res * base) % P + base = (base * base) % P; e >>= 1 + return res + R = np.zeros((n, n), np.int64) + groups = {} + for i, b in enumerate(Bs): groups.setdefault(len(b), []).append(i) + for da, ia in groups.items(): + A = np.stack([Bs[i] for i in ia]) + for db, ib in groups.items(): + Bm = np.stack([Bs[i] for i in ib]) + M = np.einsum('aik,bjk->abij', A, Bm).reshape(-1, da, db) + r = modrank(M).reshape(len(ia), len(ib)); assert (r % 2 == 0).all() + R[np.ix_(ia, ib)] = r // 2 + print(f'PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all {n}x{n} flat pairs, histogram {np.bincount(R.flatten()).tolist()}') + assert (hits <= 192 * R).all() + worst = np.argwhere((hits == 192 * R) & (R > 0)) + print('PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks)') + a, b = worst[0] + print(f' binding example: |X| = {len(fl[a][0])} (dim {fl[a][1]}), |Y| = {len(fl[b][0])} (dim {fl[b][1]}), rank {R[a,b]}, hits {hits[a,b]}; {len(worst)} binding pairs') + print('RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43)') + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_c15.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_c15.py new file mode 100644 index 0000000000..93289c3fd4 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_c15.py @@ -0,0 +1,56 @@ +"""Gaussian E8 (60 lines, w = 4): every decoder with exactly c = minrank = 15 totals costs exactly 60 (Theorem C). + +B = W U^T with W, U in F2^{60x15}; rank B >= 15 (minrank, S1: c = 14 UNSAT) so W, U have full column rank and every +total j has w_j = W e_j != 0, u_j != 0. If every nonzero vector of col(W) and of col(U) has a support spanning K^4, +then every total pairs span(R_j) = span(A_j) = K^4 with rank 4 (positive definite), so sum_j beta_j = 15 * 4 = 60. +SAT question (asymmetric decoders, clique gauge): exists a valid 15-total decoder and x != 0 with supp(W x) inside a +3-dim flat? (Every proper flat lies in one of the 1500 3-flats.) The U side follows by transposition (B^T is valid +for the transposed decoder, the pattern being symmetric). argv: [optional 'calib'] -> drop the flat constraint.""" +import sys, time, pickle, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from flats import flats +from pysat.formula import IDPool +from pysat.card import CardEnc, EncType +from pysat.solvers import Cadical153 +import networkx as nx + +def main(calib=False, c=15): + S = views()['Gauss']; v = S.v + F = flats(S) + hyp = [sorted(f) for f, d in F.items() if d == 3] + assert len(hyp) == 1500 + g = nx.Graph([(i, j) for i in range(v) for j in range(i + 1, v) if S.NO[i, j]]) + K = sorted(max(nx.find_cliques(g), key=len)); assert len(K) == 12 + pool = IDPool(); cl = [] + U = [[pool.id(('u', T, j)) for j in range(c)] for T in range(v)] + W = [[pool.id(('w', T, j)) for j in range(c)] for T in range(v)] + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + def xorlist(ls): + acc = ls[0] + for l in ls[1:]: acc = XOR(acc, l) + return acc + for T in range(v): + for S_ in range(v): + if T == S_ or S.NO[T, S_]: + b = xorlist([AND(W[T][j], U[S_][j]) for j in range(c)]) + cl.append([b] if T == S_ else [-b]) + for i, T in enumerate(K): + for j in range(c): + cl.append([U[T][j]] if j == i else [-U[T][j]]); cl.append([W[T][j]] if j == i else [-W[T][j]]) + if not calib: + x = [pool.id(('x', j)) for j in range(c)]; cl.append(x) + y = [xorlist([AND(W[T][j], x[j]) for j in range(c)]) for T in range(v)] + sel = [pool.id(('s', h)) for h in range(len(hyp))]; cl.append(sel) + for h, H in enumerate(hyp): + Hs = set(H) + for T in range(v): + if T not in Hs: cl.append([-sel[h], -y[T]]) + s = Cadical153(bootstrap_with=cl); t = time.time(); r = s.solve() + print(('calibration (no flat constraint) ' if calib else 'support-in-a-3-flat ') + ('SAT' if r else 'UNSAT'), round(time.time() - t, 1), 's', flush=True) + return r + +if __name__ == '__main__': + main(calib=len(sys.argv) > 1 and sys.argv[1] == 'calib') diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_lp3.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_lp3.py new file mode 100644 index 0000000000..8c589e9a51 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_lp3.py @@ -0,0 +1,23 @@ +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +import networkx as nx +from scipy.optimize import linprog +d = np.load('rankmat.npz'); R = d['R'].astype(float); L = d['lines_of'].astype(np.float32) +S = views()['Gauss']; v = S.v +g = nx.Graph([(i, j) for i in range(v) for j in range(i+1, v) if S.NO[i, j]]) +fams = {} +for c in nx.find_cliques(g): fams.setdefault('max%d' % len(c), []).append(c) +fams['single'] = [[T] for T in range(v)] +fams['nopair'] = [[a, b] for a in range(v) for b in range(a+1, v) if S.NO[a, b]] +fams['all'] = [list(range(v))] +mval = {k: (15 if k == 'all' else len(fams[k][0])) for k in fams} +names = sorted(fams); H = [] +for k in names: + inc = np.zeros((v, len(fams[k])), np.float32) + for j, c in enumerate(fams[k]): inc[c, j] = 1 + meet = (L @ inc > 0).astype(np.float32); H.append(meet @ meet.T) +H = np.stack(H, -1).reshape(-1, len(names)); Rf = R.reshape(-1) +rows = np.unique(np.concatenate([H, Rf[:, None]], 1), axis=0); rows = rows[rows[:, :-1].sum(1) > 0] +c = -np.array([mval[k] * len(fams[k]) for k in names], float) +res = linprog(c, A_ub=rows[:, :-1], b_ub=rows[:, -1], bounds=[(0, None)]*len(names), method='highs') +print('LP bound', -res.fun, {k: round(x*len(fams[k]), 4) for k, x in zip(names, res.x) if x > 1e-12}) diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_pi.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_pi.py new file mode 100644 index 0000000000..57eb8cd6fb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/gauss_pi.py @@ -0,0 +1,28 @@ +"""Gaussian view: the 2-adic (mod pi) sum-rank bound is at most 12, so it cannot reach 43. +The unit-pair graph (|hG|^2 = 1, 32 per line) has a partition into 12 five-cliques; with N/2 odd exactly inside the +cliques, (hG o N)/2 mod pi = block-diag(J_5) of F2-rank 12. Also: max set with no unit pair = 12 (lower bound 12).""" +import sys; sys.path.insert(0, sys.path[0]) +from lines import * +import networkx as nx +from pysat.solvers import Cadical153 +from pysat.card import CardEnc +W = views(); S = W['Gauss']; a, b = gauss_gram(S.reps, W['_J']); N = a*a + b*b; v = S.v +U = nx.Graph([(i, j) for i in range(v) for j in range(i+1, v) if N[i, j] == 1]) +cl5 = [c for c in nx.find_cliques(U) if len(c) == 5] +# exact cover by SAT +x = list(range(1, len(cl5)+1)); cls = [] +for T in range(v): + lits = [x[k] for k, c in enumerate(cl5) if T in c] + cls.append(lits) + for i in range(len(lits)): + for j in range(i+1, len(lits)): cls.append([-lits[i], -lits[j]]) +s = Cadical153(bootstrap_with=cls); assert s.solve() +part = [cl5[k] for k in range(len(cl5)) if s.get_model()[k] > 0] +assert len(part) == 12 and sorted(sum(part, [])) == list(range(v)) +M = np.zeros((v, v), int) +for c in part: + for i in c: + for j in c: M[i, j] = 1 +from lines import rank_q +# F2 rank of block-diag J_5 is 12 (one per block) +print('PASS gauss.pi: unit graph has', len(cl5), 'five-cliques and a partition into 12 of them -> mod-pi bound <= 12') diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_direct.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_direct.py new file mode 100644 index 0000000000..b96a7ed5e2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_direct.py @@ -0,0 +1,40 @@ +"""Direct SAT (S1's open item): is there any Hurwitz decoder with n <= C terms and sum of exact costs <= C? +Terms (w_j, u_j) arbitrary; cost_j from the exact formula (checked in hurwitz_floor.py); cost-0 terms are useless +(they touch only free entries) so n <= C. Symmetry: terms sorted by (cost desc) not imposed; lex order on w_j words.""" +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from pysat.formula import IDPool +from pysat.card import CardEnc, EncType +from pysat.solvers import Cadical153 +C = int(sys.argv[1]) +S = views()['Hurwitz']; v = S.v +rel = [(T, U_) for T in range(v) for U_ in range(v) if T == U_ or S.NO[T, U_]] +pool = IDPool(); cl = [] +W = [[pool.id(('w', j, T)) for T in range(v)] for j in range(C)] +U = [[pool.id(('u', j, T)) for T in range(v)] for j in range(C)] +P = {} +for j in range(C): + for T, S_ in rel: + p = pool.id(('p', j, T, S_)); P[j, T, S_] = p + cl += [[-p, W[j][T]], [-p, U[j][S_]], [p, -W[j][T], -U[j][S_]]] +def xor_eq(lits, target): + # chain xor with aux vars + acc = lits[0] + for l in lits[1:]: + z = pool.id(('x', len(cl), acc, l)) + cl.extend([[-z, acc, l], [-z, -acc, -l], [z, -acc, l], [z, acc, -l]]); acc = z + cl.append([acc] if target else [-acc]) +for T, S_ in rel: + xor_eq([P[j, T, S_] for j in range(C)], 1 if T == S_ else 0) +costlits = [] +for j in range(C): + r1 = pool.id(('r1', j)); r2 = pool.id(('r2', j)) + for T, S_ in rel: cl.append([-P[j, T, S_], r1]) + # r2 >= (|R|>=2 & |A|>=2 & relevant): any 2 rows, 2 cols -> relevant automatically (each line orth to one other) + for T1, T2 in itertools.combinations(range(v), 2): + for S1, S2 in itertools.combinations(range(v), 2): + cl.append([-W[j][T1], -W[j][T2], -U[j][S1], -U[j][S2], r2]) + costlits += [r1, r2] +enc = CardEnc.atmost(lits=costlits, bound=C, vpool=pool, encoding=EncType.seqcounter) +s = Cadical153(bootstrap_with=cl + enc.clauses) +print('C', C, 'SAT' if s.solve() else 'UNSAT', flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_floor.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_floor.py new file mode 100644 index 0000000000..8f634fb7f1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/hurwitz_floor.py @@ -0,0 +1,75 @@ +"""Hurwitz E8 (10 quaternionic lines in H^2, w = 2): every decoder on the pattern costs >= 10 = v (exact). + +Model: decoder B = sum_j w_j u_j^T over F2 (R_j = supp w_j readers, A_j = supp u_j sources), B_TT = 1, B_TS = 0 on +H-non-orthogonal pairs, free on orthogonal pairs; cost_j = Theorem-C pairing rank of span(R_j) x span(A_j) in H-units. +Checks: + 1. structure: v = 10, w = 2, every line is H-orthogonal to exactly one other (5 orthogonal pairs), any two distinct + lines span H^2. + 2. cost formula, exact for all |R|,|A| <= 2 and implied for larger sets (span saturates): + cost = 0 if R x A has no equal-or-non-orthogonal pair; else 2 if |R| >= 2 and |A| >= 2; else 1. + 3. Proof (argued, README): split terms into cost-2 'general' terms X (2 per term >= 2 rank X), cost-1 single-row / + single-column terms (rho, gamma), cost-0 terms (absorbed in free entries). B + X vanishes off rows rho and columns + gamma, so on rho' x gamma' (complements) X equals the pattern; its C x C principal part (C = rho' & gamma') is block + diagonal with blocks [1] / [[1,*],[*,1]] (non-orthogonal pairs are 0), so rank X >= |C|/2 and + cost >= |C| + 20 - |rho'| - |gamma'| >= 10. + 4. SAT cross-check of the reduced form: no (X of rank <= g, rho, gamma) with 2g + |rho| + |gamma| <= 9 (CaDiCaL). + 5. Attained: the 5 orthogonal pairs as frames, cost 5 x 2 = 10. +""" +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from pysat.formula import IDPool +from pysat.card import CardEnc, EncType +from pysat.solvers import Cadical153 + +def formula(S, R, A): + if not any(T == U_ or S.NO[T, U_] for T in R for U_ in A): return 0 + return 2 if len(R) >= 2 and len(A) >= 2 else 1 + +def main(): + S = views()['Hurwitz']; v = S.v + assert v == 10 and S.w == 2 + orth = ~S.NO & ~np.eye(v, dtype=bool) + assert (orth.sum(1) == 1).all() + assert all(S.arank([a, b]) == 2 for a, b in itertools.combinations(range(v), 2)) + print('PASS hurwitz.structure: v 10, w 2, 5 orthogonal pairs (perfect matching), any two lines span H^2') + small = [c for k in (1, 2) for c in itertools.combinations(range(v), k)] + for R in small: + for A in small: + assert S.pair_rank(R, A) == formula(S, R, A) + print(f'PASS hurwitz.costformula: exact pairing rank == formula on all {len(small)**2} (R,A) with |R|,|A| <= 2') + # 4. reduced SAT: B + X supported on rows rho / cols gamma; X = sum_{k x_K = 1') + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/lines.py b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/lines.py new file mode 100644 index 0000000000..fc4eac011d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/scripts/lines.py @@ -0,0 +1,166 @@ +"""Self-contained exact construction of the E8 root lines over Q(i), Q(omega) and the Hurwitz quaternions. + +Real model: the 240 E8 roots in doubled standard coordinates (norm 8; in {0,+-4,+-8}). + * Gaussian view : J = left multiplication by i on R^8 = H^2 (coords 1,i,j,k per block); J preserves E8. + Hermitian Gram hG(x,y) = ( + i )/4 in Z[i] (diag 2). + * Eisenstein view: rho = fixed-point-free order-3 isometry of E8 (product of Coxeter elements of 4 orthogonal A2's). + Sesquilinear Gram hE(x,y) = ( - w )/4 in Z[w] (w^2+w+1=0; diag 2+w, a unit at 2). + * Hurwitz view : left multiplication by i, j, k; quaternionic lines of real dim 4. +Elements of Z[i] / Z[w] are stored as integer pairs (a, b) = a + b*i / a + b*w. +Everything is exact integer arithmetic; asserts check every structural claim used downstream. +""" +import itertools +from fractions import Fraction +import numpy as np + + +def e8_roots(): + R = [] + for i, j in itertools.combinations(range(8), 2): + for si in (2, -2): + for sj in (2, -2): + r = [0] * 8; r[i], r[j] = si, sj; R.append(r) + for s in itertools.product((1, -1), repeat=8): + if s.count(-1) % 2 == 0: + R.append(list(s)) + R = np.array(R, dtype=np.int64) + assert len(R) == 240 and set((R * R).sum(1)) == {8} + return R + + +def rank_q(rows): + """exact rank over Q of an integer matrix (fraction-free Gaussian elimination).""" + M = [list(map(int, r)) for r in rows] + if not M: + return 0 + n, m = len(M), len(M[0]); r = 0 + for c in range(m): + p = next((i for i in range(r, n) if M[i][c] != 0), None) + if p is None: + continue + M[r], M[p] = M[p], M[r] + for i in range(r + 1, n): + if M[i][c]: + f, g = M[i][c], M[r][c] + M[i] = [g * a - f * b for a, b in zip(M[i], M[r])] + r += 1 + if r == n: + break + return r + + +def left_quat(): + Li = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 0, -1], [0, 0, 1, 0]]) + Lj = np.array([[0, 0, -1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, -1, 0, 0]]) + Lk = Li @ Lj + blk = lambda M: np.kron(np.eye(2, dtype=np.int64), M).astype(np.int64) + return blk(Li), blk(Lj), blk(Lk) + + +def a2_rotation(roots): + """rho, an order-3 fixed-point-free isometry preserving E8 (returned as an exact integer matrix 4*rho).""" + chosen = [] + for a in roots: + if len(chosen) == 4: + break + if any(np.dot(a, x) != 0 for p in chosen for x in p): + continue + for b in roots: + if np.dot(a, b) == -4 and all(np.dot(a, x) == 0 and np.dot(b, x) == 0 for p in chosen for x in p): + chosen.append((a, b)); break + assert len(chosen) == 4 + g = np.eye(8, dtype=object) + for a, b in chosen: + sa = np.eye(8, dtype=object) * 8 - 2 * np.outer(a, a).astype(object) + sb = np.eye(8, dtype=object) * 8 - 2 * np.outer(b, b).astype(object) + g = g.dot(sa).dot(sb) + D = 8 ** 8 // 4 + assert all(x % D == 0 for x in g.flatten()) + g = (g // D).astype(np.int64) + I = np.eye(8, dtype=np.int64) + assert (g @ g @ g == 64 * I).all() and (g @ g + 4 * g + 16 * I == 0).all() + return g + + +def lines_over(roots, ops): + """group roots into A-lines, A spanned by the integer matrices ops (identity first; ops may be scaled by 4). + Returns representatives (first root met of each line, in root order).""" + S = {tuple(r) for r in roots} + reps, seen = [], set() + for r in roots: + if tuple(r) in seen: + continue + span = [B @ r for B in ops] + assert rank_q(span) == len(ops) + reps.append(r) + members = [s for s in roots if rank_q(span + [s]) == len(ops)] + for s in members: + seen.add(tuple(s)) + return np.array(reps) + + +class View: + """name, reps (v x 8), ops (real matrices spanning A, scaled so that ops[i] @ root is a root times scale[i]).""" + def __init__(self, name, reps, ops, d): + self.name, self.reps, self.ops, self.d = name, reps, ops, d + self.v = len(reps) + NO = np.zeros((self.v, self.v), bool) + for B in ops: + NO |= (reps @ B @ reps.T) != 0 + np.fill_diagonal(NO, False) + self.NO = NO # A-non-orthogonal pairs (the decoder pattern forces B_TS = 0 there) + self.w = self.arank(range(self.v)) + + def arank(self, idx): + rows = [B @ self.reps[i] for i in idx for B in self.ops] + r = rank_q(rows) if rows else 0 + assert r % self.d == 0 + return r // self.d + + def pair_rank(self, R, A): + """A-rank of the pairing span(R) x span(A) = real rank of the real Gram block / dim A (Theorem C cost).""" + BR = [B @ self.reps[i] for i in R for B in self.ops] + BA = [B @ self.reps[i] for i in A for B in self.ops] + if not BR or not BA: + return 0 + M = [[int(x @ y) for y in BA] for x in BR] + r = rank_q(M); assert r % self.d == 0 + return r // self.d + + +def views(): + roots = e8_roots(); I8 = np.eye(8, dtype=np.int64) + Li, Lj, Lk = left_quat() + S = {tuple(r) for r in roots} + for M in (Li, Lj, Lk): + assert all(tuple(M @ r) in S for r in roots) + g = a2_rotation(roots) # 4*rho + assert all(tuple(g @ r // 4) in S and ((g @ r) % 4 == 0).all() for r in roots) + rho = g # 4*rho (exact integer matrix); ranks are scale-free + out = {} + out['Gauss'] = View('Gauss', lines_over(roots, [I8, Li]), [I8, Li], 2) + out['Eisen'] = View('Eisen', lines_over(roots, [I8, rho]), [I8, rho], 2) + out['Hurwitz'] = View('Hurwitz', lines_over(roots, [I8, Li, Lj, Lk]), [I8, Li, Lj, Lk], 4) + out['_J'], out['_rho'] = Li, rho + return out + + +def gauss_gram(V, J): + """hG(r_T, r_S) = ( + i)/4 as integer pairs (a, b).""" + A = V @ V.T; Bm = V @ J @ V.T + assert (A % 4 == 0).all() and (Bm % 4 == 0).all() + return A // 4, Bm // 4 + + +def eisen_gram(V, rho4): + """hE(r_T, r_S) = ( - w)/4 as integer pairs (a, b) = a + b w (rho4 = 4*rho).""" + A = V @ V.T; Bm = V @ rho4 @ V.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + + +if __name__ == '__main__': + W = views() + for nm in ('Gauss', 'Eisen', 'Hurwitz'): + S = W[nm] + print(nm, 'v', S.v, 'w', S.w, 'nonorth degree', sorted(set(S.NO.sum(1)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/verify.sh new file mode 100644 index 0000000000..f2bf8316cf --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s5-extension-ring-floors/verify.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# Re-run the checked claims and compare PASS/RESULT lines with expected.txt. Usage: PY=python3.11 sh verify.sh +set -e +PY=${PY:-python3} +D=$(cd "$(dirname "$0")" && pwd) +OUT=$(mktemp -d) +cd "$D/scripts" +for s in lines eisen_floor hurwitz_floor gauss_pi gauss_bound40; do + echo "== $s" >&2 + "$PY" -B "$s.py" | grep -E '^(PASS|FAIL|RESULT|Gauss|Eisen|Hurwitz) ' | sed 's/np\.int64(\([0-9]*\))/\1/g' >> "$OUT/got.txt" +done +"$PY" -B gauss_c15.py calib >> "$OUT/got.txt" +sed -E 's/ [0-9.]+ s$//' "$OUT/got.txt" > "$OUT/got2.txt" +if diff -u "$D/expected.txt" "$OUT/got2.txt"; then echo "ALL MATCH ($(grep -c '^PASS' "$OUT/got2.txt") PASS lines)"; else echo "MISMATCH"; exit 1; fi diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/README.md b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/README.md new file mode 100644 index 0000000000..97fd51dd3e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/README.md @@ -0,0 +1,42 @@ +# S6 — Gaussian E8: every decoder with exactly 15 totals costs exactly 60 (question G2 of §10 closed) + +Context: `s5-extension-ring-floors/` (#620 §10). The Gaussian view of E8 has 60 Q(i)-lines in K^4. Its F2 decoder pattern has +minrank 15 (S1: c = 14 UNSAT). S5 proved that every decoder costs >= 40. Excluding the Gaussian view needs 43; beating real E8 needs <= 42. +S5 left question G2 open (`gauss_c15.py`, a monolithic CaDiCaL run stopped after 18 h without an answer). The question was: +can a 15-total decoder have a column-space vector `W x` (or `U x`) supported inside a 3-flat? A "no" means every 15-total +decoder costs exactly 15 * 4 = 60. + +## Result +**G2 is UNSAT. Every Gaussian-E8 decoder with exactly 15 totals has Theorem-C cost exactly 60** (x_K = 1). The cheap +decoders that remain possible (cost 40..42) must have >= 16 totals. + +## Proof +1. **[argued] Reduction to small minrank instances.** Let B = W U^T be valid (B_TT = 1, B_TS = 0 on K-non-orthogonal pairs), with W, U in F2^{60x15}. + Suppose x != 0 and supp(W x) lies in a 3-flat H. Put O = the lines outside H. + - Then W[O] x = 0, so rank W[O] <= 14. + - So B[O,O] = W[O] U[O]^T has rank <= 14, and it obeys the decoder pattern induced on O. + - So if minrank(pattern on O) >= 15 for every 3-flat H, no such x exists. The same holds for U by transposition. + - Coverage: every proper flat lies in a 3-flat spanned by lines (extend by lines one at a time). W has full column rank, so W x != 0. + - Once no such x exists, every total has span(R_j) = span(A_j) = K^4. The Hermitian form is definite, so beta_j = 4 and the sum is 60 (S5 Theorem-C argument). +2. **[checked] Symmetry.** + - `orbits.py` builds the 60 complex reflections s_r s_{Jr}. Each is exact, preserves E8 and commutes with J, and each is checked to preserve the line set and the non-orthogonality relation. + - Closing them gives a permutation group of order 11520 on the lines (G31 modulo scalars). + - The 1500 3-flats form 3 orbits: 60 of size 15, 480 of size 6 and 960 of size 4. + - minrank(pattern on O) is invariant under the group, so one H per orbit is enough. +3. **[checked] SAT.** `prove_c15.py orbits` runs on each orbit representative's complement O (45, 54 and 56 lines). + - The encoding is S1's clique gauge: a 12-clique of O gets W = U = [I_12 0]. This is valid for any clique size <= c. + - c = 14 is **UNSAT** with each of three independent solvers: CaDiCaL 1.5.3, Glucose 4 and MapleChrono (5-120 s each). + - Calibrations: c = 15 is SAT on every O, and on all 60 lines c = 14 is UNSAT and c = 15 is SAT. So the encoding is not trivially infeasible. +4. **[in progress, group-free] All 1500 3-flats.** `prove_c15.py all` repeats the c = 14 test on every 3-flat complement, without using the group. Status: 700/1500 UNSAT so far and no SAT. The log is at s6/prove_all.log, and the advisor will append the final count. +5. **[script provided, not yet run] Sanity.** `price_c15.py` takes a SAT-found 15-total decoder and prices it, expecting every beta_j = 4 and a sum of 60. It was not run, to stay under the 3-process cap while the sweep runs. + +What made it tractable: S5's encoding asked for a decoder together with a witness x and a selector over 1500 flats in a single instance, and it ran 18 h or more without an answer. The rank-drop reformulation turns this into 3 orbit instances, each a 45-56-line minrank test that is UNSAT in about 5 s. + +## Side lemma (argued, from the same runs; possible input for the >= 43 question) +Take any decoder with c totals and any 3-flat H. At most c - 15 totals have R_j inside H, and the same holds for A_j. +- Proof: those columns vanish on O, so B[O,O] has rank <= c - #. +- More generally, for any family of 3-flats H_1..H_m: #{j : R_j inside some H_i} <= c - minrank(pattern on the complement of the union of the H_i). + +## Reproduce +`PATH=$SP/venv/bin:$PATH sh verify.sh` takes about 3-5 min and uses 3 processes. `FULL=1` adds the 1500-flat sweep, which takes about 45 min. Compare the output with `expected.txt`. +Logs from the original runs are in `logs/` (`prove_orbits.log`; `prove_all.log` is still running in s6/). diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/expected.txt b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/expected.txt new file mode 100644 index 0000000000..1fef093acf --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/expected.txt @@ -0,0 +1,12 @@ +Gauss Counter(...) (flats.py; includes 1500 dimension-3 flats: 60 of size 15, 480 of size 6, 960 of size 4) +group order on lines 11520 +orbits on 3-flats: [(15, 60), (4, 960), (6, 480)] total 1500 +ALL AS EXPECTED +all 60 lines c=14 UNSAT +all 60 lines c=15 SAT +|H|= 4 |O|=56 c=14 {cadical153,glucose4,maplechrono} UNSAT (clique-gauge 12) +|H|= 4 |O|=56 c=15 {cadical153,glucose4,maplechrono} SAT +|H|= 6 |O|=54 c=14 ... UNSAT ; c=15 ... SAT +|H|=15 |O|=45 c=14 ... UNSAT ; c=15 ... SAT +rank_F2 check: W,U columns 15 betas [4, 4, ..., 4] sum 60 +FULL=1: ALL AS EXPECTED (1500 of 1500 3-flats, c=14 UNSAT on every complement) diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_all.log b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_all.log new file mode 100644 index 0000000000..a6edbd5ea6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_all.log @@ -0,0 +1,16 @@ +|H|= 6 |O|=54 c=14 cadical153 UNSAT 5.1s clique-gauge 12 [100/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 9.1s clique-gauge 12 [200/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 5.0s clique-gauge 12 [300/1500] +|H|= 6 |O|=54 c=14 cadical153 UNSAT 3.1s clique-gauge 12 [400/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 7.2s clique-gauge 12 [500/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 5.7s clique-gauge 12 [600/1500] +|H|= 6 |O|=54 c=14 cadical153 UNSAT 3.0s clique-gauge 12 [700/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 3.2s clique-gauge 12 [800/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 6.3s clique-gauge 12 [900/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 3.2s clique-gauge 12 [1000/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 4.8s clique-gauge 12 [1100/1500] +|H|= 6 |O|=54 c=14 cadical153 UNSAT 4.4s clique-gauge 12 [1200/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 4.2s clique-gauge 12 [1300/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 5.7s clique-gauge 12 [1400/1500] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 3.3s clique-gauge 12 [1500/1500] +ALL AS EXPECTED diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_orbits.log b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_orbits.log new file mode 100644 index 0000000000..d9774a8206 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/logs/prove_orbits.log @@ -0,0 +1,21 @@ +all 60 lines c=14 UNSAT 5.6s +all 60 lines c=15 SAT 1.3s +|H|=15 |O|=45 c=14 cadical153 UNSAT 6.9s clique-gauge 12 [1/18] +|H|=15 |O|=45 c=15 cadical153 SAT 2.5s clique-gauge 12 [2/18] +|H|=15 |O|=45 c=14 glucose4 UNSAT 11.6s clique-gauge 12 [3/18] +|H|=15 |O|=45 c=15 glucose4 SAT 4.1s clique-gauge 12 [4/18] +|H|= 4 |O|=56 c=14 cadical153 UNSAT 7.4s clique-gauge 12 [5/18] +|H|= 4 |O|=56 c=14 glucose4 UNSAT 21.4s clique-gauge 12 [6/18] +|H|=15 |O|=45 c=15 maplechrono SAT 37.8s clique-gauge 12 [7/18] +|H|= 4 |O|=56 c=15 cadical153 SAT 1.2s clique-gauge 12 [8/18] +|H|=15 |O|=45 c=14 maplechrono UNSAT 57.6s clique-gauge 12 [9/18] +|H|= 4 |O|=56 c=15 maplechrono SAT 41.5s clique-gauge 12 [10/18] +|H|= 6 |O|=54 c=14 cadical153 UNSAT 4.9s clique-gauge 12 [11/18] +|H|= 6 |O|=54 c=14 glucose4 UNSAT 13.1s clique-gauge 12 [12/18] +|H|= 4 |O|=56 c=14 maplechrono UNSAT 122.7s clique-gauge 12 [13/18] +|H|= 6 |O|=54 c=15 cadical153 SAT 4.5s clique-gauge 12 [14/18] +|H|= 6 |O|=54 c=15 glucose4 SAT 2.3s clique-gauge 12 [15/18] +|H|= 6 |O|=54 c=14 maplechrono UNSAT 77.9s clique-gauge 12 [16/18] +|H|= 6 |O|=54 c=15 maplechrono SAT 55.1s clique-gauge 12 [17/18] +|H|= 4 |O|=56 c=15 glucose4 SAT 227.2s clique-gauge 12 [18/18] +ALL AS EXPECTED diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/flats.py b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/flats.py new file mode 100644 index 0000000000..b064c81896 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/flats.py @@ -0,0 +1,26 @@ +"""All flats (line sets of K-subspaces spanned by configuration lines) of a view, with dims. Exact.""" +import sys, pickle; sys.path.insert(0,sys.path[0]) +from lines import * +def flats(S): + v=S.v; out={} + def close(idx): + d=S.arank(idx); return frozenset(T for T in range(v) if S.arank(list(idx)+[T])==d), d + level=[ (frozenset([T]),1) for T in range(v)] + for f,d in level: out[f]=d + frontier=list(out.items()) + while frontier: + new={} + for f,d in frontier: + if d>=S.w: continue + for T in range(v): + if T in f or any(T in g and f in {0,+-4,+-8}). + * Gaussian view : J = left multiplication by i on R^8 = H^2 (coords 1,i,j,k per block); J preserves E8. + Hermitian Gram hG(x,y) = ( + i )/4 in Z[i] (diag 2). + * Eisenstein view: rho = fixed-point-free order-3 isometry of E8 (product of Coxeter elements of 4 orthogonal A2's). + Sesquilinear Gram hE(x,y) = ( - w )/4 in Z[w] (w^2+w+1=0; diag 2+w, a unit at 2). + * Hurwitz view : left multiplication by i, j, k; quaternionic lines of real dim 4. +Elements of Z[i] / Z[w] are stored as integer pairs (a, b) = a + b*i / a + b*w. +Everything is exact integer arithmetic; asserts check every structural claim used downstream. +""" +import itertools +from fractions import Fraction +import numpy as np + + +def e8_roots(): + R = [] + for i, j in itertools.combinations(range(8), 2): + for si in (2, -2): + for sj in (2, -2): + r = [0] * 8; r[i], r[j] = si, sj; R.append(r) + for s in itertools.product((1, -1), repeat=8): + if s.count(-1) % 2 == 0: + R.append(list(s)) + R = np.array(R, dtype=np.int64) + assert len(R) == 240 and set((R * R).sum(1)) == {8} + return R + + +def rank_q(rows): + """exact rank over Q of an integer matrix (fraction-free Gaussian elimination).""" + M = [list(map(int, r)) for r in rows] + if not M: + return 0 + n, m = len(M), len(M[0]); r = 0 + for c in range(m): + p = next((i for i in range(r, n) if M[i][c] != 0), None) + if p is None: + continue + M[r], M[p] = M[p], M[r] + for i in range(r + 1, n): + if M[i][c]: + f, g = M[i][c], M[r][c] + M[i] = [g * a - f * b for a, b in zip(M[i], M[r])] + r += 1 + if r == n: + break + return r + + +def left_quat(): + Li = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 0, -1], [0, 0, 1, 0]]) + Lj = np.array([[0, 0, -1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, -1, 0, 0]]) + Lk = Li @ Lj + blk = lambda M: np.kron(np.eye(2, dtype=np.int64), M).astype(np.int64) + return blk(Li), blk(Lj), blk(Lk) + + +def a2_rotation(roots): + """rho, an order-3 fixed-point-free isometry preserving E8 (returned as an exact integer matrix 4*rho).""" + chosen = [] + for a in roots: + if len(chosen) == 4: + break + if any(np.dot(a, x) != 0 for p in chosen for x in p): + continue + for b in roots: + if np.dot(a, b) == -4 and all(np.dot(a, x) == 0 and np.dot(b, x) == 0 for p in chosen for x in p): + chosen.append((a, b)); break + assert len(chosen) == 4 + g = np.eye(8, dtype=object) + for a, b in chosen: + sa = np.eye(8, dtype=object) * 8 - 2 * np.outer(a, a).astype(object) + sb = np.eye(8, dtype=object) * 8 - 2 * np.outer(b, b).astype(object) + g = g.dot(sa).dot(sb) + D = 8 ** 8 // 4 + assert all(x % D == 0 for x in g.flatten()) + g = (g // D).astype(np.int64) + I = np.eye(8, dtype=np.int64) + assert (g @ g @ g == 64 * I).all() and (g @ g + 4 * g + 16 * I == 0).all() + return g + + +def lines_over(roots, ops): + """group roots into A-lines, A spanned by the integer matrices ops (identity first; ops may be scaled by 4). + Returns representatives (first root met of each line, in root order).""" + S = {tuple(r) for r in roots} + reps, seen = [], set() + for r in roots: + if tuple(r) in seen: + continue + span = [B @ r for B in ops] + assert rank_q(span) == len(ops) + reps.append(r) + members = [s for s in roots if rank_q(span + [s]) == len(ops)] + for s in members: + seen.add(tuple(s)) + return np.array(reps) + + +class View: + """name, reps (v x 8), ops (real matrices spanning A, scaled so that ops[i] @ root is a root times scale[i]).""" + def __init__(self, name, reps, ops, d): + self.name, self.reps, self.ops, self.d = name, reps, ops, d + self.v = len(reps) + NO = np.zeros((self.v, self.v), bool) + for B in ops: + NO |= (reps @ B @ reps.T) != 0 + np.fill_diagonal(NO, False) + self.NO = NO # A-non-orthogonal pairs (the decoder pattern forces B_TS = 0 there) + self.w = self.arank(range(self.v)) + + def arank(self, idx): + rows = [B @ self.reps[i] for i in idx for B in self.ops] + r = rank_q(rows) if rows else 0 + assert r % self.d == 0 + return r // self.d + + def pair_rank(self, R, A): + """A-rank of the pairing span(R) x span(A) = real rank of the real Gram block / dim A (Theorem C cost).""" + BR = [B @ self.reps[i] for i in R for B in self.ops] + BA = [B @ self.reps[i] for i in A for B in self.ops] + if not BR or not BA: + return 0 + M = [[int(x @ y) for y in BA] for x in BR] + r = rank_q(M); assert r % self.d == 0 + return r // self.d + + +def views(): + roots = e8_roots(); I8 = np.eye(8, dtype=np.int64) + Li, Lj, Lk = left_quat() + S = {tuple(r) for r in roots} + for M in (Li, Lj, Lk): + assert all(tuple(M @ r) in S for r in roots) + g = a2_rotation(roots) # 4*rho + assert all(tuple(g @ r // 4) in S and ((g @ r) % 4 == 0).all() for r in roots) + rho = g # 4*rho (exact integer matrix); ranks are scale-free + out = {} + out['Gauss'] = View('Gauss', lines_over(roots, [I8, Li]), [I8, Li], 2) + out['Eisen'] = View('Eisen', lines_over(roots, [I8, rho]), [I8, rho], 2) + out['Hurwitz'] = View('Hurwitz', lines_over(roots, [I8, Li, Lj, Lk]), [I8, Li, Lj, Lk], 4) + out['_J'], out['_rho'] = Li, rho + return out + + +def gauss_gram(V, J): + """hG(r_T, r_S) = ( + i)/4 as integer pairs (a, b).""" + A = V @ V.T; Bm = V @ J @ V.T + assert (A % 4 == 0).all() and (Bm % 4 == 0).all() + return A // 4, Bm // 4 + + +def eisen_gram(V, rho4): + """hE(r_T, r_S) = ( - w)/4 as integer pairs (a, b) = a + b w (rho4 = 4*rho).""" + A = V @ V.T; Bm = V @ rho4 @ V.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + + +if __name__ == '__main__': + W = views() + for nm in ('Gauss', 'Eisen', 'Hurwitz'): + S = W[nm] + print(nm, 'v', S.v, 'w', S.w, 'nonorth degree', sorted(set(S.NO.sum(1)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/orbits.py b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/orbits.py new file mode 100644 index 0000000000..11e4c31837 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/orbits.py @@ -0,0 +1,53 @@ +"""Permutation group of the 60 Gaussian-E8 lines generated by the 60 complex reflections s_r s_{Jr}; orbits on the 1500 3-flats.""" +import sys, pickle +sys.path.insert(0, sys.path[0]) +import numpy as np +from lines import views, e8_roots + +def line_index(S): + J = S.ops[1]; idx = {} + for T, r in enumerate(S.reps): + for v in (r, -r, J @ r, -(J @ r)): idx[tuple(v)] = T + return idx + +def generators(S): + J = S.ops[1]; idx = line_index(S); gens = [] + for r in S.reps: + out = [] + for T in range(S.v): + x = S.reps[T] + for a in (r, J @ r): # real reflections in r and Jr (norm 8): x -> x - 2/8 a + x = x - (2 * int(x @ a) // 8) * a + out.append(idx[tuple(x)]) + assert sorted(out) == list(range(S.v)) + gens.append(tuple(out)) + return gens + +def closure(gens, n): + G = {tuple(range(n))}; frontier = list(G) + while frontier: + new = [] + for p in frontier: + for g in gens: + q = tuple(g[p[i]] for i in range(n)) + if q not in G: G.add(q); new.append(q) + frontier = new + return G + +if __name__ == '__main__': + S = views()['Gauss'] + gens = generators(S) + # check: generators preserve non-orthogonality + for g in gens: + P = np.array(g); assert (S.NO[np.ix_(P, P)] == S.NO).all() + G = closure(gens, S.v); print('group order on lines', len(G)) + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = sorted(sorted(f) for f, d in F.items() if d == 3) + hs = {frozenset(h) for h in hyp}; seen = set(); reps = [] + for h in hyp: + if frozenset(h) in seen: continue + orb = {frozenset(g[i] for i in h) for g in G} + assert orb <= hs + seen |= orb; reps.append((h, len(orb))) + print('orbits on 3-flats:', [(len(h), n) for h, n in reps], 'total', sum(n for _, n in reps)) + pickle.dump({'G': sorted(G), 'hyp_reps': reps}, open('orbits.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/price_c15.py b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/price_c15.py new file mode 100644 index 0000000000..89b103fdd7 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/price_c15.py @@ -0,0 +1,19 @@ +"""Sanity: a 15-total decoder found by SAT (calibration) has Theorem-C cost sum_j beta_j = 60, as the G2 theorem forces.""" +import sys +sys.path.insert(0, sys.path[0]) +import numpy as np +from lines import views +from sub_minrank import encode +from pysat.solvers import Solver + +S = views()['Gauss']; c = 15 +cl, pool, _ = encode(S, range(60), c) +with Solver(name='cadical153', bootstrap_with=cl) as s: + assert s.solve(); m = set(l for l in s.get_model() if l > 0) +W = np.array([[pool.id(('w', T, j)) in m for j in range(c)] for T in range(60)], int) +U = np.array([[pool.id(('u', T, j)) in m for j in range(c)] for T in range(60)], int) +B = W @ U.T % 2 +assert all(B[T, T] == 1 for T in range(60)) and not (B[S.NO] != 0).any(), 'invalid decoder' +beta = [S.pair_rank([T for T in range(60) if W[T, j]], [T for T in range(60) if U[T, j]]) for j in range(c)] +print('rank_F2 check: W,U columns', c, ' betas', beta, ' sum', sum(beta)) +assert sum(beta) == 60 diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/prove_c15.py b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/prove_c15.py new file mode 100644 index 0000000000..625d34bc02 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/prove_c15.py @@ -0,0 +1,44 @@ +"""G2 settled: no 15-total Gaussian-E8 decoder has a column-space vector supported in a 3-flat. + +Reduction (argued, elementary): if B = W U^T (W, U in F2^{60x15}) is valid and x != 0 has supp(W x) inside a 3-flat H, +then with O = lines outside H, W[O] x = 0, so rank W[O] <= 14 and B[O,O] = W[O] U[O]^T is a rank-<=14 matrix obeying +the decoder pattern induced on O. Hence it suffices that minrank(pattern on O) >= 15 for every 3-flat H (same for U by +transposition). By symmetry (group of order 11520 on the lines, orbits.py) one H per orbit suffices; argv 'all' runs all +1500 3-flats instead (group-free check). Calibrations: c = 15 on O must be SAT; c = 14 / 15 on all 60 lines UNSAT / SAT.""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from multiprocessing import Pool +from lines import views +from sub_minrank import run + +S = views()['Gauss'] + +def job(args): + H, c, solver = args + O = [T for T in range(60) if T not in set(H)] + r, t, k = run(S, O, c, solver) + return H, c, solver, r, t, k + +if __name__ == '__main__': + mode = sys.argv[1] if len(sys.argv) > 1 else 'orbits' + solvers = sys.argv[2].split(',') if len(sys.argv) > 2 else ['cadical153'] + if mode == 'all': + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + Hs = sorted(sorted(f) for f, d in F.items() if d == 3); assert len(Hs) == 1500 + jobs = [(H, 14, solvers[0]) for H in Hs] + else: + reps = pickle.load(open('orbits.pkl', 'rb'))['hyp_reps'] + Hs = [h for h, _ in reps] + jobs = [(H, c, s) for H in Hs for c in (14, 15) for s in solvers] + for c in (14, 15): + r, t, k = run(S, range(60), c, solvers[0]) + print('all 60 lines c=%d %s %.1fs' % (c, 'SAT' if r else 'UNSAT', t), flush=True) + bad = 0; n = 0 + with Pool(3) as p: + for H, c, s, r, t, k in p.imap_unordered(job, jobs): + n += 1 + exp = (c == 15) + bad += (r != exp) + if mode != 'all' or r or n % 100 == 0: + print('|H|=%2d |O|=%2d c=%d %-11s %s %.1fs clique-gauge %d [%d/%d]' % (len(H), 60 - len(H), c, s, 'SAT' if r else 'UNSAT', t, k, n, len(jobs)), flush=True) + print('ALL AS EXPECTED' if bad == 0 else 'MISMATCHES: %d' % bad, flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/sub_minrank.py b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/sub_minrank.py new file mode 100644 index 0000000000..771a4690b0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/scripts/sub_minrank.py @@ -0,0 +1,50 @@ +"""F2 minrank test of the Gaussian-E8 decoder pattern induced on a line subset O (asymmetric B = W U^T, clique gauge).""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from lines import views +from pysat.formula import IDPool +from pysat.solvers import Solver +import networkx as nx + +def encode(S, O, c, gauge=True): + O = list(O); pool = IDPool(); cl = [] + U = {T: [pool.id(('u', T, j)) for j in range(c)] for T in O} + W = {T: [pool.id(('w', T, j)) for j in range(c)] for T in O} + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + def xorlist(ls): + acc = ls[0] + for l in ls[1:]: acc = XOR(acc, l) + return acc + for T in O: + for S_ in O: + if T == S_ or S.NO[T, S_]: + b = xorlist([AND(W[T][j], U[S_][j]) for j in range(c)]) + cl.append([b] if T == S_ else [-b]) + K = [] + if gauge: + g = nx.Graph([(i, j) for i in O for j in O if i < j and S.NO[i, j]]) + K = sorted(max(nx.find_cliques(g), key=len))[:c] + for i, T in enumerate(K): + for j in range(c): + cl.append([U[T][j]] if j == i else [-U[T][j]]); cl.append([W[T][j]] if j == i else [-W[T][j]]) + return cl, pool, len(K) + +def run(S, O, c, solver='cadical153'): + cl, pool, k = encode(S, O, c) + t = time.time() + with Solver(name=solver, bootstrap_with=cl) as s: r = s.solve() + return r, time.time() - t, k + +if __name__ == '__main__': + S = views()['Gauss'] + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = [sorted(f) for f, d in F.items() if d == 3] + c = int(sys.argv[1]) if len(sys.argv) > 1 else 14 + print('full 60, c', c, run(S, range(60), c), flush=True) + for size in (15, 6, 4): + H = next(h for h in hyp if len(h) == size) + O = [T for T in range(60) if T not in H] + print('|H|', size, '|O|', len(O), 'c', c, run(S, O, c), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/verify.sh new file mode 100644 index 0000000000..de30fb27d8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s6-gaussian-c15/verify.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# Usage: PATH=/bin:$PATH sh verify.sh (python3.11 + numpy, networkx, python-sat). ~3 min (orbit mode), 3 processes. +# Optional: FULL=1 also runs the group-free sweep over all 1500 3-flats (~45 min with 3 processes). +set -e +export OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 +cd "$(dirname "$0")/scripts" +python -B flats.py Gauss # writes flats_Gauss.pkl +python -B orbits.py # group of order 11520, orbits on 3-flats +python -B prove_c15.py orbits cadical153,glucose4,maplechrono | sed 's/ [0-9.]*s / Ts /; s/\[[0-9]*\/18\]//' | sort +python -B price_c15.py +if [ -n "$FULL" ]; then python -B prove_c15.py all | tail -1; fi diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/README.md b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/README.md new file mode 100644 index 0000000000..166f186b6b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/README.md @@ -0,0 +1,96 @@ +# S7 — Gaussian E8: no decoder with exactly 16 totals costs ≤ 42 + +Context: #620 §10 (`s5-extension-ring-floors/`) and §13 (S6, every 15-total decoder costs exactly 60). +- The setting is the Gaussian view of E8: 60 Q(i)-lines in K⁴ with the F2 decoder pattern (B_TT = 1, and B_TS = 0 on K-non-orthogonal pairs). The minrank is 15. +- Theorem-C cost is Σβ_j, where β_j is the K-rank of the pairing span(R_j) × span(A_j). +- Proven floor: 40 for every decoder (S5). +- To beat real E8 a decoder needs Σβ ≤ 42. By S5 and S6, such a decoder needs at least 16 totals. + +## Result + +**Theorem (S7).** Every Gaussian-E8 decoder with exactly 16 totals has Σβ_j ≥ 43. + +So a Gaussian decoder that beats real E8 needs at least 17 totals. + +## Definitions + +- **Level-k set.** A set of k distinct 3-flats Z_1..Z_k such that every subset J of it satisfies + minrank(pattern on O_J) ≤ 16 − |J|, where O_J is the set of lines outside ∪_{i∈J} Z_i. +- **Cheap total.** Total j is cheap when β_j < 4, and k is the number of cheap totals. + +## Proof + +### 1. [argued] Cost by type +The form is Hermitian definite on K⁴. With X = span R_j and Y = span A_j, β_j = dim X − dim(X ∩ Y^⊥) ≥ dim X + dim Y − 4. +- If neither side is proper, β_j = 4. +- Otherwise β_j < 4, and a side lies in a proper flat, hence in a 3-flat spanned by lines. + +Cheap totals fall into three types: +- **a**: one side has dim ≤ 2, so it lies in a 2-flat Z. **Every** 3-flat through Z is a legal 3-flat for this total. β ≥ 0. +- **p**: dims (3,3) with β = 2. Then X ≠ Y, and both X and Y are legal 3-flats (a "dumbbell"). +- **q**: everything else. Here β ≥ 3: either (3,4), or (3,3) with β = 3. + +With f = 16 − k full totals: +- cost ≥ 4f + 2p + 3q. +- So cost ≤ 42 forces **3a + p ≥ 22 − k**. + +### 2. [argued] Rank drop over several flats (extends the S6 lemma) +Pick one legal 3-flat Z_j for each cheap total j. +- For any subset J, the columns j ∈ J of W (or U) vanish on O_J. So B[O_J, O_J] has rank ≤ 16 − |J| and obeys the induced pattern. +- Hence **every** legal choice gives a level-k set. +- The flats are distinct. If one 3-flat H were legal for two totals, then rank B[O,O] ≤ 14 on O = the lines outside H, but S6 proved minrank 15 there. + +### 3. [checked] Level tables up to the line group (order 11520, from S6 `orbits.py`) +- `levels2.py` / `levels3.py` compute all orbit representatives of level-k sets, k = 1..16. The run uses these steps: + - extend each level-(k−1) rep by one 3-flat; + - canonicalise (least image under the group); + - keep the set only if all of its (k−1)-subsets are level-(k−1) sets; + - SAT-test the whole union at rank ≤ 16 − k. The SAT encoding is S6's clique-gauge encoding. +- Rep counts by level: + + | k | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | + |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| + | reps | 3 | 28 | 120 | 368 | 498 | 464 | 315 | 206 | 124 | 74 | 32 | 24 | 10 | 6 | 1 | 0 | + +- Level 2: 28 of the 178 pair orbits pass. +- From level 7 on, subset closure does all of the pruning. +- Consistency check: the slow canonicaliser (`levels.py`) and the fast one (`levels2.py`/`levels3.py`) agree on levels 2–4. +- **Cross-check.** Every UNSAT pruning (levels 2–6, 200 instances) was re-solved with Glucose 4: see `logs/cross_glucose.log`. Only UNSAT verdicts remove configurations; a wrong SAT verdict would only keep more. + +### 4. [checked] (A): a = 0 whenever k ≥ 5 +A member of a level-k rep is *a-capable* if some 2-flat Z inside it has the following property: substituting each 3-flat through Z still gives a level-k set (6, 12 or 14 substitutions, depending on the 2-flat's size). +- Reps with an a-capable member: 3, 19, 18 and 3 at k = 1..4, and **0 for every k ≥ 5**. +- So a decoder with k ≥ 5 cheap totals has no total with a side of dim ≤ 2. +- The case of a line side is covered too: the line lies in a 2-flat inside the chosen 3-flat, and every 3-flat through that 2-flat contains the line. + +### 5. [checked] (B): few dumbbells +- β between 3-flats is computed for all 1500² pairs: β = 2 on 102180 ordered pairs and 3 on the rest. The method is SVD of products of orthonormal bases; an assert checks that no singular value is ambiguous. +- p* is the largest number of members of a level-k rep that can be doubled at once. Doubling means each chosen member gets a partner Y with β = 2, and **all 2^p** transversals are level-k sets. +- Results for k = 11..15: p* = 5, 4, 3, 2, 1. The requirement 22 − k is 11, 10, 9, 8, 7. + +### 6. Arithmetic +| k | why 3a + p ≥ 22 − k fails | +|---|---| +| ≤ 4 | 3a + p ≤ 3k < 22 − k | +| 5..10 | a = 0, so p ≤ k < 22 − k | +| 11..15 | p ≤ p* < 22 − k | +| 16 | no level-16 set exists | + +The 2-flat search (`lev_c16_d2`) is not needed for the theorem. It shows that at most 2 totals can have a side of dim ≤ 2, for any k. + +## Calibrations +- c = 16 decoders exist (SAT in 0.1 s, `calib.log`). A SAT-found c = 16 decoder is priced in `logs/price16.log`: all β_j = 4, sum 64. +- Level 1 at c = 15 is empty (rank ≤ 14 on each 3-flat complement is UNSAT), which reproduces S6. +- The direct monolithic encoding (`direct.py`: 16 columns, 9 of them confined to flats, no gauge) gave no answer in 20 min per instance. The orbit and level route is what made the problem tractable. + +## Related hub work (cited; no overlap) +- #767 (cbethin, planar and incidence obstructions) and #2066 (adamgall, whole-hyperplane reciprocity ≥ 180, symmetric B) are scoped subclass bounds. +- #738 and #770/#868 are about the clique-LP floor of 40. +- None of these treats 16-total decoders. + +## Reproduce +`PATH=$SP/venv/bin:$PATH sh verify.sh` uses 8 processes. It rebuilds the level tables from scratch (about 15 min), runs `prove_c16.py` (about 13 min), `price16.py` and the c = 15 calibration. Compare the output with `expected.txt`. A from-scratch run reproduced `expected.txt`, and the rebuilt tables are identical to `data/lev_c16_d3_hi.pkl`. `CROSS=1` adds the Glucose cross-check (about 45 min). + +## Notes +- `scripts/sub_minrank.py` is S6's encoder with one change: isolated lines are added to the clique graph, so a line set with no non-orthogonal pair still gets a 1-clique gauge. +- `refine.py`, `acap_sanity.py` and `dumbbell.py` are the exploratory forms of (A) and (B). `prove_c16.py` is the consolidated check. diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/flats_Gauss.pkl b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/flats_Gauss.pkl new file mode 100644 index 0000000000..1dd3283f66 Binary files /dev/null and b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/flats_Gauss.pkl differ diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d2.pkl b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d2.pkl new file mode 100644 index 0000000000..b9561358df Binary files /dev/null and b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d2.pkl differ diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d3_hi.pkl b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d3_hi.pkl new file mode 100644 index 0000000000..912743cd0c Binary files /dev/null and b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/lev_c16_d3_hi.pkl differ diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/orbits.pkl b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/orbits.pkl new file mode 100644 index 0000000000..c48f4194f8 Binary files /dev/null and b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/orbits.pkl differ diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/pair_orbits.pkl b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/pair_orbits.pkl new file mode 100644 index 0000000000..037f677d58 Binary files /dev/null and b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/data/pair_orbits.pkl differ diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/expected.txt b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/expected.txt new file mode 100644 index 0000000000..4324b68797 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/expected.txt @@ -0,0 +1,43 @@ +Gauss v 60 w 4 nonorth degree [np.int64(44)] +LEVEL 1: 3 good orbit reps (3s) +LEVEL 2: 28 good orbit reps (131s) +LEVEL 3: 120 good orbit reps (157s) +LEVEL 4: 368 good orbit reps +LEVEL 5: 498 good orbit reps +LEVEL 6: 464 good orbit reps +LEVEL 7: 315 good orbit reps +LEVEL 8: 206 good orbit reps +LEVEL 9: 124 good orbit reps +LEVEL 10: 74 good orbit reps +LEVEL 11: 32 good orbit reps +LEVEL 12: 24 good orbit reps +LEVEL 13: 10 good orbit reps +LEVEL 14: 6 good orbit reps +LEVEL 15: 1 good orbit reps +LEVEL 16: 0 good orbit reps +EMPTY at level 16 => at most 15 such flats/totals for c=16, dims 3 +level sizes: {1: 3, 2: 28, 3: 120, 4: 368, 5: 498, 6: 464, 7: 315, 8: 206, 9: 124, 10: 74, 11: 32, 12: 24, 13: 10, 14: 6, 15: 1, 16: 0} +(A) k= 1: 3 of 3 reps have an a-capable member +(A) k= 2: 19 of 28 reps have an a-capable member +(A) k= 3: 18 of 120 reps have an a-capable member +(A) k= 4: 3 of 368 reps have an a-capable member +(A) k= 5: 0 of 498 reps have an a-capable member +(A) k= 6: 0 of 464 reps have an a-capable member +(A) k= 7: 0 of 315 reps have an a-capable member +(A) k= 8: 0 of 206 reps have an a-capable member +(A) k= 9: 0 of 124 reps have an a-capable member +(A) k=10: 0 of 74 reps have an a-capable member +(A) k=11: 0 of 32 reps have an a-capable member +(A) k=12: 0 of 24 reps have an a-capable member +(A) k=13: 0 of 10 reps have an a-capable member +(A) k=14: 0 of 6 reps have an a-capable member +(A) k=15: 0 of 1 reps have an a-capable member +(B) k=11: max p* = 5 (need >= 11 for cost <= 42) +(B) k=12: max p* = 4 (need >= 10 for cost <= 42) +(B) k=13: max p* = 3 (need >= 9 for cost <= 42) +(B) k=14: max p* = 2 (need >= 8 for cost <= 42) +(B) k=15: max p* = 1 (need >= 7 for cost <= 42) +THEOREM: no Gaussian-E8 decoder with exactly 16 totals has sum beta_j <= 42 +c=16 decoder valid; betas [4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4] sum 64 cheap totals 0 +LEVEL 1: 0 good orbit reps +EMPTY at level 1 => at most 0 such flats/totals for c=15, dims 3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib.log new file mode 100644 index 0000000000..ab9ce64757 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib.log @@ -0,0 +1,2 @@ +calib no-gauge c=16 True 0.1s +calib no-gauge c=15 True 0.1s diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib15.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib15.log new file mode 100644 index 0000000000..8424c26255 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/calib15.log @@ -0,0 +1,4 @@ +level 1: candidates 3, subset-closed 3; testing minrank <= 14 (0s) +LEVEL 1: 0 good orbit reps (12s) + (flat sizes, |O|) histogram: [] +EMPTY at level 1 => at most 0 such flats/totals for c=15, dims 3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/cross_glucose.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/cross_glucose.log new file mode 100644 index 0000000000..a0b73265bc --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/cross_glucose.log @@ -0,0 +1,208 @@ +level 2: 178 candidates, 28 good, 150 UNSAT prunings to recheck with glucose4 + (0, 2) |O|=36 minrank<=14: UNSAT 10.1s + (0, 23) |O|=43 minrank<=14: UNSAT 12.6s + (0, 19) |O|=41 minrank<=14: UNSAT 14.2s + (0, 132) |O|=42 minrank<=14: UNSAT 14.7s + (0, 64) |O|=40 minrank<=14: UNSAT 24.2s + (0, 134) |O|=42 minrank<=14: UNSAT 16.8s + (0, 129) |O|=42 minrank<=14: UNSAT 27.8s + (0, 208) |O|=42 minrank<=14: UNSAT 14.3s + (0, 136) |O|=44 minrank<=14: UNSAT 16.0s + (0, 212) |O|=42 minrank<=14: UNSAT 14.3s + (0, 63) |O|=42 minrank<=14: UNSAT 32.1s + (0, 232) |O|=42 minrank<=14: UNSAT 9.8s + (0, 233) |O|=42 minrank<=14: UNSAT 15.8s + (15, 21) |O|=52 minrank<=14: UNSAT 19.6s + (15, 20) |O|=52 minrank<=14: UNSAT 23.6s + (0, 15) |O|=43 minrank<=14: UNSAT 54.5s + (15, 19) |O|=52 minrank<=14: UNSAT 26.3s + (0, 952) |O|=41 minrank<=14: UNSAT 29.0s + (15, 17) |O|=54 minrank<=14: UNSAT 32.4s + (15, 22) |O|=52 minrank<=14: UNSAT 28.7s + (15, 29) |O|=53 minrank<=14: UNSAT 17.8s + (15, 25) |O|=54 minrank<=14: UNSAT 23.0s + (15, 26) |O|=54 minrank<=14: UNSAT 23.2s + (15, 23) |O|=54 minrank<=14: UNSAT 35.6s + (15, 33) |O|=51 minrank<=14: UNSAT 28.1s + (15, 24) |O|=54 minrank<=14: UNSAT 39.9s + (15, 35) |O|=51 minrank<=14: UNSAT 29.9s + (15, 44) |O|=51 minrank<=14: UNSAT 15.0s + (15, 37) |O|=53 minrank<=14: UNSAT 20.2s + (15, 43) |O|=51 minrank<=14: UNSAT 26.1s + (15, 31) |O|=53 minrank<=14: UNSAT 47.2s + (15, 47) |O|=51 minrank<=14: UNSAT 13.7s + (15, 49) |O|=53 minrank<=14: UNSAT 15.8s + (15, 45) |O|=53 minrank<=14: UNSAT 36.4s + (15, 46) |O|=53 minrank<=14: UNSAT 27.9s + (15, 50) |O|=53 minrank<=14: UNSAT 25.7s + (15, 129) |O|=50 minrank<=14: UNSAT 23.5s + (15, 132) |O|=50 minrank<=14: UNSAT 24.0s + (15, 140) |O|=51 minrank<=14: UNSAT 20.2s + (15, 141) |O|=51 minrank<=14: UNSAT 20.6s + (15, 143) |O|=51 minrank<=14: UNSAT 22.0s + (15, 136) |O|=52 minrank<=14: UNSAT 42.0s + (15, 144) |O|=51 minrank<=14: UNSAT 23.7s + (15, 146) |O|=51 minrank<=14: UNSAT 14.8s + (15, 48) |O|=51 minrank<=14: UNSAT 73.3s + (15, 145) |O|=51 minrank<=14: UNSAT 30.7s + (15, 154) |O|=50 minrank<=14: UNSAT 18.0s + (15, 155) |O|=50 minrank<=14: UNSAT 19.6s + (15, 142) |O|=53 minrank<=14: UNSAT 55.7s + (15, 153) |O|=52 minrank<=14: UNSAT 24.7s + (15, 157) |O|=50 minrank<=14: UNSAT 20.3s + (15, 148) |O|=53 minrank<=14: UNSAT 41.8s + (15, 159) |O|=52 minrank<=14: UNSAT 18.8s + (15, 158) |O|=52 minrank<=14: UNSAT 29.2s + (15, 165) |O|=50 minrank<=14: UNSAT 24.0s + (15, 134) |O|=50 minrank<=14: UNSAT 102.1s + (15, 167) |O|=50 minrank<=14: UNSAT 19.2s + (15, 164) |O|=52 minrank<=14: UNSAT 36.4s + (15, 166) |O|=50 minrank<=14: UNSAT 30.3s + (15, 156) |O|=50 minrank<=14: UNSAT 55.2s + (15, 170) |O|=52 minrank<=14: UNSAT 25.4s + (15, 169) |O|=52 minrank<=14: UNSAT 26.6s + (15, 168) |O|=50 minrank<=14: UNSAT 55.3s + (15, 205) |O|=52 minrank<=14: UNSAT 28.9s + (15, 202) |O|=50 minrank<=14: UNSAT 40.6s + (15, 200) |O|=50 minrank<=14: UNSAT 45.6s + (15, 208) |O|=52 minrank<=14: UNSAT 29.9s + (15, 201) |O|=50 minrank<=14: UNSAT 48.4s + (15, 203) |O|=50 minrank<=14: UNSAT 44.4s + (15, 206) |O|=52 minrank<=14: UNSAT 37.4s + (15, 211) |O|=52 minrank<=14: UNSAT 24.6s + (15, 209) |O|=52 minrank<=14: UNSAT 38.0s + (15, 232) |O|=52 minrank<=14: UNSAT 26.3s + (15, 214) |O|=52 minrank<=14: UNSAT 44.7s + (15, 215) |O|=52 minrank<=14: UNSAT 42.5s + (15, 213) |O|=52 minrank<=14: UNSAT 51.8s + (15, 210) |O|=52 minrank<=14: UNSAT 68.2s + (15, 239) |O|=52 minrank<=14: UNSAT 18.1s + (15, 212) |O|=52 minrank<=14: UNSAT 73.3s + (15, 235) |O|=52 minrank<=14: UNSAT 46.5s + (15, 234) |O|=52 minrank<=14: UNSAT 62.3s + (15, 238) |O|=52 minrank<=14: UNSAT 50.8s + (15, 242) |O|=52 minrank<=14: UNSAT 45.1s + (15, 406) |O|=53 minrank<=14: UNSAT 23.9s + (15, 405) |O|=53 minrank<=14: UNSAT 32.1s + (15, 409) |O|=51 minrank<=14: UNSAT 15.6s + (15, 407) |O|=51 minrank<=14: UNSAT 27.4s + (15, 396) |O|=50 minrank<=14: UNSAT 51.3s + (15, 398) |O|=50 minrank<=14: UNSAT 44.0s + (15, 408) |O|=51 minrank<=14: UNSAT 25.5s + (15, 442) |O|=50 minrank<=14: UNSAT 16.2s + (15, 411) |O|=53 minrank<=14: UNSAT 25.0s + (15, 410) |O|=51 minrank<=14: UNSAT 30.4s + (15, 412) |O|=53 minrank<=14: UNSAT 31.9s + (15, 443) |O|=50 minrank<=14: UNSAT 30.6s + (15, 444) |O|=50 minrank<=14: UNSAT 31.4s + (15, 445) |O|=50 minrank<=14: UNSAT 41.4s + (15, 450) |O|=52 minrank<=14: UNSAT 41.6s + (15, 451) |O|=52 minrank<=14: UNSAT 38.2s + (15, 462) |O|=53 minrank<=14: UNSAT 27.5s + (15, 454) |O|=52 minrank<=14: UNSAT 38.5s + (15, 455) |O|=52 minrank<=14: UNSAT 42.1s + (15, 463) |O|=53 minrank<=14: UNSAT 36.0s + (15, 476) |O|=52 minrank<=14: UNSAT 22.4s + (15, 477) |O|=52 minrank<=14: UNSAT 25.5s + (15, 474) |O|=52 minrank<=14: UNSAT 42.5s + (15, 1129) |O|=51 minrank<=14: UNSAT 14.9s + (15, 481) |O|=52 minrank<=14: UNSAT 31.4s + (15, 1133) |O|=50 minrank<=14: UNSAT 18.2s + (15, 478) |O|=52 minrank<=14: UNSAT 45.4s + (15, 1128) |O|=51 minrank<=14: UNSAT 34.3s + (15, 1149) |O|=53 minrank<=14: UNSAT 24.4s + (15, 484) |O|=52 minrank<=14: UNSAT 55.2s + (15, 1160) |O|=51 minrank<=14: UNSAT 13.5s + (15, 1132) |O|=50 minrank<=14: UNSAT 38.9s + (15, 488) |O|=52 minrank<=14: UNSAT 55.9s + (15, 1148) |O|=53 minrank<=14: UNSAT 42.7s + (15, 1166) |O|=51 minrank<=14: UNSAT 28.9s + (15, 1167) |O|=51 minrank<=14: UNSAT 28.8s + (15, 1150) |O|=55 minrank<=14: UNSAT 56.3s + (15, 1182) |O|=53 minrank<=14: UNSAT 25.6s + (15, 1163) |O|=53 minrank<=14: UNSAT 46.9s + (15, 1180) |O|=53 minrank<=14: UNSAT 38.1s + (15, 1169) |O|=53 minrank<=14: UNSAT 51.4s + (15, 1161) |O|=51 minrank<=14: UNSAT 62.5s + (15, 1284) |O|=52 minrank<=14: UNSAT 19.3s + (15, 1276) |O|=52 minrank<=14: UNSAT 32.4s + (19, 21) |O|=50 minrank<=14: UNSAT 22.1s + (15, 1186) |O|=53 minrank<=14: UNSAT 47.7s + (19, 43) |O|=49 minrank<=14: UNSAT 20.5s + (15, 1208) |O|=50 minrank<=14: UNSAT 57.7s + (15, 1281) |O|=52 minrank<=14: UNSAT 43.5s + (19, 35) |O|=49 minrank<=14: UNSAT 39.6s + (19, 72) |O|=51 minrank<=14: UNSAT 6.9s + (19, 66) |O|=49 minrank<=14: UNSAT 18.4s + (19, 75) |O|=49 minrank<=14: UNSAT 8.6s + (15, 1209) |O|=50 minrank<=14: UNSAT 64.4s + (19, 79) |O|=49 minrank<=14: UNSAT 10.3s + (19, 88) |O|=49 minrank<=14: UNSAT 8.4s + (19, 67) |O|=49 minrank<=14: UNSAT 17.0s + (19, 44) |O|=49 minrank<=14: UNSAT 32.7s + (19, 64) |O|=51 minrank<=14: UNSAT 30.6s + (19, 129) |O|=48 minrank<=14: UNSAT 13.4s + (19, 149) |O|=48 minrank<=14: UNSAT 9.6s + (19, 150) |O|=48 minrank<=14: UNSAT 12.8s + (19, 161) |O|=48 minrank<=14: UNSAT 11.4s + (19, 83) |O|=51 minrank<=14: UNSAT 24.6s + (19, 134) |O|=48 minrank<=14: UNSAT 17.6s + (19, 160) |O|=48 minrank<=14: UNSAT 15.8s + (19, 154) |O|=48 minrank<=14: UNSAT 27.9s +level 3: 159 candidates, 120 good, 39 UNSAT prunings to recheck with glucose4 + (0, 127, 138) |O|=23 minrank<=13: UNSAT 0.1s + (0, 172, 690) |O|=36 minrank<=13: UNSAT 0.1s + (0, 172, 615) |O|=36 minrank<=13: UNSAT 0.3s + (0, 172, 694) |O|=38 minrank<=13: UNSAT 0.5s + (0, 172, 617) |O|=38 minrank<=13: UNSAT 0.5s + (0, 176, 617) |O|=40 minrank<=13: UNSAT 0.4s + (0, 173, 617) |O|=38 minrank<=13: UNSAT 0.6s + (0, 173, 694) |O|=38 minrank<=13: UNSAT 0.6s + (0, 176, 694) |O|=41 minrank<=13: UNSAT 0.4s + (15, 16, 27) |O|=53 minrank<=13: UNSAT 0.4s + (15, 16, 228) |O|=50 minrank<=13: UNSAT 0.3s + (15, 16, 395) |O|=50 minrank<=13: UNSAT 0.3s + (15, 16, 1197) |O|=50 minrank<=13: UNSAT 0.4s + (15, 16, 1196) |O|=52 minrank<=13: UNSAT 0.5s + (15, 16, 393) |O|=50 minrank<=13: UNSAT 0.7s + (15, 16, 1198) |O|=50 minrank<=13: UNSAT 0.5s + (15, 16, 1221) |O|=48 minrank<=13: UNSAT 0.3s + (15, 16, 1220) |O|=50 minrank<=13: UNSAT 0.4s + (15, 16, 1222) |O|=48 minrank<=13: UNSAT 0.3s + (15, 16, 131) |O|=50 minrank<=13: UNSAT 1.1s + (15, 16, 1223) |O|=50 minrank<=13: UNSAT 0.4s + (15, 131, 299) |O|=48 minrank<=13: UNSAT 0.6s + (15, 147, 348) |O|=49 minrank<=13: UNSAT 0.4s + (15, 393, 1155) |O|=47 minrank<=13: UNSAT 0.5s + (15, 393, 1157) |O|=47 minrank<=13: UNSAT 0.4s + (15, 395, 555) |O|=48 minrank<=13: UNSAT 0.3s + (19, 33, 416) |O|=46 minrank<=13: UNSAT 0.1s + (19, 33, 414) |O|=46 minrank<=13: UNSAT 0.2s + (15, 131, 925) |O|=48 minrank<=13: UNSAT 1.1s + (15, 147, 1159) |O|=50 minrank<=13: UNSAT 1.3s + (19, 172, 327) |O|=42 minrank<=13: UNSAT 0.6s + (19, 414, 899) |O|=42 minrank<=13: UNSAT 0.3s + (19, 74, 442) |O|=43 minrank<=13: UNSAT 2.4s + (19, 74, 119) |O|=44 minrank<=13: UNSAT 2.6s + (19, 65, 114) |O|=44 minrank<=13: UNSAT 3.0s + (19, 65, 468) |O|=43 minrank<=13: UNSAT 3.0s + (0, 131, 142) |O|=44 minrank<=13: UNSAT 8.3s + (19, 74, 1149) |O|=44 minrank<=13: UNSAT 7.3s + (19, 65, 1198) |O|=44 minrank<=13: UNSAT 11.3s +level 4: 375 candidates, 368 good, 7 UNSAT prunings to recheck with glucose4 + (0, 127, 294, 1148) |O|=28 minrank<=12: UNSAT 0.0s + (0, 172, 693, 1151) |O|=32 minrank<=12: UNSAT 0.0s + (0, 173, 615, 1200) |O|=32 minrank<=12: UNSAT 0.0s + (0, 172, 692, 1157) |O|=37 minrank<=12: UNSAT 0.0s + (0, 172, 692, 1158) |O|=37 minrank<=12: UNSAT 0.0s + (15, 393, 394, 404) |O|=48 minrank<=12: UNSAT 0.1s + (19, 20, 33, 34) |O|=49 minrank<=12: UNSAT 0.1s +level 5: 500 candidates, 498 good, 2 UNSAT prunings to recheck with glucose4 + (0, 127, 152, 372, 524) |O|=12 minrank<=11: UNSAT 0.0s + (19, 20, 33, 1221, 1226) |O|=48 minrank<=11: UNSAT 0.0s +level 6: 466 candidates, 464 good, 2 UNSAT prunings to recheck with glucose4 + (0, 127, 272, 402, 552, 1293) |O|=16 minrank<=10: UNSAT 0.0s + (0, 127, 272, 382, 538, 1170) |O|=16 minrank<=10: UNSAT 0.0s +level 7: 315 candidates, 315 good, 0 UNSAT prunings to recheck with glucose4 +level 8: 206 candidates, 206 good, 0 UNSAT prunings to recheck with glucose4 +CROSSCHECK glucose4: 200 UNSAT prunings rechecked, 0 mismatches (731s) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/dumbbell.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/dumbbell.log new file mode 100644 index 0000000000..d77386278a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/dumbbell.log @@ -0,0 +1,9 @@ +beta histogram over ordered pairs [0, 0, 102180, 2147820] (8s) +k=9 reps 124 (p*, #doublable members) histogram [((0, 0), 3), ((1, 1), 8), ((1, 2), 6), ((1, 3), 4), ((2, 2), 12), ((2, 3), 4), ((2, 5), 1), ((3, 5), 35), ((3, 9), 2), ((4, 8), 8), ((4, 9), 4), ((5, 5), 8), ((5, 9), 14), ((7, 9), 15)] feasible 15 (84s) +k=10 reps 74 (p*, #doublable members) histogram [((0, 0), 8), ((1, 1), 1), ((1, 2), 2), ((1, 3), 1), ((2, 2), 3), ((2, 3), 10), ((2, 6), 1), ((3, 6), 14), ((4, 6), 6), ((4, 10), 6), ((6, 6), 9), ((6, 10), 13)] feasible 22 (177s) +k=11 reps 32 (p*, #doublable members) histogram [((0, 0), 2), ((1, 1), 2), ((1, 3), 1), ((2, 4), 8), ((3, 3), 2), ((3, 7), 3), ((5, 7), 8), ((5, 11), 6)] feasible 14 (209s) +k=12 reps 24 (p*, #doublable members) histogram [((0, 0), 1), ((1, 2), 4), ((2, 4), 2), ((4, 4), 5), ((4, 8), 7), ((4, 12), 5)] feasible 17 (235s) +k=13 reps 10 (p*, #doublable members) histogram [((0, 0), 2), ((1, 1), 1), ((3, 5), 2), ((3, 9), 5)] feasible 7 (238s) +k=14 reps 6 (p*, #doublable members) histogram [((2, 2), 2), ((2, 6), 4)] feasible 6 (238s) +k=15 reps 1 (p*, #doublable members) histogram [((1, 3), 1)] feasible 1 (238s) +FEASIBLE CONFIGURATIONS: 82 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev2.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev2.log new file mode 100644 index 0000000000..655afa8a63 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev2.log @@ -0,0 +1,10 @@ +level 1: candidates 3, subset-closed 3; testing minrank <= 15 (0s) +LEVEL 1: 3 good orbit reps (3s) + (flat sizes, |O|) histogram: [(((2,), 58), 1), (((3,), 57), 1), (((6,), 54), 1)] +level 2: candidates 62, subset-closed 62; testing minrank <= 14 (3s) +LEVEL 2: 2 good orbit reps (155s) + (flat sizes, |O|) histogram: [(((2, 2), 56), 1), (((6, 6), 48), 1)] +level 3: candidates 37, subset-closed 0; testing minrank <= 13 (156s) +LEVEL 3: 0 good orbit reps (156s) + (flat sizes, |O|) histogram: [] +EMPTY at level 3 => at most 2 such flats/totals for c=16, dims 2 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3.log new file mode 100644 index 0000000000..f044e77536 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3.log @@ -0,0 +1,31 @@ +level 1: candidates 3, subset-closed 3; testing minrank <= 15 (0s) +LEVEL 1: 3 good orbit reps (4s) + (flat sizes, |O|) histogram: [(((4,), 56), 1), (((6,), 54), 1), (((15,), 45), 1)] +level 2: candidates 178, subset-closed 178; testing minrank <= 14 (5s) + 100/178 good 16 (148s) +LEVEL 2: 28 good orbit reps (243s) + (flat sizes, |O|) histogram: [(((4, 4), 52), 5), (((4, 4), 53), 1), (((4, 4), 54), 1), (((4, 6), 50), 2), (((4, 6), 51), 3), (((4, 15), 42), 2), (((4, 15), 44), 1), (((6, 6), 48), 3), (((6, 6), 49), 2), (((6, 6), 50), 1), (((6, 6), 51), 1), (((6, 15), 40), 3), (((15, 15), 32), 1), (((15, 15), 33), 1), (((15, 15), 36), 1)] +level 3: candidates 13029, subset-closed 159; testing minrank <= 13 (259s) + 100/159 good 78 (268s) +LEVEL 3: 120 good orbit reps (275s) + (flat sizes, |O|) histogram: [(((4, 4, 4), 48), 1), (((4, 4, 4), 49), 2), (((4, 4, 4), 50), 1), (((4, 4, 6), 48), 1), (((4, 4, 15), 39), 3), (((4, 4, 15), 40), 2), (((4, 4, 15), 41), 1), (((4, 6, 6), 44), 2), (((4, 6, 6), 46), 4), (((4, 6, 6), 48), 3), (((4, 6, 6), 49), 1), (((4, 6, 15), 37), 6), (((4, 6, 15), 38), 3), (((4, 15, 15), 30), 3), (((4, 15, 15), 31), 1), (((4, 15, 15), 32), 2), (((4, 15, 15), 34), 1), (((6, 6, 6), 42), 4), (((6, 6, 6), 43), 4), (((6, 6, 6), 44), 7), (((6, 6, 6), 46), 2), (((6, 6, 6), 48), 1), (((6, 6, 6), 49), 1), (((6, 6, 15), 35), 18), (((6, 6, 15), 36), 11), (((6, 6, 15), 37), 3), (((6, 6, 15), 38), 2), (((6, 15, 15), 28), 10), (((6, 15, 15), 29), 10), (((6, 15, 15), 32), 2), (((15, 15, 15), 21), 1), (((15, 15, 15), 22), 1), (((15, 15, 15), 23), 2), (((15, 15, 15), 24), 2), (((15, 15, 15), 26), 1), (((15, 15, 15), 32), 1)] +level 4: candidates 114809, subset-closed 375; testing minrank <= 12 (379s) + 100/375 good 99 (380s) + 200/375 good 196 (381s) + 300/375 good 294 (382s) +LEVEL 4: 368 good orbit reps (383s) +level 5: candidates 379250, subset-closed 500; testing minrank <= 11 (813s) + 100/500 good 100 (813s) + 200/500 good 200 (814s) + 300/500 good 299 (814s) + 400/500 good 399 (814s) + 500/500 good 498 (815s) +LEVEL 5: 498 good orbit reps (815s) +level 6: candidates 573713, subset-closed 466; testing minrank <= 10 (2794s) + 100/466 good 99 (2795s) + 200/466 good 199 (2795s) + 300/466 good 299 (2795s) + 400/466 good 399 (2795s) +LEVEL 6: 464 good orbit reps (2795s) +/opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown + warnings.warn('resource_tracker: There appear to be %d ' diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3_hi.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3_hi.log new file mode 100644 index 0000000000..01e0fcf0a1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/lev3_hi.log @@ -0,0 +1,21 @@ +level 7: subset-closed candidates 315; testing minrank <= 9 (16s) +LEVEL 7: 315 good orbit reps (17s); |O| hist [(0, 8), (2, 5), (3, 5), (4, 15), (5, 15), (6, 21), (7, 48), (8, 56), (9, 44), (10, 28), (11, 36), (12, 22), (13, 4), (14, 1), (15, 3), (16, 2), (18, 2)] +level 8: subset-closed candidates 206; testing minrank <= 8 (34s) +LEVEL 8: 206 good orbit reps (34s); |O| hist [(0, 2), (1, 2), (2, 4), (3, 4), (4, 16), (5, 33), (6, 49), (7, 42), (8, 23), (9, 12), (10, 11), (11, 6), (12, 1), (16, 1)] +level 9: subset-closed candidates 124; testing minrank <= 7 (50s) +LEVEL 9: 124 good orbit reps (50s); |O| hist [(0, 2), (1, 4), (2, 1), (4, 28), (5, 43), (6, 27), (7, 9), (8, 5), (9, 3), (10, 1), (11, 1)] +level 10: subset-closed candidates 74; testing minrank <= 6 (65s) +LEVEL 10: 74 good orbit reps (65s); |O| hist [(0, 3), (1, 2), (4, 34), (5, 25), (6, 4), (7, 1), (8, 4), (10, 1)] +level 11: subset-closed candidates 32; testing minrank <= 5 (79s) +LEVEL 11: 32 good orbit reps (79s); |O| hist [(4, 23), (5, 7), (6, 1), (8, 1)] +level 12: subset-closed candidates 24; testing minrank <= 4 (92s) +LEVEL 12: 24 good orbit reps (93s); |O| hist [(4, 20), (5, 3), (8, 1)] +level 13: subset-closed candidates 10; testing minrank <= 3 (106s) +LEVEL 13: 10 good orbit reps (106s); |O| hist [(4, 9), (5, 1)] +level 14: subset-closed candidates 6; testing minrank <= 2 (119s) +LEVEL 14: 6 good orbit reps (119s); |O| hist [(4, 6)] +level 15: subset-closed candidates 1; testing minrank <= 1 (132s) +LEVEL 15: 1 good orbit reps (132s); |O| hist [(4, 1)] +level 16: subset-closed candidates 1; testing minrank <= 0 (145s) +LEVEL 16: 0 good orbit reps (145s); |O| hist [] +EMPTY at level 16 => at most 15 such flats/totals for c=16, dims 3 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/levels_c16.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/levels_c16.log new file mode 100644 index 0000000000..56e7eba137 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/levels_c16.log @@ -0,0 +1,15 @@ +level 1 reps 3 +level 2: candidates 178, after subset closure 178; SAT tests at rank <= 14 + level 2 progress 50/178 good 9 (52s) + level 2 progress 100/178 good 16 (94s) + level 2 progress 150/178 good 24 (144s) + level 2 progress 178/178 good 28 (174s) +LEVEL 2: 28 good orbit reps; |O| histogram [(32, 1), (33, 1), (36, 1), (40, 3), (42, 2), (44, 1), (48, 3), (49, 2), (50, 3), (51, 4), (52, 5), (53, 1), (54, 1)] +level 3: candidates 13029, after subset closure 159; SAT tests at rank <= 13 + level 3 progress 50/159 good 41 (312s) + level 3 progress 100/159 good 79 (316s) + level 3 progress 150/159 good 114 (319s) + level 3 progress 159/159 good 120 (322s) +LEVEL 3: 120 good orbit reps; |O| histogram [(21, 1), (22, 1), (23, 2), (24, 2), (26, 1), (28, 10), (29, 10), (30, 3), (31, 1), (32, 5), (34, 1), (35, 18), (36, 11), (37, 9), (38, 5), (39, 3), (40, 2), (41, 1), (42, 4), (43, 4), (44, 9), (46, 6), (48, 6), (49, 4), (50, 1)] +/opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown + warnings.warn('resource_tracker: There appear to be %d ' diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/pairs_c14.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/pairs_c14.log new file mode 100644 index 0000000000..dc09597a8b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/pairs_c14.log @@ -0,0 +1,179 @@ +pair ( 0, 1) |H1|=15 |H2|=15 |O|=36 orbit 90 c=14 cadical153 SAT 0.1s [1/178] +pair ( 0, 2) |H1|=15 |H2|=15 |O|=36 orbit 360 c=14 cadical153 UNSAT 2.8s [2/178] +pair ( 0, 11) |H1|=15 |H2|=15 |O|=32 orbit 360 c=14 cadical153 SAT 0.1s [3/178] +pair ( 0, 15) |H1|=15 |H2|= 4 |O|=43 orbit 2880 c=14 cadical153 UNSAT 18.4s [4/178] +pair ( 0, 19) |H1|=15 |H2|= 6 |O|=41 orbit 2880 c=14 cadical153 UNSAT 5.0s [5/178] +pair ( 0, 23) |H1|=15 |H2|= 4 |O|=43 orbit 2880 c=14 cadical153 UNSAT 3.7s [6/178] +pair ( 0, 63) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 UNSAT 17.5s [7/178] +pair ( 0, 64) |H1|=15 |H2|= 6 |O|=40 orbit 5760 c=14 cadical153 UNSAT 6.3s [8/178] +pair ( 0, 127) |H1|=15 |H2|=15 |O|=33 orbit 960 c=14 cadical153 SAT 0.5s [9/178] +pair ( 0, 129) |H1|=15 |H2|= 6 |O|=42 orbit 1920 c=14 cadical153 UNSAT 11.1s [10/178] +pair ( 0, 131) |H1|=15 |H2|= 4 |O|=44 orbit 960 c=14 cadical153 SAT 2.4s [11/178] +pair ( 0, 132) |H1|=15 |H2|= 6 |O|=42 orbit 1920 c=14 cadical153 UNSAT 7.3s [12/178] +pair ( 0, 134) |H1|=15 |H2|= 6 |O|=42 orbit 1920 c=14 cadical153 UNSAT 6.0s [13/178] +pair ( 0, 136) |H1|=15 |H2|= 4 |O|=44 orbit 1920 c=14 cadical153 UNSAT 6.4s [14/178] +pair ( 0, 172) |H1|=15 |H2|= 6 |O|=40 orbit 5760 c=14 cadical153 SAT 1.0s [15/178] +pair ( 0, 173) |H1|=15 |H2|= 6 |O|=40 orbit 5760 c=14 cadical153 SAT 1.4s [16/178] +pair ( 0, 176) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 SAT 1.8s [17/178] +pair ( 0, 208) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 UNSAT 9.2s [18/178] +pair ( 0, 212) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 UNSAT 7.5s [19/178] +pair ( 0, 224) |H1|=15 |H2|= 6 |O|=40 orbit 2880 c=14 cadical153 SAT 0.1s [20/178] +pair ( 0, 228) |H1|=15 |H2|= 4 |O|=42 orbit 2880 c=14 cadical153 SAT 4.6s [21/178] +pair ( 0, 232) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 UNSAT 4.5s [22/178] +pair ( 0, 233) |H1|=15 |H2|= 4 |O|=42 orbit 5760 c=14 cadical153 UNSAT 5.3s [23/178] +pair ( 0, 952) |H1|=15 |H2|= 4 |O|=41 orbit 11520 c=14 cadical153 UNSAT 10.6s [24/178] +pair ( 15, 16) |H1|= 4 |H2|= 4 |O|=54 orbit 1440 c=14 cadical153 SAT 2.0s [25/178] +pair ( 15, 17) |H1|= 4 |H2|= 4 |O|=54 orbit 2880 c=14 cadical153 UNSAT 12.4s [26/178] +pair ( 15, 19) |H1|= 4 |H2|= 6 |O|=52 orbit 2880 c=14 cadical153 UNSAT 9.2s [27/178] +pair ( 15, 20) |H1|= 4 |H2|= 6 |O|=52 orbit 2880 c=14 cadical153 UNSAT 6.2s [28/178] +pair ( 15, 21) |H1|= 4 |H2|= 6 |O|=52 orbit 2880 c=14 cadical153 UNSAT 8.6s [29/178] +pair ( 15, 22) |H1|= 4 |H2|= 6 |O|=52 orbit 2880 c=14 cadical153 UNSAT 6.4s [30/178] +pair ( 15, 23) |H1|= 4 |H2|= 4 |O|=54 orbit 1440 c=14 cadical153 UNSAT 10.7s [31/178] +pair ( 15, 24) |H1|= 4 |H2|= 4 |O|=54 orbit 1440 c=14 cadical153 UNSAT 38.9s [32/178] +pair ( 15, 25) |H1|= 4 |H2|= 4 |O|=54 orbit 1440 c=14 cadical153 UNSAT 9.0s [33/178] +pair ( 15, 26) |H1|= 4 |H2|= 4 |O|=54 orbit 1440 c=14 cadical153 UNSAT 9.5s [34/178] +pair ( 15, 29) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 15.7s [35/178] +pair ( 15, 31) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 21.9s [36/178] +pair ( 15, 33) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.9s [37/178] +pair ( 15, 35) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 8.9s [38/178] +pair ( 15, 37) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 17.1s [39/178] +pair ( 15, 43) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.1s [40/178] +pair ( 15, 44) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 13.5s [41/178] +pair ( 15, 45) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 15.9s [42/178] +pair ( 15, 46) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 11.1s [43/178] +pair ( 15, 47) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.1s [44/178] +pair ( 15, 48) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 13.1s [45/178] +pair ( 15, 49) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 5.7s [46/178] +pair ( 15, 50) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 10.9s [47/178] +pair ( 15, 129) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 6.4s [48/178] +pair ( 15, 131) |H1|= 4 |H2|= 4 |O|=52 orbit 2880 c=14 cadical153 SAT 0.9s [49/178] +pair ( 15, 132) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 9.6s [50/178] +pair ( 15, 134) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 26.8s [51/178] +pair ( 15, 136) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 14.1s [52/178] +pair ( 15, 140) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 7.1s [53/178] +pair ( 15, 141) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 4.6s [54/178] +pair ( 15, 142) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 15.7s [55/178] +pair ( 15, 143) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.7s [56/178] +pair ( 15, 144) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 8.9s [57/178] +pair ( 15, 145) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 4.5s [58/178] +pair ( 15, 146) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 10.2s [59/178] +pair ( 15, 147) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 SAT 2.6s [60/178] +pair ( 15, 148) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 14.0s [61/178] +pair ( 15, 153) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 8.1s [62/178] +pair ( 15, 154) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 8.2s [63/178] +pair ( 15, 155) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 10.3s [64/178] +pair ( 15, 156) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 18.3s [65/178] +pair ( 15, 157) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 7.1s [66/178] +pair ( 15, 158) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 10.7s [67/178] +pair ( 15, 159) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 7.8s [68/178] +pair ( 15, 164) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 7.1s [69/178] +pair ( 15, 165) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 6.9s [70/178] +pair ( 15, 166) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 5.9s [71/178] +pair ( 15, 167) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 3.7s [72/178] +pair ( 15, 168) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 17.8s [73/178] +pair ( 15, 169) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.3s [74/178] +pair ( 15, 170) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 6.1s [75/178] +pair ( 15, 200) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 10.7s [76/178] +pair ( 15, 201) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 13.6s [77/178] +pair ( 15, 202) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 9.3s [78/178] +pair ( 15, 203) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 12.6s [79/178] +pair ( 15, 204) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 SAT 2.0s [80/178] +pair ( 15, 205) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 8.3s [81/178] +pair ( 15, 206) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 13.5s [82/178] +pair ( 15, 208) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 6.6s [83/178] +pair ( 15, 209) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 13.0s [84/178] +pair ( 15, 210) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 20.1s [85/178] +pair ( 15, 211) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 7.1s [86/178] +pair ( 15, 212) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 18.5s [87/178] +pair ( 15, 213) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 9.3s [88/178] +pair ( 15, 214) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 9.8s [89/178] +pair ( 15, 215) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 10.1s [90/178] +pair ( 15, 232) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.3s [91/178] +pair ( 15, 234) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 10.8s [92/178] +pair ( 15, 235) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.3s [93/178] +pair ( 15, 238) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 7.6s [94/178] +pair ( 15, 239) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 8.0s [95/178] +pair ( 15, 242) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.6s [96/178] +pair ( 15, 393) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 SAT 0.4s [97/178] +pair ( 15, 395) |H1|= 4 |H2|= 4 |O|=52 orbit 2880 c=14 cadical153 SAT 0.1s [98/178] +pair ( 15, 396) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 7.3s [99/178] +pair ( 15, 398) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 21.2s [100/178] +pair ( 15, 405) |H1|= 4 |H2|= 6 |O|=53 orbit 1920 c=14 cadical153 UNSAT 7.9s [101/178] +pair ( 15, 406) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 7.6s [102/178] +pair ( 15, 407) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 4.6s [103/178] +pair ( 15, 408) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.8s [104/178] +pair ( 15, 409) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 3.4s [105/178] +pair ( 15, 410) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 9.6s [106/178] +pair ( 15, 411) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 8.4s [107/178] +pair ( 15, 412) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 6.6s [108/178] +pair ( 15, 442) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 4.6s [109/178] +pair ( 15, 443) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 7.6s [110/178] +pair ( 15, 444) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 6.5s [111/178] +pair ( 15, 445) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 8.1s [112/178] +pair ( 15, 450) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.2s [113/178] +pair ( 15, 451) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 9.8s [114/178] +pair ( 15, 454) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 8.4s [115/178] +pair ( 15, 455) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 7.5s [116/178] +pair ( 15, 462) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 7.6s [117/178] +pair ( 15, 463) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 6.3s [118/178] +pair ( 15, 474) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 5.3s [119/178] +pair ( 15, 476) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 4.2s [120/178] +pair ( 15, 477) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 6.4s [121/178] +pair ( 15, 478) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 10.2s [122/178] +pair ( 15, 481) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 8.3s [123/178] +pair ( 15, 484) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 9.1s [124/178] +pair ( 15, 488) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 12.1s [125/178] +pair ( 15,1116) |H1|= 4 |H2|= 4 |O|=52 orbit 2880 c=14 cadical153 SAT 0.6s [126/178] +pair ( 15,1117) |H1|= 4 |H2|= 4 |O|=52 orbit 2880 c=14 cadical153 SAT 1.5s [127/178] +pair ( 15,1120) |H1|= 4 |H2|= 6 |O|=50 orbit 5760 c=14 cadical153 SAT 1.1s [128/178] +pair ( 15,1121) |H1|= 4 |H2|= 6 |O|=50 orbit 5760 c=14 cadical153 SAT 0.1s [129/178] +pair ( 15,1128) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 5.4s [130/178] +pair ( 15,1129) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 5.6s [131/178] +pair ( 15,1132) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 5.0s [132/178] +pair ( 15,1133) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 4.7s [133/178] +pair ( 15,1148) |H1|= 4 |H2|= 6 |O|=53 orbit 1920 c=14 cadical153 UNSAT 8.2s [134/178] +pair ( 15,1149) |H1|= 4 |H2|= 6 |O|=53 orbit 1920 c=14 cadical153 UNSAT 3.5s [135/178] +pair ( 15,1150) |H1|= 4 |H2|= 4 |O|=55 orbit 960 c=14 cadical153 UNSAT 16.4s [136/178] +pair ( 15,1154) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 SAT 1.3s [137/178] +pair ( 15,1155) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 SAT 1.5s [138/178] +pair ( 15,1160) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 3.6s [139/178] +pair ( 15,1161) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 15.9s [140/178] +pair ( 15,1163) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 15.5s [141/178] +pair ( 15,1166) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 10.2s [142/178] +pair ( 15,1167) |H1|= 4 |H2|= 6 |O|=51 orbit 5760 c=14 cadical153 UNSAT 5.8s [143/178] +pair ( 15,1169) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 11.0s [144/178] +pair ( 15,1180) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 6.3s [145/178] +pair ( 15,1182) |H1|= 4 |H2|= 4 |O|=53 orbit 5760 c=14 cadical153 UNSAT 7.0s [146/178] +pair ( 15,1186) |H1|= 4 |H2|= 4 |O|=53 orbit 2880 c=14 cadical153 UNSAT 10.5s [147/178] +pair ( 15,1208) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 7.8s [148/178] +pair ( 15,1209) |H1|= 4 |H2|= 6 |O|=50 orbit 11520 c=14 cadical153 UNSAT 17.0s [149/178] +pair ( 15,1276) |H1|= 4 |H2|= 4 |O|=52 orbit 11520 c=14 cadical153 UNSAT 6.9s [150/178] +pair ( 15,1281) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 10.1s [151/178] +pair ( 15,1284) |H1|= 4 |H2|= 4 |O|=52 orbit 5760 c=14 cadical153 UNSAT 6.7s [152/178] +pair ( 19, 20) |H1|= 6 |H2|= 6 |O|=50 orbit 720 c=14 cadical153 SAT 0.8s [153/178] +pair ( 19, 21) |H1|= 6 |H2|= 6 |O|=50 orbit 1440 c=14 cadical153 UNSAT 5.3s [154/178] +pair ( 19, 33) |H1|= 6 |H2|= 6 |O|=51 orbit 960 c=14 cadical153 SAT 1.0s [155/178] +pair ( 19, 35) |H1|= 6 |H2|= 6 |O|=49 orbit 2880 c=14 cadical153 UNSAT 10.1s [156/178] +pair ( 19, 43) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 UNSAT 4.4s [157/178] +pair ( 19, 44) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 UNSAT 13.4s [158/178] +pair ( 19, 64) |H1|= 6 |H2|= 6 |O|=51 orbit 960 c=14 cadical153 UNSAT 10.2s [159/178] +pair ( 19, 65) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 SAT 0.7s [160/178] +pair ( 19, 66) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 UNSAT 6.2s [161/178] +pair ( 19, 67) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 UNSAT 6.4s [162/178] +pair ( 19, 72) |H1|= 6 |H2|= 6 |O|=51 orbit 1920 c=14 cadical153 UNSAT 3.1s [163/178] +pair ( 19, 74) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 SAT 0.9s [164/178] +pair ( 19, 75) |H1|= 6 |H2|= 6 |O|=49 orbit 2880 c=14 cadical153 UNSAT 3.7s [165/178] +pair ( 19, 79) |H1|= 6 |H2|= 6 |O|=49 orbit 5760 c=14 cadical153 UNSAT 2.8s [166/178] +pair ( 19, 83) |H1|= 6 |H2|= 6 |O|=51 orbit 960 c=14 cadical153 UNSAT 9.8s [167/178] +pair ( 19, 88) |H1|= 6 |H2|= 6 |O|=49 orbit 2880 c=14 cadical153 UNSAT 4.9s [168/178] +pair ( 19, 129) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 7.6s [169/178] +pair ( 19, 134) |H1|= 6 |H2|= 6 |O|=48 orbit 11520 c=14 cadical153 UNSAT 7.0s [170/178] +pair ( 19, 149) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 6.8s [171/178] +pair ( 19, 150) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 10.2s [172/178] +pair ( 19, 154) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 16.3s [173/178] +pair ( 19, 160) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 5.6s [174/178] +pair ( 19, 161) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 UNSAT 6.7s [175/178] +pair ( 19, 172) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 SAT 1.2s [176/178] +pair ( 19, 414) |H1|= 6 |H2|= 6 |O|=48 orbit 5760 c=14 cadical153 SAT 1.1s [177/178] +pair ( 19, 922) |H1|= 6 |H2|= 6 |O|=48 orbit 1440 c=14 cadical153 SAT 0.4s [178/178] +SAT pairs: 28 of 178 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/price16.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/price16.log new file mode 100644 index 0000000000..5c7ee073bc --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/price16.log @@ -0,0 +1 @@ +c=16 decoder valid; betas [4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4] sum 64 cheap totals 0 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/prove_c16.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/prove_c16.log new file mode 100644 index 0000000000..6b91d6ad44 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/prove_c16.log @@ -0,0 +1,22 @@ +level sizes: {1: 3, 2: 28, 3: 120, 4: 368, 5: 498, 6: 464, 7: 315, 8: 206, 9: 124, 10: 74, 11: 32, 12: 24, 13: 10, 14: 6, 15: 1, 16: 0} +(A) k= 1: 3 of 3 reps have an a-capable member +(A) k= 2: 19 of 28 reps have an a-capable member +(A) k= 3: 18 of 120 reps have an a-capable member +(A) k= 4: 3 of 368 reps have an a-capable member +(A) k= 5: 0 of 498 reps have an a-capable member +(A) k= 6: 0 of 464 reps have an a-capable member +(A) k= 7: 0 of 315 reps have an a-capable member +(A) k= 8: 0 of 206 reps have an a-capable member +(A) k= 9: 0 of 124 reps have an a-capable member +(A) k=10: 0 of 74 reps have an a-capable member +(A) k=11: 0 of 32 reps have an a-capable member +(A) k=12: 0 of 24 reps have an a-capable member +(A) k=13: 0 of 10 reps have an a-capable member +(A) k=14: 0 of 6 reps have an a-capable member +(A) k=15: 0 of 1 reps have an a-capable member +(B) k=11: max p* = 5 (need >= 11 for cost <= 42) +(B) k=12: max p* = 4 (need >= 10 for cost <= 42) +(B) k=13: max p* = 3 (need >= 9 for cost <= 42) +(B) k=14: max p* = 2 (need >= 8 for cost <= 42) +(B) k=15: max p* = 1 (need >= 7 for cost <= 42) +THEOREM: no Gaussian-E8 decoder with exactly 16 totals has sum beta_j <= 42 (780s) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/refine.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/refine.log new file mode 100644 index 0000000000..57ef47b57a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/refine.log @@ -0,0 +1,83 @@ +k= 9 T=(0, 1, 6, 127, 139, 152, 162, 372, 382) doublable 9 a-capable 0 -> excluded (2s) +k= 9 T=(0, 1, 6, 127, 139, 152, 162, 372, 492) doublable 9 a-capable 0 -> excluded (4s) +k= 9 T=(0, 1, 6, 127, 139, 152, 162, 492, 504) doublable 9 a-capable 0 -> excluded (6s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 372, 382) doublable 9 a-capable 0 -> excluded (7s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 372, 492) doublable 9 a-capable 0 -> excluded (9s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 372, 504) doublable 9 a-capable 0 -> excluded (11s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 382, 492) doublable 9 a-capable 0 -> excluded (12s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 382, 504) doublable 9 a-capable 0 -> excluded (14s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 392, 524) doublable 9 a-capable 0 -> excluded (15s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 402, 512) doublable 9 a-capable 0 -> excluded (17s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 402, 524) doublable 9 a-capable 0 -> excluded (18s) +k= 9 T=(0, 1, 6, 127, 139, 152, 272, 492, 504) doublable 9 a-capable 0 -> excluded (20s) +k= 9 T=(0, 1, 6, 127, 139, 272, 284, 372, 382) doublable 9 a-capable 0 -> excluded (22s) +k= 9 T=(0, 1, 6, 127, 139, 372, 392, 504, 512) doublable 9 a-capable 0 -> excluded (23s) +k= 9 T=(0, 1, 6, 127, 139, 372, 402, 504, 524) doublable 9 a-capable 0 -> excluded (25s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 249, 372, 382) doublable 10 a-capable 0 -> excluded (27s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 249, 372, 492) doublable 10 a-capable 0 -> excluded (29s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 249, 372, 504) doublable 10 a-capable 0 -> excluded (31s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 249, 382, 504) doublable 10 a-capable 0 -> excluded (33s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 372, 382, 492) doublable 10 a-capable 0 -> excluded (34s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 372, 382, 1115) doublable 6 a-capable 0 -> excluded (36s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 372, 492, 504) doublable 10 a-capable 0 -> excluded (38s) +k=10 T=(0, 1, 6, 127, 139, 152, 162, 372, 492, 1115) doublable 6 a-capable 0 -> excluded (40s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 372, 504) doublable 10 a-capable 0 -> excluded (42s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 382, 492) doublable 10 a-capable 0 -> excluded (44s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 382, 504) doublable 10 a-capable 0 -> excluded (45s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 402, 512) doublable 10 a-capable 0 -> excluded (47s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 402, 524) doublable 10 a-capable 0 -> excluded (49s) +k=10 T=(0, 1, 6, 127, 139, 152, 249, 272, 512, 524) doublable 10 a-capable 0 -> excluded (51s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 284, 382, 492) doublable 10 a-capable 0 -> excluded (53s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 372, 382, 1115) doublable 6 a-capable 0 -> excluded (55s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 372, 492, 1115) doublable 6 a-capable 0 -> excluded (57s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 372, 504, 1115) doublable 6 a-capable 0 -> excluded (58s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 392, 524, 1115) doublable 6 a-capable 0 -> excluded (60s) +k=10 T=(0, 1, 6, 127, 139, 152, 272, 402, 512, 1115) doublable 6 a-capable 0 -> excluded (62s) +k=10 T=(0, 1, 6, 127, 139, 372, 392, 504, 512, 1115) doublable 6 a-capable 0 -> excluded (64s) +k=10 T=(0, 1, 6, 127, 139, 372, 402, 504, 524, 1115) doublable 6 a-capable 0 -> excluded (66s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 259, 372, 382) doublable 7 a-capable 0 -> excluded (68s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 259, 372, 492) doublable 7 a-capable 0 -> excluded (71s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 259, 372, 504) doublable 7 a-capable 0 -> excluded (73s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 259, 492, 504) doublable 7 a-capable 0 -> excluded (75s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 272, 372, 382) doublable 11 a-capable 0 -> excluded (78s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 272, 372, 504) doublable 11 a-capable 0 -> excluded (80s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 272, 392, 402) doublable 11 a-capable 0 -> excluded (82s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 272, 392, 512) doublable 11 a-capable 0 -> excluded (85s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 249, 372, 382, 504) doublable 11 a-capable 0 -> excluded (87s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 372, 382, 492, 504) doublable 7 a-capable 0 -> excluded (89s) +k=11 T=(0, 1, 6, 127, 139, 152, 162, 372, 382, 492, 1115) doublable 7 a-capable 0 -> excluded (91s) +k=11 T=(0, 1, 6, 127, 139, 152, 249, 259, 272, 372, 504) doublable 7 a-capable 0 -> excluded (94s) +k=11 T=(0, 1, 6, 127, 139, 152, 249, 259, 272, 402, 524) doublable 7 a-capable 0 -> excluded (96s) +k=11 T=(0, 1, 6, 127, 139, 152, 249, 272, 284, 382, 492) doublable 11 a-capable 0 -> excluded (99s) +k=12 T=(0, 1, 6, 127, 139, 152, 162, 249, 259, 372, 492, 1115) doublable 4 a-capable 0 -> excluded (102s) +k=12 T=(0, 1, 6, 127, 139, 152, 162, 249, 272, 392, 512, 524) doublable 12 a-capable 0 -> excluded (106s) +k=12 T=(0, 1, 6, 127, 139, 152, 249, 259, 272, 372, 492, 1115) doublable 4 a-capable 0 -> excluded (109s) +k=12 T=(0, 1, 6, 127, 139, 152, 249, 272, 284, 372, 382, 492) doublable 12 a-capable 0 -> excluded (112s) +k=12 T=(0, 1, 6, 127, 139, 162, 249, 259, 272, 372, 492, 1115) doublable 4 a-capable 0 -> excluded (114s) +k=12 T=(0, 1, 6, 127, 139, 162, 249, 272, 284, 372, 382, 492) doublable 12 a-capable 0 -> excluded (117s) +k=12 T=(0, 1, 6, 127, 139, 162, 249, 272, 284, 372, 382, 504) doublable 12 a-capable 0 -> excluded (120s) +k=12 T=(0, 1, 6, 127, 139, 249, 392, 402, 492, 504, 512, 524) doublable 8 a-capable 0 -> excluded (123s) +k=12 T=(0, 1, 6, 127, 139, 249, 392, 402, 492, 504, 512, 1115) doublable 8 a-capable 0 -> excluded (127s) +k=12 T=(0, 1, 6, 127, 139, 259, 382, 402, 492, 504, 512, 524) doublable 12 a-capable 0 -> excluded (131s) +k=12 T=(0, 1, 6, 127, 139, 259, 392, 402, 492, 504, 512, 524) doublable 8 a-capable 0 -> excluded (135s) +k=12 T=(0, 1, 6, 127, 139, 372, 392, 402, 492, 504, 512, 524) doublable 8 a-capable 0 -> excluded (138s) +k=12 T=(0, 1, 6, 127, 139, 372, 392, 402, 492, 504, 512, 1115) doublable 8 a-capable 0 -> excluded (141s) +k=12 T=(0, 1, 6, 127, 139, 372, 392, 402, 492, 512, 524, 1115) doublable 4 a-capable 0 -> excluded (143s) +k=12 T=(0, 1, 6, 127, 139, 382, 392, 402, 492, 504, 512, 524) doublable 8 a-capable 0 -> excluded (146s) +k=12 T=(0, 1, 6, 127, 139, 382, 392, 402, 492, 504, 512, 1115) doublable 8 a-capable 0 -> excluded (148s) +k=12 T=(0, 1, 6, 127, 139, 382, 392, 402, 492, 512, 524, 1115) doublable 4 a-capable 0 -> excluded (151s) +k=13 T=(0, 1, 6, 127, 139, 249, 259, 392, 402, 492, 504, 512, 524) doublable 5 a-capable 0 -> excluded (154s) +k=13 T=(0, 1, 6, 127, 139, 249, 372, 392, 402, 492, 504, 512, 524) doublable 9 a-capable 0 -> excluded (158s) +k=13 T=(0, 1, 6, 127, 139, 249, 372, 392, 402, 492, 504, 512, 1115) doublable 9 a-capable 0 -> excluded (161s) +k=13 T=(0, 1, 6, 127, 139, 249, 382, 392, 402, 492, 504, 512, 524) doublable 9 a-capable 0 -> excluded (164s) +k=13 T=(0, 1, 6, 127, 139, 259, 372, 392, 402, 492, 504, 512, 524) doublable 9 a-capable 0 -> excluded (167s) +k=13 T=(0, 1, 6, 127, 139, 259, 382, 392, 402, 492, 504, 512, 524) doublable 9 a-capable 0 -> excluded (170s) +k=13 T=(0, 1, 6, 127, 139, 372, 382, 392, 402, 492, 504, 512, 524) doublable 5 a-capable 0 -> excluded (173s) +k=14 T=(0, 1, 6, 127, 139, 249, 259, 372, 392, 402, 492, 504, 512, 524) doublable 6 a-capable 0 -> excluded (177s) +k=14 T=(0, 1, 6, 127, 139, 249, 259, 372, 392, 402, 492, 512, 524, 1115) doublable 2 a-capable 0 -> excluded (181s) +k=14 T=(0, 1, 6, 127, 139, 249, 259, 382, 392, 402, 492, 504, 512, 524) doublable 6 a-capable 0 -> excluded (184s) +k=14 T=(0, 1, 6, 127, 139, 249, 259, 382, 392, 402, 492, 512, 524, 1115) doublable 2 a-capable 0 -> excluded (188s) +k=14 T=(0, 1, 6, 127, 139, 249, 372, 382, 392, 402, 492, 504, 512, 524) doublable 6 a-capable 0 -> excluded (191s) +k=14 T=(0, 1, 6, 127, 139, 259, 372, 382, 392, 402, 492, 504, 512, 524) doublable 6 a-capable 0 -> excluded (195s) +k=15 T=(0, 1, 6, 127, 139, 249, 259, 372, 382, 392, 402, 492, 504, 512, 524) doublable 3 a-capable 0 -> excluded (199s) +SURVIVING CONFIGURATIONS: 0 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/verify_run.log b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/verify_run.log new file mode 100644 index 0000000000..43822c853c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/logs/verify_run.log @@ -0,0 +1,44 @@ +Gauss v 60 w 4 nonorth degree [np.int64(44)] +LEVEL 1: 3 good orbit reps (3s) +LEVEL 2: 28 good orbit reps (131s) +LEVEL 3: 120 good orbit reps (157s) +LEVEL 4: 368 good orbit reps +LEVEL 5: 498 good orbit reps +LEVEL 6: 464 good orbit reps +LEVEL 7: 315 good orbit reps +LEVEL 8: 206 good orbit reps +LEVEL 9: 124 good orbit reps +LEVEL 10: 74 good orbit reps +LEVEL 11: 32 good orbit reps +LEVEL 12: 24 good orbit reps +LEVEL 13: 10 good orbit reps +LEVEL 14: 6 good orbit reps +LEVEL 15: 1 good orbit reps +LEVEL 16: 0 good orbit reps +EMPTY at level 16 => at most 15 such flats/totals for c=16, dims 3 +level sizes: {1: 3, 2: 28, 3: 120, 4: 368, 5: 498, 6: 464, 7: 315, 8: 206, 9: 124, 10: 74, 11: 32, 12: 24, 13: 10, 14: 6, 15: 1, 16: 0} +(A) k= 1: 3 of 3 reps have an a-capable member +(A) k= 2: 19 of 28 reps have an a-capable member +(A) k= 3: 18 of 120 reps have an a-capable member +(A) k= 4: 3 of 368 reps have an a-capable member +(A) k= 5: 0 of 498 reps have an a-capable member +(A) k= 6: 0 of 464 reps have an a-capable member +(A) k= 7: 0 of 315 reps have an a-capable member +(A) k= 8: 0 of 206 reps have an a-capable member +(A) k= 9: 0 of 124 reps have an a-capable member +(A) k=10: 0 of 74 reps have an a-capable member +(A) k=11: 0 of 32 reps have an a-capable member +(A) k=12: 0 of 24 reps have an a-capable member +(A) k=13: 0 of 10 reps have an a-capable member +(A) k=14: 0 of 6 reps have an a-capable member +(A) k=15: 0 of 1 reps have an a-capable member +(B) k=11: max p* = 5 (need >= 11 for cost <= 42) +(B) k=12: max p* = 4 (need >= 10 for cost <= 42) +(B) k=13: max p* = 3 (need >= 9 for cost <= 42) +(B) k=14: max p* = 2 (need >= 8 for cost <= 42) +(B) k=15: max p* = 1 (need >= 7 for cost <= 42) +THEOREM: no Gaussian-E8 decoder with exactly 16 totals has sum beta_j <= 42 +c=16 decoder valid; betas [4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4] sum 64 cheap totals 0 +LEVEL 1: 0 good orbit reps +EMPTY at level 1 => at most 0 such flats/totals for c=15, dims 3 +work dir: /private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/s7/vf diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/acap_sanity.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/acap_sanity.py new file mode 100644 index 0000000000..e3a03f25df --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/acap_sanity.py @@ -0,0 +1,12 @@ +"""Sanity: how many level-k reps have a member that is 'a-capable' (some 2-flat Z inside it with every 3-flat through Z a valid substitute).""" +import sys; sys.path.insert(0, sys.path[0]) +from refine import good, inside, through, ok +for k in range(1, 16): + n = 0; mem = 0 + for T in good[k]: + T = list(T); c = 0 + for i in range(k): + subst = lambda h: sorted(T[:i] + [h] + T[i + 1:]) + if any(all(h == T[i] or (h not in T and ok(subst(h))) for h in through[z]) for z in inside[T[i]]): c += 1 + n += c > 0; mem += c + print('k=%2d reps %4d with a-capable member %4d (total a-capable members %d)' % (k, len(good[k]), n, mem), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/crosscheck.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/crosscheck.py new file mode 100644 index 0000000000..c0261eb1c1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/crosscheck.py @@ -0,0 +1,37 @@ +"""Re-derive every UNSAT pruning of the 3-flat level search (levels 2..maxlev) and re-solve it with another solver. +Only UNSAT verdicts can remove configurations, so only they need cross-checking (a wrong SAT would only keep more). +argv: c dims pkl maxlev solver nproc (first three as levels3.py, which reads them at import)""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from multiprocessing import Pool +import levels3 +from levels3 import gen, init, compl, sat_job +from sub_minrank import run +from levels2 import S + + +def job(args): + X, O, cc, solver = args + r, t, _ = run(S, O, cc, solver) + return X, r, t + + +if __name__ == '__main__': + good = levels3.good; maxlev = int(sys.argv[4]); solver = sys.argv[5]; npr = int(sys.argv[6]) + c = levels3.c + bad = 0; tot = 0; t0 = time.time() + for m in range(2, maxlev + 1): + with Pool(npr, initializer=init, initargs=(good[m - 1],)) as pool: + cand = set() + if m == 2: # all pair orbits (gen() prefilters by pair compatibility, so use the orbit list directly) + cand = {levels3.canon(levels3.P, [a, b]) for a, b, _ in pickle.load(open('pair_orbits.pkl', 'rb'))['reps']} + assert len(cand) == 178 + else: + for out in pool.imap_unordered(gen, sorted(good[m - 1]), chunksize=4): cand.update(out) + assert good[m] <= cand + uns = sorted(cand - good[m]) + print('level %d: %d candidates, %d good, %d UNSAT prunings to recheck with %s' % (m, len(cand), len(good[m]), len(uns), solver), flush=True) + for X, r, t in pool.imap_unordered(job, [(X, compl(X), c - m, solver) for X in uns]): + tot += 1; bad += bool(r) + print(' %s |O|=%d minrank<=%d: %s %.1fs' % (X, len(compl(X)), c - m, 'SAT (MISMATCH)' if r else 'UNSAT', t), flush=True) + print('CROSSCHECK %s: %d UNSAT prunings rechecked, %d mismatches (%.0fs)' % (solver, tot, bad, time.time() - t0), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/direct.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/direct.py new file mode 100644 index 0000000000..8046ec4104 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/direct.py @@ -0,0 +1,60 @@ +"""S7 step 3: full decoder SAT on all 60 lines with c totals, where for each flat Z_i of a configuration (i = 0..k-1), +total i has a whole side inside Z_i: sigma_i -> W[T,i] = 0 for T outside Z_i; not sigma_i -> U[T,i] = 0 for T outside Z_i. +Totals k..c-1 are free. sigma_0 = 1 by the global transpose symmetry (W,U) -> (U,W) (the pattern is symmetric). +No GL gauge (columns are distinguished). Free columns: lexicographic order is NOT imposed (keeps the encoding simple). +argv: c pkl level nproc [solver] [limit]: runs every orbit rep of good[level] in pkl.""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from multiprocessing import Pool +from pysat.formula import IDPool +from pysat.solvers import Solver +from lines import views + +S = views()['Gauss'] + + +def encode(Zs, c): + pool = IDPool(); cl = [] + W = [[pool.id(('w', T, j)) for j in range(c)] for T in range(60)] + U = [[pool.id(('u', T, j)) for j in range(c)] for T in range(60)] + sig = [pool.id(('s', i)) for i in range(len(Zs))] + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + for T in range(60): + for S_ in range(60): + if T == S_ or S.NO[T, S_]: + acc = AND(W[T][0], U[S_][0]) + for j in range(1, c): acc = XOR(acc, AND(W[T][j], U[S_][j])) + cl.append([acc] if T == S_ else [-acc]) + for i, Z in enumerate(Zs): + Zs_ = set(Z) + for T in range(60): + if T not in Zs_: + cl.append([-sig[i], -W[T][i]]); cl.append([sig[i], -U[T][i]]) + if Zs: cl.append([sig[0]]) + return cl, pool + + +def job(args): + X, Zs, c, solver = args + cl, pool = encode(Zs, c) + t = time.time() + with Solver(name=solver, bootstrap_with=cl) as s: + r = s.solve() + return X, r, time.time() - t + + +if __name__ == '__main__': + c = int(sys.argv[1]); d = pickle.load(open(sys.argv[2], 'rb')); lev = int(sys.argv[3]); npr = int(sys.argv[4]) + solver = sys.argv[5] if len(sys.argv) > 5 else 'cadical153' + lim = int(sys.argv[6]) if len(sys.argv) > 6 else None + fl = d['flats']; reps = sorted(d['good'][lev])[:lim] + jobs = [(X, [fl[f][1] for f in X], c, solver) for X in reps] + nsat = 0; t0 = time.time() + with Pool(npr) as p: + for k, (X, r, t) in enumerate(p.imap_unordered(job, jobs)): + nsat += bool(r) + print('%s sizes %s %s %.1fs [%d/%d] (%.0fs)' % (X, [len(fl[f][1]) for f in X], 'SAT' if r else 'UNSAT', t, k + 1, len(jobs), time.time() - t0), flush=True) + print('level %d c=%d %s: %d SAT of %d' % (lev, c, solver, nsat, len(jobs)), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/dumbbell.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/dumbbell.py new file mode 100644 index 0000000000..c5601f33f8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/dumbbell.py @@ -0,0 +1,112 @@ +"""S7 step 2c: cost-aware refinement of the 3-flat level search ("dumbbells"). + +Per total j (X_j = span R_j, Y_j = span A_j, Hermitian definite form on K^4, beta_j = dim X - dim(X cap Y^perp)): + full (4,4) beta = 4 + 'a' a side of dim <= 2 beta >= 0 (at most 2 such totals: 2-flat level search, lev_c16_d2.pkl) + 'p' (3,3), beta = 2 both X_j, Y_j are 3-flats containing a side, X_j != Y_j -> a DUMBBELL + 'q' otherwise cheap beta >= 3 (one 3-flat side; also (3,3) with beta = 3) +cost <= 42 with 16 totals: 4f + 2p + 3q <= 42, f = 16 - a - p - q <=> 3a + p >= 22 - k, k = a + p + q. +Every transversal (one 3-flat per cheap total: for p-totals either X_j or Y_j; for a-totals any 3-flat containing the +small side) is a valid level-k set (all hyperplanes distinct since one 3-flat cannot host two totals). +So a <= 42 decoder needs a level-k rep T (k = 9..15) and a choice of p members with partners Y (beta(t, Y) = 2) such that +all 2^p transversals are level-k sets, with 3*min(2, k-p) + p >= 22 - k. +This script computes, for every level-k rep, the maximum number p* of simultaneously doubled members.""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from levels2 import load_flats, canon_many, canon +from lines import views + +S = views()['Gauss'] +J = S.ops[1] + + +def hyperplane_bases(fl): + out = [] + for d, f in fl: + rows = np.array([v for T in f for v in (S.reps[T], J @ S.reps[T])], float) + u, s, vt = np.linalg.svd(rows) + r = int((s > 1e-9 * s[0]).sum()); assert r == 2 * d + out.append(vt[:r]) + return np.array(out) # NF x 6 x 8 orthonormal real bases + + +def beta_matrix(Bs): + """beta(X,Y) = real rank(Bx By^T)/2 for all pairs, by SVD; true rank-2 (real rank 4) pairs have sigma_5 ~ 1e-15.""" + NF = len(Bs); beta = np.zeros((NF, NF), np.int8) + for i in range(NF): + M = np.einsum('ik,njk->nij', Bs[i], Bs) + s = np.linalg.svd(M, compute_uv=False) + r = (s > 1e-7).sum(1) + amb = ((s > 1e-11) & (s < 1e-3)).any(1) + assert not amb.any(), 'ambiguous rank' + assert (r % 2 == 0).all() + beta[i] = r // 2 + return beta + + +def main(): + G, fl, P = load_flats('3') + NF = len(fl) + good = pickle.load(open(sys.argv[1], 'rb'))['good'] + t0 = time.time() + beta = beta_matrix(hyperplane_bases(fl)) + print('beta histogram over ordered pairs', np.bincount(beta.flatten()).tolist(), '(%.0fs)' % (time.time() - t0), flush=True) + comp = np.zeros((NF, NF), bool) + for (a, b) in good[2]: + comp[P[:, a], P[:, b]] = True; comp[P[:, b], P[:, a]] = True + feasible = [] + for k in range(9, 16): + need_best = None + stats = [] + for T in sorted(good[k]): + T = list(T) + partners = {} + for idx, t in enumerate(T): + rest = T[:idx] + T[idx + 1:] + ys = [y for y in range(NF) if y not in T and beta[t, y] == 2 and comp[y, rest].all()] + ok = [] + if ys: + cs = canon_many(P, rest, ys) + ok = [y for y, cf in zip(ys, cs) if cf in good[k]] + if ok: partners[idx] = ok + # max number of members doubled simultaneously, all transversals valid (backtracking) + best = [0] + mem = sorted(partners) + + def valid_all(choice): + # choice: dict idx -> partner; check every transversal + items = list(choice.items()) + for mask in range(1 << len(items)): + X = list(T) + for b, (idx, y) in enumerate(items): + if mask >> b & 1: X[idx] = y + if canon(P, X) not in good[k]: return False + return True + + def rec(pos, choice): + best[0] = max(best[0], len(choice)) + if len(choice) + (len(mem) - pos) <= best[0]: return + for q in range(pos, len(mem)): + idx = mem[q] + for y in partners[idx]: + if y in choice.values(): continue + # newly added transversals: those using y + choice[idx] = y + if valid_all(choice): rec(q + 1, choice) + del choice[idx] + rec(0, {}) + p = best[0] + # feasibility: exists p' <= p, a' <= min(2, k - p') with 3a' + p' >= 22 - k + feas = any(3 * min(2, k - pp) + pp >= 22 - k for pp in range(p + 1)) + stats.append((p, len(partners))) + if feas: feasible.append((k, tuple(T), p)) + from collections import Counter + print('k=%d reps %d (p*, #doublable members) histogram %s feasible %d (%.0fs)' % ( + k, len(good[k]), sorted(Counter(stats).items()), sum(1 for f in feasible if f[0] == k), time.time() - t0), flush=True) + pickle.dump({'feasible': feasible, 'beta': beta}, open('dumbbell_out.pkl', 'wb')) + print('FEASIBLE CONFIGURATIONS:', len(feasible), flush=True) + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/flats.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/flats.py new file mode 100644 index 0000000000..b064c81896 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/flats.py @@ -0,0 +1,26 @@ +"""All flats (line sets of K-subspaces spanned by configuration lines) of a view, with dims. Exact.""" +import sys, pickle; sys.path.insert(0,sys.path[0]) +from lines import * +def flats(S): + v=S.v; out={} + def close(idx): + d=S.arank(idx); return frozenset(T for T in range(v) if S.arank(list(idx)+[T])==d), d + level=[ (frozenset([T]),1) for T in range(v)] + for f,d in level: out[f]=d + frontier=list(out.items()) + while frontier: + new={} + for f,d in frontier: + if d>=S.w: continue + for T in range(v): + if T in f or any(T in g and f distinct). Level m+1 candidates: extend each level-m rep by a flat that is +pair-compatible with all members; keep if all m-subsets are level-m sets; then SAT-test the full union at c-(m+1). +argv: c maxlevel nproc. Writes levels_c{c}.pkl and prints per-level counts.""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from multiprocessing import Pool +from lines import views +from sub_minrank import run +from pairs import load + +S = views()['Gauss'] + + +def sat_job(args): + key, O, cc = args + if cc < 0: + return key, False, 0.0 + if len(O) == 0: + return key, True, 0.0 + if cc == 0: + return key, False, 0.0 # nonempty O needs rank >= 1 (B_TT = 1) + r, t, _ = run(S, O, cc, 'cadical153') + return key, r, t + + +if __name__ == '__main__': + c = int(sys.argv[1]); maxlev = int(sys.argv[2]); npr = int(sys.argv[3]) + G, hyp = load() + hid = {h: k for k, h in enumerate(hyp)} + P = np.array([[hid[tuple(sorted(g[i] for i in h))] for h in hyp] for g in G], dtype=np.int32) # |G| x 1500 + NF = len(hyp) + lines_of = [set(h) for h in hyp] + + def canon(fs): + A = np.sort(P[:, list(fs)], axis=1) + k = np.lexsort(A.T[::-1])[0] + return tuple(int(x) for x in A[k]) + + def compl(fs): + U = set().union(*(lines_of[f] for f in fs)) + return [T for T in range(60) if T not in U] + + # level 1: the 3 orbit reps, minrank 15 on complement (S6) -> valid iff 15 <= c-1 + reps1 = sorted({canon([f]) for f in range(NF)}) + good = {1: set(reps1) if 15 <= c - 1 else set()} + print('level 1 reps', len(good[1]), flush=True) + # level 2: from pairs.py orbit results (c=14 test) only valid for c=16; recompute generically via SAT + union_cache = {} + t0 = time.time() + with Pool(npr) as pool: + for m in range(2, maxlev + 1): + cand = set() + for R in good[m - 1]: + for f in range(NF): + if f in R: continue + cand.add(canon(list(R) + [f])) + # subset closure + cand2 = [] + for X in cand: + if all(canon(list(Y)) in good[m - 1] for Y in itertools.combinations(X, m - 1)): + cand2.append(X) + jobs = [] + for X in cand2: + O = compl(X) + jobs.append((X, O, c - m)) + print('level %d: candidates %d, after subset closure %d; SAT tests at rank <= %d' % (m, len(cand), len(jobs), c - m), flush=True) + good[m] = set(); n = 0 + for X, r, t in pool.imap_unordered(sat_job, jobs, chunksize=1): + n += 1 + if r: good[m].add(X) + if n % 50 == 0 or n == len(jobs): + print(' level %d progress %d/%d good %d (%.0fs)' % (m, n, len(jobs), len(good[m]), time.time() - t0), flush=True) + pickle.dump(good, open('levels_c%d.pkl' % c, 'wb')) + sizes = {} + for X in good[m]: + k = len(compl(X)); sizes[k] = sizes.get(k, 0) + 1 + print('LEVEL %d: %d good orbit reps; |O| histogram %s' % (m, len(good[m]), sorted(sizes.items())), flush=True) + if not good[m]: + print('EMPTY at level %d => at most %d cheap totals for c=%d' % (m, m - 1, c), flush=True) + break diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels2.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels2.py new file mode 100644 index 0000000000..5efc500057 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels2.py @@ -0,0 +1,110 @@ +"""S7 step 2 (fast, general): level search over sets Z = {Z_1..Z_m} of distinct flats (dims chosen by argv), up to the +line group (order 11520), such that EVERY subset J of Z satisfies + + minrank(decoder pattern on O_J) <= c - |J|, O_J = lines outside U_{i in J} Z_i. + +Why: if a decoder B = W U^T has c totals and Z_i contains a whole side (R_j or A_j) of total j_i (distinct totals), then +those |J| columns of W (or U) vanish on O_J, so B[O_J, O_J] has rank <= c - |J| (and obeys the pattern on O_J). +minrank is monotone under taking sub-line-sets, and Z = 3-flats is a relaxation of any proper flat. + +argv: c dims(e.g. '3' or '2' or '23') maxlevel nproc [tag] +Writes lev_.pkl = {m: set of canonical sets (tuples of flat ids into the 'flats' list)} and prints counts.""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from multiprocessing import Pool +from lines import views +from sub_minrank import run + +S = views()['Gauss'] +B = 12 # bits per flat id (< 4096) + + +def load_flats(dims): + G = pickle.load(open('orbits.pkl', 'rb'))['G'] + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + fl = sorted((d, tuple(sorted(f))) for f, d in F.items() if str(d) in dims) + fid = {f: k for k, (d, f) in enumerate(fl)} + P = np.array([[fid[tuple(sorted(g[i] for i in f))] for d, f in fl] for g in G], dtype=np.int64) + return G, fl, P + + +def keys(A): + """A: (..., |G|, m) sorted ids -> lexicographic key pair (k1, k2), 5 ids per key.""" + m = A.shape[-1] + k1 = np.zeros(A.shape[:-1], np.int64); k2 = np.zeros(A.shape[:-1], np.int64) + for i in range(m): + if i < 5: k1 = (k1 << B) | A[..., i] + else: k2 = (k2 << B) | A[..., i] + return k1, k2 + + +def canon_many(P, R, fs): + """canonical forms of R u {f} for each f in fs (vectorised). Canonical = lexicographically least sorted image.""" + out = [] + PR = P[:, list(R)] # |G| x m + for s in range(0, len(fs), 64): + ch = fs[s:s + 64] + A = np.concatenate([np.broadcast_to(PR, (len(ch),) + PR.shape), P[:, ch].T[:, :, None]], axis=2) + A = np.sort(A, axis=2) + k1, k2 = keys(A) + m1 = k1.min(1, keepdims=True) + k2m = np.where(k1 == m1, k2, np.iinfo(np.int64).max) + g = k2m.argmin(1) + for t in range(len(ch)): + out.append(tuple(int(x) for x in A[t, g[t]])) + return out + + +def canon(P, X): + X = list(X) + return canon_many(P, X[:-1], [X[-1]])[0] + + +def sat_job(args): + key, O, cc = args + if len(O) == 0: return key, True, 0.0 + if cc <= 0: return key, False, 0.0 # nonempty O has rank >= 1 + r, t, _ = run(S, O, cc, 'cadical153') + return key, r, t + + +def main(): + c = int(sys.argv[1]); dims = sys.argv[2]; maxlev = int(sys.argv[3]); npr = int(sys.argv[4]) + tag = sys.argv[5] if len(sys.argv) > 5 else 'c%d_d%s' % (c, dims) + G, fl, P = load_flats(dims) + NF = len(fl); assert NF < (1 << B) + lines_of = [set(f) for d, f in fl] + compl = lambda X: [T for T in range(60) if T not in set().union(*(lines_of[f] for f in X))] + t0 = time.time() + good = {0: {()}} + with Pool(npr) as pool: + for m in range(1, maxlev + 1): + cand = set() + for R in good[m - 1]: + fs = [f for f in range(NF) if f not in R] + cand.update(canon_many(P, R, fs) if R else [canon(P, [f]) for f in fs]) + cand2 = [X for X in cand + if m == 1 or all(canon(P, Y) in good[m - 1] for Y in itertools.combinations(X, m - 1))] + jobs = [(X, compl(X), c - m) for X in cand2] + print('level %d: candidates %d, subset-closed %d; testing minrank <= %d (%.0fs)' % ( + m, len(cand), len(jobs), c - m, time.time() - t0), flush=True) + good[m] = set(); n = 0 + for X, r, t in pool.imap_unordered(sat_job, jobs, chunksize=1): + n += 1 + if r: good[m].add(X) + if n % 100 == 0: print(' %d/%d good %d (%.0fs)' % (n, len(jobs), len(good[m]), time.time() - t0), flush=True) + pickle.dump({'good': good, 'flats': fl}, open('lev_%s.pkl' % tag, 'wb')) + hist = {} + for X in good[m]: + key = (tuple(sorted(len(fl[f][1]) for f in X)), len(compl(X))) + hist[key] = hist.get(key, 0) + 1 + print('LEVEL %d: %d good orbit reps (%.0fs)' % (m, len(good[m]), time.time() - t0), flush=True) + if len(hist) < 40: print(' (flat sizes, |O|) histogram:', sorted(hist.items()), flush=True) + if not good[m]: + print('EMPTY at level %d => at most %d such flats/totals for c=%d, dims %s' % (m, m - 1, c, dims), flush=True) + break + + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels3.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels3.py new file mode 100644 index 0000000000..a966313c40 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/levels3.py @@ -0,0 +1,62 @@ +"""Continuation of levels2.py (same definitions) with two exact speedups: + (1) extension flats f must be pair-compatible with every member (pair {Z_i, f} is a 2-subset, so it must be a level-2 set); + (2) canonicalisation and subset-closure checks run in the worker pool. +argv: c dims infile startlevel maxlevel nproc tag. Loads good[] from infile (written by levels2.py) and continues.""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from multiprocessing import Pool +from levels2 import load_flats, canon_many, canon, sat_job + +c = int(sys.argv[1]); dims = sys.argv[2] +G, fl, P = load_flats(dims) +NF = len(fl) +lines_of = [set(f) for d, f in fl] +good = pickle.load(open(sys.argv[3], 'rb'))['good'] +# pair compatibility matrix from level-2 orbit reps +comp = np.zeros((NF, NF), bool) +for (a, b) in good[2]: + comp[P[:, a], P[:, b]] = True; comp[P[:, b], P[:, a]] = True + + +def compl(X): + U = set().union(*(lines_of[f] for f in X)) + return [T for T in range(60) if T not in U] + + +def gen(R): + """canonical (m+1)-sets extending rep R that pass the full (m)-subset closure test, given global PREV.""" + fs = [f for f in range(NF) if f not in R and comp[f, list(R)].all()] + out = [] + for X in set(canon_many(P, R, fs)) if fs else []: + if all(canon(P, Y) in PREV for Y in itertools.combinations(X, len(X) - 1)): + out.append(X) + return out + + +def init(prev): + global PREV + PREV = prev + + +if __name__ == '__main__': + start = int(sys.argv[4]); maxlev = int(sys.argv[5]); npr = int(sys.argv[6]); tag = sys.argv[7] + t0 = time.time() + for m in range(start, maxlev + 1): + with Pool(npr, initializer=init, initargs=(good[m - 1],)) as pool: + cand = set() + for out in pool.imap_unordered(gen, sorted(good[m - 1]), chunksize=4): + cand.update(out) + jobs = [(X, compl(X), c - m) for X in sorted(cand)] + print('level %d: subset-closed candidates %d; testing minrank <= %d (%.0fs)' % (m, len(jobs), c - m, time.time() - t0), flush=True) + good[m] = set() + for X, r, t in pool.imap_unordered(sat_job, jobs, chunksize=1): + if r: good[m].add(X) + pickle.dump({'good': good, 'flats': fl}, open('lev_%s.pkl' % tag, 'wb')) + hist = {} + for X in good[m]: + k = len(compl(X)); hist[k] = hist.get(k, 0) + 1 + print('LEVEL %d: %d good orbit reps (%.0fs); |O| hist %s' % (m, len(good[m]), time.time() - t0, sorted(hist.items())), flush=True) + if not good[m]: + print('EMPTY at level %d => at most %d such flats/totals for c=%d, dims %s' % (m, m - 1, c, dims), flush=True) + break diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/lines.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/lines.py new file mode 100644 index 0000000000..fc4eac011d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/lines.py @@ -0,0 +1,166 @@ +"""Self-contained exact construction of the E8 root lines over Q(i), Q(omega) and the Hurwitz quaternions. + +Real model: the 240 E8 roots in doubled standard coordinates (norm 8; in {0,+-4,+-8}). + * Gaussian view : J = left multiplication by i on R^8 = H^2 (coords 1,i,j,k per block); J preserves E8. + Hermitian Gram hG(x,y) = ( + i )/4 in Z[i] (diag 2). + * Eisenstein view: rho = fixed-point-free order-3 isometry of E8 (product of Coxeter elements of 4 orthogonal A2's). + Sesquilinear Gram hE(x,y) = ( - w )/4 in Z[w] (w^2+w+1=0; diag 2+w, a unit at 2). + * Hurwitz view : left multiplication by i, j, k; quaternionic lines of real dim 4. +Elements of Z[i] / Z[w] are stored as integer pairs (a, b) = a + b*i / a + b*w. +Everything is exact integer arithmetic; asserts check every structural claim used downstream. +""" +import itertools +from fractions import Fraction +import numpy as np + + +def e8_roots(): + R = [] + for i, j in itertools.combinations(range(8), 2): + for si in (2, -2): + for sj in (2, -2): + r = [0] * 8; r[i], r[j] = si, sj; R.append(r) + for s in itertools.product((1, -1), repeat=8): + if s.count(-1) % 2 == 0: + R.append(list(s)) + R = np.array(R, dtype=np.int64) + assert len(R) == 240 and set((R * R).sum(1)) == {8} + return R + + +def rank_q(rows): + """exact rank over Q of an integer matrix (fraction-free Gaussian elimination).""" + M = [list(map(int, r)) for r in rows] + if not M: + return 0 + n, m = len(M), len(M[0]); r = 0 + for c in range(m): + p = next((i for i in range(r, n) if M[i][c] != 0), None) + if p is None: + continue + M[r], M[p] = M[p], M[r] + for i in range(r + 1, n): + if M[i][c]: + f, g = M[i][c], M[r][c] + M[i] = [g * a - f * b for a, b in zip(M[i], M[r])] + r += 1 + if r == n: + break + return r + + +def left_quat(): + Li = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 0, -1], [0, 0, 1, 0]]) + Lj = np.array([[0, 0, -1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, -1, 0, 0]]) + Lk = Li @ Lj + blk = lambda M: np.kron(np.eye(2, dtype=np.int64), M).astype(np.int64) + return blk(Li), blk(Lj), blk(Lk) + + +def a2_rotation(roots): + """rho, an order-3 fixed-point-free isometry preserving E8 (returned as an exact integer matrix 4*rho).""" + chosen = [] + for a in roots: + if len(chosen) == 4: + break + if any(np.dot(a, x) != 0 for p in chosen for x in p): + continue + for b in roots: + if np.dot(a, b) == -4 and all(np.dot(a, x) == 0 and np.dot(b, x) == 0 for p in chosen for x in p): + chosen.append((a, b)); break + assert len(chosen) == 4 + g = np.eye(8, dtype=object) + for a, b in chosen: + sa = np.eye(8, dtype=object) * 8 - 2 * np.outer(a, a).astype(object) + sb = np.eye(8, dtype=object) * 8 - 2 * np.outer(b, b).astype(object) + g = g.dot(sa).dot(sb) + D = 8 ** 8 // 4 + assert all(x % D == 0 for x in g.flatten()) + g = (g // D).astype(np.int64) + I = np.eye(8, dtype=np.int64) + assert (g @ g @ g == 64 * I).all() and (g @ g + 4 * g + 16 * I == 0).all() + return g + + +def lines_over(roots, ops): + """group roots into A-lines, A spanned by the integer matrices ops (identity first; ops may be scaled by 4). + Returns representatives (first root met of each line, in root order).""" + S = {tuple(r) for r in roots} + reps, seen = [], set() + for r in roots: + if tuple(r) in seen: + continue + span = [B @ r for B in ops] + assert rank_q(span) == len(ops) + reps.append(r) + members = [s for s in roots if rank_q(span + [s]) == len(ops)] + for s in members: + seen.add(tuple(s)) + return np.array(reps) + + +class View: + """name, reps (v x 8), ops (real matrices spanning A, scaled so that ops[i] @ root is a root times scale[i]).""" + def __init__(self, name, reps, ops, d): + self.name, self.reps, self.ops, self.d = name, reps, ops, d + self.v = len(reps) + NO = np.zeros((self.v, self.v), bool) + for B in ops: + NO |= (reps @ B @ reps.T) != 0 + np.fill_diagonal(NO, False) + self.NO = NO # A-non-orthogonal pairs (the decoder pattern forces B_TS = 0 there) + self.w = self.arank(range(self.v)) + + def arank(self, idx): + rows = [B @ self.reps[i] for i in idx for B in self.ops] + r = rank_q(rows) if rows else 0 + assert r % self.d == 0 + return r // self.d + + def pair_rank(self, R, A): + """A-rank of the pairing span(R) x span(A) = real rank of the real Gram block / dim A (Theorem C cost).""" + BR = [B @ self.reps[i] for i in R for B in self.ops] + BA = [B @ self.reps[i] for i in A for B in self.ops] + if not BR or not BA: + return 0 + M = [[int(x @ y) for y in BA] for x in BR] + r = rank_q(M); assert r % self.d == 0 + return r // self.d + + +def views(): + roots = e8_roots(); I8 = np.eye(8, dtype=np.int64) + Li, Lj, Lk = left_quat() + S = {tuple(r) for r in roots} + for M in (Li, Lj, Lk): + assert all(tuple(M @ r) in S for r in roots) + g = a2_rotation(roots) # 4*rho + assert all(tuple(g @ r // 4) in S and ((g @ r) % 4 == 0).all() for r in roots) + rho = g # 4*rho (exact integer matrix); ranks are scale-free + out = {} + out['Gauss'] = View('Gauss', lines_over(roots, [I8, Li]), [I8, Li], 2) + out['Eisen'] = View('Eisen', lines_over(roots, [I8, rho]), [I8, rho], 2) + out['Hurwitz'] = View('Hurwitz', lines_over(roots, [I8, Li, Lj, Lk]), [I8, Li, Lj, Lk], 4) + out['_J'], out['_rho'] = Li, rho + return out + + +def gauss_gram(V, J): + """hG(r_T, r_S) = ( + i)/4 as integer pairs (a, b).""" + A = V @ V.T; Bm = V @ J @ V.T + assert (A % 4 == 0).all() and (Bm % 4 == 0).all() + return A // 4, Bm // 4 + + +def eisen_gram(V, rho4): + """hE(r_T, r_S) = ( - w)/4 as integer pairs (a, b) = a + b w (rho4 = 4*rho).""" + A = V @ V.T; Bm = V @ rho4 @ V.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + + +if __name__ == '__main__': + W = views() + for nm in ('Gauss', 'Eisen', 'Hurwitz'): + S = W[nm] + print(nm, 'v', S.v, 'w', S.w, 'nonorth degree', sorted(set(S.NO.sum(1)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/orbits.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/orbits.py new file mode 100644 index 0000000000..11e4c31837 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/orbits.py @@ -0,0 +1,53 @@ +"""Permutation group of the 60 Gaussian-E8 lines generated by the 60 complex reflections s_r s_{Jr}; orbits on the 1500 3-flats.""" +import sys, pickle +sys.path.insert(0, sys.path[0]) +import numpy as np +from lines import views, e8_roots + +def line_index(S): + J = S.ops[1]; idx = {} + for T, r in enumerate(S.reps): + for v in (r, -r, J @ r, -(J @ r)): idx[tuple(v)] = T + return idx + +def generators(S): + J = S.ops[1]; idx = line_index(S); gens = [] + for r in S.reps: + out = [] + for T in range(S.v): + x = S.reps[T] + for a in (r, J @ r): # real reflections in r and Jr (norm 8): x -> x - 2/8 a + x = x - (2 * int(x @ a) // 8) * a + out.append(idx[tuple(x)]) + assert sorted(out) == list(range(S.v)) + gens.append(tuple(out)) + return gens + +def closure(gens, n): + G = {tuple(range(n))}; frontier = list(G) + while frontier: + new = [] + for p in frontier: + for g in gens: + q = tuple(g[p[i]] for i in range(n)) + if q not in G: G.add(q); new.append(q) + frontier = new + return G + +if __name__ == '__main__': + S = views()['Gauss'] + gens = generators(S) + # check: generators preserve non-orthogonality + for g in gens: + P = np.array(g); assert (S.NO[np.ix_(P, P)] == S.NO).all() + G = closure(gens, S.v); print('group order on lines', len(G)) + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = sorted(sorted(f) for f, d in F.items() if d == 3) + hs = {frozenset(h) for h in hyp}; seen = set(); reps = [] + for h in hyp: + if frozenset(h) in seen: continue + orb = {frozenset(g[i] for i in h) for g in G} + assert orb <= hs + seen |= orb; reps.append((h, len(orb))) + print('orbits on 3-flats:', [(len(h), n) for h, n in reps], 'total', sum(n for _, n in reps)) + pickle.dump({'G': sorted(G), 'hyp_reps': reps}, open('orbits.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/pairs.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/pairs.py new file mode 100644 index 0000000000..17ef07df54 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/pairs.py @@ -0,0 +1,61 @@ +"""S7 step 2a: orbits of unordered pairs of distinct 3-flats under the line group (order 11520), and the minrank test +minrank(pattern on O) <= c - 2 with O = lines outside H1 u H2 (c = 16 -> test 14). + +Reduction (argued): a total j is 'cheap' (beta_j < 4) iff R_j or A_j lies in a 3-flat H_j. For any m cheap totals, +on O = complement of H_1 u ... u H_m those m columns of W (or U) vanish, so B[O,O] = W[O] U[O]^T has rank <= c - m. +Output: pairs.pkl {'pair_reps': [(H1, H2, orbit_size, minrank_le_14: bool)]}.""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from multiprocessing import Pool +from lines import views +from sub_minrank import run + +S = views()['Gauss'] + + +def load(): + G = pickle.load(open('orbits.pkl', 'rb'))['G'] + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = sorted(tuple(sorted(f)) for f, d in F.items() if d == 3) + assert len(hyp) == 1500 + return G, hyp + + +def act(g, h): + return tuple(sorted(g[i] for i in h)) + + +def pair_orbits(G, hyp): + """canonical orbit reps of unordered pairs {H1,H2}, by brute force: canonical form = min over G of sorted image pair.""" + hid = {h: k for k, h in enumerate(hyp)} + # permutation of flats induced by each group element + P = [[hid[act(g, h)] for h in hyp] for g in G] + seen = set(); reps = [] + n = len(hyp) + for a in range(n): + for b in range(a + 1, n): + if (a, b) in seen: continue + orb = set() + for p in P: + x, y = p[a], p[b] + orb.add((x, y) if x < y else (y, x)) + seen |= orb + reps.append((a, b, len(orb))) + assert sum(r[2] for r in reps) == n * (n - 1) // 2 + return reps, P + + +def job(args): + H, c, solver = args + Hs = set(H) + O = [T for T in range(60) if T not in Hs] + r, t, k = run(S, O, c, solver) + return H, c, solver, r, t, len(O) + + +if __name__ == '__main__': + G, hyp = load() + t = time.time() + reps, P = pair_orbits(G, hyp) + print('pair orbits', len(reps), 'time %.0fs' % (time.time() - t), flush=True) + pickle.dump({'reps': reps}, open('pair_orbits.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/price16.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/price16.py new file mode 100644 index 0000000000..d518940485 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/price16.py @@ -0,0 +1,18 @@ +"""Calibration: a SAT-found 16-total decoder (S6 clique-gauge encoding, no cost cap) is valid and costs >= 43 (here: its exact +Theorem-C cost, with the number of cheap totals).""" +import sys; sys.path.insert(0, sys.path[0]) +import numpy as np +from lines import views +from sub_minrank import encode +from pysat.solvers import Solver +S = views()['Gauss']; c = 16 +cl, pool, _ = encode(S, range(60), c) +with Solver(name='cadical153', bootstrap_with=cl) as s: + assert s.solve(); m = set(l for l in s.get_model() if l > 0) +W = np.array([[pool.id(('w', T, j)) in m for j in range(c)] for T in range(60)], int) +U = np.array([[pool.id(('u', T, j)) in m for j in range(c)] for T in range(60)], int) +B = W @ U.T % 2 +assert all(B[T, T] == 1 for T in range(60)) and not (B[S.NO] != 0).any(), 'invalid decoder' +beta = [S.pair_rank([T for T in range(60) if W[T, j]], [T for T in range(60) if U[T, j]]) for j in range(c)] +print('c=16 decoder valid; betas', beta, 'sum', sum(beta), 'cheap totals', sum(b < 4 for b in beta)) +assert sum(beta) >= 43 diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/prove_c16.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/prove_c16.py new file mode 100644 index 0000000000..7f19b82c36 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/prove_c16.py @@ -0,0 +1,88 @@ +"""S7 theorem: every Gaussian-E8 decoder with exactly 16 totals has Theorem-C cost >= 43. + +Inputs: lev_c16_d3_hi.pkl (3-flat level tables good[k], k = 1..16, from levels2.py / levels3.py; a level-k set is a set of +k distinct 3-flats every subset J of which has minrank(pattern on lines outside the union of J) <= 16 - |J|). + +Argument (see README): + k = #cheap totals (beta < 4). Each cheap total has a side in a proper flat, hence in a 3-flat; one 3-flat per cheap + total gives a level-k set (S6 lemma for distinctness), and any legal choice of 3-flats does. Types: + a: a side of dim <= 2 (inside a 2-flat Z): every 3-flat through Z is a legal choice; beta >= 0 + p: (3,3) with beta = 2: both X_j != Y_j legal (dumbbell); beta = 2 + q: rest (beta >= 3). + cost <= 42 <=> 3a + p >= 22 - k. + (A) a = 0 whenever k >= 5: no level-k rep (k >= 5) has an 'a-capable' member. [checked below] + (B) for k >= 11 the maximum number p* of jointly doubled members is < 22 - k. [checked below] + Arithmetic: k <= 4: 3a + p <= 3k < 22 - k. 5 <= k <= 10: a = 0, p <= k < 22 - k. k = 16: good[16] empty. +""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from levels2 import load_flats, canon, canon_many +from dumbbell import hyperplane_bases, beta_matrix + +G, fl3, P = load_flats('3') +_, fl2, _ = load_flats('2') +NF = len(fl3) +good = pickle.load(open(sys.argv[1] if len(sys.argv) > 1 else 'lev_c16_d3_hi.pkl', 'rb'))['good'] +set3 = [set(f) for d, f in fl3]; set2 = [set(f) for d, f in fl2] +through = [[h for h in range(NF) if set2[z] <= set3[h]] for z in range(len(fl2))] +inside = [[z for z in range(len(fl2)) if set2[z] <= set3[h]] for h in range(NF)] +cache = {} + + +def ok(X): + X = tuple(sorted(X)) + if len(set(X)) < len(X): return False + if X not in cache: cache[X] = canon(P, X) in good[len(X)] + return cache[X] + + +def a_capable(T, i): + return any(all(h == T[i] or (h not in T and ok(T[:i] + [h] + T[i + 1:])) for h in through[z]) for z in inside[T[i]]) + + +def pstar(T, beta, cap): + k = len(T) + part = {i: [y for y in range(NF) if y not in T and beta[T[i], y] == 2 and ok(T[:i] + [y] + T[i + 1:])] for i in range(k)} + mem = sorted(i for i in part if part[i]); best = [0] + + def valid(choice): + items = list(choice.items()) + for mask in range(1 << len(items)): + X = list(T) + for b, (i, y) in enumerate(items): + if mask >> b & 1: X[i] = y + if not ok(X): return False + return True + + def rec(pos, choice): + best[0] = max(best[0], len(choice)) + if best[0] >= cap or len(choice) + len(mem) - pos <= best[0]: return + for q in range(pos, len(mem)): + i = mem[q] + for y in part[i]: + if y in choice.values(): continue + choice[i] = y + if valid(choice): rec(q + 1, choice) + del choice[i] + rec(0, {}) + return best[0] + + +if __name__ == '__main__': + t0 = time.time() + assert len(good[1]) == 3 and not good[16] + print('level sizes:', {k: len(good[k]) for k in sorted(good) if k}, flush=True) + for k in range(1, 16): + n = sum(1 for T in good[k] if any(a_capable(list(T), i) for i in range(k))) + print('(A) k=%2d: %4d of %4d reps have an a-capable member' % (k, n, len(good[k])), flush=True) + if k >= 5: assert n == 0 + beta = beta_matrix(hyperplane_bases(fl3)) + for k in range(11, 16): + ps = [pstar(list(T), beta, 22 - k) for T in sorted(good[k])] + print('(B) k=%2d: max p* = %d (need >= %d for cost <= 42)' % (k, max(ps), 22 - k), flush=True) + assert max(ps) < 22 - k + for k in range(0, 17): + if k <= 4: assert 3 * k < 22 - k + elif k <= 10: assert k < 22 - k + print('THEOREM: no Gaussian-E8 decoder with exactly 16 totals has sum beta_j <= 42 (%.0fs)' % (time.time() - t0), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/refine.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/refine.py new file mode 100644 index 0000000000..d25baf408d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/refine.py @@ -0,0 +1,97 @@ +"""S7 step 2d: exact joint refinement of the configurations surviving dumbbell.py. + +For a level-k rep T (one 3-flat per cheap total) assign each member a type: + 'a' (a side of dim <= 2): choose a 2-flat Z inside T[i] containing that side; EVERY 3-flat H containing Z is a legal choice + for this total, so every transversal using any such H must be a level-k set. At most 2 'a' members, and if 2, the + pair (Z1, Z2) must be a level-2 set of the 2-flat search (lev_c16_d2.pkl). + 'p' (dumbbell): partner Y with beta(T[i], Y) = 2; both T[i] and Y are legal. + 'q': only T[i]. +Requirement from cost <= 42: 3*#a + #p >= 22 - k. ALL transversals of the joint choice must be level-k sets. +(A side of dim 1, a line l, is inside some 2-flat Z inside T[i], and every 3-flat through Z contains l, so 'a' with Z is +a relaxation of the line case.) Prints the configurations that survive (should be none for exclusion).""" +import sys, time, pickle, itertools +sys.path.insert(0, sys.path[0]) +import numpy as np +from levels2 import load_flats, canon + +G, fl3, P3 = load_flats('3') +_, fl2, P2 = load_flats('2') +NF = len(fl3) +good = pickle.load(open('lev_c16_d3_hi.pkl', 'rb'))['good'] +good2 = pickle.load(open('lev_c16_d2.pkl', 'rb'))['good'] +dd = pickle.load(open('dumbbell_out.pkl', 'rb')) +beta = dd['beta']; feasible = dd['feasible'] +set3 = [set(f) for d, f in fl3]; set2 = [set(f) for d, f in fl2] +through = [[h for h in range(NF) if set2[z] <= set3[h]] for z in range(len(fl2))] +inside = [[z for z in range(len(fl2)) if set2[z] <= set3[h]] for h in range(NF)] +cache = {} + + +def ok(X): + X = tuple(X) + if len(set(X)) < len(X): return False + if X not in cache: cache[X] = canon(P3, X) in good[len(X)] + return cache[X] + + +def joint_ok(T, opts): + """opts[i] = list of legal 3-flats for member i; every transversal must be a level-k set.""" + for X in itertools.product(*opts): + if not ok(sorted(X)): return False + return True + + +def analyse(k, T): + T = list(T) + need = 22 - k + subst = lambda i, h: sorted(T[:i] + [h] + T[i + 1:]) + part = {i: [y for y in range(NF) if y not in T and beta[T[i], y] == 2 and ok(subst(i, y))] for i in range(k)} + aopt = {i: [z for z in inside[T[i]] if all(h == T[i] or (h not in T and ok(subst(i, h))) for h in through[z])] for i in range(k)} + sols = [] + # choose A (<= 2 members with a 2-flat), then dumbbells among the rest with backtracking + for na in range(0, 3): + if 3 * na + (k - na) < need: continue + for A in itertools.combinations(range(k), na): + if any(not aopt[i] for i in A): continue + for Zs in itertools.product(*[aopt[i] for i in A]): + if na == 2: + if Zs[0] == Zs[1] or canon(P2, list(Zs)) not in good2[2]: continue + base = [[T[i]] for i in range(k)] + for i, z in zip(A, Zs): base[i] = through[z] + if not joint_ok(T, base): continue + rest = [i for i in range(k) if i not in A and part[i]] + pneed = need - 3 * na + if pneed <= 0: + sols.append((A, Zs, {})); continue + # backtracking over dumbbells + found = [] + def rec(pos, choice): + if found: return + if len(choice) >= pneed: + found.append(dict(choice)); return + if len(choice) + len(rest) - pos < pneed: return + for q in range(pos, len(rest)): + i = rest[q] + for y in part[i]: + if y in [v for v in choice.values()]: continue + opts = [list(o) for o in base] + for j, yy in list(choice.items()) + [(i, y)]: opts[j] = [T[j], yy] + if joint_ok(T, opts): + choice[i] = y; rec(q + 1, choice); del choice[i] + if found: return + rec(0, {}) + if found: sols.append((A, Zs, found[0])) + if sols: return sols, part, aopt + return sols, part, aopt + + +if __name__ == '__main__': + t0 = time.time(); surv = [] + for k, T, p in feasible: + sols, part, aopt = analyse(k, T) + na = sum(1 for i in aopt if aopt[i]) + print('k=%2d T=%s doublable %d a-capable %d -> %s (%.0fs)' % ( + k, T, sum(1 for i in part if part[i]), na, 'SURVIVES %s' % (sols[0],) if sols else 'excluded', time.time() - t0), flush=True) + if sols: surv.append((k, T, sols[0])) + pickle.dump(surv, open('refine_out.pkl', 'wb')) + print('SURVIVING CONFIGURATIONS:', len(surv), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/sub_minrank.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/sub_minrank.py new file mode 100644 index 0000000000..2f9383e0c2 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/sub_minrank.py @@ -0,0 +1,50 @@ +"""F2 minrank test of the Gaussian-E8 decoder pattern induced on a line subset O (asymmetric B = W U^T, clique gauge).""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from lines import views +from pysat.formula import IDPool +from pysat.solvers import Solver +import networkx as nx + +def encode(S, O, c, gauge=True): + O = list(O); pool = IDPool(); cl = [] + U = {T: [pool.id(('u', T, j)) for j in range(c)] for T in O} + W = {T: [pool.id(('w', T, j)) for j in range(c)] for T in O} + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + def xorlist(ls): + acc = ls[0] + for l in ls[1:]: acc = XOR(acc, l) + return acc + for T in O: + for S_ in O: + if T == S_ or S.NO[T, S_]: + b = xorlist([AND(W[T][j], U[S_][j]) for j in range(c)]) + cl.append([b] if T == S_ else [-b]) + K = [] + if gauge: + g = nx.Graph([(i, j) for i in O for j in O if i < j and S.NO[i, j]]); g.add_nodes_from(O) # S7: isolated lines too + K = sorted(max(nx.find_cliques(g), key=len))[:c] + for i, T in enumerate(K): + for j in range(c): + cl.append([U[T][j]] if j == i else [-U[T][j]]); cl.append([W[T][j]] if j == i else [-W[T][j]]) + return cl, pool, len(K) + +def run(S, O, c, solver='cadical153'): + cl, pool, k = encode(S, O, c) + t = time.time() + with Solver(name=solver, bootstrap_with=cl) as s: r = s.solve() + return r, time.time() - t, k + +if __name__ == '__main__': + S = views()['Gauss'] + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = [sorted(f) for f, d in F.items() if d == 3] + c = int(sys.argv[1]) if len(sys.argv) > 1 else 14 + print('full 60, c', c, run(S, range(60), c), flush=True) + for size in (15, 6, 4): + H = next(h for h in hyp if len(h) == size) + O = [T for T in range(60) if T not in H] + print('|H|', size, '|O|', len(O), 'c', c, run(S, O, c), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/test_pairs.py b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/test_pairs.py new file mode 100644 index 0000000000..ad5d796d4b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/scripts/test_pairs.py @@ -0,0 +1,19 @@ +"""Run minrank(O) <= c tests on complements of unions of pair-orbit reps. argv: c solver nproc.""" +import sys, time, pickle +sys.path.insert(0, sys.path[0]) +from multiprocessing import Pool +from pairs import load, job + +if __name__ == '__main__': + c = int(sys.argv[1]); solver = sys.argv[2]; npr = int(sys.argv[3]) + G, hyp = load() + reps = pickle.load(open('pair_orbits.pkl', 'rb'))['reps'] + jobs = [(tuple(sorted(set(hyp[a]) | set(hyp[b]))), c, solver) for a, b, _ in reps] + res = {} + with Pool(npr) as p: + for k, ((U, c_, s, r, t, nO), (a, b, n)) in enumerate(zip(p.imap(job, jobs), reps)): + res[(a, b)] = r + print('pair (%4d,%4d) |H1|=%2d |H2|=%2d |O|=%2d orbit %5d c=%d %s %s %.1fs [%d/%d]' % ( + a, b, len(hyp[a]), len(hyp[b]), nO, n, c, s, 'SAT' if r else 'UNSAT', t, k + 1, len(jobs)), flush=True) + pickle.dump(res, open('pair_res_c%d_%s.pkl' % (c, solver), 'wb')) + print('SAT pairs:', sum(res.values()), 'of', len(res), flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/verify.sh new file mode 100755 index 0000000000..99ef81e07c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s7-gaussian-c16/verify.sh @@ -0,0 +1,17 @@ +#!/bin/sh +# S7 verification: rebuild the 3-flat level tables from scratch, then check the theorem. ~15-25 min, 8 processes. +# Needs python with numpy, networkx, python-sat on PATH. CROSS=1 adds the glucose4 re-solve of all UNSAT prunings (~15 min). +set -e +export OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 +HERE=$(cd "$(dirname "$0")" && pwd); W=${WORK:-$(mktemp -d)} +cp "$HERE"/scripts/*.py "$HERE"/data/flats_Gauss.pkl "$HERE"/data/orbits.pkl "$HERE"/data/pair_orbits.pkl "$W"/ +cd "$W" +python lines.py | head -1 +python levels2.py 16 3 3 8 vf | grep -E '^(LEVEL|EMPTY)' +python levels3.py 16 3 lev_vf.pkl 4 16 8 vf2 | grep -E '^(LEVEL|EMPTY)' | sed 's/ (.*//' +cp lev_vf2.pkl lev_c16_d3_hi.pkl +python prove_c16.py lev_vf2.pkl | sed 's/ (.*s)$//' +python price16.py +python levels2.py 15 3 1 1 calib15 | grep -E '^(LEVEL|EMPTY)' | sed 's/ (.*//' +if [ -n "$CROSS" ]; then python crosscheck.py 16 3 lev_vf2.pkl 8 glucose4 8 | grep CROSSCHECK | sed 's/ (.*//'; fi +echo "work dir: $W" diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/README.md b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/README.md new file mode 100644 index 0000000000..30d4737fce --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/README.md @@ -0,0 +1,113 @@ +# S8 — Gaussian E8: no decoder with at most 18 totals costs ≤ 42 + +Context: #620 §10 (S5: Σβ ≥ 40 for every decoder), §13 (S6: every 15-total decoder costs 60) and §14 (S7: no 16-total decoder costs ≤ 42). +- The setting is the Gaussian view of E8: 60 Q(i)-lines in K⁴, with the F2 decoder pattern (B_TT = 1, and B_TS = 0 on K-non-orthogonal pairs). +- The cost is the Theorem-C cost Σ_j β_j. +- To beat real E8 a decoder needs Σβ ≤ 42. + +Labels: +- **checked**: `verify.sh` recomputes the claim, with exact integers and certified rank lower bounds. +- **argued**: the proof is in this README. + +## Results + +**Theorem 1 (two counting facets; argued + checked).** Every Gaussian-E8 decoder with c totals has + + Σβ ≥ max(40, 60 − c). + +So every decoder with c ≤ 17 totals costs ≥ 43. This re-proves §14 (c = 16) with the stronger bound 44, and settles **c = 17**. No level tables or SAT are needed. + +**Theorem 2 (c = 18; argued + checked).** No 18-total decoder costs ≤ 42. + +**Theorem 3 (tight face).** A decoder with Σβ = 60 − c exists for no c: for 15 ≤ c ≤ 20, μ = (c − 12)/12 is non-integral, and c ≤ 14 or c ≥ 21 is impossible (minrank 15; cost ≥ 40). In particular: +- c = 19 needs cost ≥ 42, so the only value open there is cost exactly 42; +- c = 20 needs cost ≥ 41. + +**Corollary.** +- A total of cost 0 touches only free entries, so it can be dropped. That gives a decoder with fewer totals and the same cost. +- Therefore a Gaussian decoder that beats real E8 has **between 19 and 42 totals of nonzero cost**. +- The upper end holds because each such total costs ≥ 1. +- **The all-c theorem is NOT proved. The open range is 19 ≤ c ≤ 42.** See "Gap" below. + +## Proof + +### 0. Data (checked: `hits_table.py`, which is S5's `gauss_bound40.py` plus a pickle dump) +For all 2271² pairs of flats (X, Y), the table holds: +- the exact clique hits h(X, Y): the number of the 640 twelve-line cliques of pairwise non-orthogonal lines that meet both X and Y; +- a certified lower bound R(X, Y) on the K-rank of the pairing, computed as an exact integer rank mod 2³¹ − 1. + +### 1. Theorem 1 +**Per clique (S5, argued).** On a clique C the pattern forces B[C, C] = I₁₂. A total contributes to this block only if it *hits* C, meaning R_j ∩ C ≠ ∅ and A_j ∩ C ≠ ∅, and then it contributes a rank-1 matrix. So N_C, the number of totals hitting C, satisfies N_C ≥ 12. Summing over the 640 cliques gives Σ_j hit_j ≥ 7680. + +**Per total (checked, `facets.py`).** Let X_j, Y_j be the spans of the two sides of total j. Then +- hit_j ≤ h(X_j, Y_j), and β_j ≥ R(X_j, Y_j); +- by rank, the largest h is 0, 192, 384, 512, 640 for R = 0, 1, 2, 3, 4; +- the upper concave envelope of these values has two facets, so on **every** flat pair + + h ≤ 192·R and h ≤ 128·R + 128. + +Summing over totals gives 7680 ≤ 192·Σβ and 7680 ≤ 128·Σβ + 128·c. ∎ + +The two facets give two slack identities, each term of which is ≥ 0: +- (D) Σ_j d_j + E = 192·cost − 7680, where d_j = 192β_j − hit_j; +- (E) Σ_j e_j + E = 128·(cost + c) − 7680, where e_j = 128β_j + 128 − hit_j; +- in both, E = Σ_C (N_C − 12). + +### 2. Theorem 2 (`c18.py`) +1. **Tightness.** At c = 18 and cost ≤ 42, the right side of (E) is ≤ 0. Hence cost = 42, E = 0 (every clique is hit **exactly** 12 times), and every e_j = 0. +2. **The span pair is tight.** hit_j ≤ h(X_j, Y_j) ≤ 128·R + 128 ≤ 128β_j + 128 = hit_j, so (X_j, Y_j) is a *tight* flat pair, and total j hits exactly the cliques that its span pair hits. +3. **The tight pairs (checked).** There are 1081 of them, and they are exactly: + - (ℓ^⊥, m^⊥) with ℓ ⊥ m, β = 2; + - (ℓ^⊥, ℓ^⊥), β = 3; + - (ℓ^⊥, K⁴) and (K⁴, ℓ^⊥), β = 3; + - (K⁴, K⁴), β = 4. + + Here ℓ^⊥ is a 15-line 3-flat. For each tight pair t, the set of cliques it **misses** is exactly {C : C meets x_t}, where x_t is a set of pairwise orthogonal lines ({ℓ, m}, {ℓ} or ∅). In matrix form, 1 − h_t = M·x_t, where M is the 640 × 60 clique/line incidence matrix. +4. **Equation.** So 12·**1** = N = Σ_j h_j = 18·**1** − M·μ, with μ = Σ_j x_{t(j)} ∈ Z₊⁶⁰. That is, Mμ = 6·**1**. +5. **Contradiction.** M has exact rank 60 (checked, sympy), and M·(½·**1**) = 6·**1** because every clique has 12 lines. So μ = ½·**1**, which is not integral. ∎ + +**Theorem 3.** The same argument applies whenever cost = 60 − c. That case needs (E) = 0, and with cost ≥ 40 it forces c ≤ 20. Then Mμ = (c − 12)·**1**, so μ = (c − 12)/12. For c = 19, 20 this is 7/12 and 2/3, neither integral. + +### 3. Consequences for 19 ≤ c ≤ 42 (argued) +- From (D), a total with β ≥ 3 has d ≥ 64 and a full total has d = 128 (checked: the smallest d at R = 3 is 64, at R = 4 it is 128). So there are at most 3 full totals and at most 6 totals with β ≥ 3, **for every c**. +- From (E), a rank-1 total has e ≥ 64. So there are at most 2(c − 18) rank-1 totals, while cost ≤ 42 needs at least 2c − 42 of them. +- At c = 19 with cost 42, the slack in (E) is only 128. + +## Gap: what is not proved, and why +- **Linear counting cannot go further.** `lp2.py` is an LP over uniform orbit weights. It uses cliques of every size (7, 8, 10, 11, 12), single lines, complements of all flats (minrank 15), the whole set, and a free c-term μ·c. Its value is exactly max(40, 60 − c) (float LP, recorded). So linear counting with these sets is exhausted; what goes further must be integrality, as in Theorem 2, or F2 structure. +- **Level tables do not scale.** The S7 level-table route does not reach c ≥ 19. Every multiset of at most c − 15 three-flats is trivially a level set, so the bottom levels alone have about 1500^(c−15)/11520 orbits. The hyperplane-only version at c = 18 grew as 1, 5, 21, 129, 921, 7917 reps at levels 1–6. +- **Integer counting is not enough either from c = 21 on (checked).** + - `tight_ilp.py 21` (CP-SAT) found 21 dumbbell totals (ℓ^⊥, m^⊥) with ℓ ⊥ m. They cost 42 in total, and they hit every 12-clique at least 12 times (between 12 and 16 times). + - `check_c21cover.py` re-checks this exactly; see `logs/check_c21cover.log`. + - So clique counting, even with integrality, cannot exclude c ≥ 21. That range needs the F2 pattern itself, for example S7-style rank-drop or level arguments on the dumbbell hyperplanes, or a SAT search. +- **c = 19 and 20 are open.** + - c = 19: cost exactly 42 with (E)-slack 128. + - c = 20: cost 41 or 42. + - CP-SAT returned UNKNOWN within its time limit on two runs: + - the full covering ILP over 124,841 rank-maximal types (`types.py`, `cover_ilp.py`, all c, 900 s); + - the tight-types-only version (`tight_ilp.py 19/20`, 300 s each). + + See `logs/`. + +## Validation +- c = 15: Theorem 1 gives ≥ 45, consistent with §13's exact 60. +- c = 16: it gives ≥ 44, consistent with §14's ≥ 43. +- The identity decoder (60 single-line totals, cost 60) meets 7680 = Σ hits exactly, with 128 hits per total, and satisfies both facets. +- §14's machinery was rerun from scratch with a multiset-aware level search (`levels_ms.py` in the work dir, exact multi-key canonical form). It reproduced S7's level counts 3, 28, 120, 368 for k = 1–4; the run was stopped after that because it is superseded. +- **Caveat on S7 (not affecting any S8 claim).** S7's `levels2.keys` packs ids 6, 7, … into one int64 with 12-bit shifts. That overflows for k > 10, so the canonical form there is a min over a truncated key. False prunes would need two distinct images to tie on the truncated key. None were observed, but none were excluded either. Theorem 1 re-proves §14 without the tables. +- No new SAT/UNSAT verdicts are used, so no Glucose cross-check is needed. All checks are exact integer computations, plus a sympy rank. + +## Files (`scripts/`) +| file | what | +|---|---| +| `lines.py`, `flats.py` | S5's exact construction of the lines and flats | +| `hits_table.py` | S5's `gauss_bound40.py`, which also dumps `hits_table.pkl` | +| `facets.py` | Theorem 1: both facets on all flat pairs; the tight pairs | +| `c18.py` | Theorems 2 and 3: tight-pair miss sets equal M·x_t; rank M = 60; the parity contradiction | +| `lp2.py` | the LP showing linear counting stops at max(40, 60 − c) | +| `types.py`, `cover_ilp.py`, `tight_ilp.py` | integer clique-covering relaxations (exploratory; CP-SAT) | +| `check_c21cover.py` | exact check of the c = 21, cost-42 integral cover | + +```sh +PATH=$SP/venv/bin:$PATH sh verify.sh # ~6 min, 1 process; compare with expected.txt +``` diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/expected.txt b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/expected.txt new file mode 100644 index 0000000000..a2aefc4ba0 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/expected.txt @@ -0,0 +1,18 @@ +PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all 2271x2271 flat pairs, histogram [5450, 369780, 2631390, 2150820, 1] +PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks) +RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43) +PASS facet1: hits <= 192*rank on all 5157441 flat pairs +PASS facet2: hits <= 128*rank + 128 on all 5157441 flat pairs +max hits by certified rank: {0: 0, 1: 192, 2: 384, 3: 512, 4: 640} +tight pairs (hits = 128 rank + 128): [(('dim3/15lines', 'dim3/15lines', 2), 900), (('dim3/15lines', 'dim3/15lines', 3), 60), (('dim3/15lines', 'dim4/60lines', 3), 60), (('dim4/60lines', 'dim3/15lines', 3), 60), (('dim4/60lines', 'dim4/60lines', 4), 1)] +c = 15: cost >= 45 (> 42: excluded) +c = 16: cost >= 44 (> 42: excluded) +c = 17: cost >= 43 (> 42: excluded) +c = 18: cost >= 42 +c = 19: cost >= 41 +PASS clique/line incidence M (640 x 60) has exact rank 60; M(1/2) = 6 +PASS every tight flat pair t has 1 - h_t = M x_t with x_t pairwise-orthogonal lines: [((15, 15, 2, 2), 900), ((15, 15, 3, 1), 60), ((15, 60, 3, 1), 60), ((60, 15, 3, 1), 60), ((60, 60, 4, 0), 1)] +RESULT c18: M mu = 6*1 has the unique rational solution mu = 1/2 (non-integral) => no 18-total decoder costs <= 42 +RESULT tight c=19: cost = 41 would need mu = 7/12 (non-integral) => cost >= 42 +RESULT tight c=20: cost = 40 would need mu = 2/3 (non-integral) => cost >= 41 +c = 21 dumbbell cover: cost 42 min clique hits 12 max 16 PASS diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/check_c21cover.log b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/check_c21cover.log new file mode 100644 index 0000000000..aab245b59b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/check_c21cover.log @@ -0,0 +1 @@ +c = 21 dumbbell cover: cost 42 min clique hits 12 max 16 PASS diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/cover_ilp.log b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/cover_ilp.log new file mode 100644 index 0000000000..cd72b5cc5c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/cover_ilp.log @@ -0,0 +1 @@ +status UNKNOWN 906s diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/lp2.log b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/lp2.log new file mode 100644 index 0000000000..a6825ce4c5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/lp2.log @@ -0,0 +1,11 @@ +Counter({10: 2304, 11: 1920, 8: 1920, 7: 1920, 12: 640}) +['clq7', 'clq8', 'clq10', 'clq11', 'clq12', 'line', 'cmp1', 'cmp2', 'cmp3', 'all'] +rows 313 +17 43.0 {'clq12': np.float64(0.00781)} mu 1.0 +18 42.0 {'clq12': np.float64(0.00781)} mu 1.0 +19 41.0 {'clq12': np.float64(0.00781)} mu 1.0 +20 40.0 {'clq12': np.float64(0.00521)} mu -0.0 +22 40.0 {'clq12': np.float64(0.00521)} mu -0.0 +25 40.0 {'clq12': np.float64(0.00521)} mu -0.0 +30 40.0 {'clq12': np.float64(0.00521)} mu -0.0 +42 40.0 {'clq12': np.float64(0.00521)} mu 0.0 diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/tight_ilp.log b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/tight_ilp.log new file mode 100644 index 0000000000..f59f25e8eb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/tight_ilp.log @@ -0,0 +1,6 @@ +c 21 status OPTIMAL +D [((0, 13), 1), ((4, 22), 1), ((5, 50), 1), ((6, 19), 1), ((6, 28), 1), ((7, 38), 1), ((8, 19), 1), ((11, 56), 1), ((12, 17), 1), ((18, 37), 1), ((18, 58), 1), ((20, 30), 1), ((21, 33), 1), ((25, 26), 1), ((35, 55), 1), ((36, 45), 1), ((37, 47), 1), ((41, 44), 1), ((42, 51), 1), ((44, 51), 1), ((48, 53), 1)] S [] F 0 +c 20 status UNKNOWN +c 19 status UNKNOWN + +[exited with code 0] diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/verify_run.log b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/verify_run.log new file mode 100644 index 0000000000..d26d5cead6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/logs/verify_run.log @@ -0,0 +1,19 @@ +PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all 2271x2271 flat pairs, histogram [5450, 369780, 2631390, 2150820, 1] +PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks) +RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43) +PASS facet1: hits <= 192*rank on all 5157441 flat pairs +PASS facet2: hits <= 128*rank + 128 on all 5157441 flat pairs +max hits by certified rank: {0: 0, 1: 192, 2: 384, 3: 512, 4: 640} +tight pairs (hits = 128 rank + 128): [(('dim3/15lines', 'dim3/15lines', 2), 900), (('dim3/15lines', 'dim3/15lines', 3), 60), (('dim3/15lines', 'dim4/60lines', 3), 60), (('dim4/60lines', 'dim3/15lines', 3), 60), (('dim4/60lines', 'dim4/60lines', 4), 1)] +c = 15: cost >= 45 (> 42: excluded) +c = 16: cost >= 44 (> 42: excluded) +c = 17: cost >= 43 (> 42: excluded) +c = 18: cost >= 42 +c = 19: cost >= 41 +PASS clique/line incidence M (640 x 60) has exact rank 60; M(1/2) = 6 +PASS every tight flat pair t has 1 - h_t = M x_t with x_t pairwise-orthogonal lines: [((15, 15, 2, 2), 900), ((15, 15, 3, 1), 60), ((15, 60, 3, 1), 60), ((60, 15, 3, 1), 60), ((60, 60, 4, 0), 1)] +RESULT c18: M mu = 6*1 has the unique rational solution mu = 1/2 (non-integral) => no 18-total decoder costs <= 42 +RESULT tight c=19: cost = 41 would need mu = 7/12 (non-integral) => cost >= 42 +RESULT tight c=20: cost = 40 would need mu = 2/3 (non-integral) => cost >= 41 +c = 21 dumbbell cover: cost 42 min clique hits 12 max 16 PASS +work dir: /private/tmp/claude-501/-Users-joelpulikkan-integer-mult-bounds/3231f0a2-7112-4dee-8ef4-f1fca97ae281/scratchpad/s8/vw diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/c18.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/c18.py new file mode 100644 index 0000000000..ec1bd133bb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/c18.py @@ -0,0 +1,41 @@ +"""S8: no 18-total Gaussian-E8 decoder costs <= 42. Input hits_table.pkl (see facets.py). + +Argument. e_j := 128 beta_j + 128 - hit_j >= 0 for every total (facet2), and + sum_j e_j + E = 128 cost + 128 c - 7680, E = sum_C (N_C - 12) >= 0 (N_C = #totals hitting clique C on both sides). +At c = 18 and cost <= 42 the right side is <= 0, so cost = 42, E = 0 (every clique hit EXACTLY 12 times) and every total +is tight: hit(R_j,A_j) = 128 beta_j + 128. Since hit(R,A) <= hit(X,Y) <= 128 R(X,Y) + 128 <= 128 beta + 128 for the spans +(X,Y), the span pair is a tight flat pair and the cliques hit by total j are exactly those hit by (X_j,Y_j). +Checked below: for every tight flat pair t the set of cliques NOT hit is {C : C meets x_t} for a set x_t of pairwise +orthogonal lines (so that 1 - h_t = M x_t, M = 640 x 60 clique/line incidence). Then + 12 * 1 = N = sum_j h_j = 18 * 1 - M mu, mu = sum_j x_{t(j)} in Z_{>=0}^60, i.e. M mu = 6 * 1. +M has rank 60 (checked, exact) and M (1/2 * 1) = 6 * 1 (every clique has 12 lines), so mu = 1/2 * 1: not integral. +Contradiction. +Same argument on the tight face for any c (cost = 60 - c, needs c <= 20 since cost >= 40): M mu = (c-12) 1, mu = (c-12)/12; +for c = 19 (cost 41) and c = 20 (cost 40) this is non-integral, so c = 19 needs cost >= 42 and c = 20 cost >= 41.""" +import pickle, numpy as np, sympy +from lines import views +S = views()['Gauss']; NO = S.NO +d = pickle.load(open('hits_table.pkl', 'rb')); fl = d['fl']; R = d['R']; H = d['hits']; C = d['C'] +M = np.zeros((640, 60), np.int64) +for k, c in enumerate(C): M[k, c] = 1 +assert (M.sum(1) == 12).all() +assert sympy.Matrix(M).rank() == 60 +print('PASS clique/line incidence M (640 x 60) has exact rank 60; M(1/2) = 6') +L = [set(f) for f, dd in fl] +tight = np.argwhere(H == 128 * R + 128) +from collections import Counter +kinds = Counter() +for a, b in tight: + X, Y = L[a], L[b] + h = np.array([bool(X & set(c)) and bool(Y & set(c)) for c in C]) + x = sorted(T for T in range(60) if all(not h[k] for k, c in enumerate(C) if T in c)) # lines all of whose cliques are missed + assert all(not NO[s, t] for s in x for t in x if s != t) # pairwise orthogonal => cliques meet x at most once + assert ((M[:, x].sum(1) if x else np.zeros(640, int)) == (~h).astype(int)).all() # 1 - h = M x exactly + kinds[(len(L[a]), len(L[b]), int(R[a, b]), len(x))] += 1 +print('PASS every tight flat pair t has 1 - h_t = M x_t with x_t pairwise-orthogonal lines:', sorted(kinds.items())) +print('RESULT c18: M mu = 6*1 has the unique rational solution mu = 1/2 (non-integral) => no 18-total decoder costs <= 42') +# Tight face in general: cost = 60 - c forces the same structure with M mu = (c - 12) * 1, mu = (c - 12)/12 * 1. +for c in (19, 20): + from fractions import Fraction as Fr + mu = Fr(c - 12, 12) + print('RESULT tight c=%d: cost = %d would need mu = %s (non-integral) => cost >= %d' % (c, 60 - c, mu, 61 - c if 61 - c > 40 else 41)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/check_c21cover.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/check_c21cover.py new file mode 100644 index 0000000000..9237a61705 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/check_c21cover.py @@ -0,0 +1,11 @@ +"""Exact check: the 21 dumbbell totals (l^perp, m^perp), l ⊥ m (beta = 2 each, cost 42) found by tight_ilp.py 21 hit every +12-clique >= 12 times, i.e. the clique-covering relaxation is integrally feasible at c = 21, cost 42.""" +import pickle, numpy as np +from lines import views +S=views()['Gauss']; NO=S.NO +C=pickle.load(open('hits_table.pkl','rb'))['C'] +D=[(0,13),(4,22),(5,50),(6,19),(6,28),(7,38),(8,19),(11,56),(12,17),(18,37),(18,58),(20,30),(21,33),(25,26),(35,55),(36,45),(37,47),(41,44),(42,51),(44,51),(48,53)] +perp=lambda l: {T for T in range(60) if T!=l and not NO[l,T]} +assert all(not NO[a,b] and a!=b for a,b in D) +N=[sum(1 for a,b in D if perp(a)&set(c) and perp(b)&set(c)) for c in C] +print('c = 21 dumbbell cover: cost', 2*len(D), 'min clique hits', min(N), 'max', max(N), 'PASS' if min(N)>=12 else 'FAIL') diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/cover_ilp.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/cover_ilp.py new file mode 100644 index 0000000000..ab503dfc43 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/cover_ilp.py @@ -0,0 +1,30 @@ +"""Clique-covering ILP relaxation (c-free): integer multiplicities n_t of rank-maximal flat-pair types with deficit <= 384, +every 12-clique covered >= 12 times, sum rank*n <= 42. Feasible => relaxation cannot close all c; infeasible => all c closed.""" +import pickle, numpy as np, time, sys +from ortools.sat.python import cp_model +d=pickle.load(open('hits_table.pkl','rb')); fl=d['fl']; C=d['C'] +T=pickle.load(open('types.pkl','rb')); ts=T['types']; R=T['R']; D=T['D'] +L=np.zeros((len(fl),60),np.int64) +for i,(f,dd) in enumerate(fl): L[i,sorted(f)]=1 +inc=np.zeros((60,640),np.int64) +for k,c in enumerate(C): inc[c,k]=1 +meet=(L@inc>0) +m=cp_model.CpModel(); n=[] +for k,(a,b) in enumerate(ts): + ub=42//int(R[k]) if D[k]==0 else min(42//int(R[k]),384//int(D[k])) + n.append(m.NewIntVar(0,ub,'n%d'%k)) +cols=[np.nonzero(meet[a]&meet[b])[0] for a,b in ts] +rows=[[] for _ in range(640)] +for k,cl in enumerate(cols): + for q in cl: rows[q].append(k) +for q in range(640): m.Add(sum(n[k] for k in rows[q])>=12) +m.Add(sum(int(R[k])*n[k] for k in range(len(n)))<=int(sys.argv[1]) if len(sys.argv)>1 else 42) +m.Add(sum(int(D[k])*n[k] for k in range(len(n)))<=384) +s=cp_model.CpSolver(); s.parameters.num_workers=8; s.parameters.max_time_in_seconds=float(sys.argv[2]) if len(sys.argv)>2 else 900 +t=time.time(); r=s.Solve(m); print('status',s.StatusName(r),'%.0fs'%(time.time()-t)) +if r in (cp_model.FEASIBLE,cp_model.OPTIMAL): + sol=[(k,s.Value(n[k])) for k in range(len(n)) if s.Value(n[k])] + print('solution: totals',sum(v for k,v in sol),'cost',sum(int(R[k])*v for k,v in sol)) + from collections import Counter + print(Counter((len(fl[ts[k][0]][0]),len(fl[ts[k][1]][0]),int(R[k]),int(D[k])) for k,v in sol for _ in range(v))) + pickle.dump(sol,open('cover_sol.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/facets.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/facets.py new file mode 100644 index 0000000000..56cc4c00cd --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/facets.py @@ -0,0 +1,23 @@ +"""S8 counting lemma. Input: hits_table.pkl (from hits_table.py = S5 gauss_bound40.py + dump): for all 2271^2 flat pairs +(X, Y): exact clique hits h(X,Y) (number of the 640 twelve-cliques meeting both X and Y) and a certified LOWER bound +R(X,Y) on the Theorem-C pairing rank. + +Checks: h <= 192 R (S5) and h <= 128 R + 128 (new) on every pair. +Since a total j with spans X_j, Y_j has hit_j <= h(X_j, Y_j) and beta_j >= R(X_j, Y_j), and every clique is hit >= 12 +times, 7680 <= sum_j hit_j <= sum_j min(192 beta_j, 128 beta_j + 128) => cost >= 40 and cost >= 60 - c. +Also prints the tight pairs (h = 128 R + 128) and the deficit facts used for c = 18.""" +import pickle, numpy as np +from collections import Counter +d = pickle.load(open('hits_table.pkl', 'rb')); fl = d['fl']; R = d['R']; H = d['hits'] +assert len(fl) == 2271 and len(d['C']) == 640 +assert (H <= 192 * R).all() +print('PASS facet1: hits <= 192*rank on all %d flat pairs' % H.size) +assert (H <= 128 * R + 128).all() +print('PASS facet2: hits <= 128*rank + 128 on all %d flat pairs' % H.size) +print('max hits by certified rank:', {int(r): int(H[R == r].max()) for r in range(5)}) +desc = lambda i: 'dim%d/%dlines' % (fl[i][1], len(fl[i][0])) +tight = np.argwhere(H == 128 * R + 128) +print('tight pairs (hits = 128 rank + 128):', sorted(Counter((desc(a), desc(b), int(R[a, b])) for a, b in tight).items())) +for c in range(15, 20): + lb = max(40, -(-(7680 - 128 * c) // 128)) + print('c = %d: cost >= %d%s' % (c, lb, ' (> 42: excluded)' if lb > 42 else '')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/flats.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/flats.py new file mode 100644 index 0000000000..b064c81896 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/flats.py @@ -0,0 +1,26 @@ +"""All flats (line sets of K-subspaces spanned by configuration lines) of a view, with dims. Exact.""" +import sys, pickle; sys.path.insert(0,sys.path[0]) +from lines import * +def flats(S): + v=S.v; out={} + def close(idx): + d=S.arank(idx); return frozenset(T for T in range(v) if S.arank(list(idx)+[T])==d), d + level=[ (frozenset([T]),1) for T in range(v)] + for f,d in level: out[f]=d + frontier=list(out.items()) + while frontier: + new={} + for f,d in frontier: + if d>=S.w: continue + for T in range(v): + if T in f or any(T in g and f= 40 (x_K >= 2/3). + +Argument. C ranges over the 640 twelve-line sets of pairwise K-non-orthogonal lines. On C x C the pattern forces +B[C,C] = I_12 (diagonal 1, all off-diagonal pairs non-orthogonal), and each total contributes a rank-1 F2 matrix there, +so at least 12 totals hit C on both sides (R_j & C != {} and A_j & C != {}). Summing: sum_j hit(R_j, A_j) >= 12*640. +For fixed spans X = span R_j, Y = span A_j the hit count is largest for R = lines(X), A = lines(Y) (flats), and this +script shows hit(X, Y) <= 192 * rank(X, Y) for all 2271^2 flat pairs, rank = K-rank of the pairing (Theorem C cost). +Hence sum_j beta_j >= 12*640/192 = 40. Binding pair: X = 6-line 2-flat containing l, Y = the 15-line hyperplane l^perp +(pairing rank 1, 192 hits). Mixing all clique sizes, singletons, non-orthogonal pairs and the whole set in an LP over +uniform orbit weights does not improve 40 (gauss_lp3.py in the work dir). +Ranks: exact integer pairing matrices, rank mod p = 2^31-1 (a certified lower bound on the rank over Q).""" +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from flats import flats +import networkx as nx + +def main(): + W = views(); S = W['Gauss']; J = W['_J']; v = S.v + F = flats(S); fl = sorted(F.items(), key=lambda t: (t[1], len(t[0]), sorted(t[0]))) + n = len(fl); assert n == 2271 + g = nx.Graph([(i, j) for i in range(v) for j in range(i+1, v) if S.NO[i, j]]) + C = [c for c in nx.find_cliques(g) if len(c) == 12] + assert len(C) == 640 and all(S.NO[a, b] for c in C for a, b in itertools.combinations(c, 2)) + Lo = np.zeros((n, v), np.int64) + for i, (f, d) in enumerate(fl): Lo[i, sorted(f)] = 1 + inc = np.zeros((v, len(C)), np.int64) + for k, c in enumerate(C): inc[c, k] = 1 + meet = (Lo @ inc > 0).astype(np.int64); hits = meet @ meet.T # exact integers + # exact integer bases (vectors r_T, J r_T), pairing ranks by batched elimination mod p = 2^31-1 + # (rank mod p <= rank over Q, so these are certified LOWER bounds; the claim only needs lower bounds) + def ibasis(f): + rows = [] + for T in sorted(f): + for x in (S.reps[T], J @ S.reps[T]): + if rank_q(rows + [x]) > len(rows): rows.append(x) + return np.array(rows, np.int64) + Bs = [ibasis(f) for f, d in fl] + assert all(len(b) == 2 * d for b, (f, d) in zip(Bs, fl)) + P = 2147483647 + def modrank(A): + A = A % P; N, r, c = A.shape; used = np.zeros((N, r), bool); idx = np.arange(N) + for col in range(c): + cand = (A[:, :, col] != 0) & ~used; has = cand.any(1) + if not has.any(): continue + pr = cand.argmax(1); it = idx[has]; prow = A[it, pr[has]] + inv = np.array([pow(int(x), P - 2, P) for x in prow[:, col]], np.int64) if len(it) < 64 else modinv(prow[:, col]) + prow = (prow * inv[:, None]) % P + fac = A[it, :, col].copy(); fac[np.arange(len(it)), pr[has]] = 0 + A[it] = (A[it] - (fac[:, :, None] * prow[:, None, :]) % P) % P + used[it, pr[has]] = True + return used.sum(1) + def modinv(x): + res = np.ones_like(x); base = x % P; e = P - 2 + while e: + if e & 1: res = (res * base) % P + base = (base * base) % P; e >>= 1 + return res + R = np.zeros((n, n), np.int64) + groups = {} + for i, b in enumerate(Bs): groups.setdefault(len(b), []).append(i) + for da, ia in groups.items(): + A = np.stack([Bs[i] for i in ia]) + for db, ib in groups.items(): + Bm = np.stack([Bs[i] for i in ib]) + M = np.einsum('aik,bjk->abij', A, Bm).reshape(-1, da, db) + r = modrank(M).reshape(len(ia), len(ib)); assert (r % 2 == 0).all() + R[np.ix_(ia, ib)] = r // 2 + import pickle; pickle.dump({'fl':fl,'R':R,'hits':hits,'C':C},open('hits_table.pkl','wb')) + print(f'PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all {n}x{n} flat pairs, histogram {np.bincount(R.flatten()).tolist()}') + assert (hits <= 192 * R).all() + worst = np.argwhere((hits == 192 * R) & (R > 0)) + print('PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks)') + a, b = worst[0] + print(f' binding example: |X| = {len(fl[a][0])} (dim {fl[a][1]}), |Y| = {len(fl[b][0])} (dim {fl[b][1]}), rank {R[a,b]}, hits {hits[a,b]}; {len(worst)} binding pairs') + print('RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43)') + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lines.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lines.py new file mode 100644 index 0000000000..fc4eac011d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lines.py @@ -0,0 +1,166 @@ +"""Self-contained exact construction of the E8 root lines over Q(i), Q(omega) and the Hurwitz quaternions. + +Real model: the 240 E8 roots in doubled standard coordinates (norm 8; in {0,+-4,+-8}). + * Gaussian view : J = left multiplication by i on R^8 = H^2 (coords 1,i,j,k per block); J preserves E8. + Hermitian Gram hG(x,y) = ( + i )/4 in Z[i] (diag 2). + * Eisenstein view: rho = fixed-point-free order-3 isometry of E8 (product of Coxeter elements of 4 orthogonal A2's). + Sesquilinear Gram hE(x,y) = ( - w )/4 in Z[w] (w^2+w+1=0; diag 2+w, a unit at 2). + * Hurwitz view : left multiplication by i, j, k; quaternionic lines of real dim 4. +Elements of Z[i] / Z[w] are stored as integer pairs (a, b) = a + b*i / a + b*w. +Everything is exact integer arithmetic; asserts check every structural claim used downstream. +""" +import itertools +from fractions import Fraction +import numpy as np + + +def e8_roots(): + R = [] + for i, j in itertools.combinations(range(8), 2): + for si in (2, -2): + for sj in (2, -2): + r = [0] * 8; r[i], r[j] = si, sj; R.append(r) + for s in itertools.product((1, -1), repeat=8): + if s.count(-1) % 2 == 0: + R.append(list(s)) + R = np.array(R, dtype=np.int64) + assert len(R) == 240 and set((R * R).sum(1)) == {8} + return R + + +def rank_q(rows): + """exact rank over Q of an integer matrix (fraction-free Gaussian elimination).""" + M = [list(map(int, r)) for r in rows] + if not M: + return 0 + n, m = len(M), len(M[0]); r = 0 + for c in range(m): + p = next((i for i in range(r, n) if M[i][c] != 0), None) + if p is None: + continue + M[r], M[p] = M[p], M[r] + for i in range(r + 1, n): + if M[i][c]: + f, g = M[i][c], M[r][c] + M[i] = [g * a - f * b for a, b in zip(M[i], M[r])] + r += 1 + if r == n: + break + return r + + +def left_quat(): + Li = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 0, -1], [0, 0, 1, 0]]) + Lj = np.array([[0, 0, -1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, -1, 0, 0]]) + Lk = Li @ Lj + blk = lambda M: np.kron(np.eye(2, dtype=np.int64), M).astype(np.int64) + return blk(Li), blk(Lj), blk(Lk) + + +def a2_rotation(roots): + """rho, an order-3 fixed-point-free isometry preserving E8 (returned as an exact integer matrix 4*rho).""" + chosen = [] + for a in roots: + if len(chosen) == 4: + break + if any(np.dot(a, x) != 0 for p in chosen for x in p): + continue + for b in roots: + if np.dot(a, b) == -4 and all(np.dot(a, x) == 0 and np.dot(b, x) == 0 for p in chosen for x in p): + chosen.append((a, b)); break + assert len(chosen) == 4 + g = np.eye(8, dtype=object) + for a, b in chosen: + sa = np.eye(8, dtype=object) * 8 - 2 * np.outer(a, a).astype(object) + sb = np.eye(8, dtype=object) * 8 - 2 * np.outer(b, b).astype(object) + g = g.dot(sa).dot(sb) + D = 8 ** 8 // 4 + assert all(x % D == 0 for x in g.flatten()) + g = (g // D).astype(np.int64) + I = np.eye(8, dtype=np.int64) + assert (g @ g @ g == 64 * I).all() and (g @ g + 4 * g + 16 * I == 0).all() + return g + + +def lines_over(roots, ops): + """group roots into A-lines, A spanned by the integer matrices ops (identity first; ops may be scaled by 4). + Returns representatives (first root met of each line, in root order).""" + S = {tuple(r) for r in roots} + reps, seen = [], set() + for r in roots: + if tuple(r) in seen: + continue + span = [B @ r for B in ops] + assert rank_q(span) == len(ops) + reps.append(r) + members = [s for s in roots if rank_q(span + [s]) == len(ops)] + for s in members: + seen.add(tuple(s)) + return np.array(reps) + + +class View: + """name, reps (v x 8), ops (real matrices spanning A, scaled so that ops[i] @ root is a root times scale[i]).""" + def __init__(self, name, reps, ops, d): + self.name, self.reps, self.ops, self.d = name, reps, ops, d + self.v = len(reps) + NO = np.zeros((self.v, self.v), bool) + for B in ops: + NO |= (reps @ B @ reps.T) != 0 + np.fill_diagonal(NO, False) + self.NO = NO # A-non-orthogonal pairs (the decoder pattern forces B_TS = 0 there) + self.w = self.arank(range(self.v)) + + def arank(self, idx): + rows = [B @ self.reps[i] for i in idx for B in self.ops] + r = rank_q(rows) if rows else 0 + assert r % self.d == 0 + return r // self.d + + def pair_rank(self, R, A): + """A-rank of the pairing span(R) x span(A) = real rank of the real Gram block / dim A (Theorem C cost).""" + BR = [B @ self.reps[i] for i in R for B in self.ops] + BA = [B @ self.reps[i] for i in A for B in self.ops] + if not BR or not BA: + return 0 + M = [[int(x @ y) for y in BA] for x in BR] + r = rank_q(M); assert r % self.d == 0 + return r // self.d + + +def views(): + roots = e8_roots(); I8 = np.eye(8, dtype=np.int64) + Li, Lj, Lk = left_quat() + S = {tuple(r) for r in roots} + for M in (Li, Lj, Lk): + assert all(tuple(M @ r) in S for r in roots) + g = a2_rotation(roots) # 4*rho + assert all(tuple(g @ r // 4) in S and ((g @ r) % 4 == 0).all() for r in roots) + rho = g # 4*rho (exact integer matrix); ranks are scale-free + out = {} + out['Gauss'] = View('Gauss', lines_over(roots, [I8, Li]), [I8, Li], 2) + out['Eisen'] = View('Eisen', lines_over(roots, [I8, rho]), [I8, rho], 2) + out['Hurwitz'] = View('Hurwitz', lines_over(roots, [I8, Li, Lj, Lk]), [I8, Li, Lj, Lk], 4) + out['_J'], out['_rho'] = Li, rho + return out + + +def gauss_gram(V, J): + """hG(r_T, r_S) = ( + i)/4 as integer pairs (a, b).""" + A = V @ V.T; Bm = V @ J @ V.T + assert (A % 4 == 0).all() and (Bm % 4 == 0).all() + return A // 4, Bm // 4 + + +def eisen_gram(V, rho4): + """hE(r_T, r_S) = ( - w)/4 as integer pairs (a, b) = a + b w (rho4 = 4*rho).""" + A = V @ V.T; Bm = V @ rho4 @ V.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + + +if __name__ == '__main__': + W = views() + for nm in ('Gauss', 'Eisen', 'Hurwitz'): + S = W[nm] + print(nm, 'v', S.v, 'w', S.w, 'nonorth degree', sorted(set(S.NO.sum(1)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lp2.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lp2.py new file mode 100644 index 0000000000..2f24885d2a --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/lp2.py @@ -0,0 +1,38 @@ +import pickle, numpy as np, networkx as nx +from scipy.optimize import linprog +from lines import views +S=views()['Gauss']; NO=S.NO +d=pickle.load(open('hits_table.pkl','rb')); fl=d['fl']; R=d['R']; H=d['hits'] +n=len(fl); v=60 +L=np.zeros((n,v),np.int64) +for i,(f,dd) in enumerate(fl): L[i,sorted(f)]=1 +dims=np.array([dd for f,dd in fl]) +g=nx.Graph([(i,j) for i in range(v) for j in range(i+1,v) if NO[i,j]]) +cl=list(nx.find_cliques(g)) +fams={} +def famhits(sets): + inc=np.zeros((v,len(sets)),np.int64) + for k,s in enumerate(sets): inc[list(s),k]=1 + meet=(L@inc>0).astype(np.int64); return meet@meet.T +from collections import Counter +print(Counter(len(c) for c in cl)) +for s in sorted(set(len(c) for c in cl)): + sets=[c for c in cl if len(c)==s]; fams['clq%d'%s]=(famhits(sets),s,len(sets)) +fams['line']=(famhits([[T] for T in range(v)]),1,60) +for dd in (1,2,3): + idx=np.where(dims==dd)[0] + inc=((L@L[idx].T)==L.sum(1)[:,None]).astype(np.int64) + a=inc.sum(1); N=len(idx)-a[:,None]-a[None,:]+inc@inc.T + fams['cmp%d'%dd]=(N,15,len(idx)) +fams['all']=(((L.sum(1)[:,None]>0)&(L.sum(1)[None,:]>0)).astype(np.int64),15,1) +names=list(fams); print(names) +mask=R>0 +rows=np.stack([fams[k][0][mask] for k in names],1); rr=R[mask] +u=np.unique(np.concatenate([rows,rr[:,None]],1),axis=0); print('rows',len(u)) +dem=np.array([fams[k][1]*fams[k][2] for k in names],float) +# vars: y (per family, uniform), mu>=0 ; constraint rows: hits.y - mu <= r ; maximize dem.y - c*mu +for c in (17,18,19,20,22,25,30,42): + obj=np.concatenate([-dem,[c]]) + A=np.concatenate([u[:,:-1],-np.ones((len(u),1))],1) + res=linprog(obj,A_ub=A,b_ub=u[:,-1],bounds=[(0,None)]*(len(names)+1),method='highs') + y=res.x; print(c, round(-res.fun,4), {names[i]:round(y[i],5) for i in range(len(names)) if y[i]>1e-9}, 'mu',round(y[-1],4)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/tight_ilp.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/tight_ilp.py new file mode 100644 index 0000000000..6f6e19c668 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/tight_ilp.py @@ -0,0 +1,22 @@ +"""Clique covering using only tight types (dumbbells (l^perp,m^perp) l⊥m beta 2; (l^perp,l^perp)/(l^perp,K4) beta 3; (K4,K4) beta 4), +for a fixed number c of totals: misses per clique = sum over chosen totals of |C ∩ x_t| <= c - 12, cost <= 42. +argv: c""" +import pickle, sys, numpy as np +from ortools.sat.python import cp_model +from lines import views +S=views()['Gauss']; NO=S.NO +C=pickle.load(open('hits_table.pkl','rb'))['C'] +c=int(sys.argv[1]) +pairs=[(a,b) for a in range(60) for b in range(a+1,60) if not NO[a,b]] +m=cp_model.CpModel() +nD={p:m.NewIntVar(0,21,'d%d_%d'%p) for p in pairs}; nS={l:m.NewIntVar(0,14,'s%d'%l) for l in range(60)}; nF=m.NewIntVar(0,3,'f') +m.Add(sum(nD.values())+sum(nS.values())+nF==c) +m.Add(2*sum(nD.values())+3*sum(nS.values())+4*nF<=42) +mu={l:[] for l in range(60)} +for (a,b),v in nD.items(): mu[a].append(v); mu[b].append(v) +for l,v in nS.items(): mu[l].append(v) +for cl in C: m.Add(sum(v for l in cl for v in mu[l])<=c-12) +s=cp_model.CpSolver(); s.parameters.num_workers=8; s.parameters.max_time_in_seconds=300 +r=s.Solve(m); print('c',c,'status',s.StatusName(r)) +if r in (cp_model.OPTIMAL,cp_model.FEASIBLE): + print('D',[(p,s.Value(v)) for p,v in nD.items() if s.Value(v)],'S',[(l,s.Value(v)) for l,v in nS.items() if s.Value(v)],'F',s.Value(nF)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/types.py b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/types.py new file mode 100644 index 0000000000..16e2eab88f --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/scripts/types.py @@ -0,0 +1,18 @@ +"""Rank-maximal flat-pair types with deficit <= 384 (any total of a cost<=42 decoder can be replaced, for the clique-covering +relaxation, by its span pair, then by a rank-maximal enlargement: hits only grow, rank (cost) unchanged).""" +import pickle, numpy as np +d=pickle.load(open('hits_table.pkl','rb')); fl=d['fl']; R=d['R']; H=d['hits']; C=d['C'] +n=len(fl); L=[set(f) for f,dd in fl] +sup=[[j for j in range(n) if L[i] < L[j]] for i in range(n)] +D=192*R-H +ok=(R>0)&(D<=384) +enlX=np.zeros((n,n),bool); enlY=np.zeros((n,n),bool) +for i in range(n): + if sup[i]: + enlX[i]=(R[sup[i],:]==R[i][None,:]).any(0) + enlY[:,i]=(R[:,sup[i]]==R[:,i][:,None]).any(1) +mx=ok&~enlX&~enlY +ts=np.argwhere(mx); print('types',len(ts)) +from collections import Counter +print(Counter((len(fl[a][0]),len(fl[b][0]),int(R[a,b]),int(D[a,b])) for a,b in ts).most_common()) +pickle.dump({'types':ts,'R':R[mx],'D':D[mx]},open('types.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/verify.sh new file mode 100755 index 0000000000..649678a685 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s8-gaussian-all-c/verify.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# S8 verification (~6 min, 1 process; prints PASS/RESULT lines identical to expected.txt): rebuild the S5 flat-pair table (exact hits, certified ranks), then check +# the two counting facets (cost >= max(40, 60 - c)) and the c = 18 parity argument. Compare with expected.txt. +# Needs python 3.11 with numpy, scipy, sympy, networkx on PATH. +set -e +export OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 +HERE=$(cd "$(dirname "$0")" && pwd); W=${WORK:-$(mktemp -d)} +cp "$HERE"/scripts/*.py "$W"/ +cd "$W" +python hits_table.py | grep -E '^(PASS|RESULT)' +python facets.py +python c18.py +python check_c21cover.py +echo "work dir: $W" diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/README.md b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/README.md new file mode 100644 index 0000000000..eb74ceb4eb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/README.md @@ -0,0 +1,126 @@ +# S9 — Gaussian E8: integral clique packing closes c = 19 (and part of c = 20); c ≥ 21 needs F2 structure + +Context: #620 §13–§15 (S6, S7, S8). Setting: the Gaussian view of E8, 60 Q(i)-lines in K⁴, F2 decoder pattern +(B_TT = 1, B_TS = 0 on K-non-orthogonal pairs), minrank 15, line group of order 11520, Theorem-C cost Σβ_j. +Beating real E8 needs Σβ ≤ 42. Before S9: cost ≥ max(40, 60 − c); c ≤ 18 excluded; c = 19 open only at cost 42; +c = 20 open at 41–42; c ≥ 21 open (and S8 found an integral clique cover at c = 21, so counting alone cannot close c ≥ 21). + +Labels: **checked** = recomputed by `verify.sh` (exact integers / certified ranks for the data; HiGHS MIP and CP-SAT for +the integer programs, see "Trust" below); **argued** = proof in this README. + +## Status (PARTIAL — written 2026-10-15 01:40 EDT, solver jobs still running) +- **Proven (checked):** the μ-packing lemma (§1) and its corollary: a decoder whose totals are all tight costs ≥ 44 at c = 18–20 and ≥ 42 at c = 21. +- **c = 19, cost 42: 17 of 20 cases excluded. Three cases are still running:** orbits 17, 18 and 20. Their logs are (HiGHS, optimisation form) and (CP-SAT on case 20). The 10 cases that went to the time limit in optimisation form all had dual bounds ≤ 33 < 34, and analogous cases gave dual bound 33 within 1200 s. **c = 19 is NOT yet closed.** +- **c = 20:** cost 41 has cases −1 and 2 excluded; cases 4, 5 and 6 are running (), and cases 9 and 11–23 were never run. Cost 42 has cases −1 and 21 excluded; case 22 is running (, sequential −1, 21, 22, 23, 19, 20). The heavy cases at cost 42 (orbits 1, 2, 4–6 and 9–18, 20k–54k columns) were never run; they look out of reach without a better formulation. +- **c ≥ 21:** counting is exhausted (proven). The F2 pattern is what fails on the counting covers (2×2 support lemma, §3). This is not a proof for any c ≥ 21. + +## 1. The μ-packing lemma (checked: `mu_ilp.py` HiGHS optimal, `mu_cpsat.py` CP-SAT cross-check) +M = 640 × 60 clique/line incidence (the 640 twelve-cliques of pairwise non-orthogonal lines; each line lies in 128). + + f(k) := max { |μ| : μ ∈ Z≥0⁶⁰, Mμ ≤ k·1 } = 28, 32, 36, 42, 48, 52, 60 for k = 6, …, 12, + +while the LP value is 5k (μ = k/12·1). CP-SAT independently: k = 7, |μ| ≥ 33 INFEASIBLE; k = 8, |μ| ≥ 37 INFEASIBLE. +The integrality gap (3 at k = 7, 4 at k = 8) is what S8's linear counting could not see. + +**Corollary (all-tight decoders; argued).** S8: a tight total t misses exactly the cliques M·x_t with x_t pairwise +orthogonal lines and gain g_t = 4 − β_t = |x_t|. N_C ≥ 12 gives Σ_j M x_j ≤ (c − 12)·1, so Σ g ≤ f(c − 12): +cost ≥ 4c − f(c − 12) = 44 (c = 18), 44 (c = 19), 44 (c = 20), 42 (c = 21). The c = 21 value is attained by +S8's 21-dumbbell cover, so counting is exactly exhausted there. + +## 2. Lifted packing relaxation for all decoders (argued; data checked) +For a decoder with c totals and cost ≤ cm: +1. **WLOG β_j ≥ 1** (S8: a cost-0 total can be dropped; c − 1 is already excluded at the same cost). +2. **Types.** Replace total j by a rank-maximal enlargement t of its span pair (X_j, Y_j): certified rank R_t ≤ β_j, + its miss set m_t (cliques not met on both sides) ⊆ the total's miss set, and e_t = 128R_t + 128 − hits_t ≤ e_j. + All rank-maximal types with 1 ≤ R ≤ 3 have e ≤ 132; deduplicated by (R, miss set) there are **58,340**, in **27 orbits** + under the line group (`types_e.py`, `dom.py`). +3. **Packing.** N_C ≥ 12 ⇔ Σ_j m_j ≤ (c − 12)·1. Cost ≤ cm ⇔ Σ_j g_j ≥ 4c − cm with g = 4 − β ≤ g_t. + Identity (E) of S8: Σ_j e_j + E = 128(cm + c) − 7680 =: slack, so Σ_j e_t(j) ≤ slack. +4. **Lifting.** Z_t = lines all of whose 128 cliques are missed by t; x_t = a maximum pairwise-orthogonal subset (α_t = |x_t|). + A clique contains at most one line of x_t, so M x_t ≤ m_t. If α_t ≥ g_t ("non-deficient") replace t by x_t (fewer + misses, gain ≥ g_t): these totals only enter through μ = Σ x_t ∈ Z≥0⁶⁰. "Deficient" types (δ_t = g_t − α_t > 0) keep their + exact miss set. Every deficient type has e_t ≥ 16 (`lift_prep.py`; table below). +5. **Case split (symmetry).** No deficient total: Σ g ≤ f(c − 12). Otherwise apply a group element so that the + deficient total of **largest e** is its orbit representative t₁; every other deficient total then has + e ≤ min(e₁, slack − e₁). Each case is the ILP M μ + m_{t₁} + Σ n_t m_t ≤ (c−12)·1, Σ n_t e_t ≤ slack − e₁, + |μ| + g_{t₁} + Σ n_t g_t ≥ 4c − cm (μ, n integer) — `branch1.py` (HiGHS; "feas" adds the last row as a + constraint, "opt" maximises the left side and reports the dual bound). + +**Theorem 1 (c = 19), partial.** Cases −1, 2, 4, 5, 6, 9–16, 19, 21, 22 and 23 are excluded: HiGHS proved them infeasible, or the optimisation form gave a dual bound ≤ 33 < 34. Cases 17, 18 and 20 are still running. If they are excluded, then no 19-total decoder costs ≤ 42 (logs: ). + +**Theorem 2 (c = 20), partial.** Cost ≤ 41: cases −1 and 2 are excluded. Cost ≤ 42: cases −1 and 21 are excluded. Everything else is open or running (see Status). + +Single-deficient diagnostics (`vsingle.py`, c = 19): with one deficient total fixed to each orbit representative, +max(|μ| + g) ≤ 33 < 34 for every orbit, i.e. deficient totals always cost more clique capacity than they gain. + +| orbit | R | e | α | δ | size | | orbit | R | e | α | δ | size | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| 0 | 1 | 128 | 3 | 0 | 60 | 14 | 2 | 84 | 0 | 2 | 5760 | +| 1 | 1 | 132 | 2 | 1 | 2880 | 15 | 2 | 70 | 0 | 2 | 5760 | +| 2 | 1 | 100 | 2 | 1 | 1440 | 16 | 2 | 70 | 0 | 2 | 5760 | +| 3 | 2 | 128 | 2 | 0 | 360 | 17 | 2 | 76 | 1 | 1 | 2880 | +| 4 | 1 | 84 | 1 | 2 | 1920 | 18 | 3 | 76 | 0 | 1 | 960 | +| 5 | 1 | 76 | 2 | 1 | 960 | 19 | 2 | 24 | 0 | 2 | 720 | +| 6 | 2 | 76 | 1 | 1 | 320 | 20 | 2 | 34 | 0 | 2 | 5760 | +| 7 | 1 | 64 | 3 | 0 | 180 | 21 | 2 | 16 | 0 | 2 | 1440 | +| 8 | 2 | 64 | 2 | 0 | 30 | 22 | 2 | 20 | 1 | 1 | 2880 | +| 9 | 2 | 108 | 0 | 2 | 2880 | 23 | 3 | 20 | 0 | 1 | 480 | +| 10 | 2 | 114 | 0 | 2 | 5760 | 24 | 2 | 0 | 2 | 0 | 90 | +| 11 | 2 | 112 | 0 | 2 | 2880 | 25 | 2 | 0 | 2 | 0 | 360 | +| 12 | 2 | 112 | 0 | 2 | 2880 | 26 | 3 | 0 | 1 | 0 | 60 | +| 13 | 2 | 112 | 0 | 2 | 2880 | | | | | | | + +## 3. c ≥ 21: counting is exhausted; the F2 pattern is what fails (checked: `f2check.py`, `try_cand.py`, `core.py`, `tight_diag.py`) +- For c ≥ 21 the lifted relaxation is feasible already with μ alone (f(9) = 42 = 4·21 − 42, f(10) = 48 ≥ 46, …): no counting + argument of this kind can exclude c ≥ 21. +- `f2check.py` solves the actual F2 problem with every total confined to given line sets (R_j ⊆ X_j, A_j ⊆ Y_j, optional + alternatives such as both orientations of a dumbbell). Calibration: the identity decoder (60 singleton totals) is SAT and + re-checks as a valid decoder. +- **The counting covers are F2-infeasible.** S8's 21-dumbbell cover (fixed orientation: UNSAT immediately — some line has + no total covering its diagonal) and the cover found by `tight_diag.py` (21 dumbbells satisfying clique counting AND + diagonal coverage; free orientation per dumbbell): UNSAT in 0.1 s. +- **Minimal reason** (`core.py`, UNSAT core over per-line assumptions + deletion): a single non-orthogonal pair {23, 26}. + On it the decoder needs B[{a,b},{a,b}] = I₂. Exactly one total covers both diagonals, and the only other total hitting the pair + can give just one off-diagonal entry. Counting is satisfied (2 totals hit the pair), but I₂ − e_a e_bᵀ has rank 2. +- **Exact 2×2 support lemma** (`pairkinds.py`): classify each total on a non-orthogonal pair {a,b} by its kind + (X∩{a,b}, Y∩{a,b}). I₂ is reachable iff the multiset of kinds contains one of **17 minimal patterns**: + 16 two-element patterns (two totals that can supply the two diagonal entries) and the triple + {(ab,ab), (a→b), (b→a)}. This is a local F2 constraint that counting (N_K ≥ |K|) does not capture. + +## Remaining open cases +- c = 19 (cost 42): cases 17, 18 and 20 (running). +- c = 20, cost 41: cases 4, 5 and 6 (running), and 9, 11–23 (not run). +- c = 20, cost 42: every case except −1 and 21. Case 22 is running. +- 21 ≤ c ≤ 42: everything. Suggested next step: a cut loop that adds the exact 2×2 lemma constraints (, written but not run) plus F2 cores to the type ILP. + +## Trust and validation +- Inputs are exact (lines, flats, cliques, hits) with certified rank lower bounds (S5/S8 `hits_table.py`). The relaxation + only uses R as a lower bound on β and hits as upper bounds, so every step is in the safe direction. +- The integer programs are solved in floating point by HiGHS (scipy 1.x `milp`) and have no proof certificates. The μ optima + were cross-checked with CP-SAT. The case ILPs have not been cross-checked independently; one CP-SAT run () is in progress. +- Calibrations (must be feasible, are feasible): c = 21, cost ≤ 42, case "no deficient" (μ = 42); c = 19, cost ≤ 44, + same case. The F2 checker accepts the identity decoder. +- Not used: level tables (S7), the S6 sweep. + +## Files (`scripts/`) +| file | role | +|---|---| +| `lines.py`, `flats.py`, `hits_table.py`, `orbits.py` | S5/S6/S8 exact data: lines, 2271 flats, 640 cliques, hits, certified ranks, the order-11520 group | +| `mu_ilp.py`, `mu_cpsat.py` | μ-packing optima f(k) (HiGHS) and CP-SAT cross-check | +| `types_e.py`, `dom.py`, `lift_prep.py` | type catalog (58,340), orbits (27) and dominance, lifting data (x_t, δ_t) | +| `branch1.py` | one case ILP of the lifted relaxation (`feas` / optimisation form) | +| `vsingle.py` | single-deficient diagnostics | +| `f2check.py`, `try_cand.py`, `core.py`, `tight_diag.py`, `pairkinds.py`, `check_c21cover.py` | F2 confinement SAT, UNSAT cores, tight covering ILP with diagonal coverage, 2×2 lemma, S8's c = 21 cover | + +```sh +PATH=$SP/venv/bin:$PATH sh verify.sh # data + μ lemma + calibrations + F2 checks (~15 min) +FULL=1 PAR=6 PATH=$SP/venv/bin:$PATH sh verify.sh # + every case ILP of Theorems 1-2 (hours) +``` + +## Addendum (2026-10-15, after the agent stopped): final case results +- **c = 19: all 20 lifted cases EXCLUDED.** The last three (cases 17, 18 and 20, 11,280 columns each) finished in the optimisation form with a HiGHS MIP dual bound of 33 < 34 needed, with best incumbents 31, 31 and 30, at about 2,400 s each (`logs/cases_19_42_opt2.log`). CP-SAT on case 20 returned UNKNOWN (`logs/cp_19_20.log`); the HiGHS bound is the one used. **Hence no 19-total Gaussian decoder costs ≤ 42.** HiGHS dual bounds are floating-point MIP bounds; they are not exact certificates. +- **c = 20, partial.** + - At cost ≤ 41: cases −1, 2, 10, 11, 12 and 13 are EXCLUDED. Cases 6 and 9 hit the time limit (OPEN). + - At cost ≤ 42: cases −1 and 21 are EXCLUDED. Cases 19, 22 and 23 hit the time limit (OPEN). + - The remaining heavy cases were not run. See `logs/cases_20_41.log` and `logs/cases_20_42_light.log`. +- `verify.sh` has not been rerun from scratch. diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/expected.txt b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/expected.txt new file mode 100644 index 0000000000..cd207039bf --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/expected.txt @@ -0,0 +1,19 @@ +# completed parts only (non-FULL verify.sh; values recorded from the work-dir runs, not yet from a fresh verify run) +mu-packing: cap 6 max |mu| = 28 (LP 30) +mu-packing: cap 7 max |mu| = 32 (LP 35) +mu-packing: cap 8 max |mu| = 36 (LP 40) +mu-packing: cap 9 max |mu| = 42 (LP 45) +mu-packing: cap 10 max |mu| = 48 (LP 50) +mu-packing: cap 11 max |mu| = 52 (LP 55) +mu-packing: cap 12 max |mu| = 60 (LP 60) +cap 7 |mu|>= 33 INFEASIBLE +cap 8 |mu|>= 37 INFEASIBLE +types: 58340 (R,miss)-distinct rank-maximal types, 27 orbits, 23 non-dominated +calibration c=21 cost<=42 case -1: FEASIBLE ; c=19 cost<=44 case -1: FEASIBLE +identity True True +S8 dumbbell cover, fixed orientation: False +c = 21 dumbbell cover: cost 42 min clique hits 12 max 16 PASS +free orientation: False +minimal infeasible O: 2 [23, 26] +17 minimal feasible kind-multisets: +# case ILPs: see logs/cases_*.log (c=19: 17/20 EXCLUDED; c=20/41: -1,2 EXCLUDED; c=20/42: -1,21 EXCLUDED) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42.log new file mode 100644 index 0000000000..77fc0249f3 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42.log @@ -0,0 +1,13 @@ +c 19 cost<= 42 case -1 e1 None cols 0 status 0 best 31.999999999999616 dual_bound 31.999999999999616 need 34 EXCLUDED 5s +c 19 cost<= 42 case 10 e1 114 cols 0 status 0 best 30.0 dual_bound 30.0 need 34 EXCLUDED 4s +c 19 cost<= 42 case 2 e1 100 cols 5520 status 0 best 31.999999999934616 dual_bound 31.999999999934616 need 34 EXCLUDED 88s +c 19 cost<= 42 case 11 e1 112 cols 1440 status 0 best 31.000000000000007 dual_bound 31.000000000000007 need 34 EXCLUDED 84s +c 19 cost<= 42 case 9 e1 108 cols 4800 status 0 best 31.000000000000014 dual_bound 31.000000000000014 need 34 EXCLUDED 312s +c 19 cost<= 42 case 13 e1 112 cols 1440 status 0 best 29.999999999999993 dual_bound 29.999999999999993 need 34 EXCLUDED 264s +c 19 cost<= 42 case 12 e1 112 cols 1440 status 0 best 31.0 dual_bound 31.0 need 34 EXCLUDED 273s +c 19 cost<= 42 case 4 e1 84 cols 11280 status 1 best 30.999999999999915 dual_bound 33.0 need 34 EXCLUDED 1201s +c 19 cost<= 42 case 5 e1 76 cols 11280 status 1 best 30.99999999999773 dual_bound 33.0 need 34 EXCLUDED 1202s +c 19 cost<= 42 case 6 e1 76 cols 11280 status 1 best 30.999999999998764 dual_bound 33.0 need 34 EXCLUDED 1202s +c 19 cost<= 42 case 14 e1 84 cols 11280 status 1 best 31.00000000000057 dual_bound 33.0 need 34 EXCLUDED 1201s +c 19 cost<= 42 case 15 e1 70 cols 11280 status 1 best 31.000000000010186 dual_bound 32.0 need 34 EXCLUDED 1201s +c 19 cost<= 42 case 16 e1 70 cols 11280 status 1 best 30.00000000000007 dual_bound 33.0 need 34 EXCLUDED 1201s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_feas.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_feas.log new file mode 100644 index 0000000000..8a17423ce5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_feas.log @@ -0,0 +1,7 @@ +c 19 cost<= 42 case 21 e1 16 cols 1440 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 26s +c 19 cost<= 42 case 22 e1 20 cols 4800 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 266s +c 19 cost<= 42 case 23 e1 20 cols 4800 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 267s +c 19 cost<= 42 case 19 e1 24 cols 5520 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 365s +c 19 cost<= 42 case 20 e1 34 cols 11280 feasibility form: status 1 Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) OPEN 2109s +c 19 cost<= 42 case 17 e1 76 cols 11280 feasibility form: status 1 Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) OPEN 2203s +c 19 cost<= 42 case 18 e1 76 cols 11280 feasibility form: status 1 Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) OPEN 2212s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_opt2.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_opt2.log new file mode 100644 index 0000000000..14d2cac35c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_19_42_opt2.log @@ -0,0 +1,3 @@ +c 19 cost<= 42 case 20 e1 34 cols 11280 status 1 best 30.000000000000192 dual_bound 33.0 need 34 EXCLUDED 2401s +c 19 cost<= 42 case 18 e1 76 cols 11280 status 1 best 31.00000000000765 dual_bound 33.0 need 34 EXCLUDED 2402s +c 19 cost<= 42 case 17 e1 76 cols 11280 status 1 best 31.000000000001165 dual_bound 33.0 need 34 EXCLUDED 2402s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_41.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_41.log new file mode 100644 index 0000000000..04102afaa5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_41.log @@ -0,0 +1,10 @@ +c 20 cost<= 41 case -1 e1 None cols 0 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 1s +c 20 cost<= 41 case 2 e1 100 cols 5520 feasibility form: status 2 The problem is infeasible. (HiGHS Status 8: model_status is Infeasible; primal_status is None) EXCLUDED 679s +c 20 cost<= 41 case 4 e1 84 cols 11280 feasibility form: status 1 Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) OPEN 10438s +c 20 cost<= 41 case 5 e1 76 cols 11280 feasibility form: status 1 Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) OPEN 10442s +c=20 cost<=41 case 10 e1 114 cols 0 -> EXCLUDED (infeasible) 0s +c=20 cost<=41 case 11 e1 112 cols 1440 -> EXCLUDED (infeasible) 316s +c=20 cost<=41 case 12 e1 112 cols 1440 -> EXCLUDED (infeasible) 683s +c=20 cost<=41 case 13 e1 112 cols 1440 -> EXCLUDED (infeasible) 2932s +c=20 cost<=41 case 9 e1 108 cols 4800 -> OPEN: Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) 12882s +c=20 cost<=41 case 6 e1 76 cols 11280 -> OPEN: Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) 31851s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_42_light.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_42_light.log new file mode 100644 index 0000000000..5c6a33307c --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cases_20_42_light.log @@ -0,0 +1,5 @@ +c=20 cost<=42 case -1 e1 None cols 0 -> EXCLUDED (infeasible) 3s +c=20 cost<=42 case 21 e1 16 cols 1440 -> EXCLUDED (infeasible) 683s +c=20 cost<=42 case 22 e1 20 cols 4800 -> OPEN: Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) 3814s +c=20 cost<=42 case 23 e1 20 cols 4800 -> OPEN: Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) 5047s +c=20 cost<=42 case 19 e1 24 cols 5520 -> OPEN: Time limit reached. (HiGHS Status 13: model_status is Time limit reached; primal_status is None) 13885s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cp_19_20.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cp_19_20.log new file mode 100644 index 0000000000..e0ccfa5edb --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/cp_19_20.log @@ -0,0 +1 @@ +CP-SAT c=19 cost<=42 case 20 e1 34 cols 11280 -> UNKNOWN 2400s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_7.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_7.log new file mode 100644 index 0000000000..b1c95628be --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_7.log @@ -0,0 +1 @@ +cap 7 |mu|>= 33 INFEASIBLE 3s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_8.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_8.log new file mode 100644 index 0000000000..40e7f74f07 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_cpsat_8.log @@ -0,0 +1 @@ +cap 8 |mu|>= 37 INFEASIBLE 170s diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_ilp_9_12.log b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_ilp_9_12.log new file mode 100644 index 0000000000..36d96aff25 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/logs/mu_ilp_9_12.log @@ -0,0 +1,8 @@ +12 0 Optimization terminated successfully. (HiGHS Status 7: Optimal) 60.000000000000014 bound -60.000000000000014 0s +[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] +10 0 Optimization terminated successfully. (HiGHS Status 7: Optimal) 48.00000000000016 bound -48.00000000000016 3s +[1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0] +9 0 Optimization terminated successfully. (HiGHS Status 7: Optimal) 41.99999999999998 bound -41.99999999999998 5s +[0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1] +11 0 Optimization terminated successfully. (HiGHS Status 7: Optimal) 52.00000000000023 bound -52.00000000000023 6s +[1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch1.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch1.py new file mode 100644 index 0000000000..2cc25369d5 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch1.py @@ -0,0 +1,42 @@ +"""Exact case split for c totals, cost <= cm (relaxation 'lifted packing'). Non-tight totals with delta>0 ('deficient'); +case 0: none -> pure mu ILP. Otherwise WLOG (line group) the deficient total of largest e is the orbit representative t1; +the other deficient totals have e <= min(e1, slack - e1). Each case: max |mu| + sum delta over +M mu + y_t1 + sum n_t y_t <= cap, sum n_t e_t <= slack - e1, mu in Z^60, n in Z. Exclusion iff every case < 4c - cm. +argv: c cm case(orbit index or -1) timelimit""" +import pickle, sys, numpy as np, time +from scipy.optimize import milp, LinearConstraint, Bounds +from scipy.sparse import csr_matrix, hstack, vstack +c=int(sys.argv[1]); cm=int(sys.argv[2]); case=int(sys.argv[3]); tl=float(sys.argv[4]) +cap=c-12; slack=128*(cm+c)-7680; need=4*c-cm +C=pickle.load(open('hits_table.pkl','rb'))['C'] +M=np.zeros((640,60)) +for k,cl in enumerate(C): M[k,cl]=1 +T=pickle.load(open('types_e256.pkl','rb')); L=pickle.load(open('lift.pkl','rb')); dom=pickle.load(open('dom.pkl','rb')); orb=dom['orb'] +fixed=np.zeros(640); base=0; e1=None +if case>=0: + r=dom['reps'][case]; a,b,R,e1,m=T[r]; x,d=L[r]; assert d>0 and e1<=slack + fixed=m-M[:,x].sum(1); base=d+len(x) # t1's own x lines are not in mu: count alpha directly + cols=[i for i in range(len(T)) if L[i][1]>0 and T[i][3]<=min(e1,slack-e1)] +else: cols=[] +n=len(cols) +Y=csr_matrix(np.array([T[i][4]-M[:,L[i][0]].sum(1) for i in cols]).T) if n else csr_matrix((640,0)) +dl=np.array([L[i][1]+len(L[i][0]) for i in cols],float) # gain g_t = alpha + delta (x lines counted here, not in mu) +el=np.array([T[i][3] for i in cols],float) +# note: for deficient types we count their x lines inside their own column (load M x + y = m_t): exact miss vector +Ycol=csr_matrix(np.array([T[i][4] for i in cols],float).T) if n else csr_matrix((640,0)) +obj=-np.concatenate([np.ones(60),dl]) +FEAS=len(sys.argv)>5 and sys.argv[5]=='feas' # feasibility form: add row |mu| + sum g >= need - base +A=vstack([hstack([csr_matrix(M),Ycol]),csr_matrix(np.concatenate([np.zeros(60),el])[None,:]),csr_matrix(obj[None,:])]) +ub=np.concatenate([cap-T[dom['reps'][case]][4] if case>=0 else np.full(640,cap),[slack-(e1 or 0)],[-(need-base) if FEAS else np.inf]]) +t0=time.time() +res=milp(obj,constraints=[LinearConstraint(A,-np.inf,ub)],integrality=np.ones(60+n),bounds=Bounds(0,cap),options={'time_limit':tl}) +val=None if res.x is None else -res.fun+base +bound=-res.mip_dual_bound+base if getattr(res,'mip_dual_bound',None) is not None else None +if FEAS: + print('c=%d cost<=%d case %d e1 %s cols %d -> %s'%(c,cm,case,e1,n,'EXCLUDED (infeasible)' if res.status==2 else ('FEASIBLE' if res.x is not None else 'OPEN: '+res.message)),'%.0fs'%(time.time()-t0),flush=True) + if res.x is not None: xx=np.round(res.x).astype(int); print(' mu',xx[:60].tolist(),'deficient',[(cols[k],int(orb[cols[k]]),xx[60+k]) for k in range(n) if xx[60+k]]) + sys.exit() +ok=(res.status==0 and val %s (opt: best %s, dual bound %s, need %d)'%(c,cm,case,e1,n,'EXCLUDED' if ok else 'OPEN',None if val is None else round(val),None if bound is None else round(bound,3),need),'%.0fs'%(time.time()-t0),flush=True) +if res.x is not None and val>=need: + xx=np.round(res.x).astype(int); print(' mu',xx[:60].tolist(),'deficient',[(cols[k],int(orb[cols[k]]),xx[60+k]) for k in range(n) if xx[60+k]]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch_cp.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch_cp.py new file mode 100644 index 0000000000..964c2722f4 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/branch_cp.py @@ -0,0 +1,24 @@ +"""CP-SAT version of branch1.py feasibility form (same case split, same columns). argv: c cm case timelimit workers""" +import pickle, sys, numpy as np, time +from ortools.sat.python import cp_model +c=int(sys.argv[1]); cm=int(sys.argv[2]); case=int(sys.argv[3]); tl=float(sys.argv[4]); wk=int(sys.argv[5]) +cap=c-12; slack=128*(cm+c)-7680; need=4*c-cm +C=pickle.load(open('hits_table.pkl','rb'))['C'] +T=pickle.load(open('types_e256.pkl','rb')); L=pickle.load(open('lift.pkl','rb')); dom=pickle.load(open('dom.pkl','rb')) +base=0; e1=0; load0=np.zeros(640,int) +if case>=0: + r=dom['reps'][case]; a,b,R,e1,m=T[r]; x,d=L[r]; assert d>0 and e1<=slack + base=4-R; load0=m.astype(int) + cols=[i for i in range(len(T)) if L[i][1]>0 and T[i][3]<=min(e1,slack-e1)] +else: cols=[] +md=cp_model.CpModel() +mu=[md.NewIntVar(0,cap,'m%d'%t) for t in range(60)] +nv=[md.NewIntVar(0,min(cap,(slack-e1)//T[i][3]),'n%d'%i) for i in cols] +for q,cl in enumerate(C): + terms=[mu[t] for t in cl]+[nv[k] for k,i in enumerate(cols) if T[i][4][q]] + md.Add(sum(terms)<=cap-int(load0[q])) +md.Add(sum(T[i][3]*nv[k] for k,i in enumerate(cols))<=slack-e1) +md.Add(sum(mu)+sum((4-T[i][2])*nv[k] for k,i in enumerate(cols))>=need-base) +s=cp_model.CpSolver(); s.parameters.num_workers=wk; s.parameters.max_time_in_seconds=tl +t=time.time(); res=s.Solve(md) +print('CP-SAT c=%d cost<=%d case %d e1 %s cols %d -> %s'%(c,cm,case,e1,len(cols),{'INFEASIBLE':'EXCLUDED (infeasible)'}.get(s.StatusName(res),s.StatusName(res))),'%.0fs'%(time.time()-t),flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/check_c21cover.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/check_c21cover.py new file mode 100644 index 0000000000..9237a61705 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/check_c21cover.py @@ -0,0 +1,11 @@ +"""Exact check: the 21 dumbbell totals (l^perp, m^perp), l ⊥ m (beta = 2 each, cost 42) found by tight_ilp.py 21 hit every +12-clique >= 12 times, i.e. the clique-covering relaxation is integrally feasible at c = 21, cost 42.""" +import pickle, numpy as np +from lines import views +S=views()['Gauss']; NO=S.NO +C=pickle.load(open('hits_table.pkl','rb'))['C'] +D=[(0,13),(4,22),(5,50),(6,19),(6,28),(7,38),(8,19),(11,56),(12,17),(18,37),(18,58),(20,30),(21,33),(25,26),(35,55),(36,45),(37,47),(41,44),(42,51),(44,51),(48,53)] +perp=lambda l: {T for T in range(60) if T!=l and not NO[l,T]} +assert all(not NO[a,b] and a!=b for a,b in D) +N=[sum(1 for a,b in D if perp(a)&set(c) and perp(b)&set(c)) for c in C] +print('c = 21 dumbbell cover: cost', 2*len(D), 'min clique hits', min(N), 'max', max(N), 'PASS' if min(N)>=12 else 'FAIL') diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/core.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/core.py new file mode 100644 index 0000000000..ab92a4032e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/core.py @@ -0,0 +1,47 @@ +"""Minimal line set O on which the candidate's confined decoder is infeasible (UNSAT core over per-line assumptions + deletion).""" +from pysat.formula import IDPool +from pysat.solvers import Solver +from f2check import NO +def build(opts): + pool=IDPool(); cl=[]; W={}; U={} + act=[pool.id(('a',T)) for T in range(60)] + for j,alts in enumerate(opts): + sels=[pool.id(('s',j,k)) for k in range(len(alts))]; cl.append(sels) + for a in range(len(sels)): + for b in range(a+1,len(sels)): cl.append([-sels[a],-sels[b]]) + for T in set().union(*[set(X) for X,Y in alts]): + W[T,j]=pool.id(('w',T,j)); cl.append([-W[T,j]]+[sels[k] for k,(X,Y) in enumerate(alts) if T in X]) + for T in set().union(*[set(Y) for X,Y in alts]): + U[T,j]=pool.id(('u',T,j)); cl.append([-U[T,j]]+[sels[k] for k,(X,Y) in enumerate(alts) if T in Y]) + def AND(a,b): + z=pool.id(('and',a,b)); cl.extend([[-z,a],[-z,b],[z,-a,-b]]); return z + def XOR(a,b): + z=pool.id(('xor',a,b)); cl.extend([[-z,a,b],[-z,-a,-b],[z,-a,b],[z,a,-b]]); return z + for T in range(60): + for S_ in range(60): + if not (T==S_ or NO[T,S_]): continue + terms=[AND(W[T,j],U[S_,j]) for j in range(len(opts)) if (T,j) in W and (S_,j) in U] + g=[-act[T]]+([-act[S_]] if S_!=T else []) + if not terms: + if T==S_: cl.append(g) + continue + acc=terms[0] + for t in terms[1:]: acc=XOR(acc,t) + cl.append(g+([acc] if T==S_ else [-acc])) + return cl,act +def min_core(opts,solver='cadical153'): + cl,act=build(opts) + with Solver(name=solver,bootstrap_with=cl) as s: + if s.solve(assumptions=act): return None + core=sorted(act.index(x) for x in s.get_core()) + cur=list(core) + for T in list(core): + trial=[x for x in cur if x!=T] + if not s.solve(assumptions=[act[x] for x in trial]): cur=trial + return cur +if __name__=='__main__': + import sys + perp=lambda l:[T for T in range(60) if T!=l and not NO[l,T]] + D=eval(sys.argv[1]) + O=min_core([[(perp(a),perp(b)),(perp(b),perp(a))] for a,b in D]) + print('minimal infeasible O:',len(O),O) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/dom.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/dom.py new file mode 100644 index 0000000000..9f43338113 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/dom.py @@ -0,0 +1,34 @@ +"""Dominance reduction of the miss-set type catalog: t is dominated if some t' has g' >= g and m' subset of m (t' != t). +Orbits under the line group (acting on cliques) are used to test one representative per orbit.""" +import pickle, numpy as np +from collections import Counter +T=pickle.load(open('types_e256.pkl','rb')) +G=np.array(pickle.load(open('orbits.pkl','rb'))['G']) +C=pickle.load(open('hits_table.pkl','rb'))['C'] +ci={frozenset(c):k for k,c in enumerate(C)} +Cperm=np.array([[ci[frozenset(int(g[t]) for t in c)] for c in C] for g in G]) # (|G|,640) +mis=np.array([t[4] for t in T]); g=np.array([4-t[2] for t in T]); e=np.array([t[3] for t in T]) +key={ (int(g[i]),mis[i].tobytes()):i for i in range(len(T))} +orb=-np.ones(len(T),int); reps=[] +for i in range(len(T)): + if orb[i]>=0: continue + reps.append(i) + idx=np.nonzero(mis[i])[0] + for p in Cperm: + m2=np.zeros(640,bool); m2[p[idx]]=True + j=key.get((int(g[i]),m2.tobytes())) + assert j is not None + orb[j]=len(reps)-1 +print('types',len(T),'orbits',len(reps)) +size=mis.sum(1) +dominated=[] +for r in reps: + # t' with g'>=g, m' subset m, t' != r + sub=~(mis[:, ~mis[r]].any(1)) # m' within m_r + cand=np.nonzero(sub & (g>=g[r]) & (np.arange(len(T))!=r))[0] + dominated.append(len(cand)>0) +keep=np.array([not dominated[orb[i]] for i in range(len(T))]) +print('orbit reps non-dominated',sum(not d for d in dominated)) +for k,r in enumerate(reps): + print(k,'R',T[r][2],'e',T[r][3],'|m|',int(size[r]),'orbit',int((orb==k).sum()),'dominated' if dominated[k] else 'KEEP') +pickle.dump({'orb':orb,'reps':reps,'keep':keep,'Cperm':Cperm},open('dom.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/f2check.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/f2check.py new file mode 100644 index 0000000000..9444d66270 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/f2check.py @@ -0,0 +1,93 @@ +"""F2 feasibility of a decoder whose totals are confined to given line sets: R_j <= X_j, A_j <= Y_j. +B = sum_j w_j u_j^T must have B_TT = 1 and B_TS = 0 for K-non-orthogonal T != S. Returns SAT model or None.""" +import sys, itertools +from pysat.formula import IDPool +from pysat.solvers import Solver +from lines import views +S = views()['Gauss']; NO = S.NO + +def encode(pairs, extra_or=None): + pool = IDPool(); cl = [] + W = {}; U = {} + for j, (X, Y) in enumerate(pairs): + for T in X: W[T, j] = pool.id(('w', T, j)) + for T in Y: U[T, j] = pool.id(('u', T, j)) + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + for T in range(60): + for S_ in range(60): + if not (T == S_ or NO[T, S_]): continue + terms = [AND(W[T, j], U[S_, j]) for j in range(len(pairs)) if (T, j) in W and (S_, j) in U] + if not terms: + if T == S_: return None, None, None, None + continue + acc = terms[0] + for t in terms[1:]: acc = XOR(acc, t) + cl.append([acc] if T == S_ else [-acc]) + return cl, pool, W, U + +def solve(pairs, solver='cadical153'): + cl, pool, W, U = encode(pairs) + if cl is None: return False, None + with Solver(name=solver, bootstrap_with=cl) as s: + r = s.solve() + if not r: return False, None + mod = set(x for x in s.get_model() if x > 0) + R = [sorted(T for T in X if W[T, j] in mod) for j, (X, Y) in enumerate(pairs)] + A = [sorted(T for T in Y if U[T, j] in mod) for j, (X, Y) in enumerate(pairs)] + return True, (R, A) + +def check_decoder(R, A): + import numpy as np + B = np.zeros((60, 60), int) + for r, a in zip(R, A): + for T in r: + for S_ in a: B[T, S_] ^= 1 + ok = all(B[T, T] == 1 for T in range(60)) and all(B[T, S_] == 0 for T in range(60) for S_ in range(60) if T != S_ and NO[T, S_]) + return ok + +if __name__ == '__main__': + # calibration: identity decoder (60 singleton totals) and the 21-dumbbell cover of S8 + ok, sol = solve([([T], [T]) for T in range(60)]); print('identity', ok, check_decoder(*sol)) + perp = lambda l: [T for T in range(60) if T != l and not NO[l, T]] + D = [(0,13),(4,22),(5,50),(6,19),(6,28),(7,38),(8,19),(11,56),(12,17),(18,37),(18,58),(20,30),(21,33),(25,26),(35,55),(36,45),(37,47),(41,44),(42,51),(44,51),(48,53)] + import time; t = time.time() + ok, sol = solve([(perp(a), perp(b)) for a, b in D]); print('S8 dumbbell cover, fixed orientation:', ok, '%.1fs' % (time.time() - t)) + +def solve_opts(opts, solver='cadical153', assumptions=()): + """opts[j] = list of (X, Y) alternatives for total j (one is selected).""" + pool = IDPool(); cl = [] + W = {}; U = {}; SEL = {} + for j, alts in enumerate(opts): + sels = [pool.id(('s', j, k)) for k in range(len(alts))] + cl.append(sels) + for a in range(len(sels)): + for b in range(a + 1, len(sels)): cl.append([-sels[a], -sels[b]]) + SEL[j] = sels + allX = set().union(*[set(X) for X, Y in alts]); allY = set().union(*[set(Y) for X, Y in alts]) + for T in allX: + W[T, j] = pool.id(('w', T, j)); cl.append([-W[T, j]] + [sels[k] for k, (X, Y) in enumerate(alts) if T in X]) + for T in allY: + U[T, j] = pool.id(('u', T, j)); cl.append([-U[T, j]] + [sels[k] for k, (X, Y) in enumerate(alts) if T in Y]) + def AND(a, b): + z = pool.id(('and', a, b)); cl.extend([[-z, a], [-z, b], [z, -a, -b]]); return z + def XOR(a, b): + z = pool.id(('xor', a, b)); cl.extend([[-z, a, b], [-z, -a, -b], [z, -a, b], [z, a, -b]]); return z + for T in range(60): + for S_ in range(60): + if not (T == S_ or NO[T, S_]): continue + terms = [AND(W[T, j], U[S_, j]) for j in range(len(opts)) if (T, j) in W and (S_, j) in U] + if not terms: + if T == S_: return False, None + continue + acc = terms[0] + for t in terms[1:]: acc = XOR(acc, t) + cl.append([acc] if T == S_ else [-acc]) + with Solver(name=solver, bootstrap_with=cl) as s: + if not s.solve(): return False, None + mod = set(x for x in s.get_model() if x > 0) + R = [sorted(T for (T, jj), v in W.items() if jj == j and v in mod) for j in range(len(opts))] + A = [sorted(T for (T, jj), v in U.items() if jj == j and v in mod) for j in range(len(opts))] + return True, (R, A) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/flats.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/flats.py new file mode 100644 index 0000000000..b064c81896 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/flats.py @@ -0,0 +1,26 @@ +"""All flats (line sets of K-subspaces spanned by configuration lines) of a view, with dims. Exact.""" +import sys, pickle; sys.path.insert(0,sys.path[0]) +from lines import * +def flats(S): + v=S.v; out={} + def close(idx): + d=S.arank(idx); return frozenset(T for T in range(v) if S.arank(list(idx)+[T])==d), d + level=[ (frozenset([T]),1) for T in range(v)] + for f,d in level: out[f]=d + frontier=list(out.items()) + while frontier: + new={} + for f,d in frontier: + if d>=S.w: continue + for T in range(v): + if T in f or any(T in g and f= 40 (x_K >= 2/3). + +Argument. C ranges over the 640 twelve-line sets of pairwise K-non-orthogonal lines. On C x C the pattern forces +B[C,C] = I_12 (diagonal 1, all off-diagonal pairs non-orthogonal), and each total contributes a rank-1 F2 matrix there, +so at least 12 totals hit C on both sides (R_j & C != {} and A_j & C != {}). Summing: sum_j hit(R_j, A_j) >= 12*640. +For fixed spans X = span R_j, Y = span A_j the hit count is largest for R = lines(X), A = lines(Y) (flats), and this +script shows hit(X, Y) <= 192 * rank(X, Y) for all 2271^2 flat pairs, rank = K-rank of the pairing (Theorem C cost). +Hence sum_j beta_j >= 12*640/192 = 40. Binding pair: X = 6-line 2-flat containing l, Y = the 15-line hyperplane l^perp +(pairing rank 1, 192 hits). Mixing all clique sizes, singletons, non-orthogonal pairs and the whole set in an LP over +uniform orbit weights does not improve 40 (gauss_lp3.py in the work dir). +Ranks: exact integer pairing matrices, rank mod p = 2^31-1 (a certified lower bound on the rank over Q).""" +import sys, itertools; sys.path.insert(0, sys.path[0]) +from lines import * +from flats import flats +import networkx as nx + +def main(): + W = views(); S = W['Gauss']; J = W['_J']; v = S.v + F = flats(S); fl = sorted(F.items(), key=lambda t: (t[1], len(t[0]), sorted(t[0]))) + n = len(fl); assert n == 2271 + g = nx.Graph([(i, j) for i in range(v) for j in range(i+1, v) if S.NO[i, j]]) + C = [c for c in nx.find_cliques(g) if len(c) == 12] + assert len(C) == 640 and all(S.NO[a, b] for c in C for a, b in itertools.combinations(c, 2)) + Lo = np.zeros((n, v), np.int64) + for i, (f, d) in enumerate(fl): Lo[i, sorted(f)] = 1 + inc = np.zeros((v, len(C)), np.int64) + for k, c in enumerate(C): inc[c, k] = 1 + meet = (Lo @ inc > 0).astype(np.int64); hits = meet @ meet.T # exact integers + # exact integer bases (vectors r_T, J r_T), pairing ranks by batched elimination mod p = 2^31-1 + # (rank mod p <= rank over Q, so these are certified LOWER bounds; the claim only needs lower bounds) + def ibasis(f): + rows = [] + for T in sorted(f): + for x in (S.reps[T], J @ S.reps[T]): + if rank_q(rows + [x]) > len(rows): rows.append(x) + return np.array(rows, np.int64) + Bs = [ibasis(f) for f, d in fl] + assert all(len(b) == 2 * d for b, (f, d) in zip(Bs, fl)) + P = 2147483647 + def modrank(A): + A = A % P; N, r, c = A.shape; used = np.zeros((N, r), bool); idx = np.arange(N) + for col in range(c): + cand = (A[:, :, col] != 0) & ~used; has = cand.any(1) + if not has.any(): continue + pr = cand.argmax(1); it = idx[has]; prow = A[it, pr[has]] + inv = np.array([pow(int(x), P - 2, P) for x in prow[:, col]], np.int64) if len(it) < 64 else modinv(prow[:, col]) + prow = (prow * inv[:, None]) % P + fac = A[it, :, col].copy(); fac[np.arange(len(it)), pr[has]] = 0 + A[it] = (A[it] - (fac[:, :, None] * prow[:, None, :]) % P) % P + used[it, pr[has]] = True + return used.sum(1) + def modinv(x): + res = np.ones_like(x); base = x % P; e = P - 2 + while e: + if e & 1: res = (res * base) % P + base = (base * base) % P; e >>= 1 + return res + R = np.zeros((n, n), np.int64) + groups = {} + for i, b in enumerate(Bs): groups.setdefault(len(b), []).append(i) + for da, ia in groups.items(): + A = np.stack([Bs[i] for i in ia]) + for db, ib in groups.items(): + Bm = np.stack([Bs[i] for i in ib]) + M = np.einsum('aik,bjk->abij', A, Bm).reshape(-1, da, db) + r = modrank(M).reshape(len(ia), len(ib)); assert (r % 2 == 0).all() + R[np.ix_(ia, ib)] = r // 2 + import pickle; pickle.dump({'fl':fl,'R':R,'hits':hits,'C':C},open('hits_table.pkl','wb')) + print(f'PASS gauss.ranks: certified (mod p) pairing-rank lower bounds for all {n}x{n} flat pairs, histogram {np.bincount(R.flatten()).tolist()}') + assert (hits <= 192 * R).all() + worst = np.argwhere((hits == 192 * R) & (R > 0)) + print('PASS gauss.hits: hit(X,Y) <= 192 * rank(X,Y) on every flat pair (exact hits, certified ranks)') + a, b = worst[0] + print(f' binding example: |X| = {len(fl[a][0])} (dim {fl[a][1]}), |Y| = {len(fl[b][0])} (dim {fl[b][1]}), rank {R[a,b]}, hits {hits[a,b]}; {len(worst)} binding pairs') + print('RESULT gauss: every decoder has sum_j beta_j >= 12*640/192 = 40 => x_K >= 2/3 (threshold to exclude: 43)') + +if __name__ == '__main__': + main() diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lift_prep.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lift_prep.py new file mode 100644 index 0000000000..fd0f131260 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lift_prep.py @@ -0,0 +1,27 @@ +"""Per type: Z_t (lines all of whose cliques are missed), x_t max pairwise-orthogonal subset, delta, y_t. Cached to lift.pkl""" +import pickle, numpy as np +from f2check import NO +C=pickle.load(open('hits_table.pkl','rb'))['C'] +M=np.zeros((640,60),bool) +for k,cl in enumerate(C): M[k,cl]=True +T=pickle.load(open('types_e256.pkl','rb')) +orth=~NO; np.fill_diagonal(orth,False) +cache={} +def maxorth(Z): + k=tuple(Z) + if k in cache: return cache[k] + best=[] + def ext(cl,cand): + nonlocal best + if len(cl)>len(best): best=cl + for i,v in enumerate(cand): ext(cl+[v],[w for w in cand[i+1:] if orth[v,w]]) + ext([],list(Z)); cache[k]=best; return best +mis=np.array([t[4] for t in T]) +Zall=~((~mis[:,:,None])&M[None,:,:]).any(1) # (n,60): no hit clique through line +out=[] +for i,(a,b,R,e,m) in enumerate(T): + x=maxorth(list(np.nonzero(Zall[i])[0])); d=(4-R)-len(x) + out.append((x,d)) +pickle.dump(out,open('lift.pkl','wb')) +from collections import Counter +print(Counter((T[i][2],T[i][3],len(x),d) for i,(x,d) in enumerate(out))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lines.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lines.py new file mode 100644 index 0000000000..fc4eac011d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/lines.py @@ -0,0 +1,166 @@ +"""Self-contained exact construction of the E8 root lines over Q(i), Q(omega) and the Hurwitz quaternions. + +Real model: the 240 E8 roots in doubled standard coordinates (norm 8; in {0,+-4,+-8}). + * Gaussian view : J = left multiplication by i on R^8 = H^2 (coords 1,i,j,k per block); J preserves E8. + Hermitian Gram hG(x,y) = ( + i )/4 in Z[i] (diag 2). + * Eisenstein view: rho = fixed-point-free order-3 isometry of E8 (product of Coxeter elements of 4 orthogonal A2's). + Sesquilinear Gram hE(x,y) = ( - w )/4 in Z[w] (w^2+w+1=0; diag 2+w, a unit at 2). + * Hurwitz view : left multiplication by i, j, k; quaternionic lines of real dim 4. +Elements of Z[i] / Z[w] are stored as integer pairs (a, b) = a + b*i / a + b*w. +Everything is exact integer arithmetic; asserts check every structural claim used downstream. +""" +import itertools +from fractions import Fraction +import numpy as np + + +def e8_roots(): + R = [] + for i, j in itertools.combinations(range(8), 2): + for si in (2, -2): + for sj in (2, -2): + r = [0] * 8; r[i], r[j] = si, sj; R.append(r) + for s in itertools.product((1, -1), repeat=8): + if s.count(-1) % 2 == 0: + R.append(list(s)) + R = np.array(R, dtype=np.int64) + assert len(R) == 240 and set((R * R).sum(1)) == {8} + return R + + +def rank_q(rows): + """exact rank over Q of an integer matrix (fraction-free Gaussian elimination).""" + M = [list(map(int, r)) for r in rows] + if not M: + return 0 + n, m = len(M), len(M[0]); r = 0 + for c in range(m): + p = next((i for i in range(r, n) if M[i][c] != 0), None) + if p is None: + continue + M[r], M[p] = M[p], M[r] + for i in range(r + 1, n): + if M[i][c]: + f, g = M[i][c], M[r][c] + M[i] = [g * a - f * b for a, b in zip(M[i], M[r])] + r += 1 + if r == n: + break + return r + + +def left_quat(): + Li = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 0, -1], [0, 0, 1, 0]]) + Lj = np.array([[0, 0, -1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, -1, 0, 0]]) + Lk = Li @ Lj + blk = lambda M: np.kron(np.eye(2, dtype=np.int64), M).astype(np.int64) + return blk(Li), blk(Lj), blk(Lk) + + +def a2_rotation(roots): + """rho, an order-3 fixed-point-free isometry preserving E8 (returned as an exact integer matrix 4*rho).""" + chosen = [] + for a in roots: + if len(chosen) == 4: + break + if any(np.dot(a, x) != 0 for p in chosen for x in p): + continue + for b in roots: + if np.dot(a, b) == -4 and all(np.dot(a, x) == 0 and np.dot(b, x) == 0 for p in chosen for x in p): + chosen.append((a, b)); break + assert len(chosen) == 4 + g = np.eye(8, dtype=object) + for a, b in chosen: + sa = np.eye(8, dtype=object) * 8 - 2 * np.outer(a, a).astype(object) + sb = np.eye(8, dtype=object) * 8 - 2 * np.outer(b, b).astype(object) + g = g.dot(sa).dot(sb) + D = 8 ** 8 // 4 + assert all(x % D == 0 for x in g.flatten()) + g = (g // D).astype(np.int64) + I = np.eye(8, dtype=np.int64) + assert (g @ g @ g == 64 * I).all() and (g @ g + 4 * g + 16 * I == 0).all() + return g + + +def lines_over(roots, ops): + """group roots into A-lines, A spanned by the integer matrices ops (identity first; ops may be scaled by 4). + Returns representatives (first root met of each line, in root order).""" + S = {tuple(r) for r in roots} + reps, seen = [], set() + for r in roots: + if tuple(r) in seen: + continue + span = [B @ r for B in ops] + assert rank_q(span) == len(ops) + reps.append(r) + members = [s for s in roots if rank_q(span + [s]) == len(ops)] + for s in members: + seen.add(tuple(s)) + return np.array(reps) + + +class View: + """name, reps (v x 8), ops (real matrices spanning A, scaled so that ops[i] @ root is a root times scale[i]).""" + def __init__(self, name, reps, ops, d): + self.name, self.reps, self.ops, self.d = name, reps, ops, d + self.v = len(reps) + NO = np.zeros((self.v, self.v), bool) + for B in ops: + NO |= (reps @ B @ reps.T) != 0 + np.fill_diagonal(NO, False) + self.NO = NO # A-non-orthogonal pairs (the decoder pattern forces B_TS = 0 there) + self.w = self.arank(range(self.v)) + + def arank(self, idx): + rows = [B @ self.reps[i] for i in idx for B in self.ops] + r = rank_q(rows) if rows else 0 + assert r % self.d == 0 + return r // self.d + + def pair_rank(self, R, A): + """A-rank of the pairing span(R) x span(A) = real rank of the real Gram block / dim A (Theorem C cost).""" + BR = [B @ self.reps[i] for i in R for B in self.ops] + BA = [B @ self.reps[i] for i in A for B in self.ops] + if not BR or not BA: + return 0 + M = [[int(x @ y) for y in BA] for x in BR] + r = rank_q(M); assert r % self.d == 0 + return r // self.d + + +def views(): + roots = e8_roots(); I8 = np.eye(8, dtype=np.int64) + Li, Lj, Lk = left_quat() + S = {tuple(r) for r in roots} + for M in (Li, Lj, Lk): + assert all(tuple(M @ r) in S for r in roots) + g = a2_rotation(roots) # 4*rho + assert all(tuple(g @ r // 4) in S and ((g @ r) % 4 == 0).all() for r in roots) + rho = g # 4*rho (exact integer matrix); ranks are scale-free + out = {} + out['Gauss'] = View('Gauss', lines_over(roots, [I8, Li]), [I8, Li], 2) + out['Eisen'] = View('Eisen', lines_over(roots, [I8, rho]), [I8, rho], 2) + out['Hurwitz'] = View('Hurwitz', lines_over(roots, [I8, Li, Lj, Lk]), [I8, Li, Lj, Lk], 4) + out['_J'], out['_rho'] = Li, rho + return out + + +def gauss_gram(V, J): + """hG(r_T, r_S) = ( + i)/4 as integer pairs (a, b).""" + A = V @ V.T; Bm = V @ J @ V.T + assert (A % 4 == 0).all() and (Bm % 4 == 0).all() + return A // 4, Bm // 4 + + +def eisen_gram(V, rho4): + """hE(r_T, r_S) = ( - w)/4 as integer pairs (a, b) = a + b w (rho4 = 4*rho).""" + A = V @ V.T; Bm = V @ rho4 @ V.T + assert (A % 4 == 0).all() and (Bm % 16 == 0).all() + return A // 4, -(Bm // 16) + + +if __name__ == '__main__': + W = views() + for nm in ('Gauss', 'Eisen', 'Hurwitz'): + S = W[nm] + print(nm, 'v', S.v, 'w', S.w, 'nonorth degree', sorted(set(S.NO.sum(1)))) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_cpsat.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_cpsat.py new file mode 100644 index 0000000000..0a408ac9e6 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_cpsat.py @@ -0,0 +1,10 @@ +"""Independent cross-check of the mu-packing optimum with CP-SAT: is there mu in Z>=0^60 with M mu <= cap and |mu| >= target?""" +import pickle, sys, numpy as np, time +from ortools.sat.python import cp_model +C=pickle.load(open('hits_table.pkl','rb'))['C'] +cap=int(sys.argv[1]); target=int(sys.argv[2]); wk=int(sys.argv[3]) if len(sys.argv)>3 else 2 +m=cp_model.CpModel(); mu=[m.NewIntVar(0,cap,'m%d'%i) for i in range(60)] +for cl in C: m.Add(sum(mu[t] for t in cl)<=cap) +m.Add(sum(mu)>=target) +s=cp_model.CpSolver(); s.parameters.num_workers=wk; s.parameters.max_time_in_seconds=3000 +t=time.time(); r=s.Solve(m); print('cap',cap,'|mu|>=',target,s.StatusName(r),'%.0fs'%(time.time()-t),flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_ilp.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_ilp.py new file mode 100644 index 0000000000..2c37ad6662 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/mu_ilp.py @@ -0,0 +1,10 @@ +import pickle, sys, numpy as np, time +from scipy.optimize import milp, LinearConstraint, Bounds +C=pickle.load(open('hits_table.pkl','rb'))['C'] +M=np.zeros((640,60)); +for k,c in enumerate(C): M[k,c]=1 +cap=int(sys.argv[1]) +t=time.time() +r=milp(-np.ones(60),constraints=[LinearConstraint(M,-np.inf,cap)],integrality=np.ones(60),bounds=Bounds(0,cap),options={'disp':False,'time_limit':float(sys.argv[2])}) +print(cap, r.status, r.message, -r.fun if r.x is not None else None, 'bound', getattr(r,'mip_dual_bound',None), '%.0fs'%(time.time()-t)) +if r.x is not None: print(np.round(r.x).astype(int).tolist()) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/orbits.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/orbits.py new file mode 100644 index 0000000000..11e4c31837 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/orbits.py @@ -0,0 +1,53 @@ +"""Permutation group of the 60 Gaussian-E8 lines generated by the 60 complex reflections s_r s_{Jr}; orbits on the 1500 3-flats.""" +import sys, pickle +sys.path.insert(0, sys.path[0]) +import numpy as np +from lines import views, e8_roots + +def line_index(S): + J = S.ops[1]; idx = {} + for T, r in enumerate(S.reps): + for v in (r, -r, J @ r, -(J @ r)): idx[tuple(v)] = T + return idx + +def generators(S): + J = S.ops[1]; idx = line_index(S); gens = [] + for r in S.reps: + out = [] + for T in range(S.v): + x = S.reps[T] + for a in (r, J @ r): # real reflections in r and Jr (norm 8): x -> x - 2/8 a + x = x - (2 * int(x @ a) // 8) * a + out.append(idx[tuple(x)]) + assert sorted(out) == list(range(S.v)) + gens.append(tuple(out)) + return gens + +def closure(gens, n): + G = {tuple(range(n))}; frontier = list(G) + while frontier: + new = [] + for p in frontier: + for g in gens: + q = tuple(g[p[i]] for i in range(n)) + if q not in G: G.add(q); new.append(q) + frontier = new + return G + +if __name__ == '__main__': + S = views()['Gauss'] + gens = generators(S) + # check: generators preserve non-orthogonality + for g in gens: + P = np.array(g); assert (S.NO[np.ix_(P, P)] == S.NO).all() + G = closure(gens, S.v); print('group order on lines', len(G)) + F = pickle.load(open('flats_Gauss.pkl', 'rb')) + hyp = sorted(sorted(f) for f, d in F.items() if d == 3) + hs = {frozenset(h) for h in hyp}; seen = set(); reps = [] + for h in hyp: + if frozenset(h) in seen: continue + orb = {frozenset(g[i] for i in h) for g in G} + assert orb <= hs + seen |= orb; reps.append((h, len(orb))) + print('orbits on 3-flats:', [(len(h), n) for h, n in reps], 'total', sum(n for _, n in reps)) + pickle.dump({'G': sorted(G), 'hyp_reps': reps}, open('orbits.pkl', 'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pair_ilp.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pair_ilp.py new file mode 100644 index 0000000000..f48491c52e --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pair_ilp.py @@ -0,0 +1,55 @@ +"""c = 21, all totals tight (then all 21 are dumbbells, oriented): clique packing + diagonal + exact 2x2 support lemma on +every non-orthogonal pair. argv: c timelimit""" +import pickle, sys, numpy as np, time +from scipy.optimize import milp, LinearConstraint, Bounds +from scipy.sparse import lil_matrix, csr_matrix +from f2check import NO +c=int(sys.argv[1]); tl=float(sys.argv[2]); cap=c-12 +C=pickle.load(open('hits_table.pkl','rb'))['C']; PK=pickle.load(open('pairkinds.pkl','rb')); kinds=PK['kinds']; minimal=PK['minimal'] +orth=lambda a,b: a!=b and not NO[a,b] +perp=lambda l: set(T for T in range(60) if orth(T,l)) +ALL=set(range(60)) +cols=[] +for a in range(60): + for b in range(60): + if orth(a,b): cols.append((2,(a,b),perp(a),perp(b))) +for l in range(60): cols+= [(3,(l,),perp(l),perp(l)),(3,(l,),perp(l),ALL),(3,(l,),ALL,perp(l))] +cols.append((4,(),ALL,ALL)) +n=len(cols); pairs=[(a,b) for a in range(60) for b in range(a+1,60) if NO[a,b]] +nz=len(pairs)*len(minimal); N=n+nz +rows=[]; lb=[]; ub=[] +A=lil_matrix((640+60+len(pairs)*(1+3*len(minimal))+2,N)); r=0 +for q,cl in enumerate(C): + for k,(bt,x,X,Y) in enumerate(cols): + v=len(set(x)&set(cl)) + if v: A[r,k]=v + lb.append(-np.inf); ub.append(cap); r+=1 +for T in range(60): + for k,(bt,x,X,Y) in enumerate(cols): + if T in X and T in Y: A[r,k]=1 + lb.append(1); ub.append(np.inf); r+=1 +kid={kd:i for i,kd in enumerate(kinds)} +for p,(a,b) in enumerate(pairs): + kcols={} + for k,(bt,x,X,Y) in enumerate(cols): + xs=tuple(i for i,t in enumerate((a,b)) if t in X); ys=tuple(i for i,t in enumerate((a,b)) if t in Y) + if xs and ys: kcols.setdefault(kid[(xs,ys)],[]).append(k) + for i,m in enumerate(minimal): + z=n+p*len(minimal)+i; A[r,z]=1 + lb.append(1); ub.append(np.inf); r+=1 + for i,m in enumerate(minimal): + z=n+p*len(minimal)+i + for kd in set(m): + for k in kcols.get(kd,[]): A[r,k]=1 + A[r,z]=-m.count(kd); lb.append(0); ub.append(np.inf); r+=1 +for k in range(n): A[r,k]=1 +lb.append(c); ub.append(c); r+=1 +for k,(bt,x,X,Y) in enumerate(cols): A[r,k]=bt +lb.append(-np.inf); ub.append(42); r+=1 +A=csr_matrix(A[:r]) +print('cols',n,'binaries',nz,'rows',r,flush=True) +t=time.time() +res=milp(np.zeros(N),constraints=[LinearConstraint(A,lb,ub)],integrality=np.ones(N),bounds=Bounds(0,np.concatenate([np.full(n,c),np.ones(nz)])),options={'time_limit':tl}) +print('c',c,'all-tight + 2x2 lemma:',res.status,res.message,'%.0fs'%(time.time()-t),flush=True) +if res.x is not None: + x=np.round(res.x[:n]).astype(int); print([(cols[k][1],cols[k][0],x[k]) for k in np.nonzero(x)[0]]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pairkinds.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pairkinds.py new file mode 100644 index 0000000000..83ece23c7d --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/pairkinds.py @@ -0,0 +1,23 @@ +"""2x2 support lemma data: kinds (Xs, Ys) with Xs, Ys nonempty subsets of {a,b} (a,b non-orthogonal lines). +A total of kind (Xs,Ys) contributes w u^T with supp w <= Xs, supp u <= Ys. B[{a,b},{a,b}] = I_2 is required. +Brute force: minimal multisets of kinds (size <= 4) for which I_2 is reachable. Feasibility is monotone under adding totals.""" +import itertools, pickle +import numpy as np +subs=[(0,),(1,),(0,1)] +kinds=[(x,y) for x in subs for y in subs] +def vecs(s): return [v for v in itertools.product((0,1),repeat=2) if all(v[i]==0 for i in range(2) if i not in s)] +I=np.eye(2,dtype=int) +def feasible(ms): + opts=[[(np.array(w),np.array(u)) for w in vecs(kinds[k][0]) for u in vecs(kinds[k][1])] for k in ms] + for ch in itertools.product(*opts): + S=sum((np.outer(w,u) for w,u in ch),np.zeros((2,2),int))%2 + if (S==I).all(): return True + return False +minimal=[] +for size in range(1,5): + for ms in itertools.combinations_with_replacement(range(9),size): + if any(all(ms.count(k)>=m.count(k) for k in set(m)) for m in minimal): continue + if feasible(ms): minimal.append(ms) +print(len(minimal),'minimal feasible kind-multisets:') +for m in minimal: print(' ',[kinds[k] for k in m]) +pickle.dump({'kinds':kinds,'minimal':minimal},open('pairkinds.pkl','wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/tight_diag.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/tight_diag.py new file mode 100644 index 0000000000..db08ab41c8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/tight_diag.py @@ -0,0 +1,30 @@ +"""Tight-types-only covering ILP (S8 tight_ilp) + diagonal coverage: every line T lies in X_j and Y_j of some total. +argv: c timelimit""" +import pickle, sys, numpy as np, time +from scipy.optimize import milp, LinearConstraint, Bounds +from lines import views +S=views()['Gauss']; NO=S.NO +C=pickle.load(open('hits_table.pkl','rb'))['C'] +c=int(sys.argv[1]); tl=float(sys.argv[2]) +orth=lambda a,b: a!=b and not NO[a,b] +pairs=[(a,b) for a in range(60) for b in range(a+1,60) if orth(a,b)] +# columns: dumbbells (beta2, x={a,b}), singles l (beta3, x={l}), full (beta4) +cols=[] +for a,b in pairs: cols.append((2,[a,b],[T for T in range(60) if orth(T,a) and orth(T,b)])) +for l in range(60): cols.append((3,[l],[T for T in range(60) if orth(T,l)])) +cols.append((4,[],list(range(60)))) +n=len(cols); A=[];lb=[];ub=[] +M=np.zeros((640,n)) +for k,(beta,x,cov) in enumerate(cols): + for q,cl in enumerate(C): M[q,k]=len(set(x)&set(cl)) +A.append(M); lb+= [-np.inf]*640; ub+=[c-12]*640 +D=np.zeros((60,n)) +for k,(beta,x,cov) in enumerate(cols): D[cov,k]=1 +A.append(D); lb+=[1]*60; ub+=[np.inf]*60 +A.append(np.ones((1,n))); lb.append(c); ub.append(c) +A.append(np.array([[cl[0] for cl in cols]])); lb.append(-np.inf); ub.append(42) +t=time.time() +r=milp(np.array([cl[0] for cl in cols],float),constraints=[LinearConstraint(np.vstack(A),lb,ub)],integrality=np.ones(n),bounds=Bounds(0,c),options={'time_limit':tl}) +print('c',c,'status',r.status,r.message,'cost',r.fun,'%.0fs'%(time.time()-t)) +if r.x is not None: + x=np.round(r.x).astype(int); print([(cols[k][0],cols[k][1],x[k]) for k in np.nonzero(x)[0]]) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/try_cand.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/try_cand.py new file mode 100644 index 0000000000..9366778a4b --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/try_cand.py @@ -0,0 +1,6 @@ +import sys, time +from f2check import * +perp = lambda l: [T for T in range(60) if T != l and not NO[l, T]] +D=eval(sys.argv[1]) +t=time.time(); ok,sol=solve_opts([[(perp(a),perp(b)),(perp(b),perp(a))] for a,b in D]); print('free orientation:',ok,'%.1fs'%(time.time()-t)) +if ok: print('decoder check',check_decoder(*sol)) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/types_e.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/types_e.py new file mode 100644 index 0000000000..99ea12fb19 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/types_e.py @@ -0,0 +1,27 @@ +"""Rank-maximal flat-pair types (1<=R<=3) with e = 128R+128-h <= EMAX; dedupe by (miss set, R). Output types_e{EMAX}.pkl""" +import pickle, sys, numpy as np +from collections import Counter +EMAX=int(sys.argv[1]) +d=pickle.load(open('hits_table.pkl','rb')); fl=d['fl']; R=d['R']; H=d['hits']; C=d['C'] +n=len(fl); L=[set(f) for f,dd in fl] +E=128*R+128-H +sup=[[j for j in range(n) if L[i] < L[j]] for i in range(n)] +enlX=np.zeros((n,n),bool); enlY=np.zeros((n,n),bool) +for i in range(n): + if sup[i]: + enlX[i]=(R[sup[i],:]==R[i][None,:]).any(0) + enlY[:,i]=(R[:,sup[i]]==R[:,i][:,None]).any(1) +ok=(R>=1)&(R<=3)&(E<=EMAX)&~enlX&~enlY +ts=np.argwhere(ok) +Lo=np.zeros((n,60),np.int64) +for i,(f,dd) in enumerate(fl): Lo[i,sorted(f)]=1 +inc=np.zeros((60,640),np.int64) +for k,c in enumerate(C): inc[c,k]=1 +meet=(Lo@inc>0) +miss={} +for a,b in ts: + m=~(meet[a]&meet[b]); key=(int(R[a,b]),np.packbits(m).tobytes()) + if key not in miss: miss[key]=(int(a),int(b)) +print('EMAX',EMAX,'rank-max pairs',len(ts),'distinct (R,miss)',len(miss)) +print(Counter((len(fl[a][0]),len(fl[b][0]),int(R[a,b]),int(E[a,b])) for (r,k),(a,b) in miss.items()).most_common()) +pickle.dump([(a,b,r,int(E[a,b]),np.unpackbits(np.frombuffer(k,np.uint8))[:640].astype(bool)) for (r,k),(a,b) in miss.items()],open('types_e%d.pkl'%EMAX,'wb')) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/vsingle.py b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/vsingle.py new file mode 100644 index 0000000000..996fa243f1 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/scripts/vsingle.py @@ -0,0 +1,13 @@ +import pickle,sys,numpy as np +from scipy.optimize import milp, LinearConstraint, Bounds +cap=int(sys.argv[1]) +C=pickle.load(open('hits_table.pkl','rb'))['C'] +M=np.zeros((640,60)) +for k,cl in enumerate(C): M[k,cl]=1 +T=pickle.load(open('types_e256.pkl','rb')); L=pickle.load(open('lift.pkl','rb')); dom=pickle.load(open('dom.pkl','rb')) +for k,r in enumerate(dom['reps']): + a,b,R,e,m=T[r]; x,d=L[r] + if d<=0: continue + y=m-M[:,x].sum(1) + res=milp(-np.ones(60),constraints=[LinearConstraint(M,-np.inf,cap-y)],integrality=np.ones(60),bounds=Bounds(0,cap),options={'time_limit':300}) + print('orbit',k,'R',R,'e',e,'alpha',len(x),'delta',d,'V',-res.fun,'V+alpha+delta',-res.fun+len(x)+d,flush=True) diff --git a/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/verify.sh b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/verify.sh new file mode 100755 index 0000000000..201f2084a8 --- /dev/null +++ b/research/e8-decoder-floors-and-cold14-notes/s9-gaussian-c19/verify.sh @@ -0,0 +1,41 @@ +#!/bin/sh +# S9 verification. Rebuilds every input from scratch (exact lines/flats/cliques, certified ranks), then +# (1) the mu-packing optima f(k) (HiGHS) + CP-SAT cross-checks, +# (2) the type catalog (58,340 types), its 27 orbits and the lifting data, +# (3) calibrations of the case ILP (must be FEASIBLE) and the F2 confinement checks of the c = 21 counting covers, +# (4) [FULL=1] every case ILP of Theorems 1-2 (HiGHS; hours; PAR parallel jobs, default 6). +# Lines printed (times stripped) should match expected.txt (FULL part: expected_full.txt). +# Needs python 3.11 + numpy scipy sympy networkx python-sat ortools on PATH. +set -e +export OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 +HERE=$(cd "$(dirname "$0")" && pwd); W=${WORK:-$(mktemp -d)}; PAR=${PAR:-6} +cp "$HERE"/scripts/*.py "$W"/; cd "$W" +strip() { sed -E 's/ [0-9]+s$//'; } +python flats.py Gauss > /dev/null +python orbits.py +python hits_table.py | grep -E '^(PASS|RESULT)' +python types_e.py 256 | head -1 +python dom.py | head -2 +python lift_prep.py > /dev/null && echo "lift.pkl built" +for cap in 6 7 8 9 10 11 12; do python mu_ilp.py $cap 600 | head -1 | awk '{printf "mu-packing: cap %s max |mu| = %d (LP %d)\n",$1,int($(NF-3)+0.5),5*$1}'; done +python mu_cpsat.py 7 33 2 | strip +python mu_cpsat.py 8 37 2 | strip +echo "calibrations (must be FEASIBLE):" +python branch1.py 21 42 -1 600 feas | head -1 | strip +python branch1.py 19 44 -1 600 feas | head -1 | strip +python pairkinds.py | head -1 +python f2check.py +python check_c21cover.py +python try_cand.py "[(1,54),(3,4),(5,59),(6,18),(7,19),(8,57),(20,32),(21,31),(23,24),(25,51),(28,58),(29,52),(33,56),(35,48),(36,50),(38,53),(39,46),(40,55),(41,47),(42,44),(43,45)]" | strip +python core.py "[(1,54),(3,4),(5,59),(6,18),(7,19),(8,57),(20,32),(21,31),(23,24),(25,51),(28,58),(29,52),(33,56),(35,48),(36,50),(38,53),(39,46),(40,55),(41,47),(42,44),(43,45)]" +if [ -n "$FULL" ]; then + for cc in "19 42" "20 41" "20 42"; do + set -- $cc + CASES=$(python -c " +import pickle; T=pickle.load(open('types_e256.pkl','rb')); L=pickle.load(open('lift.pkl','rb')); d=pickle.load(open('dom.pkl','rb')) +s=128*($2+$1)-7680 +print(' '.join(['-1']+[str(k) for k,r in enumerate(d['reps']) if L[r][1]>0 and T[r][3]<=s]))") + echo "$CASES" | tr ' ' '\n' | xargs -P $PAR -I{} python branch1.py $1 $2 {} ${TL:-14400} $MODE | strip | sort -t' ' -k4,4n + done +fi +echo "work dir: $W"