diff --git a/research/e8-complex-gauge-kernel/NOTICE.md b/research/e8-complex-gauge-kernel/NOTICE.md new file mode 100644 index 0000000000..a5682058c5 --- /dev/null +++ b/research/e8-complex-gauge-kernel/NOTICE.md @@ -0,0 +1,4 @@ +Apache-2.0. `code/audit_cxg.py`, `emit_cxg.py`, `phases_cxg.py` are modified copies of PR #433's `complex_audit.py`, +`complex_emit.py`, `complex_phases_full.py` (icekylinx with Codex; adapted from PR #425 by jacklightChen / Zhihao Chen). +`code/hostmodel.py` replicates the profile arithmetic of PR #442's `complex_hosted.py` (DaysSky) for mixed widths. +Prepared by DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. diff --git a/research/e8-complex-gauge-kernel/PROOF.md b/research/e8-complex-gauge-kernel/PROOF.md new file mode 100644 index 0000000000..78315a32e6 --- /dev/null +++ b/research/e8-complex-gauge-kernel/PROOF.md @@ -0,0 +1,59 @@ +# Lemmas used by the draft (both are NEW and are the review points) + +Setting: #442's complex word (#433's split-centre R560 word, m = 73, nine stages) with the idle-axis hosting of #442. +Notation: frames are subspaces of F2^9; T_U = canonical_frame(9, U) (clifford_exact.py); T_FULL is the full Fourier +transform; a helper normally climbs ∅ → … → FULL and its residual has width 9. + +## 0. What is not claimed +The entrance lines used here are all of EVEN weight, i.e. totally isotropic for the F2 dot product (14 distinct rank-1 +lines in the kernel word; 16 frames of rank 1, 2 and 4 in the combined word). The bit-style criterion of #440 +(`validate_isometries`: an orthonormal basis whose first e rows span E, i.e. coordinate entrances) is therefore NOT met: +no orthogonal matrix of O(9,2) sends an isotropic E to coordinate axes. `code/iso1.py` shows that, restricted to +entrances that do meet it, the kernel potential and the number of gauge-eligible helpers are both 0. #442's shipped +complex audit asserts `w == 9` for hosted residuals (`idle-axis-hosting/audit.py:92`), and #433's emit asserts +`c['start'][r]==0` for compensated registers (`complex_emit.py:55`). No shipped complex word has a nonzero helper +entrance. Everything below is therefore CONDITIONAL on Lemma A and Lemma G. + +## Lemma A (entrance adapter) — review point 1 +Let E ⊆ F2^9, e = dim E, w = 9 − e, and Φ_E = T_E^† T_FULL (the chain ∅ → E removed from the helper's climb). +(i) Φ_E = R_E (C_[w] ⊗ I) L_E with R_E, L_E explicit Gaussian monomials (address permutation, quadratic phase, X +translation), exactly, including the global phase. This is `lower_child` applied to Φ_E: it asserts rank w, +`compiled.tab == Φ_E.tab` and `compiled.zero_state() == Φ_E.zero_state()`. +(ii) Claim (assumed, not proved): a helper that starts at frame E and ends at FULL, with its compensation moved off +the zero frame (kernel: absorbed by donors; gauge: Lemma G), acts on its residual stream as the exact coordinate +factor P_g C_[w] P_g^-1 that hosting lemma (a) of #442 uses, where the monomials R_E, L_E are rank-zero local gates like +the endpoint adapters of #425. Its residual block has width w; the e remaining local coordinates carry arbitrary +dirt and are transformed by Φ_E like any other dirt coordinate, which the arbitrary-dirt restoration identity +(replayed exactly on all 800 columns) tolerates. +Evidence: `code/nec4.py` — all 16 entrance frames of `gauge80-kernel42` pass (i); `code/nec3.py` shows the tableaux: +for coordinate frames Φ_E fixes every X_i, for the isotropic and odd-weight non-coordinate frames it does not, i.e. +the monomials are genuinely non-orthogonal GL(9,2) address maps. Part (ii) is where a reviewer must look: it is the +statement that the bank block of width w plus a non-orthogonal monomial adapter is an admissible residual route. + +## Lemma K (shared-donor kernel, port of #417/#435/#434) +A pivot p with exact response relation col(p) = Σ q_d col(d) over donors d, entrance E ⊆ first frame of p and of every +donor, starts at frame E; each donor receives `d += -q_d p` at frame E before the main word (inverse after the +cleanup at the full frame); the compensation columns of pivots are dropped. Exactness is the arbitrary-dirt replay +(donor update is part of the replayed word); frames are checked by the chain assertion. Same statement as in #440, +with rational coefficients (all denominators are powers of two, as required by `primitive_bound`). + +## Lemma G (entrance gauge, nonzero-frame compensation and centre scatter) — review point 2 +For a helper j with entrance E_j (a member of a nested family) its compensation `y_t += c_tj x_j` is executed at +frame E_j instead of frame 0, as the first gates of the main word; the helper starts at E_j and every target t in its +response support climbs ∅ → L_t, L_t = the largest E_j with t in the support of j. +Constraints (all checked): (a) E_j ⊆ first frame of j and ⊆ the first main frame of every target in its support; +(b) the entrances touching one target are pairwise nested; (c) for every centre k, the levels L_t of the targets it +scatters into form a chain whose top lies inside the centre frame c_k. The centre scatter then runs as: copy at c_k; +lower stepwise c_k → (served levels, descending) → lowest served level; scatter into the targets of level L at frame L; +clear at the lowest level (reversed in the transposed orientation). The total lowering rank equals the baseline 8 +whenever some served target has level ∅, so the rank deficit stays 2496. +Claim (assumed): a scalar add at frame E between a helper at E and a target raised to E, and the centre scatter at +frame L between a copy lowered only to L and a target at L, are the same operators as the zero-frame versions up to the +common exterior window (the common-frame lemma of #425, extended from frame 0 to frame L). +Evidence: audit (exact arbitrary-dirt and inverse-transpose replay of the full program including the gauge gates), +emit (literal scalar replay of both orientations on 800 columns, centre lifetimes, `paid` equals the audit profile), +exact Gaussian recipes for every used transition (2,940 edges for the gauge word). + +## Mutation controls (to be added in the full package) +Not yet run: omit the donor update, flip a gauge coefficient sign, move an entrance out of the first frame, break a +centre chain. `audit_cxg.py` already rejects frame-chain violations by assertion. diff --git a/research/e8-complex-gauge-kernel/README.md b/research/e8-complex-gauge-kernel/README.md new file mode 100644 index 0000000000..d5c1be2e0a --- /dev/null +++ b/research/e8-complex-gauge-kernel/README.md @@ -0,0 +1,58 @@ +# Complex supplier b: shared-donor kernel and entrance gauges on the E8 complex word (DRAFT, status: work in progress) + +Not a claim yet. The claim is CONDITIONAL on two new lemmas (see `PROOF.md`) and the global package is not built. + +## Construction +On #442's complex word (#433's split-centre R560 word, m = 73, nine stages, idle-axis hosting), two tools of the +bit side are ported: +- **Shared-donor kernel** (#417/#435/#434/#439/#440 style): 79 pivots, rank-1 entrances, 14 entrance frames. +- **Entrance gauges** (#401/#408–#416 style): compensation moved to the entrance frame; 80 helpers, entrance + rank 106 (62 rank-1, 14 rank-2, 4 rank-4), nested per target and per centre. +- Not portable on this word: elimination reuse and early resets (#436/#437): 0 helpers survive the frame-monotonicity + check; block retiming (#441/#440): 0 improvement (frame cost unchanged after single-gate and closure moves). + +## Measured (hosted b, exact moment engine; the hosting step is my replica of #442's, which reproduces #442's complex b +1.267453159029119e-3, live stock 8400 and rank deficit 2496 to the last digit) +| word | helpers entering | total entrance rank | b | gain | κ cap if complex-bound | +| --- | ---: | ---: | ---: | ---: | ---: | +| #442 complex | 0 | 0 | 1.267453e-3 | | 1.265849e-3 | +| kernel only | 79 | 79 | 1.310216e-3 | +3.37% | 1.308502e-3 | +| gauge only | 80 | 106 | 1.310916e-3 | +3.43% | 1.309200e-3 | +| gauge + kernel | 122 | 148 | **1.333854e-3** | **+5.24%** | 1.332077e-3 | +The combined word hosts all 40,320 residuals (live stock 7,680, no bank). The rank deficit is 2496 in every row. + +## Checks that PASS (local word only; `sh check.sh DIR`) +- Exact arbitrary-dirt and inverse-transpose replay of the full word (800 columns), frame chain, paid profile. +- Emitted local program, literal scalar replay in both orientations on 800 columns with centre lifetimes. +- Exact Gaussian phase recipes for every used frame transition (`PHASES=1`): 2,932 / 2,940 / 2,952 edges. +- Entrance-adapter evidence: for all 16 entrance frames E→FULL = monomial · C_[9−e] · monomial exactly + (`lower_child`: tableau and state-vector equality, global phase included). + +## Plain statements about what is NOT met +- **The entrances are isotropic** (even-weight lines of F2^9). The bit-style coordinate-entrance criterion (#440's + `validate_isometries`) is NOT met, and no orthogonal map of O(9,2) can meet it for these lines. Restricted to + entrances that meet it, the gain is 0 (`code/iso1.py`). +- **#442's shipped complex audit and emit assume `w == 9` and `start == 0`.** No shipped complex word has a nonzero + helper entrance. The audit, emit and phase scripts used here are PATCHED copies (`code/audit_cxg.py`, `emit_cxg.py`, + `phases_cxg.py`: pre-ops, entrance starts, target levels, centre descent); their defaults are not yet shown to + reproduce #442's files byte for byte. +- The claim is therefore **conditional on Lemma A (entrance adapter) and Lemma G (nonzero-frame compensation and centre + scatter)**, which are the headline review points. + +## Pending +Global template, owner tables and hosting generator for mixed residual widths (the hosted b above is a model); +residual-to-bank routes for each entrance frame; written proofs of Lemma A(ii) and Lemma G; mutation controls; +finite invoices and the outer assembly (47/47) with a bit word; byte-for-byte regression of the generalised scripts at +the #442 defaults; two fresh-copy replays. + +## Files +`PROOF.md` lemmas and scope; `check.sh`; `data/words/*/FULL-WORD.json.gz` the three words; `data/selection/*.json` +frozen selections; `code/` checks (`audit_cxg.py`, `emit_cxg.py`, `phases_cxg.py`, `cxcheck.py`, `hostmodel.py`, +`final_eval.py`, `nec3.py`, `nec4.py`, `iso1.py`) and the search scripts that produced the selections +(`kern*.py`, `gauge*.py`, `retime.py`, `reuse*.py`). + +## Credits +Builds on #442 (DaysSky, idle-axis hosting), #433 (icekylinx, split centres), #425 and #386 (jacklightChen / Zhihao +Chen, mutable packets, compressed windows), the shared-donor kernel of #417/#435 (chafreaky) and #434/#440 (rohanarun, +chafreaky), the entrance gauges of #401/#408–#416, #370 (Dylan Crow) and Jacob Sussman's E8 unit. +Prepared by DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. Apache-2.0. diff --git a/research/e8-complex-gauge-kernel/check.sh b/research/e8-complex-gauge-kernel/check.sh new file mode 100755 index 0000000000..94c4cc61f5 --- /dev/null +++ b/research/e8-complex-gauge-kernel/check.sh @@ -0,0 +1,23 @@ +#!/bin/sh +# Draft local checks for the complex gauge/kernel words. +# CXG_BASE : output directory of `python -B research/idle-axis-hosting/verify.py $CXG_BASE` (PR #442 replay; contains parent/changed/) +# CXG_REPO : checkout of PR #442's tree (research/split-centres-global, research/idle-axis-hosting) +# PY : python with numpy + scipy (hosted-b model); stdlib python is enough for the other steps +# PHASES=1 : also replay the exact Gaussian phase recipes of every used frame transition (about 160 s per word) +# usage: sh check.sh /absolute/new/output/dir +set -e +: "${CXG_BASE:?}"; : "${CXG_REPO:?}"; PY=${PY:-python3}; OUT=${1:?new output dir} +PKG=$(cd "$(dirname "$0")" && pwd); export CXG_PKG=$PKG CXG_BASE CXG_REPO PYTHONDONTWRITEBYTECODE=1 +mkdir -p "$OUT" +for w in kernel79 gauge80 gauge80-kernel42; do + d="$OUT/$w"; mkdir -p "$d" + cp -RL "$CXG_BASE/parent/changed/split-centres" "$d/split-centres" + cp -RL "$CXG_BASE/parent/changed/work" "$d/work" + cp "$PKG/data/words/$w/FULL-WORD.json.gz" "$d/split-centres/FULL-WORD.json.gz" + echo "== $w: arbitrary-dirt + inverse-transpose replay, frame chain, paid profile"; (cd "$d" && python3 -B "$PKG/code/audit_cxg.py" | tail -3) + echo "== $w: emitted program, literal scalar replay (forward and backward, 800 columns, centre lifetimes)"; (cd "$d" && python3 -B "$PKG/code/emit_cxg.py" | tail -2) + if [ -n "$PHASES" ]; then echo "== $w: exact Gaussian phase recipes (all used edges)"; (cd "$d" && python3 -B "$PKG/code/phases_cxg.py" | tail -1); fi +done +echo "== entrance-adapter evidence (nec3: tableau shapes, nec4: lower_child equality for every entrance frame)" +(cd "$PKG/code" && python3 -B nec3.py && $PY -B nec4.py) +echo "== hosted b (exact moment engine) per tool"; (cd "$PKG/code" && CXG_OUT="$OUT" $PY -B final_eval.py | cut -c1-200) diff --git a/research/e8-complex-gauge-kernel/code/audit_cxg.py b/research/e8-complex-gauge-kernel/code/audit_cxg.py new file mode 100644 index 0000000000..b19202899d --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/audit_cxg.py @@ -0,0 +1,165 @@ +# Modified from PR425 by icekylinx with Codex assistance. +"""Read-back audit, independent of the Hamming producer and its row state.""" +import copy,gzip,hashlib,importlib.util,json,math,time +from collections import Counter +from fractions import Fraction as Q +from pathlib import Path + +import os +OUT=Path(os.environ.get('CXOUT','split-centres'));p=OUT/'FULL-WORD.json.gz' +word=json.load(gzip.open(p));v=120;n=240+word['report']['R'];start=time.monotonic() +def apply(e,rows): + if e[0]=='scale': + _,t,a,b,*_=e;q=Q(a,b);assert q + rows[t]={j:q*x for j,x in rows[t].items()} + else: + _,t,s,a,b,*_=e;q=Q(a,b);assert t!=s + for j,x in list(rows[s].items()): + z=rows[t].get(j,Q(0))+q*x + if z:rows[t][j]=z + else:rows[t].pop(j,None) +def ident():return [{r:Q(1)} for r in range(n)] +def inv(e): + e=copy.deepcopy(e) + if e[0]=='add':e[3]*=-1 + else:e[2],e[3]=e[3],e[2] + return e +def dual(e): + e=inv(e) + if e[0]=='add':e[1],e[2]=e[2],e[1] + return e + +main=word['A']+word['cut']+word['B'] +assert all(not 120<=e[2]<240 for e in word['A']+word['B'] if e[0]=='add' and not 120<=e[1]<240) +matrix=ident() +for e in main:apply(e,matrix) +print('full main columns',n,'replayed',flush=True) +# Compute -A_Y^-1 B_S using only rows of the explicit main operator. +dirty=[{} for r in range(n)] +for r in range(v,2*v):dirty[r]={j:x for j,x in matrix[r].items() if j>=2*v} +for e in word['precondition']:apply(e,dirty) +comp=[] +for r in range(v,2*v): + for j,q in sorted(dirty[r].items()):comp.append(['add',r,j,-q.numerator,q.denominator,[]]) + +cleanup=[inv(e) for e in reversed(word['A']+word['B']) if not v<=e[1]<2*v] +assert [e[:-1] for e in cleanup]==[e[:-1] for e in word['cleanup']] +program=word['precondition']+comp+main+cleanup +actual=ident() +for e in program:apply(e,actual) +expected=ident() +for t in range(v):expected[v+t][t]=Q(1) +assert actual==expected +print('PASS exact arbitrary dirt and restored sources/helpers',flush=True) +back=ident() +for e in program:apply(dual(e),back) +expected_back=ident() +for t in range(v):expected_back[t][v+t]=Q(-1) +assert back==expected_back +print('PASS exact inverse-transpose on every column',flush=True) +# Independent sparse vector controls avoid repeating a dense full replay. +def vector(es,x): + x=list(map(Q,x)) + for e in es: + if e[0]=='scale':x[e[1]]*=Q(e[2],e[3]) + else:x[e[1]]+=Q(e[3],e[4])*x[e[2]] + return x +sample=[((i*137+13)%101)-50 for i in range(n)] +want=vector(program,sample) +controls={ + 'missing_dirty_compensation':word['precondition']+main+cleanup, + 'missing_Y_inverse_basis':comp+main+cleanup, + 'missing_source_cleanup':word['precondition']+comp+main, + 'wrong_dual_event_order':[dual(e) for e in reversed(program)], +} +for name,es in controls.items(): + expected_sample=vector([dual(e) for e in program],sample) if name=='wrong_dual_event_order' else want + assert vector(es,sample)!=expected_sample,name + +# Orthogonal transvection and a port permutation bring the new geometry onto +# the already fixed 120-port routing family. Every incidence is transformed. +old=json.load(gzip.open('work/higher-stage-e8/verified/gcert1-e8-retimed.json.gz')) +def tau(x):return x^(43 if (x&43).bit_count()%2 else 0) +assert 43 .bit_count()%2==0 +assert set(map(tau,word['ports']))==set(old['ports']) +mapping={i:old['ports'].index(tau(q)) for i,q in enumerate(word['ports'])} +mapping.update({v+i:v+mapping[i] for i in range(v)}) +mapping.update({r:r for r in range(2*v,n)}) +def basis(vs): + b={} + for x in vs: + for k in sorted(b,reverse=True): + if x>>k&1:x^=b[k] + if x: + k=x.bit_length()-1 + for j in b: + if b[j]>>k&1:b[j]^=x + b[k]=x + return tuple(b[k] for k in sorted(b,reverse=True)) +def perp(x):return basis([z for z in range(1,512) if not (x&z).bit_count()%2]) +full=tuple(1<0 + edges.add((old,f));paid[delta]+=1;cur[r]=f +for part in ('A','B'): + if part=='B': + assert all(cur[r]==f for r,f in zip(can['centers'],can['center_frames'])) + assert all(cur[r]==can['target_levels'].get(r-v,0) for r in range(v,2*v)) + for r_c,f in zip(can['centers'],can['center_frames']): + # mass-neutral descent: cf -> served levels (descending) -> lowest served level + served={can['target_levels'].get(e[1]-v,0) for e in can['cut'] if e[2]==r_c} + o=f + for L in sorted(served,key=lambda L:-len(frames[L])): + assert basis(frames[L]+frames[o])==tuple(frames[o]),('centre frame must contain served level',r_c) + if L!=o:paid[len(frames[o])-len(frames[L])]+=1;edges.add((o,L));edges.add((L,o));o=L + if o==f:paid[0]+=0 + for e in can[part]: + rs=[e[1]]+([e[2]] if e[0]=='add' else []) + for r in rs:climb(r,e[-1]);incidences+=1 +for r,f in enumerate(can['final']):climb(r,f) +assert dict(paid)=={int(r):x for r,x in word['report']['local_profile'].items()} +assert sum(r*x for r,x in paid.items())==sum(int(r)*x for r,x in word['report']['local_profile'].items()) +can['local_profile']=dict(sorted(paid.items()));can['edges']=sorted(edges) +cp=OUT/'CANONICAL-WORD.json.gz';cp.write_bytes(gzip.compress(json.dumps(can,sort_keys=True,separators=(',',':')).encode(),mtime=0)) + +b=Q(0);grid=Q(1,10**18) +coeff=Counter(str(Q(e[3],e[4]) if e[0]=='add' else Q(e[2],e[3])) for e in program) +report={'status':'PASS_EXACT_SCALAR_AND_FRAME_RESEARCH_CANDIDATE_NOT_YET_ADMITTED', + 'full_word_sha256':hashlib.sha256(p.read_bytes()).hexdigest(), + 'canonical_word_sha256':hashlib.sha256(cp.read_bytes()).hexdigest(), + 'live_columns_each_orientation':n,'arbitrary_helper_columns':can['R'], + 'all_live_source_and_helper_rows_restored':True,'all_target_rows_correct':True, + 'negative_controls_rejected':list(controls),'old_response_additions':len(comp), + 'complete_forward_scalar_operations':len(program),'coefficient_histogram':dict(coeff), + 'frames':len(frames),'nested_directed_edges':len(edges),'common_frame_incidences':incidences, + 'orthogonal_transport_vector':43,'transported_ports_exactly_match_fixed_routes':True, + 'local_profile':dict(sorted(paid.items())), + 'two_rational_moment_engines':True,'next_b_grid_rejected':True, + 'remaining':['new general scalar-interface proof integration','exact phase and emitted local/global program','finite invoice and outer assembly'], + 'scope':'Substantial new scalar-network candidate. All parameter values remain prospective until the expanded scalar interface and complete phase-aware supplier are admitted.', + 'seconds':time.monotonic()-start} +(OUT/'SCALAR-FRAME-AUDIT.json').write_text(json.dumps(report,indent=2)+'\n') +print('PASS audit',report['seconds'],flush=True) diff --git a/research/e8-complex-gauge-kernel/code/build_final.py b/research/e8-complex-gauge-kernel/code/build_final.py new file mode 100644 index 0000000000..945d233719 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/build_final.py @@ -0,0 +1,17 @@ +import pickle,sys,json +from mkword_gauge import * +from kern6 import asg_to_groups,groups_to_pre +from kern import setup +import numpy as np +w=load();w['R']=w['report']['R'];col,first=setup(w);R=len(col) +C=np.zeros((v,R)) +for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) +which=sys.argv[1];out=sys.argv[2] +if which=='v1': + best,asg=pickle.load(open('sa_long_12.pkl','rb')) + groups=asg_to_groups(C,asg);pre,start,eps=groups_to_pre(col,groups);gauge={} +elif which=='v3': + pre,start,gauge,flag=pickle.load(open('v3_sel.pkl','rb')) +mkword_gauge(w,pre,start,gauge,out) +print(which,'pre',len(pre),'pivots',len(start),'gauge',len(gauge)) diff --git a/research/e8-complex-gauge-kernel/code/build_v4.py b/research/e8-complex-gauge-kernel/code/build_v4.py new file mode 100644 index 0000000000..e280d0ccab --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/build_v4.py @@ -0,0 +1,8 @@ +import pickle,sys +from mkword_gauge import * +w=load();w['R']=w['report']['R'] +which=sys.argv[1];out=sys.argv[2] +pre,start,gauge,_=pickle.load(open('v4_sel.pkl','rb')) +if which=='g80':pre,start=[],{} +mkword_gauge(w,pre,start,gauge,out) +print(which,'pre',len(pre),'pivots',len(start),'gauge',len(gauge)) diff --git a/research/e8-complex-gauge-kernel/code/cxcheck.py b/research/e8-complex-gauge-kernel/code/cxcheck.py new file mode 100644 index 0000000000..c0eee7f6d0 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/cxcheck.py @@ -0,0 +1,79 @@ +"""Generalised exact checker for variants of the #433 complex word (arbitrary dirt, inverse-transpose, frame chain, paid profile). +A variant = dict(A,B,cut,precondition,pre,start,ports,...). pre = list of ['add',donor,pivot,num,den,E] ops placed first in main. +start = {register: frame(list of masks)} for pivots (default () for helpers, (P[s],) for sources, () for targets). +cleanup is rebuilt: inverse of all non-target ops of pre+A+B at FULL, reversed.""" +from cxlab import * +import time +FULL=tuple(1<=2*v} + for e in prec:apply(e,dirty) + comp=[] + for r in range(v,2*v): + for j,x in sorted(dirty[r].items()):comp.append(['add',r,j,-x.numerator,x.denominator,[]]) + ncomp_cols=len({e[2] for e in comp}) + cleanup=[] + for e in reversed(list(pre)+A+B): + if not v<=e[1]<2*v: + c=inv(e);c[-1]=list(FULL);cleanup.append(c) + program=list(prec)+comp+main+cleanup + rep=dict(R=R,comp_adds=len(comp),comp_cols=ncomp_cols) + if full: + actual=ident(n) + for e in program:apply(e,actual) + expected=ident(n) + for t in range(v):expected[v+t][t]=Q(1) + assert actual==expected,'arbitrary-dirt exactness FAILED' + back=ident(n) + for e in program:apply(dual(e),back) + eb=ident(n) + for t in range(v):eb[t][v+t]=Q(-1) + assert back==eb,'inverse-transpose FAILED' + rep['exact_arbitrary_dirt']=True;rep['exact_inverse_transpose']=True + # frame chain + paid profile (mirrors complex_audit canonical climb check) + cur={} + for r in range(n): + if r=2*v) else perp(P[r-v])) + rep['local_profile']=dict(sorted(paid.items()));rep['edges']=len(edges) + rep['mass']=sum(r*z for r,z in paid.items()) + rep['widths']=[9-startrank[r] for r in range(2*v,n)] + rep['seconds']=time.time()-t0 + rep['_program']=program + return rep,edges +if __name__=='__main__': + w=load();rep,_=check(w) + print({k:v for k,v in rep.items() if k not in('_program','widths')},sum(rep['widths'])) diff --git a/research/e8-complex-gauge-kernel/code/cxlab.py b/research/e8-complex-gauge-kernel/code/cxlab.py new file mode 100644 index 0000000000..caed7a48d6 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/cxlab.py @@ -0,0 +1,38 @@ +"""cx lab helpers. Paths come from the environment: CXG_BASE = output directory of +`python -B research/idle-axis-hosting/verify.py $CXG_BASE` (PR #442 replay, which contains parent/changed/split-centres).""" +import gzip,json,copy,os,sys +from fractions import Fraction as Q +from collections import Counter,defaultdict +CX=os.environ.get('CXG_BASE','') +CH=CX+'/parent/changed/split-centres/' +v=120 +def load(path=None): + return json.load(gzip.open(path or (CH+'FULL-WORD.json.gz'))) +def apply(e,rows): + if e[0]=='scale': + _,t,a,b,*_=e;q=Q(a,b) + rows[t]={j:q*x for j,x in rows[t].items()} + else: + _,t,s,a,b,*_=e;q=Q(a,b) + for j,x in list(rows[s].items()): + z=rows[t].get(j,Q(0))+q*x + if z:rows[t][j]=z + else:rows[t].pop(j,None) +def inv(e): + e=copy.deepcopy(e) + if e[0]=='add':e[3]*=-1 + else:e[2],e[3]=e[3],e[2] + return e +def basis(vs): + b={} + for x in vs: + for k in sorted(b,reverse=True): + if x>>k&1:x^=b[k] + if x: + k=x.bit_length()-1 + for j in b: + if b[j]>>k&1:b[j]^=x + b[k]=x + return tuple(b[k] for k in sorted(b,reverse=True)) +def inside(a,b):return basis(list(a)+list(b))==tuple(b) if basis(b)==tuple(b) else basis(list(a)+list(b))==basis(b) +def ident(n):return [{r:Q(1)} for r in range(n)] diff --git a/research/e8-complex-gauge-kernel/code/emit_cxg.py b/research/e8-complex-gauge-kernel/code/emit_cxg.py new file mode 100644 index 0000000000..8075000813 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/emit_cxg.py @@ -0,0 +1,146 @@ +# Adapted from PR425 by icekylinx with Codex assistance. +"""Emit both local programs, integrate the fixed nine-stage geometry, and bill. + +The expanded scalar interface is justified in PROOF.txt. This script does not +claim a Lean proof or discharge the inherited all-size/analytic interfaces. +""" +import copy,gzip,hashlib,importlib.util,json,math +from collections import Counter +from fractions import Fraction as Q +from pathlib import Path + +OUT=Path('split-centres');cp=OUT/'CANONICAL-WORD.json.gz' +c=json.load(gzip.open(cp)) +frames=c['frames'];v,R,h=c['v'],c['R'],c['h'];n=2*v+R +def digest(x):return hashlib.sha256(json.dumps(x,sort_keys=True,separators=(',',':')).encode()).hexdigest() +def basis(vs): + b={} + for x in vs: + for p in sorted(b,reverse=True): + if x>>p&1:x^=b[p] + if x: + p=x.bit_length()-1 + for q in b: + if b[q]>>p&1:b[q]^=x + b[p]=x + return tuple(b[p] for p in sorted(b,reverse=True)) +def dual(e): + e=list(e) + if e[0]=='add':e[1],e[2]=e[2],e[1];e[3]*=-1 + else:e[2],e[3]=e[3],e[2] + return e +def scalar(e,back): + e=dual(e) if back else list(e) + if e[0]=='add':return ['add',['L',e[1]],['L',e[2]],e[3],e[4]] + return ['scale',['L',e[1]],e[2],e[3]] +def primitive_bound(e): + if e[0] in ('clear','copy'):return 1 + q=Q(e[3],e[4]) if e[0]=='add' else Q(e[2],e[3]) + a,b=abs(q.numerator),q.denominator + assert b&(b-1)==0 + if e[0]=='scale': + assert a&(a-1)==0 + return abs(a.bit_length()-b.bit_length())+int(q<0) + # Exact binary Horner multiplication on a cleared private stream, repeated + # halving, signed accumulation into the target, then clear the private stream. + return 3+(a.bit_length()-1)+a.bit_count()+(b.bit_length()-1) + +def emit(back): + out=[];cur=c['start'].copy() + def event(e):out.append(e) + def scal(e,f):event(['scalar',e,f]) + event(['phase','zero_frame_precondition']) + for e in c['precondition']+c['compensation']: + regs=[e[1]]+([e[2]] if e[0]=='add' else []) + assert all(c['start'][r]==0 for r in regs) + scal(scalar(e,back),0) + event(['phase','entry_adapters']) + for r,f in enumerate(c['start']):event(['adapter','old_to_new',r,f]) + def climb(r,f,part): + old=cur[r] + if old==f:return + assert basis(frames[old]+frames[f])==tuple(frames[f]) + rank=len(frames[f])-len(frames[old]);assert rank>0 + event(['climb',part,r,old,f,rank]);cur[r]=f + for part in ('A','centers','B','final'): + event(['phase',part]) + if part in ('A','B'): + for e in c[part]: + regs=[e[1]]+([e[2]] if e[0]=='add' else []) + for r in regs:climb(r,e[-1],part) + scal(scalar(e,back),e[-1]) + elif part=='centers': + tl=c.get('target_levels',{}) + lv=lambda r:tl.get(str(r-v),tl.get(r-v,0)) + assert all(cur[r]==lv(r) for r in range(v,2*v)) + for k,r in enumerate(c['centers']): + cf=c['center_frames'][k];assert cur[r]==cf + terms=[e for e in c['cut'] if e[2]==r] + served=sorted({lv(e[1]) for e in terms},key=lambda L:-len(frames[L])) + path=[cf]+[L for L in served if L!=cf] # descending chain of frames + C=['C',k] + if not back: + scal(['clear',C],cf);scal(['copy',C,['L',r]],cf) + for a in range(len(path)): + if a>0:event(['center',k,path[a-1],path[a],len(frames[path[a-1]])-len(frames[path[a]])]) + for e in terms: + if lv(e[1])==path[a] or (a==0 and lv(e[1])==cf):scal(['add',['L',e[1]],C,e[3],e[4]],path[a]) + scal(['clear',C],path[-1]) + else: + scal(['clear',C],path[-1]) + for a in range(len(path)-1,-1,-1): + for e in terms: + if lv(e[1])==path[a] or (a==0 and lv(e[1])==cf):scal(['add',C,['L',e[1]],e[3],e[4]],path[a]) + if a>0:event(['center',k,path[a],path[a-1],len(frames[path[a-1]])-len(frames[path[a]])]) + scal(['add',['L',r],C,-1,1],cf);scal(['clear',C],cf) + else: + for r,f in enumerate(c['final']):climb(r,f,part) + event(['phase','exit_adapters']) + for r,f in enumerate(c['final']):event(['adapter','new_to_old',r,f]) + event(['phase','full_frame_cleanup']) + for e in c['cleanup']: + regs=[e[1]]+([e[2]] if e[0]=='add' else []) + assert all(c['final'][r]==1 for r in regs) + scal(scalar(e,back),1) + event(['phase','done']) + return out + +def check_local(events,back): + # Literal scalar replay of the emitted stream, including center lifetimes. + state={('L',r):{r:Q(1)} for r in range(n)};live=[];paid=Counter();prims=0;macros=0 + for e in events: + if e[0]=='climb':assert not live;paid[e[5]]+=1 + elif e[0]=='center':assert live==[e[1]];paid[e[4]]+=1 + elif e[0]=='scalar': + op=e[1];tag=op[0];t=tuple(op[1]);macros+=1;prims+=2*primitive_bound(op) + if tag=='clear': + state[t]={} + if t[0]=='C' and t[1] in live:live.remove(t[1]) + elif tag=='copy':state[t]=dict(state.get(tuple(op[2]),{}));live.append(t[1]) + elif tag=='scale':state[t]={j:Q(op[2],op[3])*x for j,x in state[t].items()} + else: + s=tuple(op[2]);q=Q(op[3],op[4]);row=state.setdefault(t,{}) + if t[0]=='C' and t[1] not in live:live.append(t[1]) + for j,x in list(state.get(s,{}).items()): + z=row.get(j,Q(0))+q*x + if z:row[j]=z + else:row.pop(j,None) + assert not live and all(not row for r,row in state.items() if r[0]=='C') + for r in range(n): + expected={r:Q(1)} + if back and r=2 and len(G)-np.linalg.matrix_rank(C[:,G])>0] +print('kernel classes available on rest',len(Es)) +best=None +if Es: + for seed in range(2): + b_,asg=sa(col,first,C,cand,Es,iters=40000,seed=seed) + if best is None or b_>best[0]:best=(b_,asg) + groups=asg_to_groups(C,best[1]);pre,start,eps=groups_to_pre(col,groups) +else:pre,start,eps=[],{},0 +prof,widths=gauge_profile(w,pre,start,gauge,None) +r=HM.evaluate(prof,widths,exact=True) +print('gauge+kernel: kernel eps',eps,'total eps',sum(len(E) for E in gauge.values())+eps,'b',r['b'],'%+.3f%%'%(100*(r['b']/r0['b']-1)),r['W'],r['banks'],r['hosted']) +pickle.dump((pre,start,gauge,None),open('v4_sel.pkl','wb')) diff --git a/research/e8-complex-gauge-kernel/code/gauge.py b/research/e8-complex-gauge-kernel/code/gauge.py new file mode 100644 index 0000000000..45ddbacdd6 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/gauge.py @@ -0,0 +1,51 @@ +from kern import * +import random +def span(F): + s={0} + for b in F:s|={x^b for x in s} + return frozenset(s) +def perp(x):return basis([z for z in range(1,512) if not (x&z).bit_count()%2]) +def prep(w): + col,first=setup(w);R=len(col);P=w['ports'] + A,B=w['A'],w['B'];FB={} + for seq in (A,B): + for e in seq: + for r in ([e[1]]+([e[2]] if e[0]=='add' else [])): + if v<=r<2*v and e[-1] is not None:FB.setdefault(r-v,basis(e[-1])) + cap={t:span(FB[t]) if t in FB else span(perp(P[t])) for t in range(v)} + cands={} + for h in range(R): + S=list(col[h]);inter=set(span(first[h])) + for t in S:inter&=cap[t] + # candidate subspaces of first[h] inside inter + cs=[] + for E in subspaces(first[h]): + if span(E)<=inter:cs.append(E) + cands[h]=sorted(cs,key=lambda E:-len(E)) + return col,first,cap,cands,P +def nested(sa,sb):return sa<=sb or sb<=sa +def greedy(col,cands,seed=0,order=None): + R=len(col);rnd=random.Random(seed) + chain={t:[] for t in range(v)} # list of span-sets + E_of={} + hs=list(range(R)) + if order is None: + hs.sort(key=lambda h:(len(col[h]),rnd.random())) + else:hs=order + eps=0 + for h in hs: + for E in cands[h]: + sE=span(E) + if all(all(nested(sE,c) for c in chain[t]) for t in col[h]): + E_of[h]=E;eps+=len(E) + for t in col[h]:chain[t].append(sE) + break + return E_of,eps +if __name__=='__main__': + w=load();col,first,cap,cands,P=prep(w) + print('cands sizes',Counter(len(c) for c in cands.values()).most_common(6)) + print('upper (sum of max cand rank)',sum(len(c[0]) if c else 0 for c in cands.values())) + best=0 + for seed in range(30): + E_of,eps=greedy(col,cands,seed) + if eps>best:best=eps;print(seed,eps,len(E_of),Counter(len(E) for E in E_of.values())) diff --git a/research/e8-complex-gauge-kernel/code/gauge2.py b/research/e8-complex-gauge-kernel/code/gauge2.py new file mode 100644 index 0000000000..84c9c839e6 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/gauge2.py @@ -0,0 +1,72 @@ +"""Gauge variant (entrance comps at nonzero frames on a global flag; centre copies climb the flag). Profile model on top of the exact word.""" +from cxcheck import * +from gauge import prep,span,perp +import pickle +import hostmodel as HM +def gauge_profile(w,pre,start_kernel,gauge,flag): + """gauge: {helper h: E basis tuple (member of flag)}; start_kernel: {reg: E} kernel pivots (registers absolute ids); returns profile dict + widths""" + rep,edges=check(w,pre=pre,start=start_kernel,full=False) # kernel/baseline chain profile (comps ignored) + paid=Counter(rep['local_profile']) + # remove climbs saved for gauge helpers: recompute their chains with start E + A,B=w['A'],w['B'];P=w['ports'];R=w['report']['R'];n=240+R + # recompute full chain simulation with gauge starts and target levels + col,first=__import__('kern').setup(w) + supp={h:set(col[h]) for h in range(R)} + lev={t:() for t in range(v)} + for h,E in gauge.items(): + for t in supp[h]: + if len(E)>len(lev[t]): + assert span(lev[t])<=span(E) + lev[t]=E + st={} + for h,E in start_kernel.items():st[h]=E + cur={} + for r in range(n): + if r top served level -> ... -> lowest served level (mass-neutral descent) + for r,f in zip(ce,cf): + f=basis(list(f)) + tl={lev[e[1]-v] for e in w['cut'] if e[2]==r} + path=sorted(tl,key=lambda L:-len(L)) + o=f + for L in path: + assert basis(list(L)+list(o))==o,('centre frame does not contain served level',r,L,o) + if L!=o:paid[len(o)-len(L)]+=1;o=L + for e in seq: + for r in ([e[1]]+([e[2]] if e[0]=='add' else [])):climb(r,e[-1]) + for r in range(n):climb(r,FULL if (r=2*v) else perp(P[r-v])) + widths=[9-len(gauge.get(h,st.get(2*v+h,()))) for h in range(R)] + return dict(sorted(paid.items())),widths +if __name__=='__main__': + w=load();w['R']=w['report']['R'] + sc,flag,per=pickle.load(open('flag_best.pkl','rb')) + col,first,cap,cands,P=prep(w);R=len(col) + gauge={} + for h,k in per.items(): + if k>0: + E=[F for F in flag if len(F)==k][0];gauge[h]=E + prof,widths=gauge_profile(w,(),{},gauge,flag) + print('gauge helpers',len(gauge),'eps',sum(len(E) for E in gauge.values()),'profile',prof) + r0=HM.evaluate({1:1912,2:784,3:168,4:224,5:392,8:24},[9]*560) + r=HM.evaluate(prof,widths) + print('b',r['b'],'%+.3f%%'%(100*(r['b']/r0['b']-1)),r['W'],r['banks']) diff --git a/research/e8-complex-gauge-kernel/code/gauge3.py b/research/e8-complex-gauge-kernel/code/gauge3.py new file mode 100644 index 0000000000..5b17140c0c --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/gauge3.py @@ -0,0 +1,53 @@ +"""per-target nested + per-centre chain gauge assignment (greedy / SA-lite)""" +from gauge import * +import random,pickle +def setup_g(w): + col,first,cap,cands,P=prep(w);R=len(col) + served={} + for i,r in enumerate(w['centers']): + served[i]=sorted({e[1]-v for e in w['cut'] if e[2]==r}) + cf=[span(f) for f in w['center_frames']] + tcent=defaultdict(list) + for i,ts in served.items(): + for t in ts:tcent[t].append(i) + return col,first,cap,cands,P,served,cf,tcent +def greedy_g(col,cands,served,cf,tcent,order,minrank=1): + R=len(col) + L={t:frozenset({0}) for t in range(v)} # span sets of current level + Lall={t:[] for t in range(v)} + Lb={t:() for t in range(v)} + E_of={};eps=0 + for h in order: + for E in cands[h]: + sE=span(E) + ok=True;newL={} + for t in col[h]: + cur=L[t] + if not all(sE<=c or c<=sE for c in Lall[t]):ok=False;break + newL[t]=sE if len(sE)>len(cur) else cur + if not ok:continue + # centre chains + containment in cf + cs={i for t in newL for i in tcent[t]} + for i in cs: + levels=[newL.get(u,L[u]) for u in served[i]] + top=max(levels,key=len) + if not all(a<=b or b<=a for a in set(levels) for b in set(levels)):ok=False;break + if not top<=cf[i]:ok=False;break + if not ok:continue + E_of[h]=E;eps+=len(E) + for t,s in newL.items(): + L[t]=s;Lb[t]=E if len(s)>len(span(Lb[t])) else Lb[t] + for t in col[h]:Lall[t].append(sE) + break + return E_of,eps +if __name__=='__main__': + w=load();w['R']=w['report']['R'] + col,first,cap,cands,P,served,cf,tcent=setup_g(w);R=len(col) + print('centres served sizes',[len(served[i]) for i in served]) + best=(0,None) + for seed in range(300): + rnd=random.Random(seed);order=list(range(R));rnd.shuffle(order) + if seed%3==0:order.sort(key=lambda h:len(col[h])) + E_of,eps=greedy_g(col,cands,served,cf,tcent,order) + if eps>best[0]:best=(eps,E_of);print(seed,eps,len(E_of),Counter(len(E) for E in E_of.values()),flush=True) + pickle.dump(best,open('gauge3_best.pkl','wb')) diff --git a/research/e8-complex-gauge-kernel/code/gauge4.py b/research/e8-complex-gauge-kernel/code/gauge4.py new file mode 100644 index 0000000000..f1be84aba6 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/gauge4.py @@ -0,0 +1,53 @@ +from gauge3 import * +import time +def state_from(E_of,col,tcent,served,cf): + pass +def refill(col,cands,served,cf,tcent,E_of,drop,rnd): + keep={h:E for h,E in E_of.items() if h not in drop} + R=len(col) + # rebuild L from keep + L={t:frozenset({0}) for t in range(v)} + Lall={t:[] for t in range(v)} + for h,E in keep.items(): + sE=span(E) + for t in col[h]: + Lall[t].append(sE) + if len(sE)>len(L[t]):L[t]=sE + cur=dict(keep);eps=sum(len(E) for E in keep.values()) + order=[h for h in range(R) if h not in keep];rnd.shuffle(order) + for h in order: + for E in cands[h]: + sE=span(E);ok=True;newL={} + for t in col[h]: + c=L[t] + if not all(sE<=x or x<=sE for x in Lall[t]):ok=False;break + newL[t]=sE if len(sE)>len(c) else c + if not ok:continue + cs={i for t in newL for i in tcent[t]} + for i in cs: + levels=[newL.get(u,L[u]) for u in served[i]] + top=max(levels,key=len) + if not all(a<=b or b<=a for a in set(levels) for b in set(levels)) or not top<=cf[i]:ok=False;break + if not ok:continue + cur[h]=E;eps+=len(E) + for t,s in newL.items():L[t]=s + for t in col[h]:Lall[t].append(sE) + break + return cur,eps +if __name__=='__main__': + import sys + seed=int(sys.argv[1]);iters=int(sys.argv[2]) + w=load();w['R']=w['report']['R'] + col,first,cap,cands,P,served,cf,tcent=setup_g(w);R=len(col) + eps0,E_of=pickle.load(open('gauge3_best.pkl','rb')) + rnd=random.Random(seed);best=(eps0,E_of);t0=time.time() + cands_h=[h for h in range(R) if cands[h]] + for it in range(iters): + cur=best[1] + drop=set(rnd.sample(list(cur),max(1,int(len(cur)*rnd.choice([0.05,0.1,0.2,0.4]))))) + n,e=refill(col,cands,served,cf,tcent,cur,drop,rnd) + if e>=best[0]: + if e>best[0]:print(it,e,len(n),Counter(len(E) for E in n.values()),round(time.time()-t0),flush=True) + best=(e,n) + pickle.dump(best,open('gauge4_best_%d.pkl'%seed,'wb')) + print('final',best[0]) diff --git a/research/e8-complex-gauge-kernel/code/hostmodel.py b/research/e8-complex-gauge-kernel/code/hostmodel.py new file mode 100644 index 0000000000..6d7723269d --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/hostmodel.py @@ -0,0 +1,88 @@ +"""Hosted complex model: replicates idle-axis-hosting/complex_hosted.py profile arithmetic for arbitrary local paid profile +and per-helper residual widths (same for every stage/replica). Needs scipy (venv python).""" +import gzip,json,sys,math,importlib.util,time +from collections import Counter +from pathlib import Path +from fractions import Fraction as Q +import os +CX=os.environ['CXG_BASE'];REPO=os.environ.get('CXG_REPO','.') +sys.path.insert(0,REPO+'/research/idle-axis-hosting') +import hosting_core as HC +V,K,T,L,M=120,4,8,9,73 +DATA=2*K*V*T +PAR=Path(CX+'/parent/changed') +_t=json.loads(gzip.decompress((PAR/'split-centres/GLOBAL-TEMPLATE.json.gz').read_bytes())) +def _eng(name): + sp=importlib.util.spec_from_file_location('h_'+name,PAR/'work/pr361/code/engines'/(name+'.py'));mod=importlib.util.module_from_spec(sp);sp.loader.exec_module(mod);return mod +ENG=_eng('moment') +rank={};pair_of_stage={e['stage']:e['pair'] for e in _t['recipe'] if e['kind']=='local'} +sched=[];side_of={};pair_of={} +for it in _t['idle_children']:rank.setdefault(it['child'],it['rank']) +for e in _t['execution_schedule']: + if e['kind']=='local_cover_sweep':sched.append(dict(kind='local',stage=e['stage'],pair=pair_of_stage[e['stage']])) + elif e['kind']=='bridge':sched.append(dict(kind='bridge',dst=e['dst'],src=e['src'])) + elif e['kind']=='idle_cover_sweep': + for sd,ch in enumerate(e['child_indices']): + if ch in rank:sched.append(dict(kind='idle',pair=e['pair'],rank=rank[ch],idx=ch));side_of[ch]=sd;pair_of[ch]=e['pair'] +WINALL=HC.windows(sched) +IDLE=_t['idle_children'] +def evaluate(local_profile,widths,discover=True,sel=None,verbose=False,pad_val=None,exact=False): + """local_profile: {rank:count per call}; widths: list of residual widths (len R) -> dict(b,W,profile)""" + R=len(widths);wmin=min(widths) + win={k:x for k,x in WINALL.items() if x[1]>=wmin and x[2]} + cnt=Counter(widths) + avail={(s,w):c for s in range(L) for w,c in cnt.items()} + # identify helper ids per width + hl={}; + ids=sorted(range(R),key=lambda h:(-widths[h],h)) + helpers={s:[(h,widths[h]) for h in ids] for s in range(L)} + if discover: + ys,zs,val=HC.solve(win,V,avail,M,pad_value=(HC.cost(1,M)/8 if pad_val is None else pad_val)) + else:ys,zs,val=sel + slots,rem=HC.expand(win,ys,zs,helpers,V) + hosted={(s,h) for i,port,off,w,s,h in slots} # per replica? slots are per (idx,copy) -> one replica set + hosted_cnt=Counter((s,h) for i,c,o,w,s,h in slots) + # each slot hosts replicas T times (same for all replicas) + base=DATA;partial=Counter();banks=[] + paid=Counter() + W=DATA + for s in range(L): + left=[(rep,h) for rep in range(T) for h in range(R) if (s,h) not in {(s2,h2) for (_,_,_,_,s2,h2) in []}] + # build hosted set + hset={(s,h) for (_,_,_,_,s,h) in slots} + for s in range(L): + items=sorted([widths[h] for rep in range(T) for h in range(R) if (s,h) not in hset],reverse=True) + bins=[] + for w in items: + for i,b in enumerate(bins): + if b+w<=M:bins[i]+=w;break + else:bins.append(w) + W+=len(bins);banks.append(len(bins)) + for b in bins: + if M-b:paid[M-b]+=1 + for r_,z in local_profile.items():paid[int(r_)]+=int(z)*L*T + nid=0 + for it in IDLE: + ch,port=it['child'],it['port'];rr=rem.get((ch,port),it['rank']) if ch in win else it['rank'] + if rr:paid[rr]+=T + mass=sum(r*z for r,z in paid.items());D=M*W-mass + out=dict(W=W,D=D,profile=dict(sorted(paid.items())),hosted=len(slots)*T,banks=banks,val=val,sel=(ys,zs,val)) + if exact: + rc=ENG.certify({int(r):n for r,n in paid.items()},M,W,True);out['b']=float(Q(int(Q(rc['lower'])*10**18),10**18)) + else:out['b']=float(bfloat(paid,W)) + return out +def bfloat(paid,W,m=M): + lo,hi=0.,.01 + for _ in range(80): + q=(lo+hi)/2 + mom=sum(r*z/(m*W)*(m/r)**q for r,z in paid.items())+32*m*sum(paid.values())/(10**16*W)*m**q + if mom<1:lo=q + else:hi=q + return (lo+hi)/2 +if __name__=='__main__': + t0=time.time() + loc={1:1912,2:784,3:168,4:224,5:392,8:24} + r=evaluate(loc,[9]*560,discover=False,sel=HC.load_selection(REPO+'/research/idle-axis-hosting/SELECTION.json','complex'),exact=True) + print('frozen sel:',r['W'],r['D'],r['b'],r['banks'],r['hosted'],time.time()-t0) + r=evaluate(loc,[9]*560,discover=True,exact=True) + print('discover:',r['W'],r['D'],r['b'],r['banks'],r['hosted'],time.time()-t0) diff --git a/research/e8-complex-gauge-kernel/code/iso1.py b/research/e8-complex-gauge-kernel/code/iso1.py new file mode 100644 index 0000000000..715255d443 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/iso1.py @@ -0,0 +1,53 @@ +from kern import * +import numpy as np +def gram(E): + return [[bin(a&b).count('1')&1 for b in E] for a in E] +def det2(G): + n=len(G);M=[r[:] for r in G] + for c in range(n): + p=next((i for i in range(c,n) if M[i][c]),None) + if p is None:return 0 + M[c],M[p]=M[p],M[c] + for i in range(n): + if i!=c and M[i][c]:M[i]=[x^y for x,y in zip(M[i],M[c])] + return 1 +def spanvecs(E): + s={0} + for b in E:s|={x^b for x in s} + return s +ONE=511 +def isometric(E): + """E has orthonormal basis and E^perp (in F2^9) too""" + if not E:return True + if not det2(gram(E)):return False + S=spanvecs(E) + if ONE in S:return False # complement alternating + if all(bin(x).count('1')%2==0 for x in S):return False # E alternating + # E^perp contains ONE? ONE in E^perp iff all vectors of E even-weight -> excluded above + return True +if __name__=='__main__': + w=load();col,first=setup(w);R=len(col) + # which first-frame subspaces are isometric + cand={} + for h,F in first.items(): + for E in subspaces(F):cand.setdefault(E,[]).append(h) + iso={E:G for E,G in cand.items() if isometric(E)} + print('first-frame subspaces',len(cand),'isometric',len(iso),Counter(len(E) for E in iso)) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + pot=[] + for E,G in iso.items(): + if len(G)>=2: + rk=np.linalg.matrix_rank(C[:,G]); + if len(G)-rk>0:pot.append((len(E)*(len(G)-rk),E,len(G),rk)) + print('kernel potentials with isometric E',sorted(pot,reverse=True)[:10]) + # gauge: eligible helpers per isometric E + from gauge import prep,span + col2,first2,cap,cands,P=prep(w) + best=[] + for E in iso: + sE=span(E) + el=[h for h in range(R) if sE<=span(first2[h]) and all(sE<=cap[t] for t in col2[h])] + best.append((len(E)*len(el),len(el),E)) + best.sort(reverse=True);print('gauge best isometric E (eps,n,E)',best[:8]) diff --git a/research/e8-complex-gauge-kernel/code/kcap.py b/research/e8-complex-gauge-kernel/code/kcap.py new file mode 100644 index 0000000000..37e9724fcc --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kcap.py @@ -0,0 +1,13 @@ +from fractions import Fraction as Q +def kappa_of(s_in,eta=Q(1,10**18),beta=Q(1,10**18)): + s=Q(s_in)-Q(1,10**24) + a=Q(384599,10**10) + for _ in range(6):a=(1-s)*s+s*a + q=a*(1-2*eta);margin=(1-eta)*q/(1+q);grid=Q(1,10**18);ticks=margin/grid + return grid*((ticks.numerator-1)//ticks.denominator) +def cap_from_b(b):return float(kappa_of((1-Q(1,10**18))*Q(b))) +if __name__=='__main__': + c=Q('1239085165241180038597')/Q(10**24) + print('check kappa(c bit)',float(kappa_of(c)),'expected 0.001237551733247294') + for name,b in (('baseline cx hosted',1.267453159029119e-3),('kernel77',1.309121737234019e-3),('gauge70+kernel44 (model)',1.329163e-3)): + print(name,'b',b,'kappa cap',cap_from_b(Q(b).limit_denominator(10**18))) diff --git a/research/e8-complex-gauge-kernel/code/kern.py b/research/e8-complex-gauge-kernel/code/kern.py new file mode 100644 index 0000000000..3ca9dfeb0d --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern.py @@ -0,0 +1,53 @@ +from cxlab import * +import numpy as np, itertools, sys +def subspaces(F): + """all nonzero subspaces of span(F) as canonical bases, any rank""" + F=list(F);k=len(F);out=set() + vecs=[] + for mask in range(1,1<>i&1:x^=F[i] + vecs.append(x) + # enumerate subspaces by generating from subsets of vectors up to rank k + seen=set() + def rec(cur,start): + for i in range(start,len(vecs)): + b=basis(list(cur)+[vecs[i]]) + if len(b)==len(cur)+1 and b not in seen: + seen.add(b);rec(b,i+1) + rec((),0) + return seen +def setup(w): + A,B=w['A'],w['B'];n=240+w['report']['R'];R=w['report']['R'] + M=ident(n) + for e in A+w['cut']+B:apply(e,M) + dirty=[{} for r in range(n)] + for r in range(v,2*v):dirty[r]={j:x for j,x in M[r].items() if j>=2*v} + for e in w['precondition']:apply(e,dirty) + col=[dict() for _ in range(R)] + for r in range(v,2*v): + for j,x in dirty[r].items():col[j-2*v][r-v]=x + first={} + for seq in (A,B): + for e in seq: + F=basis(e[-1]) + for r in ([e[1]]+([e[2]] if e[0]=='add' else [])): + if r>=2*v:first.setdefault(r-2*v,F) + return col,first +if __name__=='__main__': + w=load();col,first=setup(w);R=len(col) + cand={} + for h,F in first.items(): + for E in subspaces(F):cand.setdefault(E,[]).append(h) + print('first-frame subspaces',len(cand),Counter(len(E) for E in cand)) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + pot=[] + for E,G in cand.items(): + if len(G)<2:continue + rk=np.linalg.matrix_rank(C[:,G]);pot.append((len(E)*(len(G)-rk),E,len(G),rk)) + pot.sort(reverse=True) + print('E with positive potential',sum(1 for p in pot if p[0]>0),Counter((len(p[1]),p[2]-p[3]) for p in pot if p[0]>0)) + print(pot[:5]) diff --git a/research/e8-complex-gauge-kernel/code/kern2.py b/research/e8-complex-gauge-kernel/code/kern2.py new file mode 100644 index 0000000000..23ac0ea211 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern2.py @@ -0,0 +1,43 @@ +from kern import * +from cxcheck import check +import hostmodel as HM +def span(F): + s={0} + for b in F:s|={x^b for x in s} + return s +def dup_pairs(col): + key=defaultdict(list) + for h,c in enumerate(col): + if not c:continue + ts=sorted(c);f=c[ts[0]] + key[tuple((t,c[t]/f) for t in ts)].append(h) + out=[] + for g in key.values(): + if len(g)==2:out.append(tuple(g)) + return out +def build_pre(col,plan,first): + """plan: list of (pivot, donor, E_basis_tuple) with proportional columns. returns pre ops (register ids absolute) and start dict""" + pre=[];start={} + for p,d,E in sorted(plan,key=lambda x:len(x[2])): + cp,cd=col[p],col[d] + t0=sorted(cd)[0];c=cp[t0]/cd[t0] + assert all(cp.get(t,0)==c*cd.get(t,0) for t in set(cp)|set(cd)) + q=-c + pre.append(['add',2*v+d,2*v+p,q.numerator,q.denominator,list(E)]) + start[2*v+p]=list(E) + return pre,start +if __name__=='__main__': + w=load();w['R']=w['report']['R'];col,first=setup(w) + base,_=check(w);print('base profile',base['local_profile']) + pairs=dup_pairs(col);print(len(pairs)) + plan=[] + for a,b in pairs: + I=span(first[a])&span(first[b]);x=[y for y in I if y] + assert len(x)==1 + plan.append((a,b,(x[0],))) + pre,start=build_pre(col,plan,first) + rep,edges=check(w,pre=pre,start=start) + print('exact ok',rep.get('exact_arbitrary_dirt'),rep['exact_inverse_transpose'],'comp cols',rep['comp_cols'],'profile',rep['local_profile'],'sumwidth',sum(rep['widths'])) + r0=HM.evaluate({1:1912,2:784,3:168,4:224,5:392,8:24},[9]*560) + r=HM.evaluate(rep['local_profile'],rep['widths']) + print('b base',r0['b'],'b kernel56',r['b'],'%+.3f%%'%(100*(r['b']/r0['b']-1)),'W',r['W'],r['banks']) diff --git a/research/e8-complex-gauge-kernel/code/kern3.py b/research/e8-complex-gauge-kernel/code/kern3.py new file mode 100644 index 0000000000..eaf0ff7d28 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern3.py @@ -0,0 +1,73 @@ +"""General exact kernel (shared-donor) for the complex word: partition helpers by rank-1 entrance E; donors independent, pivots dependent.""" +from kern2 import * +import itertools,random +def solve_rel(col,donors,p): + """exact rational coefficients c_d with col[p]=sum c_d col[d] (or None)""" + cols=sorted({t for d in donors for t in col[d]}|set(col[p])) + if not cols:return {} + idx={t:i for i,t in enumerate(cols)} + M=[[Q(0)]*(len(donors)+1) for _ in cols] + for ci,d in enumerate(donors): + for t,x in col[d].items():M[idx[t]][ci]=x + for t,x in col[p].items():M[idx[t]][-1]=x + piv=[];r=0 + for c in range(len(donors)): + pr=next((i for i in range(r,len(M)) if M[i][c]),None) + if pr is None:continue + M[r],M[pr]=M[pr],M[r];q=M[r][c];M[r]=[x/q for x in M[r]] + for i in range(len(M)): + if i!=r and M[i][c]: + f=M[i][c];M[i]=[a-f*b for a,b in zip(M[i],M[r])] + piv.append(c);r+=1 + for i in range(r,len(M)): + if M[i][-1]:return None + return {donors[c]:M[i][-1] for i,c in enumerate(piv) if M[i][-1]} +def plan_greedy(col,first,seed=0,minE=1): + R=len(col) + cand={} + for h,F in first.items(): + for E in subspaces(F):cand.setdefault(E,[]).append(h) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + Es=[E for E,G in cand.items() if len(G)>=2 and len(G)-np.linalg.matrix_rank(C[:,G])>0] + free=set(range(R));groups=[] + while True: + best=None + for E in Es: + G=[h for h in cand[E] if h in free] + if len(G)<2:continue + rk=np.linalg.matrix_rank(C[:,G]);pv=len(G)-rk + val=pv*len(E) + if best is None or val>best[0]:best=(val,E,G) + if not best or best[0]<=0:break + val,E,G=best + # donors: greedy independent in order of G; pivots rest + donors=[];rows=np.zeros((v,0)) + for h in sorted(G): + c=C[:,h:h+1] + if np.linalg.matrix_rank(np.hstack([rows,c]))>rows.shape[1]:rows=np.hstack([rows,c]);donors.append(h) + pivots=[h for h in G if h not in donors] + groups.append((E,donors,pivots));free-=set(G) + return groups +def groups_to_pre(col,groups): + pre=[];start={};eps=0 + for E,donors,pivots in sorted(groups,key=lambda g:len(g[0])): + for p in pivots: + rel=solve_rel(col,donors,p) + assert rel is not None,(E,p) + start[2*v+p]=list(E);eps+=len(E) + for d,c in rel.items(): + q=-c;pre.append(['add',2*v+d,2*v+p,q.numerator,q.denominator,list(E)]) + return pre,start,eps +if __name__=='__main__': + w=load();w['R']=w['report']['R'];col,first=setup(w) + groups=plan_greedy(col,first) + print([(E,len(d),len(p)) for E,d,p in groups]) + pre,start,eps=groups_to_pre(col,groups) + print('pre ops',len(pre),'eps',eps) + rep,edges=check(w,pre=pre,start=start) + print('exact',rep['exact_arbitrary_dirt'],rep['exact_inverse_transpose'],rep['local_profile'],'sumwidth',sum(rep['widths']),'newedges',len(edges)) + r=HM.evaluate(rep['local_profile'],rep['widths']) + r0=HM.evaluate({1:1912,2:784,3:168,4:224,5:392,8:24},[9]*560) + print('b',r['b'],'%+.3f%%'%(100*(r['b']/r0['b']-1)),'W',r['W'],r['banks']) diff --git a/research/e8-complex-gauge-kernel/code/kern4.py b/research/e8-complex-gauge-kernel/code/kern4.py new file mode 100644 index 0000000000..04d7cec6e3 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern4.py @@ -0,0 +1,46 @@ +from kern3 import * +def rand_plan(col,first,C,cand,Es,rnd): + R=len(col);free=set(range(R));groups=[] + Eorder=Es[:] + while True: + opts=[] + for E in Eorder: + G=[h for h in cand[E] if h in free] + if len(G)<2:continue + rk=np.linalg.matrix_rank(C[:,G]);pv=len(G)-rk + if pv>0:opts.append((pv,E,G)) + if not opts:break + # pick randomly weighted by pivots + tot=sum(o[0] for o in opts);x=rnd.random()*tot;acc=0 + for o in opts: + acc+=o[0] + if acc>=x:pick=o;break + pv,E,G=pick + donors=[];rows=np.zeros((v,0));Gs=G[:];rnd.shuffle(Gs) + for h in Gs: + c=C[:,h:h+1] + if np.linalg.matrix_rank(np.hstack([rows,c]))>rows.shape[1]:rows=np.hstack([rows,c]);donors.append(h) + pivots=[h for h in G if h not in donors] + groups.append((E,donors,pivots));free-=set(G) + # completion: free helpers that can pivot onto existing classes + for h in sorted(free): + for gi,(E,donors,pivots) in enumerate(groups): + if not span(E)<=span(first[h]):continue + M=np.hstack([C[:,donors],C[:,h:h+1]]) + if np.linalg.matrix_rank(M)==len(donors):pivots.append(h);free.discard(h);break + return groups +if __name__=='__main__': + w=load();w['R']=w['report']['R'];col,first=setup(w);R=len(col) + cand={} + for h,F in first.items(): + for E in subspaces(F):cand.setdefault(E,[]).append(h) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + Es=[E for E,G in cand.items() if len(G)>=2 and len(G)-np.linalg.matrix_rank(C[:,G])>0] + best=0;bg=None + for seed in range(300): + rnd=random.Random(seed);g=rand_plan(col,first,C,cand,Es,rnd) + tot=sum(len(p)*len(E) for E,d,p in g) + if tot>best:best=tot;bg=g;print(seed,tot,[(E[0],len(d),len(p)) for E,d,p in g],flush=True) + import pickle;pickle.dump(bg,open('kern4_best.pkl','wb')) diff --git a/research/e8-complex-gauge-kernel/code/kern5.py b/research/e8-complex-gauge-kernel/code/kern5.py new file mode 100644 index 0000000000..5064b882cc --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern5.py @@ -0,0 +1,50 @@ +from kern4 import * +import math,pickle,time +def nullity(C,S): + return len(S)-(np.linalg.matrix_rank(C[:,sorted(S)],tol=1e-8) if S else 0) +def sa(col,first,C,cand,Es,iters=30000,seed=0,init=None,T0=0.6): + rnd=random.Random(seed);big=sorted({h for E in Es for h in cand[E]}) + allowed={h:[E for E in Es if h in set(cand[E])] for h in big} + cls={E:set() for E in Es};asg={} + if init: + for E,d,p in init: + for h in d+p: + if E in cls:cls[E].add(h);asg[h]=E + val={E:nullity(C,cls[E]) for E in Es} + cur=sum(val.values());best=cur;bestasg=dict(asg);t0=time.time() + for it in range(iters): + T=T0*(1-it/iters)+0.02 + h=rnd.choice(big);old=asg.get(h) + choices=allowed[h]+[None] + new=rnd.choice(choices) + if new==old:continue + dv=0;newvals={} + if old is not None: + S=cls[old]-{h};nv=nullity(C,S);newvals[old]=nv;dv+=nv-val[old] + if new is not None: + S=cls[new]|{h};nv=nullity(C,S);newvals[new]=nv;dv+=nv-val[new] + if dv>=0 or rnd.random()best:best=cur;bestasg=dict(asg) + return best,bestasg +if __name__=='__main__': + w=load();w['R']=w['report']['R'];col,first=setup(w);R=len(col) + cand={} + for h,F in first.items(): + for E in subspaces(F):cand.setdefault(E,[]).append(h) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + Es=[E for E,G in cand.items() if len(G)>=2 and len(G)-np.linalg.matrix_rank(C[:,G])>0] + init=pickle.load(open('kern4_best.pkl','rb')) + print('init',sum(len(p) for E,d,p in init)) + b0=None + for seed in range(4): + best,asg=sa(col,first,C,cand,Es,iters=40000,seed=seed,init=init) + print('seed',seed,'best nullity sum',best,flush=True) + if b0 is None or best>b0[0]:b0=(best,asg) + pickle.dump(b0,open('kern5_best.pkl','wb')) diff --git a/research/e8-complex-gauge-kernel/code/kern6.py b/research/e8-complex-gauge-kernel/code/kern6.py new file mode 100644 index 0000000000..e2699ebaff --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/kern6.py @@ -0,0 +1,29 @@ +from kern5 import * +def asg_to_groups(C,asg): + cls={} + for h,E in asg.items():cls.setdefault(E,[]).append(h) + groups=[] + for E,G in cls.items(): + G=sorted(G);donors=[];rows=np.zeros((v,0)) + for h in G: + c=C[:,h:h+1] + if np.linalg.matrix_rank(np.hstack([rows,c]),tol=1e-8)>rows.shape[1]:rows=np.hstack([rows,c]);donors.append(h) + pivots=[h for h in G if h not in donors] + if pivots:groups.append((E,donors,pivots)) + return groups +if __name__=='__main__': + w=load();w['R']=w['report']['R'];col,first=setup(w);R=len(col) + C=np.zeros((v,R)) + for h in range(R): + for t,x in col[h].items():C[t,h]=float(x) + best,asg=pickle.load(open('kern5_best.pkl','rb')) + groups=asg_to_groups(C,asg) + print([(E[0],len(d),len(p)) for E,d,p in groups]) + pre,start,eps=groups_to_pre(col,groups) + print('pre ops',len(pre),'eps',eps) + rep,edges=check(w,pre=pre,start=start) + print('exact',rep['exact_arbitrary_dirt'],rep['exact_inverse_transpose'],rep['local_profile'],'sumwidth',sum(rep['widths']),'newedges',len(edges)) + r=HM.evaluate(rep['local_profile'],rep['widths'],exact=True) + r0=HM.evaluate({1:1912,2:784,3:168,4:224,5:392,8:24},[9]*560,exact=True) + print('b',r['b'],'%+.3f%%'%(100*(r['b']/r0['b']-1)),'W',r['W'],r['banks'],'hosted',r['hosted']) + pickle.dump((pre,start),open('kern6_word.pkl','wb')) diff --git a/research/e8-complex-gauge-kernel/code/mkword.py b/research/e8-complex-gauge-kernel/code/mkword.py new file mode 100644 index 0000000000..1d6c1b51c8 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/mkword.py @@ -0,0 +1,26 @@ +"""Write a modified FULL-WORD.json.gz (kernel variant): A = pre + A, cleanup rebuilt, start_extra for pivots, report updated.""" +import gzip,json,sys,pickle,os +from cxlab import * +from cxcheck import check,FULL +def mkword(w,pre,start,outdir): + w2=copy.deepcopy(w) + w2['R']=w['report']['R'] + rep,edges=check(w,pre=pre,start=start,full=True) + w2['A']=[list(e) for e in pre]+w2['A'] + # cleanup: inverse of non-target ops of A+B reversed at FULL + cl=[] + for e in reversed(w2['A']+w2['B']): + if not v<=e[1]<2*v: + c=inv(e);c[-1]=list(FULL);cl.append(c) + w2['cleanup']=cl + w2['start_extra']={str(r):list(E) for r,E in start.items()} + w2['report']['local_profile']={str(k):x for k,x in rep['local_profile'].items()} + w2['report']['pivots']=len(start);w2['report']['entrance_rank']=sum(len(E) for E in start.values()) + os.makedirs(outdir,exist_ok=True) + (open(outdir+'/FULL-WORD.json.gz','wb')).write(gzip.compress(json.dumps(w2,sort_keys=True,separators=(',',':')).encode(),mtime=0)) + return rep +if __name__=='__main__': + w=load() + pre,start=pickle.load(open('kern6_word.pkl','rb')) + rep=mkword(w,pre,start,sys.argv[1]) + print('written',rep['local_profile'],len(pre),len(start)) diff --git a/research/e8-complex-gauge-kernel/code/mkword_gauge.py b/research/e8-complex-gauge-kernel/code/mkword_gauge.py new file mode 100644 index 0000000000..a92d986dab --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/mkword_gauge.py @@ -0,0 +1,30 @@ +import gzip,json,sys,pickle,os +from cxlab import * +from cxcheck import check,FULL +from kern import setup +def mkword_gauge(w,pre,start_k,gauge,outdir): + w2=copy.deepcopy(w);R=w['report']['R'];w2['R']=R + col,first=setup(w) + gc=[] + for h,E in sorted(gauge.items(),key=lambda kv:(len(kv[1]),kv[0])): + for t,x in sorted(col[h].items()): + c=-x;gc.append(['add',v+t,2*v+h,c.numerator,c.denominator,list(E)]) + lev={t:() for t in range(v)} + for h,E in gauge.items(): + for t in col[h]: + if len(E)>len(lev[t]):lev[t]=E + w2['A']=gc+[list(e) for e in pre]+w2['A'] + cl=[] + for e in reversed(w2['A']+w2['B']): + if not v<=e[1]<2*v: + c=inv(e);c[-1]=list(FULL);cl.append(c) + w2['cleanup']=cl + st=dict(start_k);st.update({2*v+h:E for h,E in gauge.items()}) + w2['start_extra']={str(r):list(E) for r,E in st.items()} + w2['target_levels']={str(t):list(E) for t,E in lev.items() if E} + from gauge2 import gauge_profile + prof,widths=gauge_profile(w,pre,start_k,gauge,None) + w2['report']['local_profile']={str(k):x for k,x in prof.items()} + os.makedirs(outdir,exist_ok=True) + open(outdir+'/FULL-WORD.json.gz','wb').write(gzip.compress(json.dumps(w2,sort_keys=True,separators=(',',':')).encode(),mtime=0)) + return w2 diff --git a/research/e8-complex-gauge-kernel/code/nec3.py b/research/e8-complex-gauge-kernel/code/nec3.py new file mode 100644 index 0000000000..4673e03fb0 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/nec3.py @@ -0,0 +1,15 @@ +import sys +import os +sys.path.insert(0,os.environ['CXG_REPO']+'/research/split-centres-global/vendor') +from clifford_exact import * +n=9 +FULL=tuple(1<=2: + kp+=max(0,len(G)-np.linalg.matrix_rank(C[:,G]))*len(E) +print('relaxed-isometric kernel potential',kp) +col2,first2,cap,cands,P=prep(w) +ge=0 +for E in cand: + if relaxed(E): + sE=span(E);el=[h for h in range(R) if sE<=span(first2[h]) and all(sE<=cap[t] for t in col2[h])] + ge=max(ge,len(el)*len(E)) +print('relaxed-isometric gauge best eps',ge) +# (2) adapter factorisation for every entrance used in v4 +d=json.load(gzip.open(os.path.join(os.environ['CXG_PKG'],'data/words/gauge80-kernel42/FULL-WORD.json.gz'))) +Es={tuple(E) for E in d['start_extra'].values()} +print('distinct entrance frames in v4',len(Es)) +ok=0 +for E in sorted(Es,key=lambda e:(len(e),e)): + op=canonical_frame(n,tuple(E)).adjoint().then(canonical_frame(n,FULL)) + rec,compiled=lower_child(op) + assert rec['rank']==n-len(E) and compiled.tab==op.tab and compiled.zero_state()==op.zero_state() + ok+=1 +print('E->FULL composite = monomial . C_[9-e] . monomial exactly for',ok,'frames (adapters entrance/exit give C_[w] on w coordinates)') diff --git a/research/e8-complex-gauge-kernel/code/phases_cxg.py b/research/e8-complex-gauge-kernel/code/phases_cxg.py new file mode 100644 index 0000000000..b5a5bd64b4 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/phases_cxg.py @@ -0,0 +1,39 @@ +# Adapted from PR425 for heterogeneous centres by icekylinx with Codex assistance. +"""Exact phase-aware lowering of every used edge of the new structural word.""" +import gzip,hashlib,json,sys,time +from collections import Counter +from pathlib import Path +sys.path.insert(0,str(Path('work/odd-stages').resolve())) +from clifford_exact import * + +OUT=Path('split-centres');P=OUT/'CANONICAL-WORD.json.gz' +c=json.load(gzip.open(P));n=c['h'];frames=c['frames'];start=time.monotonic();ops=[] +for U in frames: + op=canonical_frame(n,U) + L=[u|(u<=2*v) else perp(self.P[r-v]) + self.pos=defaultdict(list) + for i,rs in enumerate(self.regs): + for r in rs:self.pos[r].append(i) + self.mk={} + self.m=m + self.centers_pin={} + # pinned: centre registers final frame within A (last A gate) + for r,f in zip(w['centers'],w['center_frames']): + for i in reversed(self.pos[r]): + if i0 else 0.0 + def chain_cost(self,r,F=None): + """cost of register r's chain with current frames (F: optional override list)""" + F=F or self.F + cur=self.start[r];c=0.0;cnt=Counter() + for i in self.pos[r]: + f=F[i] + if f!=cur: + d=len(f)-len(cur);c+=self.phi(d);cnt[d]+=1;cur=f + f=self.final[r] + if f!=cur: + d=len(f)-len(cur);c+=self.phi(d);cnt[d]+=1 + return c,cnt + def total(self): + c=0.;cnt=Counter() + for r in range(self.n): + a,b=self.chain_cost(r);c+=a;cnt+=b + return c,cnt + def legal(self): + for r in range(self.n): + cur=self.sm(self.start[r]) + for i in self.pos[r]: + f=self.sm(self.F[i]) + if cur&~f:return False,(r,i) + cur=f + if cur&~self.sm(self.final[r]):return False,(r,'final') + return True,None + +def family(W): + fam={} + for F in W.F:fam[W.sm(F)]=tuple(F) + for r in range(W.n): + for F in (W.start[r],W.final[r]):fam[W.sm(F)]=tuple(F) + return fam +def bounds(W,i): + rs=W.regs[i];L=0;U=None + for r in rs: + ps=W.pos[r];k=ps.index(i) + prev=W.F[ps[k-1]] if k>0 else W.start[r] + L|=0 + nxt=W.F[ps[k+1]] if k+10 else W.start[r]) + Lb=basis(Lb) + return Lb,U +def local_cost(W,i,F,rs=None): + rs=rs or W.regs[i];old=W.F[i];W.F[i]=F + c=sum(W.chain_cost(r)[0] for r in rs);W.F[i]=old;return c +def descent(W,fam,seed=0,passes=10,verbose=True): + rnd=random.Random(seed);gates=[i for i in range(len(W.ops)) if i not in W.centers_pin] + famitems=list(fam.items()) + for p in range(passes): + rnd.shuffle(gates);moved=0 + for i in gates: + Lb,U=bounds(W,i);Lm=W.sm(Lb) + cur=W.F[i];best=cur;bc=local_cost(W,i,cur) + for m_,F in famitems: + if m_==W.sm(cur):continue + if Lm&~m_ or m_&~U:continue + c=local_cost(W,i,F) + if c>x&1] + return basis(vs) +def descent2(W,seed=0,passes=10,verbose=True,allow_new=True): + rnd=random.Random(seed);gates=[i for i in range(len(W.ops)) if i not in W.centers_pin] + for p in range(passes): + rnd.shuffle(gates);moved=0 + for i in gates: + Lb,U=bounds(W,i) + cur=W.F[i];best=cur;bc=local_cost(W,i,cur) + cands=[Lb,basis_of_mask(U)] + for F in cands: + if W.sm(F)==W.sm(cur):continue + c=local_cost(W,i,F) + if cframe or None if illegal.""" + ch={i:Fnew};queue=[i] + while queue: + g=queue.pop();Fg=ch[g] + for r in W.regs[g]: + ps=W.pos[r];k=ps.index(g) + if k+1maxn:return None + else: + if W.sm(Fg)&~W.sm(W.final[r]):return None + return ch +def lower_closure(W,i,Fnew,maxn=60): + """set F[i]=Fnew (subframe of F[i]) and cascade lowering on predecessors whose frame must sit inside Fnew.""" + ch={i:Fnew};queue=[i] + while queue: + g=queue.pop();Fg=ch[g] + if g in W.centers_pin and g!=i:return None + for r in W.regs[g]: + ps=W.pos[r];k=ps.index(g) + if k>0: + h=ps[k-1];Fh=ch.get(h,W.F[h]) + if W.sm(Fh)&~W.sm(Fg): + if h in W.centers_pin:return None + nf=basis_of_mask(W.sm(Fh)&W.sm(Fg));ch[h]=nf;queue.append(h) + if len(ch)>maxn:return None + else: + if W.sm(W.start[r])&~W.sm(Fg):return None + return ch +def apply_ch(W,ch): + old={g:W.F[g] for g in ch} + for g,f in ch.items():W.F[g]=f + return old +def delta(W,ch): + regs=set() + for g in ch:regs|=set(W.regs[g]) + before=sum(W.chain_cost(r)[0] for r in regs) + old=apply_ch(W,ch);after=sum(W.chain_cost(r)[0] for r in regs) + for g,f in old.items():W.F[g]=f + return after-before +def closure_descent(W,seed=0,passes=6,verbose=True,maxn=60): + rnd=random.Random(seed);gates=[i for i in range(len(W.ops)) if i not in W.centers_pin] + for p in range(passes): + rnd.shuffle(gates);moved=0 + for i in gates: + best=None;bd=-1e-9 + cands=[] + for r in W.regs[i]: + ps=W.pos[r];k=ps.index(i) + if k+10:cands.append(('l',W.F[ps[k-1]])) + cands.append(('rf',W.final[r])) if k+1==len(ps) else None + for kind,F in cands: + if W.sm(F)==W.sm(W.F[i]):continue + if kind in('r','rf') and not (W.sm(W.F[i])&~W.sm(F)): + ch=raise_closure(W,i,F,maxn) + elif kind=='l' and not (W.sm(F)&~W.sm(W.F[i])): + ch=lower_closure(W,i,F,maxn) + else:continue + if ch is None:continue + d=delta(W,ch) + if d list of (idx, frame) + for i,e in enumerate(ops): + F=basis(e[-1]) if e[-1] is not None else None + for r in ([e[1]]+([e[2]] if e[0]=='add' else [])): + uses[r].append((i,F)) + for r in range(2*v,n): + if uses[r]:first[r]=uses[r][0][0];last[r]=uses[r][-1][0];ffr[r]=uses[r][0][1] if uses[r][0][1] is not None else () + # centres of A->cut etc: cut ops for center frames use None frame -> treated as () + rows=[{r:Q(1)} if r=2*v and (k not in first or first[k]>i+0 and False):pass + if not span(cur[k])<=span(F2):continue + # k's future frames must contain the frame we will force + cands.append(k) + sol=solve(rows[h1],cands,rows) + if sol is None:continue + # frame of reset ops + ok=True + for k in sol: + Fr=basis(list(F1)+list(cur[k])) + if not span(Fr)<=span(F2):ok=False;break + nf=next((fr for (ii,fr) in uses[k] if ii>i and fr is not None),None) + if nf is not None and not span(Fr)<=span(nf):ok=False;break + if ok:out.append((h1,h2,i,sol,F1)) + return out,tl +def rewrite(word,h1,h2,i,sol,tl): + """alias h2 onto h1; insert reset adds after timeline index i""" + w=copy.deepcopy(word);P=w['ports'] + # recompute frames of reset ops: need cur frames at time i -> replay frames only + n=240+(w['R'] if 'R' in w else w['report']['R']) + cur=[()]*n + for r in range(v):cur[r]=(P[r],) + for j,(ph,e) in enumerate(tl): + if j>i:break + if e[-1] is not None: + F=basis(e[-1]) + for r in ([e[1]]+([e[2]] if e[0]=='add' else [])):cur[r]=F + resets=[] + for k,c in sol.items(): + Fr=list(basis(list(cur[h1])+list(cur[k]))) + c=-c + resets.append(['add',h1,k,c.numerator,c.denominator,Fr]) + # split timeline back to lists + lst={'A':[], 'C':[], 'B':[]} + for j,(ph,e) in enumerate(tl): + e=copy.deepcopy(e) + if j>i: + if e[1]==h2:e[1]=h1 + if e[0]=='add' and e[2]==h2:e[2]=h1 + lst[ph].append(e) + if j==i:lst[ph]+=resets + # for ops before i (h2 unused) fine. compact ids: remove h2 + mp={} + R=(w['R'] if 'R' in w else w['report']['R']) + for r in range(n): + if r==h2:continue + mp[r]=r-(1 if r>h2 else 0) + for ph in lst: + for e in lst[ph]: + e[1]=mp[e[1]] + if e[0]=='add':e[2]=mp[e[2]] + w['A'],w['cut'],w['B']=lst['A'],lst['C'],lst['B'] + w['centers']=[mp[r] for r in w['centers']] if all(r!=h2 for r in w['centers']) else None + assert w['centers'] is not None + w['R']=R-1 + return w diff --git a/research/e8-complex-gauge-kernel/code/reuse2.py b/research/e8-complex-gauge-kernel/code/reuse2.py new file mode 100644 index 0000000000..b7ae091dd3 --- /dev/null +++ b/research/e8-complex-gauge-kernel/code/reuse2.py @@ -0,0 +1,79 @@ +from reuse import * +from retime import Word,raise_closure,apply_ch,join as rjoin +from cxcheck import check +import hostmodel as HM +import copy +def repair(w,maxn=400): + """make frame chains legal by raising gate frames (cascade). returns new word or None""" + w=copy.deepcopy(w) + W=Word(w) + for it in range(2000): + bad=None + for r in range(W.n): + cur=W.start[r] + for i in W.pos[r]: + if W.sm(cur)&~W.sm(W.F[i]):bad=(r,i,cur);break + cur=W.F[i] + if bad:break + if not bad: + for i,e in enumerate(W.ops): + (w['A'][i] if i=2 and len(G)-np.linalg.matrix_rank(C[:,G])>0] +init=pickle.load(open('kern4_best.pkl','rb')) +best,asg=sa(col,first,C,cand,Es,iters=iters,seed=seed,init=init,T0=0.5) +print('seed',seed,'best',best,flush=True) +pickle.dump((best,asg),open('sa_long_%d.pkl'%seed,'wb')) diff --git a/research/e8-complex-gauge-kernel/data/final_eval_reference.json b/research/e8-complex-gauge-kernel/data/final_eval_reference.json new file mode 100644 index 0000000000..eaeb796fa7 --- /dev/null +++ b/research/e8-complex-gauge-kernel/data/final_eval_reference.json @@ -0,0 +1,189 @@ +{ + "baseline": { + "b": 0.001267453159029119, + "W": 8400, + "eps": 0 + }, + "k77": { + "b": 0.001309121737234019, + "W": 7977, + "D": 2496, + "eps": 77, + "banks": [ + 143, + 152, + 0, + 1, + 0, + 1, + 0, + 0, + 0 + ], + "hosted": 37944, + "gain": 3.2875832852725217, + "kappa_cap": 0.001307410178150321, + "profile": { + "1": 1979, + "2": 928, + "3": 24, + "4": 224, + "5": 392, + "8": 24 + }, + "maxchild": 9 + }, + "v1": { + "b": 0.001310216366506891, + "W": 7966, + "D": 2496, + "eps": 79, + "banks": [ + 139, + 147, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "hosted": 38032, + "gain": 3.3739477607621415, + "kappa_cap": 0.001308501945841842, + "profile": { + "1": 1981, + "2": 932, + "3": 20, + "4": 224, + "5": 392, + "8": 24 + }, + "maxchild": 8 + }, + "g70": { + "b": 0.001305167273080109, + "W": 7966, + "D": 2496, + "eps": 96, + "banks": [ + 278, + 0, + 0, + 0, + 0, + 0, + 0, + 8, + 0 + ], + "hosted": 38032, + "gain": 2.975582472797589, + "kappa_cap": 0.001303466031873735, + "profile": { + "1": 2003, + "2": 824, + "3": 152, + "4": 276, + "5": 314, + "6": 2, + "7": 1, + "8": 17 + }, + "maxchild": 8 + }, + "v3": { + "b": 0.001328773171067089, + "W": 7724, + "D": 2496, + "eps": 140, + "banks": [ + 44, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "hosted": 39968, + "gain": 4.838049564288571, + "kappa_cap": 0.00132700987594619, + "profile": { + "1": 2055, + "2": 920, + "3": 56, + "4": 276, + "5": 314, + "6": 2, + "7": 1, + "8": 17 + }, + "maxchild": 8 + }, + "g80": { + "b": 0.001310916123494978, + "W": 7912, + "D": 2496, + "eps": 106, + "banks": [ + 232, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "hosted": 38464, + "gain": 3.4291574529785374, + "kappa_cap": 0.001309199872273537, + "profile": { + "1": 2009, + "2": 838, + "3": 140, + "4": 274, + "5": 314, + "6": 2, + "7": 1, + "8": 17 + }, + "maxchild": 8 + }, + "v4": { + "b": 0.001333854156208409, + "W": 7680, + "D": 2496, + "eps": 148, + "banks": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "hosted": 40320, + "gain": 5.2389310568410785, + "kappa_cap": 0.001332077359286333, + "profile": { + "1": 2057, + "2": 928, + "3": 50, + "4": 274, + "5": 314, + "6": 2, + "7": 1, + "8": 17 + }, + "maxchild": 8 + } +} \ No newline at end of file diff --git a/research/e8-complex-gauge-kernel/data/selection/gauge80.json b/research/e8-complex-gauge-kernel/data/selection/gauge80.json new file mode 100644 index 0000000000..93ec6b64d4 --- /dev/null +++ b/research/e8-complex-gauge-kernel/data/selection/gauge80.json @@ -0,0 +1,271 @@ +{ + "kind": "gauge80", + "gauge": { + "553": [ + 510 + ], + "305": [ + 510 + ], + "409": [ + 408 + ], + "278": [ + 510 + ], + "249": [ + 510 + ], + "275": [ + 408, + 102 + ], + "534": [ + 510 + ], + "415": [ + 408 + ], + "277": [ + 510 + ], + "164": [ + 510 + ], + "552": [ + 510 + ], + "193": [ + 510 + ], + "551": [ + 408, + 102 + ], + "250": [ + 510 + ], + "495": [ + 408 + ], + "549": [ + 408, + 102 + ], + "191": [ + 408, + 102 + ], + "194": [ + 510 + ], + "221": [ + 510 + ], + "547": [ + 510 + ], + "546": [ + 510 + ], + "139": [ + 510 + ], + "327": [ + 408 + ], + "550": [ + 408, + 102 + ], + "471": [ + 408 + ], + "279": [ + 510 + ], + "548": [ + 408, + 102 + ], + "248": [ + 510 + ], + "247": [ + 408, + 102 + ], + "276": [ + 510 + ], + "138": [ + 510 + ], + "220": [ + 510 + ], + "306": [ + 510 + ], + "167": [ + 510 + ], + "559": [ + 510 + ], + "494": [ + 408 + ], + "541": [ + 291, + 163, + 102, + 24 + ], + "246": [ + 408, + 102 + ], + "533": [ + 510 + ], + "492": [ + 408 + ], + "195": [ + 510 + ], + "165": [ + 510 + ], + "535": [ + 510 + ], + "408": [ + 408 + ], + "542": [ + 291, + 163, + 102, + 24 + ], + "166": [ + 510 + ], + "538": [ + 408, + 102 + ], + "493": [ + 408 + ], + "223": [ + 510 + ], + "307": [ + 510 + ], + "137": [ + 510 + ], + "326": [ + 408 + ], + "536": [ + 408, + 102 + ], + "470": [ + 408 + ], + "136": [ + 510 + ], + "330": [ + 408 + ], + "251": [ + 510 + ], + "544": [ + 510 + ], + "539": [ + 408, + 102 + ], + "540": [ + 291, + 163, + 102, + 24 + ], + "558": [ + 510 + ], + "556": [ + 510 + ], + "555": [ + 510 + ], + "554": [ + 510 + ], + "465": [ + 408 + ], + "532": [ + 510 + ], + "222": [ + 510 + ], + "414": [ + 408 + ], + "190": [ + 408, + 102 + ], + "464": [ + 408 + ], + "304": [ + 510 + ], + "557": [ + 510 + ], + "192": [ + 510 + ], + "331": [ + 408 + ], + "537": [ + 408, + 102 + ], + "274": [ + 408, + 102 + ], + "545": [ + 510 + ], + "543": [ + 291, + 163, + 102, + 24 + ], + "134": [ + 510 + ], + "135": [ + 510 + ] + } +} \ No newline at end of file diff --git a/research/e8-complex-gauge-kernel/data/selection/kernel79.json b/research/e8-complex-gauge-kernel/data/selection/kernel79.json new file mode 100644 index 0000000000..98ce1c66a4 --- /dev/null +++ b/research/e8-complex-gauge-kernel/data/selection/kernel79.json @@ -0,0 +1,339 @@ +{ + "kind": "kernel79", + "classes": [ + { + "E": [ + 480 + ], + "donors": [ + 134, + 135, + 270, + 271, + 272, + 273, + 298, + 300, + 384, + 390, + 410, + 519, + 536, + 558 + ], + "pivots": [ + 299, + 301, + 385, + 411, + 440, + 441, + 466, + 467 + ] + }, + { + "E": [ + 510 + ], + "donors": [ + 166, + 167, + 194, + 195, + 532, + 539, + 541, + 543 + ], + "pivots": [ + 222, + 223, + 250, + 251, + 278, + 279, + 306, + 307 + ] + }, + { + "E": [ + 408 + ], + "donors": [ + 184, + 190, + 191, + 240, + 246, + 247, + 274, + 275, + 324, + 326, + 408, + 507, + 540 + ], + "pivots": [ + 327, + 330, + 331, + 414, + 415, + 464, + 465, + 470, + 471, + 492, + 493, + 494 + ] + }, + { + "E": [ + 384 + ], + "donors": [ + 162, + 163, + 248, + 249, + 297, + 304, + 305, + 356, + 409, + 412, + 413, + 439, + 514, + 525, + 534, + 542 + ], + "pivots": [ + 357, + 360, + 361, + 416, + 417, + 436, + 437, + 442, + 443, + 495, + 502 + ] + }, + { + "E": [ + 126 + ], + "donors": [ + 160, + 161, + 533, + 535, + 552 + ], + "pivots": [ + 242, + 243 + ] + }, + { + "E": [ + 414 + ], + "donors": [ + 185, + 192, + 193, + 218, + 219, + 302, + 303, + 352, + 353, + 380, + 382, + 468, + 505, + 510, + 511, + 521, + 547, + 548 + ], + "pivots": [ + 358, + 359, + 381, + 383, + 386, + 387, + 469, + 472, + 473, + 496, + 497 + ] + }, + { + "E": [ + 390 + ], + "donors": [ + 164, + 165, + 220, + 221, + 276, + 277, + 325, + 328, + 506, + 512, + 515, + 529 + ], + "pivots": [ + 329, + 332, + 333, + 388, + 389, + 500, + 501, + 503 + ] + }, + { + "E": [ + 102 + ], + "donors": [ + 188, + 189, + 537, + 538 + ], + "pivots": [ + 244, + 245 + ] + }, + { + "E": [ + 486 + ], + "donors": [ + 136, + 137, + 334, + 335, + 354, + 362, + 363, + 509, + 526, + 531 + ], + "pivots": [ + 355, + 444, + 445, + 446, + 447, + 474, + 475 + ] + }, + { + "E": [ + 96 + ], + "donors": [ + 186, + 187, + 438, + 517, + 549 + ], + "pivots": [ + 216, + 217 + ] + }, + { + "E": [ + 30 + ], + "donors": [ + 132, + 133, + 296, + 498, + 499, + 528 + ], + "pivots": [ + 268, + 269 + ] + }, + { + "E": [ + 120 + ], + "donors": [ + 158, + 159, + 527, + 546, + 553 + ], + "pivots": [ + 214, + 215 + ] + }, + { + "E": [ + 6 + ], + "donors": [ + 128, + 129, + 138, + 139, + 508, + 513, + 554, + 555 + ], + "pivots": [ + 156, + 157 + ] + }, + { + "E": [ + 24 + ], + "donors": [ + 130, + 131, + 241, + 419, + 557 + ], + "pivots": [ + 212, + 213 + ] + } + ] +} \ No newline at end of file diff --git a/research/e8-complex-gauge-kernel/data/words/gauge80-kernel42/FULL-WORD.json.gz b/research/e8-complex-gauge-kernel/data/words/gauge80-kernel42/FULL-WORD.json.gz new file mode 100644 index 0000000000..85da3f86a2 Binary files /dev/null and b/research/e8-complex-gauge-kernel/data/words/gauge80-kernel42/FULL-WORD.json.gz differ diff --git a/research/e8-complex-gauge-kernel/data/words/gauge80/FULL-WORD.json.gz b/research/e8-complex-gauge-kernel/data/words/gauge80/FULL-WORD.json.gz new file mode 100644 index 0000000000..e9f9555d54 Binary files /dev/null and b/research/e8-complex-gauge-kernel/data/words/gauge80/FULL-WORD.json.gz differ diff --git a/research/e8-complex-gauge-kernel/data/words/kernel79/FULL-WORD.json.gz b/research/e8-complex-gauge-kernel/data/words/kernel79/FULL-WORD.json.gz new file mode 100644 index 0000000000..68820e20db Binary files /dev/null and b/research/e8-complex-gauge-kernel/data/words/kernel79/FULL-WORD.json.gz differ