Record candidate: T16 owner map from class-level re-solve and guided realisation on the selected supplier (κ=0.001673614205725695), with a proved packing ceiling

[philippeVV]

Contribution

This PR adds research/complex-t16-two-stage-cycles, a registered research
package. It is a conditional T16 owner map on the unchanged selected PR1094
supplier, with no new mechanism, plus a proved packing ceiling for that
supplier.

• κ = 0.001673614205725695 (conservative 0.001673614202872892).
• That is +5.7035e-4 relative (+9.540e-7) over the selected record
  (0.001672660197076918) and +9.334e-7 over open PR1407
  (0.001672680818492982). Over this PR's earlier versions it is +9.297e-7
  (first, 0.001672684510394236), +9.113e-7 (second, 0.001672702955495933),
  +5.600e-7 (third, 0.001673054161426948) and +2.723e-7, or +1.6271e-4
  relative (fourth, 0.001673341928508063).
• Idle remainders go from {1:2276, 3:377, 7:7, 48:4866} (selected) to
  {1:2392, 2:8, 3:107, 4:1, 7:5, 16:1, 48:4880}. Rank mass 1,041,408 and
  deficit 5,120 are unchanged. The new residual vector majorizes the selected
  one (equal totals 237,024; check.py asserts the partial-sum comparison), so
  by Karamata's inequality the complex moment strictly decreases for any
  strictly concave per-residual weight. The value of κ comes from the pinned
  exact engines and the strict replay.
• Standalone diff against the selected map: DIFF-vs-selected.json
  changes 2,537 of 69,120 rows: 2,170 helpers move to another bin and 367 rows
  only change their slice offset. Widths, call order, windows and the local and
  bit suppliers are unchanged. No open PR is needed.
• The unchanged PR1087 strict replay passes all 11 stages, with controls
  69/12/6/15. actual_73column_sigmas is 10011: the strict audit recomputes
  this chart count for the realized map and assemble.py pins it.
• The seed67 owner auditor passes R1–R7, X1, X2 and its 11 controls. Pricing
  with the pinned exact engines and the PR1069 sweep agree with the replay.

With the PR1240 local-word ceiling endpoint a_hi = 0.001690054109089, this map
realises 6.560% of the complex endpoint gap (fourth version: 4.688%; third:
2.709%; second: 0.294%; first: 0.167%; PR1407: 0.142%).

It claims no packing optimum and no unconditional theorem. All PR1094
inherited assumptions remain.

Proved packing ceiling

PROOF.md ("A proved packing ceiling") and ceiling_bound.py (standard
library, exact integer arithmetic, run by verify.py) prove two bounds for
every legal owner map on this supplier. The proof relaxes the problem to
plain bin packing: windows and the family rule are dropped.

• The instance has 32,256 bins (capacities 8, 16, 32, 48; counts 10,752,
  7,168, 7,168, 7,168) and 69,120 helpers (widths 4, 5, 7, 8, 9; counts
  144 × (56, 32, 30, 122, 240)). Both are derived from the package data.
• At most 4,887 cap-48 bins can be empty. Give each helper weight ⌊7w/9⌋
  and each bin weight ⌊7c/9⌋ (6, 12, 24, 37). The helpers in a bin never weigh
  more than the bin. The total slack is 587,776 − 406,944 = 180,832 for every
  map, and an empty cap-48 bin uses 37 of it. This map has 4,880 empty cap-48
  bins; the selected map has 4,866.
• The search surrogate Σ f ln f is at most 4,887 · 48 ln 48 + 17 ln 17 =
  908,139.013. This comes from an exact knapsack over the per-bin slack. A
  plain LP dual gives 908,156.14. This map is at 907,271.883.
• Solver-reported, not checked: the class-level window LP and the window + R4
  IP reach the same optimum, so windows do not tighten the bound.

This bounds the surrogate and the number of empty cap-48 bins, not κ. As
an indication only, pricing the bound-attaining idle multiset
{1:2431, 17:1, 48:4887} with the pinned engines gives κ 0.001674088611862992
(9.822% of the complex endpoint gap). This replaces the earlier version's
"≈908.1k" heuristic ceiling, which could not be reproduced.

How the map was found

Inside a physical family the bins' windows never overlap. So the family rule
reduces to at most one helper per bin per call, and bins with the same capacity
and the same allowed calls are interchangeable. κ is close to linear in the
surrogate Σ f ln f over idle fragments, which the search uses as its
objective; only the exact engines decide acceptance.

1. Earlier versions' search. A global class-level repack and
   cycle/LNS loop (third version), then an evacuation LNS neighbourhood
   (fourth version), up to κ 0.001673341928508063.
2. Class-level re-solve (new, search/winlp.py). The window model is an
   IP over (capacity, window) classes: pattern counts, flows of each
   (call, width) to each class, time and width balance, and aggregate R4 cuts
   for each family group and call. Each step re-solves it warm-started from
   the current map. Only a random 12–24-class subset may change, drawn from
   the classes that differ from a full-model target. Each step must gain a
   minimum surrogate amount and pays a penalty per unit of change.
3. Guided realisation (new, search/lns2.py mode guide). The neighbourhood
   is the bins holding helpers whose class (call, width) count must drop, plus
   the bins whose pattern count must drop. CP-SAT then maximises the surrogate
   exactly over it, and a result is kept only if the exact κ rises.
   search/gloop.sh and search/sloop.sh loop this. Strict-replayed
   checkpoints: 0.001673373170972473, 0.001673393061983855,
   0.001673535326871893 and 0.001673614205725695. The search was stopped
   while still improving.
4. Min-diff re-assignment at the same idle profile, hence the same κ. The
   raw map relocates 63,109 rows against the selected map. The class-level
   min-diff tools (search/mindiff_cls1.py, mindiff_cls2.py,
   mindiff_lns.py, mindiff_lns2.py, realize.py) give 63,109 → 2,210 →
   2,170 relocations, i.e. 2,537 changed rows of which 367 are offset-only.
   Two stage-1 class patterns could not be realised, because several helpers of
   one width in a bin need distinct calls. The README gives the counting
   argument for each. Both were excluded with BAN and stage 1 was re-solved
   locally. The class-level relaxation implies at least 1,548 relocations for
   any legal map with this profile (solver-reported). The min-diff stopped at
   its two-hour budget, so the shipped map is not proven minimal.

SEARCH.json records the checkpoints and the commands. lns.py, lns2.py,
the loops, check.py and verify.py compare κ as Decimal, because the
native assembler can drop trailing zeros.

Earlier versions of this PR

• First version, κ 0.001672684510394236: two-stage ejection cycles on
  PR1407's map, a 23-row successor of PR1407, with the ancestry checked by
  content.
• Second version, κ 0.001672702955495933: a longer cycle/LNS search, then
  a min-diff re-assignment to 70 changed rows (55 relocations) against the
  selected map; the PR1407 ancestry check was dropped.
• Third version, κ 0.001673054161426948: a global class-level repack and
  the cycle/LNS loop, then a class-level min-diff re-assignment to 1,661
  changed rows (1,435 relocations). Its idle profile did not majorize the
  selected one, so it relied on the exact engines alone for the sign.
• Fourth version, κ 0.001673341928508063: an evacuation LNS neighbourhood
  in the same loop, then the class-level min-diff to 1,638 changed rows (1,415
  relocations); its profile majorized the selected one again.

This fifth version replaces the map, the diff and the search record, and adds
the proved packing ceiling. PR1407 (pai-scaffolde) remains the starting point
of the search line and the source of the whole-helper semantics and all-input
argument. PR2077 (lydakis) remains the origin of the vacancy-chain idea. Their
four literal moves still do not compose (search/compose.py).

Provenance

• Chooser: this work was not drawn by scripts/choose_research_task.py. It
  was self-selected as the review-and-compose task: a review of PR1407 and
  PR2077, a test of whether they compose, and then a search on the stronger
  map. TASK.json records it under the closest card, selected-packing, with
  scope_status: changed and this reason.
• Base: 0fdd7d82. The 287 pinned inputs are listed in INPUTS.json and
  SOURCES.json.
• Credit:
  • pai-scaffolde (PR1407): starting map of the search line, whole-helper
    semantics and all-input argument.
  • George Lydakis (lydakis, PR2077): the vacancy-chain idea.
  • Rohan Arun and Jeremy Kothe: the PR1050–PR1094 line and the selected
    supplier and map.
  • Josie Elliston: the seed67 auditor, PR1091.
  • Sagar Thakrar: the independent pricing sweep, PR1069.
• Prepared by Philippe Van Velzen with substantial Claude Code (Claude
  Opus 5.5) assistance. No independent human review is claimed.

Reproduce

python3 scripts/research_package.py --package research/complex-t16-two-stage-cycles --check-only
python3.12 scripts/research_package.py --package research/complex-t16-two-stage-cycles

The full replay takes about 60 s and uses only the standard library. The
search is optional and needs numpy, ortools, scipy and highspy; the README and
SEARCH.json give the commands.

🤖 Generated with Claude Code

[philippeVV]

Scope note relating this PR to open #2081 (T16 invoice-boundary audit):

• Audit selected T16 invoice input and caller boundaries #2081's unchanged invoice_allcold.run()/assemble() boundary audit also passes on this map (PASS_T16_INVOICE_BOUNDARY_AUDIT, 54 cases, same finite implications), using a wrapper that substitutes only this map's strict replay. The 10 cases that pass assembly are the same ones as on the selected map, and κ is unchanged.
• There is one difference from the selected fixture. The "negative rank count with preserved mass" case is rejected earlier, at invoice_allcold.py:254 rather than at assemble.py:130, because this map has no rank-7 entries.
• Following Audit selected T16 invoice input and caller boundaries #2081, admission here relies on the complete verify_t16.py chain: the controller audit(), the controller-flag assertion and the UPSTREAM-PINS check, which this package's replay runs. Invoice plus assembly alone would not be enough.
• The frozen PR1087 code is byte-identical to the selected and Conditional T16 κ gain through neutral whole-helper hosting paths #1407 code.
• Audit selected T16 invoice input and caller boundaries #2081's open items (local-provider input checks, all-size forms) apply here unchanged.

[philippeVV]

Updated to κ = 0.001672702955495933, +2.556e-5 relative over the selected record. The map now comes from CP-SAT large-neighbourhood search alternating with ejection cycles, then a min-diff reassignment at the same κ, so it changes only 70 of 69,120 rows relative to the selected PR1094 map (55 relocations, 15 offset-only).

The package no longer depends on open PR1407. PR1407 and PR2077 remain credited as the starting point and the idea.

The unchanged PR1087 strict replay passes all 11 stages (controls 69/12/6/15), and the seed67 audit (11 controls), pinned pricing and the PR1069 sweep all agree. Both registered package checks pass.

The search was stopped while still improving, so a further update may follow.

[philippeVV]

Update (349e985): κ = 0.001673054161426948, +2.3553e-4 relative over the selected record. On the PR1240 endpoint this is 2.709% of the complex packing gap (the previous version was 0.294%).

• Method. The family rule reduces to "at most one helper per bin per call", because windows inside a family never overlap. That makes the packing a class-level pattern/flow model, which is re-solved globally and then refined with the existing cycle/LNS loop. A class-level min-diff MIP then brings the diff against the selected map down to 1,661 rows (1,435 relocations), with the same profile and therefore the same κ.
• Checks. Both registered checks pass: PASS_T16_OWNER_MAP … changed 1661 relocated 1435 and PASS_REGISTERED_RESEARCH_PACKAGE. The strict replay passes all 11 stages (controls 69/12/6/15), with actual_73column_sigmas = 10055. I also re-ran the strict replay and the owner/pricing check from a separate driver, and they agree.
• PROOF.md change. The new idle profile no longer majorizes the selected one: 37 fully idle bins now hold a few small helpers. So the sign of the cost change now rests on the pinned exact engines and the strict replay, not on Karamata's inequality. PROOF.md says this explicitly.
• Caveats. The ≈908.1k surrogate ceiling in SEARCH.json is a heuristic figure that was not reproduced, and it is labelled that way. The search was stopped while still improving, and a continuation is running.

[philippeVV]

Update (01b15ae): κ = 0.001673341928508063, +4.0757e-4 relative over the selected record. On the PR1240 endpoint this is 4.688% of the complex packing gap (the previous version was 2.709%).

• What changed. I added a new LNS neighbourhood, "evac". It frees every helper in a few nearly-empty cap-48 bins, together with the bins whose windows cover those helpers' calls, and re-packs them exactly with CP-SAT. One solve emptied 35 cap-48 bins, and the cycle/LNS loop did the rest. The stranded bins with 20–44 free slots are gone. Fully idle bins went from 4,866 (selected) to 4,874.
• Proof. The idle profile majorizes the selected one again, strictly. So the concavity (Karamata) argument for the strict cost decrease is back in PROOF.md, and check.py now asserts it mechanically on the audited vectors. κ itself still comes from the exact engines and the strict replay.
• Checks. The min-diff against the selected map is 1,638 changed rows (1,415 relocations). Both registered checks pass: PASS_T16_OWNER_MAP 0.001673341928508063 … changed 1638 relocated 1415 and PASS_REGISTERED_RESEARCH_PACKAGE. The strict replay passes 11 stages with controls 69/12/6/15 and actual_73column_sigmas = 10006. I also re-ran the strict replay and the owner/pricing check myself.
• Other fix. All κ comparisons in the package and the search tooling now use Decimal. String comparison could reject valid κ values that have trailing zeros.
• Caveat. The search was again stopped while still improving.

[philippeVV]

Update (4452c98): κ = 0.001673614205725695, now with a proved packing ceiling. On the PR1240 endpoint this is +5.7035e-4 relative to the selected record and 6.560% of the complex gap.

• Proved ceiling. ceiling_bound.py uses the standard library only and exact integers. verify.py runs it. It uses weights a(w)=⌊7w/9⌋ and A(cap)=⌊7·cap/9⌋. Every one of the 1,163 bin patterns satisfies Σa ≤ A(cap), so the slack sums to 587,776 − 406,944 = 180,832. That allows at most 4,887 empty cap-48 bins; this map has 4,880 and the selected map has 4,866. The same weights bound the surrogate E ≤ 4,887·g(48) + g(17) = 908,139.013, using integer comparisons of products f^f. The bound-attaining idle profile prices at κ 0.001674088611862992 with the pinned engines. That price is indicative and is not a κ bound. Windows and R4 do not lower the aggregate optimum (solver-reported), so the remaining gap is a search gap, not a counting gap.
• Method. A stage-1 class IP is re-solved, warm-started from the current map and with only a random subset of classes freed. A guided CP-SAT realisation follows. The idle profile still majorizes the selected one, so the Karamata argument applies.
• Checks. Both registered checks pass: PASS_T16_OWNER_MAP … changed 2537 relocated 2170 ceiling: empty cap-48 <= 4887 E <= 908139.013 and PASS_REGISTERED_RESEARCH_PACKAGE. Strict replay passes 11 stages with controls 69/12/6/15, sigmas 10011. I re-ran the strict replay and the pricing check myself and got the same result.
• Context. The frontier has since moved to the T74 alias line (Conditional κ = 0.001703713039257905: a sixth #2436-type alias (helper 656 borrows lender 88) on the entrance-raised T74 words #2526). This PR remains a selected-supplier packing result.