Tooling: checked marshal copies make the selected final stage's audit sub-stage 2.47x faster (no new kappa)

[jackmgaller]

Tooling (verifier engineering). No κ, certificate, supplier, frontier or pinned-source change.

Card replay-runtime, scope refined to one profiled sub-stage. Chooser OS nonce 343b846376b457288e1b5a2336a887c0, priority revision 3. Base 0b1cccf86e2f5132ed286a26ef0a681a495740ea; selected-record SHA-256 10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32. Claim: #1166 (comment)

Bottleneck

The selected record has one stage after #1042: complex-t16-rare-pattern-exchanges/verify.py, which runs the frozen PR1087 verify_t16.py. Its slowest sub-stage is audit: 32.6 s of 57.6 s in the maintainer receipt. The script is audit_allcold.py, SHA-256 1467a2ac…. The cold baseline1042 t16 stage runs the same bytes from research/gauge44-slots-t16/code/.

cProfile puts about 80% of this sub-stage in copy.deepcopy. Each of the 15 mutation controls copies the whole hosted T16 template, and each of the 16 audits copies every source event: 4,838 top-level calls in total.

Change

New package research/audit-allcold-fast-copy/. Its fast_copy.py is an opt-in drop-in for python3 -B run_portable.py SCRIPT ARGS…:

• It checks run_portable.py and the script against a SHA-256 allow-list, then runs the pinned run_portable.py unchanged.
• It replaces only copy.deepcopy. For a top-level dict or list, the copy is marshal.loads(marshal.dumps(x)). That copy is kept only if marshal accepts the value (core builtin types only), the copy has the same type, and it compares equal. Otherwise, and for any call with a memo, the original deepcopy runs.
• marshal keeps aliasing through back-references, as deepcopy does.
• No verifier source is edited, nothing is cached, and no check or control is skipped.

Evidence (verify.py, registered replay)

1. Fixture. It extracts the archived gauge44 provider files from selected-replay.tar.gz and runs the unchanged selected final stage on them. The stage must print PASS_T16_RARE_PATTERN_EXCHANGES 0.001672660197076918, and its ASYNC-AUDIT.json must equal the archived selected receipt byte for byte.
2. A/B. It reruns the audit sub-stage on fresh copies of the same inputs with verify_t16's exact environment, alternating legacy and fast. Both must reproduce the in-stage outputs byte for byte: ASYNC-AUDIT.json, OWNERS-AND-PROFILE.json, IDLE-REMAINDER-FRAMES.json, the regenerated IDLE-REMAINDER-FRAMES.json.gz (same environment) and the merged stdout/stderr log. All 4,838 calls must take the marshal path.
3. Controls.
   • A tampered hosted template is rejected by both launchers.
   • An unpinned script is refused.
   • An identity copier (no copying) is detected. The pinned audit itself still exits 0 in that case, but writes a corrupted report (provider count 143 rather than 144, and one idle frame missing), so the byte comparison is load-bearing.
   • Copy semantics are checked: aliasing and independence, plus fallback for a dict subclass, NaN, a custom object, a cyclic list and an explicit memo.

Measured on macOS (Apple silicon), same host and fixture, 3 alternating rounds, with identical outputs every round:

Interpreter	Legacy audit (s)	Fast audit (s)	Median
CPython 3.12.13	22.37, 22.27, 22.50	9.05, 9.06, 9.21	2.47x
CPython 3.14.7	17.85, 18.31, 17.79	9.43, 9.46, 9.47	1.89x

Commands run:

• python3.12 scripts/research_package.py --package research/audit-allcold-fast-copy: PASS_REGISTERED_RESEARCH_PACKAGE, about 80 s.
• --check-only: pass.
• python3 -B -m unittest discover -s tests -p 'test_research_*.py': 42 tests OK, 2 skips.
• git diff --check: clean.

Limitations / not checked

• Not wired in. The launcher is not used by make frontier1094-verify, because the frozen verify_t16.py calls run_portable.py directly. Wiring it in needs an opt-in driver or a maintainer decision (open question 1).
• One script hash. The allow-list has the single audited script. The nine other audit_allcold.py variants were not checked.
• No fresh cold baseline. The fixture's --local input comes from the archived selected replay. It is bound by the archived ASYNC-AUDIT.json equality, not by a fresh cold baseline1042 run. A full cold make frontier1094-verify was not rerun.
• Indirect inputs. verify_t16's own UPSTREAM-PINS.json dependencies are enforced by the stage itself, but they are not declared as this package's CI inputs.
• Untested hosts. Linux and Windows timings were not measured here.

Credit

The audited code is from the retained gauge44 / PR1087 lineage:

• the audit_allcold.py controller audit and its mutation controls (OpenAI Codex in the original packages);
• Hao Yan's (@FanPUYen) PR858 T16 controller;
• the run_portable.py wrapper from complex-k492-owner-repair;
• PR1094 (Rohan Arun) on Jeremy Kothe's k8/T16 construction;
• the maintainer's frontier1094 audit receipt.

The opt-in, byte-identical adapter pattern follows #1880, #1874, #1646 and frontier1042-lec-parallel.

Prepared by jackmgaller with substantial Claude Code (Claude Opus 5.5) assistance. Apache-2.0.

Workflow feedback (optional). The task chooser and overlap helper worked well. The biggest friction was getting the final stage's --local input without a 20-minute cold baseline. The archived selected-replay.tar.gz provider files solved it. Suggestion: document that fixture route in the bottleneck brief for replay-runtime work.

🤖 Generated with Claude Code

[JosieElliston]

Cross-reference from session work on the same audit sub-stage: #1959 runs the pinned audit_allcold.py (1467a2ac…) in-process on the archived selected template. It records which line rejects each of the 15 shipped controls, 35 new semantic mutants and a 111-field census. The harness copies with a checked marshal round trip, following this PR's observation, and credits it. Finding: the shipped controls are first rejected at only 8 of the audit's 42 lines with an assert; 34 of 35 new mutants are rejected, reaching 20 more. Your open item 2 (the other audit_allcold.py hashes) is also our open question 1.

[jackmgaller]

Thanks @JosieElliston. The cross-reference is useful, and the two packages fit together.

Commit 94998804 (README only, no code change) does two things:

• It links Audit: mutant coverage of the selected T16 controller/owner audit (no gap found, no new kappa) #1959 under related work.
• It records that open question 2 here is your open question 1. One per-hash fixture for those nine audit_allcold.py variants could serve both the coverage receipt and the speed allow-list.

One connection between the results, in case it helps your line attribution: this package's identity-copier control shows that the 15 shipped controls depend on copy.deepcopy for isolation. With no copy, each control mutates the audited template in place. The pinned audit still exits 0 and writes a corrupted report (provider count 143 rather than 144, and one idle frame missing). Any harness that reruns the shipped control block therefore needs a real copy. Yours does, using the checked marshal round trip, so this should not affect your table.

[Mister-Meeseeks]

Review follow-up for frozen head b1631b1bf950f8da70510057faf55c6ed705c815.

The frozen review covered b1631b1bf950f8da70510057faf55c6ed705c815; the head changed during the round. The old execution fixture also lacked the pinned baseline558 canonical word. This PR remains open for review of the revision delta and a current-head replay with that input available. No rejection of the newer revision is inferred from the old fixture failure.

The current head is 94998804693d50c5b2d57e923a6781368e7a635e. These findings describe the frozen review; any fixes in the newer revision still need their own delta check.

Recorded scope, attribution and execution evidence. We are leaving this open with the outstanding work recorded; previously completed checks remain credited. Thank you for the contribution.