Tooling: 1.54-1.71x faster #741 complex replay by running independent pinned stages concurrently

[navinor]

Status: tooling improvement. No κ, construction, predicate, pinned file or selected record changes. make frontier741-verify is unchanged and remains the reference replay.

Base: 7ee02b7c96d85f22b38d28bd7ed49bb8c1aeb261 (main). Track: verification engineering (sprint seed 85).

What changed

The complex half of make frontier741-verify (874 s of 1158 s in the c290b7b receipt) runs the 22 stages of research/complex-sw4-frame8/verify.py one after another. Many are independent. For example, the fresh PR 558 ancestor replay shares nothing with the gauge44 → p6n → SW4 → frames chain, and three control stages read only the final word.

scripts/verify_complex741_parallel.py (opt-in make complex741-parallel-verify) executes the pinned verify.py itself, statement by statement, after checking its SHA-256 (7706065e…, the value in the package MANIFEST.json):

• The prelude runs unchanged: manifest check, vendored source pins, environment.
• Each run(...) call goes to verify.py's own run function on a worker thread once its prerequisites in STAGE_DEPS have passed. Command line, environment, run_portable.py, logs and failure handling are the original code.
• Statements that read stage output (ancestral linkage loop, all count assertions, RESULT.json, RECEIPT.json) run unchanged, in their original order, after all stages pass.
• It refuses a changed verify.py or a stage list that differs from the table.

Evidence

Schedule against a recorded trace. research/complex741-parallel-replay/record_access.py --record DIR runs the unchanged sequential verify.py under a Python audit hook loaded through a .pth file, so it also covers the -I children. The hook logs every open, listing, rename, delete and mkdir per stage. ACCESS.json covers 83 processes. No stage writes outside the work directory except into fresh tempfile directories. No pair of stages left unordered by STAGE_DEPS has a write/read, write/write, write/list or mkdir/list conflict, and deferring the ancestral linkage loop conflicts with no later stage. Two independent recordings gave identical access sets. The table keeps argument-derived edges, which are stricter than the trace requires, because audit hooks do not see stat.

Equivalence. Three full replays on one machine, each in the same absolute work directory:

Replay	Result	Total
sequential verify.py	PASS κ = 1661202643272239/10^18	594.7 s
driver --jobs 8 (default)	PASS, same κ	386.4 s (1.54×)
driver --jobs 3	PASS, same κ	347.7 s (1.71×)

compare_replays.py checked all 304 files of each tree pair. Sequential vs --jobs 8: 233 byte-identical, 50 JSON files differ only under seconds/elapsed_seconds or in SHA-256 digests of the same file where that file is itself accepted (checked to a fixed point), and 21 logs differ only in decimal elapsed times or such digests. Nothing else differs, and the other two pairs give the same result. Every generated word is byte-identical. Masking floats in logs is a relaxation; the JSON outputs are compared without masking. Negative control: changing the last digit of κ in global-replay/ASSEMBLY.json and one integer in a log produces exactly those two rejections.

The parallel time equals the dependency chain gauge44 → … → all-native-phase-edges → global admission, so more workers cannot shorten it. Applying that chain to the Linux stage times in c290b7b gives 457 of 874 s. That is an estimate, not a measurement.

Reproduce

python3 -m unittest discover -s tests -p test_complex741_parallel.py   # 15 tests
make complex741-parallel-verify                                         # PASS 0.001661202643272239
python3 research/complex741-parallel-replay/compare_replays.py SEQ_DIR PAR_DIR   # exit 0, "0 other differences"

(SEQ_DIR and PAR_DIR must be replays made at the same path, one after the other.)

Scope and limits

• Measured on Windows 11 (16 logical CPUs, CPython 3.12.14), one run each, host not isolated. Linux was not run for this change. The sequential package replay also passed on this Windows machine.
• Covers the complex supplier only. The bit replay (tune/verify.sh, needs C++17 and Boost) could not be run here and is unchanged. Running it beside this driver would hide most of its 284 s; I leave that for someone who can test it.
• The trace and comparison are evidence for this pinned verify.py; the SHA-256 pin forces re-derivation after any package change.
• Finite replay does not prove the multiplication theorem; inherited assumptions are unchanged.

Credit and overlap

Replays chafreaky's PR 741 package and the contributors credited in its PROVENANCE.md. eumemic's PR 154 proposed concurrent execution of the make verify groups; this applies the idea inside the selected complex replay, with a recorded trace. It does not overlap #824 (stage-pair audit), PR 818 (R420 RREF) or PR 838 (binding audit).

Prepared by navinor (Aaron Wang) with substantial AI assistance (Claude) in design, code, measurement and prose.