diff --git a/research/audit-allcold-fast-copy/README.md b/research/audit-allcold-fast-copy/README.md new file mode 100644 index 0000000000..3c88c0f91b --- /dev/null +++ b/research/audit-allcold-fast-copy/README.md @@ -0,0 +1,165 @@ +# Checked marshal copies: 1.89x faster audit sub-stage of the selected final stage + +**Tooling only (verifier engineering).** No κ, certificate, supplier, frontier or +pinned-source change. Track `verifier-engineering`, card `replay-runtime`, scope +refined to one profiled sub-stage (chooser OS nonce `343b846376b457288e1b5a2336a887c0`, +priority revision 3). Base `0b1cccf86e2f5132ed286a26ef0a681a495740ea`; selected-record +SHA-256 `10b9e340…85fa32`. + +## What is slow + +The selected record's only stage after its #1042 baseline, +`research/complex-t16-rare-pattern-exchanges/verify.py`, runs the frozen PR1087 +`verify_t16.py`. Its `audit` sub-stage is the slowest: 32.6 s of 57.6 s in the +maintainer receipt (`research/community-frontier1094-audit/selected-replay.tar.gz`), +18.0 s of 32.0 s here. The pinned script is `audit_allcold.py`, SHA-256 +`1467a2acb654e62552c69fb9a7c4c85f70a02120881d222313865058ab71a6cc`, the same bytes as +`research/gauge44-slots-t16/code/audit_allcold.py`, which the cold baseline1042 `t16` +stage also runs. + +cProfile attributes about 80% of the profiled sub-stage to `copy.deepcopy`. Each of +the 15 mutation controls deep-copies the whole hosted T16 template +(`q=copy.deepcopy(ct)`), and each of the 16 audits deep-copies every source event. +That is 4,838 top-level calls and about 46 million recursive `deepcopy` frames. + +## The change + +[`fast_copy.py`](fast_copy.py) is a drop-in launcher. Use +`python3 -B fast_copy.py RUN_PORTABLE SCRIPT ARGS...` in place of +`python3 -B RUN_PORTABLE SCRIPT ARGS...`. It checks both files against a SHA-256 +allow-list and runs the pinned `run_portable.py` through `runpy`, as before. It +replaces only `copy.deepcopy`. For a top-level `dict` or `list`, the copy is +`marshal.loads(marshal.dumps(x))`. That copy is used only when all of the following hold: + +- marshal accepts the value: core builtin types only, so dict/list subclasses and + custom objects raise `ValueError`; +- the copy has the same type; +- the copy compares equal (`==`; NaN, cyclic or over-deep values fall back). + +Every call with a `memo` also uses the original `deepcopy`, as does every value +that fails one of these tests. marshal format 4 writes repeated objects as +back-references, so aliasing inside the copy is kept, as deepcopy keeps it. Every +container in the copy is new. No verifier source is edited, nothing is cached, and no +check or control is skipped. + +## Evidence + +`verify.py` does the following, all in fresh directories outside the checkout: + +1. **Fixture = the selected stage itself.** It extracts the archived gauge44 + provider files from `selected-replay.tar.gz` and passes them as `--local` to the + unchanged `complex-t16-rare-pattern-exchanges/verify.py`. That runs all 11 + `verify_t16.py` sub-stages and must print `PASS_T16_RARE_PATTERN_EXCHANGES + 0.001672660197076918`. Its `ASYNC-AUDIT.json` must equal the archived selected + receipt byte for byte. +2. **A/B on the same inputs.** It reruns the `audit` sub-stage on fresh copies of + that stage's `base`/`hosted` inputs with verify_t16's exact environment. Runs + alternate between the pinned `run_portable.py` (legacy) and `fast_copy.py`. Each + run must reproduce, byte for byte, the in-stage `ASYNC-AUDIT.json`, + `OWNERS-AND-PROFILE.json`, `IDLE-REMAINDER-FRAMES.json`, the newly written + `hosted/IDLE-REMAINDER-FRAMES.json.gz` and the merged stdout/stderr `audit.log`. + All 4,838 deepcopy calls must take the marshal path (`fallback: 0`). +3. **Controls.** + - A hosted template with `physical` reduced by one is rejected (`AssertionError`) + by both launchers. + - A copy of the script with one appended comment is refused by the allow-list. + - An **identity "copy"** (no copy at all) is detected. Notably, the pinned audit + itself still exits 0 then: every control mutates the audited template in place, + and the written reports change (for example, provider count 143 instead of 144, + and one idle-remainder frame missing). Only the byte comparison catches it, so + the comparison is load-bearing. + - In-process semantics: aliasing is kept, copies are independent, and the + fallback runs for a dict subclass, NaN, a custom object, a cyclic list and an + explicit memo. + +Measured on macOS (Apple silicon), same host and fixture, with +`verify.py --work DIR --rounds 3` (alternating order). All rounds produced +byte-identical outputs: + +| Interpreter | Legacy audit, 3 rounds (s) | `fast_copy.py` audit, 3 rounds (s) | Median speedup | +|---|---|---|---:| +| CPython 3.12.13 (registered replay interpreter) | 22.37, 22.27, 22.50 | 9.05, 9.06, 9.21 | **2.47x** | +| CPython 3.14.7 | 17.85, 18.31, 17.79 | 9.43, 9.46, 9.47 | **1.89x** | + +On 3.12, the selected final stage spends about 13 s less in this sub-stage. The +maintainer's Linux receipt has a 32.6 s audit inside a 57.6 s stage, so the absolute +saving there may be larger, but that host was not measured. Timing has no pass +threshold. + +## Reproduce + +```sh +python3 scripts/research_package.py --package research/audit-allcold-fast-copy --check-only +python3.12 scripts/research_package.py --package research/audit-allcold-fast-copy +# Optional: more timing rounds and a kept RESULT.json +python3 -B research/audit-allcold-fast-copy/verify.py --work /fresh/dir/outside/repo --rounds 3 +``` + +The registered replay takes about 1–2 minutes: one selected-stage run plus one A/B +round and the controls. + +## Limitations + +- **Not wired into `make frontier1094-verify`.** The frozen `verify_t16.py` calls + `run_portable.py` directly. Using the launcher needs an opt-in driver or a + maintainer-approved hook (open question 1). +- **One script hash.** The allow-list has the single audited `audit_allcold.py`. + The other nine `audit_allcold.py` variants in `research/` are not checked here. +- **Fixture source.** The fixture's `--local` files come from the maintainer's + archived selected replay, not a fresh cold baseline1042 run. The archived + `ASYNC-AUDIT.json` check binds them to the selected receipt. A cold + `make frontier1094-verify` was not rerun. +- **Indirect inputs.** `verify_t16.py`'s upstream pins (`data/UPSTREAM-PINS.json`) + are checked by the stage itself, but they are not listed as this package's CI + inputs. +- **Same interpreter.** The `marshal` format is used only in-process, so both + sides of every copy run under the same interpreter. + +## Open next questions + +The structured list is in [TASK.json](TASK.json). + +1. **Opt-in wiring.** Should a new driver offer a source-bound, opt-in path that + runs the audit sub-stage (and the cold baseline1042 `t16` stage) through this + launcher? That is what turns the measured saving into faster reviews. First + step: an opt-in flag in a new driver, not in the frozen `verify_t16.py`, that + substitutes the launcher only for allow-listed hashes. It is blocked on a + maintainer decision about wrapping the frozen stage runner. +2. **Other scripts.** Which other pinned verifiers spend most of their time + deep-copying JSON trees for mutation controls? There are ten distinct + `audit_allcold.py` hashes, plus other `reject()` helpers. First step: profile + each on its own committed fixture and add a hash only when it has byte-identical + receipts. Blocker: many need a long cold upstream replay first. The concurrent + mutant-coverage audit [PR1959](https://github.com/CrocSwap/integer-mult-bounds/pull/1959) + lists the same nine hashes as its open question 1, for coverage rather than + speed. One per-hash fixture could serve both questions. +3. **The remaining 9 s.** These are spent mainly in `audit()` itself: 16 passes + over 69,120 helper roles. Reducing that would need an adapter that changes + audit code, with its own equivalence argument. + +## Credit + +The audited code is from the retained gauge44 / PR1087 lineage: + +- the `audit_allcold.py` controller audit and its mutation controls (OpenAI Codex + in the original packages); +- Hao Yan's (@FanPUYen) PR858 T16 controller; +- the `run_portable.py` wrapper from `complex-k492-owner-repair`; +- PR1094 (Rohan Arun) on Jeremy Kothe's k8/T16 construction; +- the maintainer's frontier1094 audit receipt (Douglas Colkitt). + +Earlier opt-in, byte-identical replay adapters set the pattern followed here: +PR1880 (joint-dual scans), PR1874 (PR589 hashing), PR1646 (phase recipes) and +`frontier1042-lec-parallel`. + +Related, concurrent: Josie Elliston's +[PR1959](https://github.com/CrocSwap/integer-mult-bounds/pull/1959) runs the same +pinned audit in-process on the same archived template. It records which assertion +rejects each shipped control and new semantic mutants, and its harness uses the +checked `marshal` copy from this package. The two are complementary. This package +shows that the shipped controls depend on real copies for isolation (the +identity-copier control). PR1959 maps which of the audit's assertions those controls +reach. + +Prepared by jackmgaller with substantial Claude Code (Claude Opus 5.5) assistance. +New files are Apache-2.0. diff --git a/research/audit-allcold-fast-copy/SOURCES.json b/research/audit-allcold-fast-copy/SOURCES.json new file mode 100644 index 0000000000..e71b074ef5 --- /dev/null +++ b/research/audit-allcold-fast-copy/SOURCES.json @@ -0,0 +1,54 @@ +{ + "schema_version": 1, + "base_commit": "0b1cccf86e2f5132ed286a26ef0a681a495740ea", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "ae6a99f28aa468649eb66a361e764d769363bf11", + "sha256": "10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "06c96f5579bb67ed1b86d334e78fbed7da6945f1", + "sha256": "6686ad58f1fbdbc87134a78901b60fd53a20d3984355e91defbbebc9e9fa8075" + }, + { + "path": "research/community-frontier1094-audit/selected-replay.tar.gz", + "usage": "input", + "git_blob": "c24daf067f9c543421d89ca2eeaa4eb78a57fb6f", + "sha256": "f1da80610bf33c58527f18ad63b4350d0b864e1d4486141ee17ac9bed4d529d6" + }, + { + "path": "research/complex-k492-owner-repair/vendor/multicold/code/run_portable.py", + "usage": "input", + "git_blob": "2fed13f432df4d76ce72185dd79fac63d96aee96", + "sha256": "322d29d43dbee8130cde15d4bf05396698b5e3dfe3735813899db91cc0ba7955" + }, + { + "path": "research/complex-t16-rare-pattern-exchanges/MANIFEST.json", + "usage": "input", + "git_blob": "3216666bd231516f8d2387e3096d6a5bd413b519", + "sha256": "16a964621e718c8da4c859ccbb280c3c00816ca72874e955910fdf0ef27a0adc" + }, + { + "path": "research/complex-t16-rare-pattern-exchanges/upstream1087.tar.gz", + "usage": "input", + "git_blob": "2970ef79b0441e937449bed30f68f2e0558186ba", + "sha256": "d4acf04daef85d4514f80ec5fcd2492d0c60e5d6c49fa71b3e6523f8db2dea86" + }, + { + "path": "research/complex-t16-rare-pattern-exchanges/verify.py", + "usage": "input", + "git_blob": "9d471cf90835bc91d32bb0216c2bb867011a7420", + "sha256": "b8c2f76b305c5c9ceb42df203deef564857d17f1f8182d0745dda9ab96e3d4de" + }, + { + "path": "research/gauge44-slots-t16/code/audit_allcold.py", + "usage": "input", + "git_blob": "923bf7b675ca09f4d2633e7e53f16a1b23b6d5bc", + "sha256": "1467a2acb654e62552c69fb9a7c4c85f70a02120881d222313865058ab71a6cc" + } + ] +} diff --git a/research/audit-allcold-fast-copy/TASK.json b/research/audit-allcold-fast-copy/TASK.json new file mode 100644 index 0000000000..366794e5e5 --- /dev/null +++ b/research/audit-allcold-fast-copy/TASK.json @@ -0,0 +1,73 @@ +{ + "schema_version": 1, + "task_id": "replay-runtime", + "priority_revision": 3, + "base_commit": "0b1cccf86e2f5132ed286a26ef0a681a495740ea", + "original_card": { + "id": "replay-runtime", + "question": "Can one measured replay bottleneck be sped up without dropping coverage?", + "starting_points": [ + "scripts/run_isolated_tests.py", + "scripts/verify_replay_coverage.py", + "certificates/selected-result.json" + ], + "methods": [ + "Profile one isolated replay and optimize the measured bottleneck.", + "Design source-bound reuse with explicit dependency invalidation and negative controls." + ], + "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Read the selected reproduction command and choose a single replay stage for profiling.", + "scope": "Positive control / source-bound orientation only; this is not a new construction or full cold replay." + }, + "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.", + "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.", + "allowed_variations": [ + "Profile one stage or design one source-bound cache boundary." + ], + "scope_change_triggers": [ + "Skipping coverage or comparing different fixtures to claim speedup." + ], + "overlap_terms": [ + "replay runtime", + "cache invalidation", + "isolated tests" + ] + }, + "scope_status": "refined", + "actual_scope": "One measured bottleneck of the selected record's only post-#1042 stage (research/complex-t16-rare-pattern-exchanges/verify.py -> frozen PR1087 verify_t16.py): the `audit` sub-stage, i.e. the pinned audit_allcold.py (SHA-256 1467a2ac...) with its 15 mutation controls, whose time is dominated by copy.deepcopy. Can an opt-in launcher replace only deepcopy by a checked marshal round trip, leaving every pinned byte, check and control unchanged, with byte-identical outputs on the same fixture?", + "scope_change_reason": "Refined within the card's allowed variation 'Profile one stage': the profiled stage is the selected final stage and the optimized object is one sub-stage's copy primitive. No coverage is removed and no result is cached.", + "bottleneck_link": "Every reviewer of the selected record runs this stage (also with --baseline1042 reuse), and the same audit script runs in the cold baseline1042 `t16` stage and in owner-map candidates built on verify_t16.py. Halving its slowest sub-stage shortens review of complex-side packing/owner candidates without weakening a check.", + "budget_minutes": 90, + "stop_rule": "Stop if any audit output or the merged stage log differs from the in-stage receipt, if the fixture's ASYNC-AUDIT.json differs from the archived selected receipt, or if the speedup required editing pinned code.", + "outcome": "completed", + "evidence": "verify.py replays the selected final stage on the archived provider files (audit equals the archived selected ASYNC-AUDIT.json), then reruns the audit sub-stage legacy vs fast in fresh copies: all three written audit files, the written IDLE-REMAINDER-FRAMES.json.gz and the merged stdout/stderr log are byte-identical to the in-stage outputs; 4,838 deepcopy calls all take the marshal path. macOS, 3 alternating rounds: CPython 3.12 legacy 22.37/22.27/22.50 s vs fast 9.05/9.06/9.21 s (2.47x median); CPython 3.14.7 legacy 17.85/18.31/17.79 s vs fast 9.43/9.46/9.47 s (1.89x). Registered replay passes under CPython 3.12. Controls: tampered hosted template rejected by both; unpinned script refused; identity copier detected by byte comparison (the pinned audit itself exits 0 with a corrupted report); fallback semantics for subclass, NaN, custom object, cyclic and memo calls.", + "author": "jackmgaller (with Claude Code, Claude Opus 5.5)", + "ai_assistance": "Substantial: Claude Code (Claude Opus 5.5) chose the task with the repository helper, profiled the stage, wrote fast_copy.py, verify.py and this package; jackmgaller's account published it.", + "open_questions": [ + { + "question": "Should the selected replay driver offer an opt-in, source-bound hook that runs the audit sub-stage (and the cold baseline1042 t16 stage) through fast_copy.py?", + "why_it_matters": "The launcher is verified here but is not wired into make frontier1094-verify, so reviewers do not yet get the time saving.", + "first_step": "Add an opt-in flag to a new driver (not the frozen verify_t16.py) that substitutes the launcher for run_portable.py only for allow-listed script hashes and compares all audit outputs against a legacy run once.", + "blocked_by": "A maintainer decision on modifying or wrapping the frozen PR1087 stage runner." + }, + { + "question": "Which other pinned verifiers spend most of their time in copy.deepcopy of JSON trees for mutation controls (ten distinct audit_allcold.py hashes exist in research/, plus other reject() helpers)?", + "why_it_matters": "The same checked substitution may speed up historical cold replays and candidate checks with no source edits.", + "first_step": "Profile each allow-list candidate on its own committed fixture and extend SCRIPTS only with byte-identical receipts.", + "blocked_by": "Each candidate needs its own runnable fixture; many require a long cold upstream replay first." + }, + { + "question": "What dominates the remaining ~9 s of the audit sub-stage, and can it be reduced without editing the pinned audit?", + "why_it_matters": "audit() itself (16 passes over 69,120 helper roles) is now the largest share of the selected final stage.", + "first_step": "cProfile the fast path; the per-role list/index scans over 73-column bases are the likely candidates.", + "blocked_by": "Further gains likely require an adapter that changes audit() code, which needs a separate equivalence argument." + } + ] +} diff --git a/research/audit-allcold-fast-copy/VERIFY.json b/research/audit-allcold-fast-copy/VERIFY.json new file mode 100644 index 0000000000..cf7ff1efc2 --- /dev/null +++ b/research/audit-allcold-fast-copy/VERIFY.json @@ -0,0 +1,25 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "research/community-frontier1094-audit/selected-replay.tar.gz", + "research/complex-k492-owner-repair/vendor/multicold/code/run_portable.py", + "research/complex-t16-rare-pattern-exchanges/MANIFEST.json", + "research/complex-t16-rare-pattern-exchanges/upstream1087.tar.gz", + "research/complex-t16-rare-pattern-exchanges/verify.py", + "research/gauge44-slots-t16/code/audit_allcold.py" + ], + "commands": [ + { + "argv": [ + "{python}", + "-B", + "{package}/verify.py" + ], + "timeout_seconds": 600, + "checks": "Replays the selected final stage on archived provider files (audit equals archived selected receipt); legacy vs fast audit sub-stage outputs and merged log byte-identical; deepcopy call counts; tampered-template, unpinned-script, identity-copier and copy-semantics controls." + } + ] +} diff --git a/research/audit-allcold-fast-copy/fast_copy.py b/research/audit-allcold-fast-copy/fast_copy.py new file mode 100644 index 0000000000..7f28c4e307 --- /dev/null +++ b/research/audit-allcold-fast-copy/fast_copy.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +"""Opt-in launcher: run a pinned retained verifier with a checked C-level deepcopy. + +Usage (drop-in for `python3 -B RUN_PORTABLE SCRIPT ARGS...`): + + python3 -B fast_copy.py RUN_PORTABLE SCRIPT ARGS... + +Both RUN_PORTABLE and SCRIPT must match the SHA-256 allow-list below. The +pinned `run_portable.py` then runs SCRIPT exactly as it would without this +launcher. The only process change is `copy.deepcopy`: for a top-level dict or +list, the copy is `marshal.loads(marshal.dumps(x))`, accepted only if marshal +serializes the value (it refuses anything but core builtin types, including +dict/list subclasses) and the copy has the same type and compares equal. +Otherwise, and for every call with a `memo`, the original `copy.deepcopy` +runs. marshal (format 4) records repeated objects as back-references, so +aliasing inside the value is kept, as it is by deepcopy; every dict and list in +the copy is a new object. + +No verifier source is edited, no check or mutation control is skipped, and no +result is cached. Set FAST_COPY_STATS=PATH to write call counts as JSON +(stdout and stderr stay untouched, so stage logs remain byte-identical). + +jackmgaller with Claude Code (Claude Opus 5.5) assistance. Apache-2.0. +""" +import copy +import hashlib +import json +import marshal +import os +import runpy +import sys +from pathlib import Path + +if not __debug__: + raise RuntimeError('Assertions must remain enabled') + +# run_portable.py retained by research/complex-k492-owner-repair/vendor/multicold. +RUN_PORTABLE = {'322d29d43dbee8130cde15d4bf05396698b5e3dfe3735813899db91cc0ba7955'} +# audit_allcold.py of the selected PR1087 T16 replay (upstream1087.tar.gz code/) +# and of research/gauge44-slots-t16/code/ (the cold baseline1042 `t16` stage). +SCRIPTS = {'1467a2acb654e62552c69fb9a7c4c85f70a02120881d222313865058ab71a6cc'} + +_original_deepcopy = copy.deepcopy +STATS = {'marshal': 0, 'fallback': 0} + + +def checked_deepcopy(x, memo=None, _nil=[]): + if memo is None and type(x) in (dict, list): + try: + y = marshal.loads(marshal.dumps(x)) + ok = type(y) is type(x) and y == x + except (ValueError, RecursionError): # unmarshallable, too deep or cyclic + ok = False + if ok: + STATS['marshal'] += 1 + return y + STATS['fallback'] += 1 + return _original_deepcopy(x) if memo is None else _original_deepcopy(x, memo, _nil) + + +def sha(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def main(argv, copier=checked_deepcopy): + if len(argv) < 2: + raise SystemExit('usage: fast_copy.py RUN_PORTABLE SCRIPT ARGS...') + launcher, script = Path(argv[0]).resolve(), Path(argv[1]).resolve() + if sha(launcher) not in RUN_PORTABLE: + raise SystemExit('fast_copy: run_portable.py is not on the allow-list: ' + str(launcher)) + if sha(script) not in SCRIPTS: + raise SystemExit('fast_copy: script is not on the allow-list: ' + str(script)) + copy.deepcopy = copier + sys.argv = [str(launcher), *argv[1:]] + try: + runpy.run_path(str(launcher), run_name='__main__') + finally: + copy.deepcopy = _original_deepcopy + out = os.environ.get('FAST_COPY_STATS') + if out: + Path(out).write_text(json.dumps(STATS, sort_keys=True) + '\n') + + +if __name__ == '__main__': + main(sys.argv[1:]) diff --git a/research/audit-allcold-fast-copy/verify.py b/research/audit-allcold-fast-copy/verify.py new file mode 100644 index 0000000000..f3afb854c9 --- /dev/null +++ b/research/audit-allcold-fast-copy/verify.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +"""Replay the selected final stage, then compare its audit sub-stage with fast_copy.py. + +1. Rebuild the selected PR1094 final stage unchanged: the archived gauge44 provider + files from research/community-frontier1094-audit/selected-replay.tar.gz are the + `--local` input of research/complex-t16-rare-pattern-exchanges/verify.py, which + runs the frozen PR1087 verify_t16.py (all 11 sub-stages). Its ASYNC-AUDIT.json + must equal the archived selected receipt byte for byte. +2. Rerun that stage's `audit` sub-stage (pinned audit_allcold.py with its 15 + mutation controls) in fresh copies of the same inputs, as the pinned + `run_portable.py` command (legacy) and through fast_copy.py, alternating order. + Every written file and the merged stdout/stderr log must equal the in-stage + audit outputs byte for byte. Wall times are recorded; there is no threshold. +3. Controls: a tampered hosted template fails under both launchers; an unpinned + script is refused; an identity "copy" (no copy) changes the outputs, so the + copy is load-bearing; in-process semantics of the checked copy and fallback. + +Timing only. No kappa, certificate, supplier, frontier or pinned source changes. +jackmgaller with Claude Code (Claude Opus 5.5) assistance. Apache-2.0. +""" +import argparse +import gzip +import hashlib +import json +import math +import os +import shutil +import statistics +import subprocess +import sys +import tarfile +import tempfile +import time +from pathlib import Path + +assert __debug__, 'Assertions required' +PKG = Path(__file__).resolve().parent +ROOT = PKG.parents[1] +sys.path.insert(0, str(PKG)) +import fast_copy # noqa: E402 + +STAGE = ROOT / 'research/complex-t16-rare-pattern-exchanges/verify.py' +ARCHIVE = ROOT / 'research/community-frontier1094-audit/selected-replay.tar.gz' +MULTICOLD = ROOT / 'research/complex-k492-owner-repair/vendor/multicold' +RUN_PORTABLE = MULTICOLD / 'code/run_portable.py' +KAPPA = '0.001672660197076918' +AUDIT_FILES = ('ASYNC-AUDIT.json', 'OWNERS-AND-PROFILE.json', 'IDLE-REMAINDER-FRAMES.json') +EXPECTED_STATS = {'fallback': 0, 'marshal': 4838} + + +def sha(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def tree(path): + path = Path(path) + return {str(f.relative_to(path)): sha(f) for f in sorted(path.rglob('*')) if f.is_file()} + + +def stage_env(spec): + # Exactly verify_t16.py's env0 plus its WITNESS_SPEC for the `audit` sub-stage. + env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1', PYTHONUTF8='1', PYTHONINTMAXSTRDIGITS='0', + MULTICOLD_SOURCE=str(MULTICOLD), GAIN_PR564_SOURCE=str(MULTICOLD)) + for k in ('PYTHONOPTIMIZE', 'WITNESS_DISCOVER', 'WITNESS_RECORD', 'WITNESS_SPEC', 'FAST_COPY_STATS'): + env.pop(k, None) + env['WITNESS_SPEC'] = str(spec) + return env + + +def build_fixture(work): + archived = work / 'archived' + archived.mkdir() + with tarfile.open(ARCHIVE) as t: + for m in t.getmembers(): + assert m.isfile() and (archived / m.name).resolve().is_relative_to(archived), 'Unsafe member' + t.extractall(archived, filter='data') + local = work / 'local' + shutil.copytree(archived / 'candidate/strict/global/base/providers/gauge44', local) + stage = work / 'stage' + env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1') + env.pop('PYTHONOPTIMIZE', None) + t0 = time.monotonic() + out = subprocess.run([sys.executable, '-B', str(STAGE), '--main', str(ROOT), '--local', str(local), + '--work', str(stage)], env=env, capture_output=True, text=True) + seconds = time.monotonic() - t0 + assert out.returncode == 0, out.stdout[-3000:] + out.stderr[-3000:] + assert out.stdout.strip().endswith('PASS_T16_RARE_PATTERN_EXCHANGES ' + KAPPA), out.stdout[-500:] + g = stage / 'strict/global' + archived_audit = archived / 'candidate/strict/global/audit/ASYNC-AUDIT.json' + assert (g / 'audit/ASYNC-AUDIT.json').read_bytes() == archived_audit.read_bytes(), \ + 'Fixture audit differs from the archived selected receipt' + substages = json.loads((stage / 'strict/STAGES.json').read_text()) + return stage, seconds, substages + + +def run_audit(stage, dest, mode, script=None, extra_env=None): + """Run the `audit` sub-stage on fresh copies of the stage's inputs.""" + g = stage / 'strict/global' + dest.mkdir(parents=True) + hosted = dest / 'hosted' + shutil.copytree(g / 'hosted', hosted) + (hosted / 'IDLE-REMAINDER-FRAMES.json.gz').unlink() # written by the audit itself + script = script or stage / 'source/code/audit_allcold.py' + args = [str(script), str(g / 'base'), str(hosted), str(dest / 'audit/ASYNC-AUDIT.json'), '--mutation-controls'] + if mode == 'legacy': + argv = [sys.executable, '-B', str(RUN_PORTABLE), *args] + elif mode == 'fast': + argv = [sys.executable, '-B', str(PKG / 'fast_copy.py'), str(RUN_PORTABLE), *args] + else: # in-process control copier selected by name + code = ('import sys;sys.path.insert(0,%r);import fast_copy;' + 'fast_copy.main(sys.argv[1:],copier=lambda x,memo=None,_nil=None:x)') % str(PKG) + argv = [sys.executable, '-B', '-c', code, str(RUN_PORTABLE), *args] + env = stage_env(stage / 'source/data/SPEC.json') + env.update(extra_env or {}) + log = dest / 'audit.log' + t0 = time.monotonic() + with log.open('w') as s: # merged like verify_t16.py's run() + rc = subprocess.run(argv, env=env, stdout=s, stderr=subprocess.STDOUT).returncode + return rc, time.monotonic() - t0 + + +def outputs(stage_or_dest, in_stage=False): + base = stage_or_dest / ('strict/global' if in_stage else '') + files = {name: sha(base / 'audit' / name) for name in AUDIT_FILES} + files['hosted/IDLE-REMAINDER-FRAMES.json.gz'] = sha(base / 'hosted/IDLE-REMAINDER-FRAMES.json.gz') + files['audit.log'] = sha((stage_or_dest / 'strict/audit.log') if in_stage else (stage_or_dest / 'audit.log')) + return files + + +def semantics(): + import marshal # noqa: F401 (documented dependency of fast_copy) + stats0 = dict(fast_copy.STATS) + shared = [1, [2]] + x = {'a': [shared, shared], 's': 'é\ud800', 'n': None, 'b': True, 'f': 0.1, 'i': 10 ** 40} + y = fast_copy.checked_deepcopy(x) + assert y == x and y is not x and y['a'] is not x['a'] and y['a'][0] is not shared + assert y['a'][0] is y['a'][1], 'aliasing kept like deepcopy' + y['a'][0][1].append(3) + assert shared == [1, [2]], 'copy is independent of the original' + + class D(dict): + pass + + class O: + pass + + cyc = [] + cyc.append(cyc) + fallbacks = [D(k=1), {'nan': math.nan}, {'o': O()}, cyc] + for v in fallbacks: + fast_copy.checked_deepcopy(v) + z = fast_copy.checked_deepcopy(cyc) + assert z is not cyc and z[0] is z + assert type(fast_copy.checked_deepcopy(D(k=1))) is D + memo = {} + fast_copy.checked_deepcopy([1], memo) + d = {k: fast_copy.STATS[k] - stats0[k] for k in stats0} + assert d == {'marshal': 1, 'fallback': len(fallbacks) + 3}, d + return dict(marshal_copies=1, fallbacks=d['fallback'], + fallback_cases=['dict subclass', 'NaN (copy not equal)', 'custom object', 'cyclic list', + 'cyclic list (identity check)', 'subclass type check', 'explicit memo']) + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument('--work', type=Path, help='fresh directory outside the repository (default: temporary)') + ap.add_argument('--rounds', type=int, default=1, help='alternating legacy/fast timing rounds') + a = ap.parse_args() + assert a.rounds >= 1 + if a.work is None: + tmp = tempfile.TemporaryDirectory(prefix='audit-allcold-fast-copy.') + work = Path(tmp.name).resolve() / 'run' # e.g. macOS /var -> /private/var + else: + tmp = None + work = a.work.resolve() + assert not work.exists() and not work.is_relative_to(ROOT), 'Use a fresh directory outside the repository' + work.mkdir(parents=True) + script = work / 'stage/source/code/audit_allcold.py' + + stage, stage_seconds, substages = build_fixture(work) + assert sha(script) in fast_copy.SCRIPTS and sha(RUN_PORTABLE) in fast_copy.RUN_PORTABLE + assert sha(script) == sha(ROOT / 'research/gauge44-slots-t16/code/audit_allcold.py'), \ + 'Same audit script as the cold baseline1042 t16 stage' + reference = outputs(stage, in_stage=True) + print('PASS fixture: selected final stage replayed, audit equals archived receipt', flush=True) + + times = {'legacy': [], 'fast': []} + stats = None + for i in range(a.rounds): + for mode in (('legacy', 'fast') if i % 2 == 0 else ('fast', 'legacy')): + dest = work / 'ab' / ('%s-%d' % (mode, i)) + extra = {'FAST_COPY_STATS': str(dest.parent / ('stats-%d.json' % i))} if mode == 'fast' else None + rc, seconds = run_audit(stage, dest, mode, extra_env=extra) + assert rc == 0, (mode, (dest / 'audit.log').read_text()[-3000:]) + assert outputs(dest) == reference, mode + ': audit outputs differ from the in-stage receipt' + times[mode].append(seconds) + if mode == 'fast': + stats = json.loads((dest.parent / ('stats-%d.json' % i)).read_text()) + assert stats == EXPECTED_STATS, stats + print('PASS round %d: legacy %.2fs, fast %.2fs, outputs identical' % + (i, times['legacy'][-1], times['fast'][-1]), flush=True) + report = json.loads((work / 'ab/fast-0/audit/ASYNC-AUDIT.json').read_text()) + assert report['mutation_controls_run'] and len(report['mutations_rejected']) == 15 + + controls = {} + # Tampered hosted template: both launchers must reject it. + bad = work / 'controls/tampered' + for part in ('source', 'strict'): + shutil.copytree(stage / part, bad / part) + t = bad / 'strict/global/hosted/GLOBAL-TEMPLATE.json.gz' + obj = json.loads(gzip.decompress(t.read_bytes())) + obj['physical'] -= 1 + t.write_bytes(gzip.compress(json.dumps(obj).encode(), mtime=0)) + for mode in ('legacy', 'fast'): + rc, _ = run_audit(bad, bad / mode, mode) + assert rc != 0 and 'AssertionError' in (bad / mode / 'audit.log').read_text(), mode + ' accepted tampering' + controls['tampered_hosted_template'] = 'rejected by legacy and fast' + # Unpinned script: one appended comment is refused before anything runs. + other = work / 'controls/unpinned/audit_allcold.py' + other.parent.mkdir(parents=True) + shutil.copytree(script.parent, other.parent, dirs_exist_ok=True) + other.write_bytes(script.read_bytes() + b'# changed\n') + rc, _ = run_audit(stage, work / 'controls/unpinned/run', 'fast', script=other) + assert rc != 0 and 'not on the allow-list' in (work / 'controls/unpinned/run/audit.log').read_text() + controls['unpinned_script'] = 'refused' + # Identity copier: without real copies the mutation controls corrupt the audited + # template, so outputs must change (or the audit must fail). + dest = work / 'controls/identity' + rc, _ = run_audit(stage, dest, 'identity') + changed = rc != 0 or not all((dest / 'audit' / n).is_file() for n in AUDIT_FILES) or outputs(dest) != reference + assert changed, 'identity copier was not detected' + controls['identity_copy'] = 'detected (exit %d)' % rc + controls['checked_copy_semantics'] = semantics() + + legacy, fast = statistics.median(times['legacy']), statistics.median(times['fast']) + result = dict( + status='PASS_AUDIT_ALLCOLD_FAST_COPY_IDENTICAL', + python=sys.version.split()[0], platform=sys.platform, + audit_allcold_sha256=sha(script), run_portable_sha256=sha(RUN_PORTABLE), + fast_copy_sha256=sha(PKG / 'fast_copy.py'), + selected_stage_seconds=round(stage_seconds, 3), + selected_stage_substages={s['name']: s['seconds'] for s in substages}, + audit_outputs_sha256=reference, rounds=a.rounds, + legacy_seconds=[round(x, 3) for x in times['legacy']], fast_seconds=[round(x, 3) for x in times['fast']], + median_speedup=round(legacy / fast, 3), deepcopy_calls=stats, mutation_controls=report['mutations_rejected'], + controls=controls, + scope='Same fixture, same pinned run_portable.py and audit_allcold.py bytes; only copy.deepcopy is replaced ' + 'by a checked marshal round trip. Timing has no pass threshold. No kappa or certificate change.') + if tmp is None: + (work / 'RESULT.json').write_text(json.dumps(result, indent=2) + '\n') + print(json.dumps({k: result[k] for k in ('legacy_seconds', 'fast_seconds', 'median_speedup', 'deepcopy_calls')})) + print(result['status'], 'median speedup %.2fx' % result['median_speedup']) + if tmp is not None: + tmp.cleanup() + + +if __name__ == '__main__': + main()