Repository navigation
ci: select affected verification inputs and preserve historical replays - #2361
Closed
thakrarsagar wants to merge 4 commits into
Closed
thakrarsagar wants to merge 4 commits into
thakrarsagar wants to merge 4 commits into
Conversation
…l 52 jobs per push Every pull-request revision and every push ran the full artifact matrix (48 Python jobs, the finite replay, three formal packages) plus four unfiltered package workflows, about 232 runner-minutes per sync, while 23 of 30 recent pull requests only add a new research package. With about 925 runs a day the queue reached 20 hours. A plan job now diffs the pull request's merge commit against its base (or a push's before..after), classifies each changed path with .github/verify-groups.json and runs only the groups it reaches, on every supported Python. The declarations are recorded, not guessed: scripts/trace_verify_inputs.py runs each group's unchanged command under strace and records the tracked files read, directories listed, existence checks, probes and whole-tree git queries; every job runs under it and fails on an undeclared input. Paths nobody declared, the root attribute files, scripts/, the Makefile, symlink or submodule changes and ignored-but-tracked paths run everything; scheduled and manual runs and the ci:full-verify label run everything; pull-request revisions supersede each other, pushes never do. The four package verifiers become a planner-selected job (native paths filters stop at a few hundred changed files), verify-summary gives one status that requires success for every selected job, and a manual workflow can cancel superseded pull-request runs from the backlog. Replaying the planner over the 60 most recently updated pull requests gives a mean of 63 runner-minutes per sync (27% of before); a research-only pull request runs the two whole-tree groups and the plan job, about 25 minutes. Not changed: pinned sources, frozen evidence, the selected result, or any verification command. Known, pre-existing: d7d8bfc and 6178ded changed pinned sources without updating their pins, so the clones group and the tests group fail on main (documented in docs/ci.md, not fixed here). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keep verification active on unrelated PR label events, honor existing full-run requests, and preserve recorded failures when checking saved trace coverage. Reject incompatible or incomplete recordings instead of accepting partial input evidence. Report failed cancellation requests and reject negative limits. Complete the historical formal dependency recording after fetching its pinned Mathlib cache. Preserve whole Lean project trees and explicitly cover external notes, certificates, upstream sections and alternate project discovery. The full target and its traced coverage pass; the original partial recording had missed inputs after a missing dependency stopped compilation. Add regressions for external references, label handling, saved traces and cancellation bounds. Preserve the original Apache-2.0 attribution, frozen archives, all source pins, selected-result record, optimized scripts, and earlier optional speedups. The historical compatibility repair authenticates exact predecessor bytes rather than changing their evidence. Document current workload estimates and the stable required-check gate in docs/ci.md. Validation on Linux/Python 3.14.4: full traced tests group (130 modules and 20 patch checks), complete traced historical formal target, actionlint, selected-result/input checks, and clean tracked-artifact drift checks. Final formal declaration also passed independent review and recorded input coverage. Earlier full clones replay passed with authenticated historical inputs. Hosted execution and Python 3.11/3.13 runs remain unverified. Prepared with substantial OpenAI Codex assistance and independent agent code review. Earlier CI implementation assistance by Claude Fable 5.1 remains credited in the preceding commit and documentation.
Retain selective traced CI and main's authenticated historical baselines, accepted live-source checks, and isolated test dispatch. Preserve all frozen manifests, supplier sources, and the selected result. Restore the pinned partial-swap and sparse-fusion helpers in historical clone replays. Run the original split-skip native producer command inside its authenticated copy so newer live optimizations do not break its source pins. Add regression coverage for tracked baselines, complete frozen input coverage, and preservation of the native Makefile recipe. Validated against canonical main ab4fcd3: actionlint, 120 focused tests, 21 compatibility tests, the complete traced clones group (both native reconstructions, 4,073,300 data pairs and 21 package tests), and selected-result/input checks. Seven broader test failures reproduce on untouched main; exact commands, evidence and limits are in docs/ci.md. Prepared with OpenAI Codex assistance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change and mathematical scope
Every revision currently launches the full artifact matrix and four unfiltered
package workflows, including checks whose inputs did not change. This change
selects affected groups from declared, runtime-checked inputs, retains every
supported Python combination for selected groups, adds superseded-PR cancellation
and a stable
verify-summarygate, and caches pinned Mathlib dependencies.Scheduled/manual/full-label runs retain complete coverage. Unknown paths and
changes to shared verification code run everything.
The independent audit fixed label events producing empty successful gates,
failed trace recordings losing their failure status, cancellation API errors
being ignored, and missing external inputs in the historical formal declaration.
The complete historical formal target has now been traced successfully.
Historical clone/split checks also replay authenticated predecessor source bytes
in private copies, fixing pre-existing source-pin failures without reverting the
current optimized implementations or weakening frozen evidence. This is CI
maintenance; no selected frontier, mathematical claim, source pin, archive or
upstream source is changed.
Task and overlap
Explicit task: reduce CI queue/load and audit the changes before submission.
Base:
0fdd7d82de03e43c9e5a1000dfbcd380d1b687d2from canonical main.Scope remains CI maintenance; no research task was drawn.
The existing optional speedups in #1041, #1059, #1151, #1433 and #1584 remain
opt-in and byte-unchanged. #2214 concerns selected-result navigation; this change
addresses execution selection and historical replay compatibility. See
docs/ci.mdfor dependency semantics, timing methodology, historical sources,required-check guidance and validation limits.
Replaying the selector over 60 recent PRs gives an estimated mean workload of
63 runner-minutes versus 232 previously. These use old per-job durations and
exclude new tracing/copy/summary overhead; they are not a hosted speedup claim.
Validation
Validation was run in an isolated worktree, with artifact suites sequentially.
Linux x86_64, Python 3.14.4, pinned Lean 4.21.0 / Mathlib
308445d7985027f538e281e18df29ca16ede2ba3.actionlint .github/workflows/*.yml.clonesgroup: native reconstructions, 4,073,300 fixed-basispairs, historical witnesses and 21 package tests.
298 machine-transfer declaration audits, Lean build, 386 project axiom audits,
source/literal checks and paired-circuit reconstruction.
checks and the declared-input coverage audit passed.
Reproduction commands and local-only evidence locations are in
docs/ci.md.Hosted runs and Python 3.11/3.13 execution have not yet been verified. The selected
cold replay was not rerun because its construction and pinned inputs are unchanged.
No live backlog cancellations or branch-protection changes were performed.
These scoped checks do not formally verify the full multiplication theorem.
Credit
Sagar Thakrar, with substantial Claude Fable 5.1 and OpenAI Codex assistance,
including independent code reviews. The authenticated historical adapter builds
on Dennis's prior work and preserves its source attribution. Wentao Li's live
circuit optimizations and equivalence controls remain unchanged. Original
snapshot headers, contributors and Apache-2.0 licensing are retained.
Open next questions
Measure hosted end-to-end duration and queue recovery after deployment. This is
maintenance tooling, not a new research package.