Skip to content

Tooling: preserve registered replay coverage for input and workflow changes - #1354

Merged
Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
cbethin:codex/replay-input-invalidation
Oct 14, 2026
Merged

Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
cbethin:codex/replay-input-invalidation

Conversation

@cbethin

@cbethin cbethin commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Registered packages can declare proof inputs anywhere in the repository, but the workflow currently filters events before that declaration reaches the planner. A change confined to a declared certificate or review file can therefore skip replay. Two later selection gaps also lose required checks: Git's default rename detection reports only the new path, and a change to the replay workflow selects no packages.

This tooling repair closes those three gaps in the same invalidation path:

  • Run the planning job for otherwise eligible pull requests and pushes to main, without path filters. The manifest-aware selector still produces no replay jobs for unrelated changes.
  • Use git diff --no-renames --name-only -z so renaming an input invalidates consumers of its old path as well as its destination.
  • Treat the replay workflow, like the runner, as a shared control that selects every registered package.

The claim concerns packages present at the compared head, with canonical declared input paths and a valid comparison base. It does not infer undeclared imports or bypass GitHub event eligibility/permissions. Package verification, source pins, exactness checks, job timeouts and concurrency limits remain unchanged. No mathematical or κ claim changes.

Concrete evidence: pending PR1262 at 4b196b40dcac3ec40f62ec285babac53a1b5a3c5 declares 16 inputs. Four fall outside both former event filters: the selected certificate, the #1042 checkpoint, and the #1047/#1099 integration receipts. The base main tree has zero registered VERIFY.json files; this establishes a defect in the registration contract, not an observed missed hosted run. PR1262's mathematics is not evaluated here.

Validation:

python3.12 -B -m unittest discover -s tests -p 'test_research_*.py' -v

Expected: 26 tests pass. On the primary CPython 3.15.0 research runtime, 25 pass and one skips because full generic replay explicitly requires 3.12. The compatibility run uses installed CPython 3.12.12. Python.org confirms 3.15.0 remains current stable.

  • Six new regression groups execute the actual planner CLI in temporary Git repositories. They cover inputs under certificates/docs/src, paths containing spaces, deletion, a Git-detected rename, both shared controls, unrelated edits, and new package-local artifacts. The JSON matrix and GitHub output count must agree.
  • Restoring only the original workflow and runner causes three regression groups to fail. A separate before/after Git-tree-object model, which neither calls git diff nor imports the planner, agrees on ten seed-94 scenarios, including shared consumers.
  • A separate Ruby YAML parser plus a bounded event model finds four excluded PR1262 inputs before the patch and none after. It is not a general Actions emulator. The no-path-filter regression is an explicit static policy guard.
  • actionlint 1.7.12 passes for the changed workflow (-shellcheck='' -pyflakes=''). The selected-record check and all 25 selected-record tests pass. No hosted CI execution, selected cold replay, Windows run, or independent human review is claimed.

Cost: the existing planning/helper-test job now runs on all otherwise eligible events, including unrelated edits. Expensive package jobs remain selective; changing the workflow deliberately replays all registered packages. Hosted startup cost was not measured.

Base bce54fb3f895cbe5cd4c0eb477c3afeed100190f; OS seed 94, verification engineering. Final overlap covered 834 open PRs, 76 discussion records, and 325 local records, with complete remote pagination and no warnings. No pinned coordination issue exists; the claim and evidence are saved locally. PR474 concerns a historical gen4 workflow. PR1298 and PR1299 change other launcher behavior; this adds a separate test file and does not include their pending fixes.

Credit Douglas Colkitt for the package tooling and Sagar Thakrar for PR1262's concrete manifest example. Existing Apache-2.0 attribution is preserved. Prepared for Charles Bethin with substantial OpenAI Codex assistance in diagnosis, implementation, tests, and writing. The guide describes the revised contract. GitHub's documented path-filter behavior: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onpushpull_requestpull_request_targetpathspaths-ignore.

@Mister-Meeseeks
Mister-Meeseeks merged commit e523fd5 into CrocSwap:main Oct 14, 2026
61 checks passed
@Mister-Meeseeks

Copy link
Copy Markdown
Contributor

100-PR batch review at 7b7bf79d30fb214e36ed80c24ea47044362a20b2.

Merged to main, preserving the original contributor commits and package attribution.

Reviewed scope: Replay planning invalidates packages for declared inputs anywhere, both sides of renames, and shared workflow changes. Real-Git tests cover deletions, renames and unrelated changes; combined runner tests preserve current provenance and assertion safeguards.

The pinned scoped command passed; the batch ledger contains its command, input hashes, environment and log. This incorporates the research/tooling contribution and does not change the selected κ or discharge the inherited multiplication-theorem assumptions.

Batch review, attribution and evidence. Thank you for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants