Skip to content

Reject staged input changes after registered research replays - #3637

Open
roger-sato wants to merge 1 commit into
CrocSwap:mainfrom
roger-sato:fix/research-ci-staged-drift
Open

roger-sato wants to merge 1 commit into
CrocSwap:mainfrom
roger-sato:fix/research-ci-staged-drift

Conversation

@roger-sato

Copy link
Copy Markdown

Summary

The registered research replay's final “Committed inputs remain unchanged” gate runs only git diff --exit-code, which compares the working tree with the index. If a replay stages a changed input, a deletion, or a newly added file, the gate currently exits successfully despite the checkout differing from the committed inputs.

Keep that worktree comparison and also run git diff --cached --exit-code against the commit. Both comparisons matter: replacing the existing command with git diff HEAD would accept a staged edit whose working bytes were subsequently restored to HEAD. Ordinary untracked replay outputs remain allowed.

The regression tests read the actual named workflow run step and execute it with fail-fast Bash in tiny real Git repositories. They cover clean and untracked-output controls, unstaged edits/deletions, staged edits/deletions/additions, and an index-only difference hidden by a restored working file. They run in the existing test_research_*.py onboarding suite.

Test plan

  • Before the workflow change, the eight new tests have exactly three failures: staged edit, deletion, and addition incorrectly pass the gate. After the change all eight pass under CPython 3.12.14 and 3.11.6 on macOS arm64.
  • python3.12 -B -m unittest discover -s tests -p 'test_research_ci_drift.py' -v — 8 passed.
  • python3.12 -B -m unittest discover -s tests -p 'test_research_matrix_batches.py' -v — 5 passed, including actual planner CLI coverage and unchanged matrix boundaries.
  • git diff --check passes. A full-checkout search found neither changed path in registered SOURCES.json or VERIFY.json declarations.

The existing planner deliberately selects all 1,629 registered packages when this shared workflow changes; this PR retains that routing and does not bypass or filter it. The bounded local checks above exercised planning and the Git gate only. No all-package mathematical replay, selected cold replay, or hosted CI success is claimed. This is an end-of-replay cleanliness check, not a sandbox or proof that inputs were never temporarily modified during execution.

Scope and attribution

Based on canonical main 20ecf2891d80b5acf18d980b2164fb44ea7faa90. The reusable workflow was introduced by ywz65 in PR #2505, head a357942c5ee5ce88948c120a0a174bb7dd0c2d2a, and retained in Douglas R. Colkitt's integration commit e182f1a175b3e7635b1c7a66fdf6f1c4a40cf3bd. This change closes the staged-file gap in its existing drift contract. PR #2602 separately checks publication bytes, and PR #3142 validates sparse planning manifests; neither supplies this post-replay CI gate.

Exact-topic overlap was checked against a fresh snapshot of 344 open PRs, 2,964 closed PRs and 2,453 Discussion records, including the current handoff hub. No equivalent fix was found; text search does not prove exclusivity. Only this reusable workflow and its new behavioral tests change. Frozen sources, replay commands, scientific artifacts and the selected result remain unchanged.

Prepared by roger-sato with substantial OpenAI Codex assistance in implementation and independent agent review. Apache-2.0.

@roger-sato
roger-sato marked this pull request as ready for review October 18, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant