[kyle-sexton]

Refs: #12302

Summary

This commits verify-legacy-fails.json, the follow-up #12302 left. Committing it turns on the verdict reader's FAIL index mode. The reader then judges a pull request from its FAIL-index artifacts plus this file, instead of paging the whole repository artifact listing on every judgment. That listing grew to about 136 pages, its page count never settled against GitHub's live count, and it caused breaker trips and the queue cycling #12472 fixed.

The file lists the 550 FAIL verdicts uploaded before the index existed: since 2026-10-17, cut over at 2026-11-02T09:16:56Z (#12302's squash commit plus the one-hour margin).

Fix

• plugins/source-control/skills/babysit-prs/scripts/verify-legacy-fails.json holds 550 entries from 550 runs. Each entry records the run's verified origin and its verdict's head branch. ci-status's sparse checkout already names the file (pr-require-checks.yml:4982).
• It was built by babysit_verify_evidence.py legacy-fails (flags as perf(source-control): find pr-verify FAILs by exact-name index instead of the whole artifact listing #12302's Follow-up gives them), with --merge-sha 23dfa3fe38, from 15,783 pr-verify runs. Owner-token reads stayed under 1,800 per rate window, across checkpoint-resumed runs, with no rate-limit or breaker stop.
• About 54% of pr-verify runs in the window were skipped runs that hold no verdict. The build reads each run's artifacts anyway, which is where most of its calls went (actions/runs/{id}/artifacts, 6,495 reads in the last resumed run). A skip filter would roughly halve a rebuild. It is not built here: no rebuild is planned, and the file only shrinks as entries expire.
• Guard: no entry was uploaded before --since. The earliest is 2026-10-17T05:11:02Z, the oldest unexpired FAIL perf(source-control): find pr-verify FAILs by exact-name index instead of the whole artifact listing #12302 measured (05:11Z). fix(source-control): refuse pre-since legacy FAILs and retry the FAIL index pre-read #12370's legacy-fails refuses this case. The file was built on the pre-fix(source-control): refuse pre-since legacy FAILs and retry the FAIL index pre-read #12370 code, so this check was applied to the output by hand.

Verification

• main's Reader._legacy loads the file: 550 entries, 550 live, LEGACY_FAILS_VERSION 1.
• With the file present: test_babysit_verify_evidence.py 429 passed, test_babysit_verify_index.py 41, test_babysit_merge_unattended.py 298, test_babysit_lane_room.py 75.
• The independent security review passed. It checked the following:
  • The file holds no secret or local path.
  • All 550 names parse as fail verdicts, and every created_at falls between since and cutover.
  • 3 entries match the API exactly.
  • The cutover matches the commit that made FAIL uploads require the index.
  • Every FAIL the listing path counted is covered: pre-since FAILs and in-flight runs refuse the build, and a re-run of an old run executes main's write-verdict.mjs, which throws on a FAIL without the index.
  • A failed index read refuses (verify:unreadable).
• AFTER: the GITHUB_TOKEN window sum against the baselines.log 04:55Z BEFORE goes to the hub once the reader runs in index mode.

Follow-up owed after merge

The review lanes count this file as read through scripts/review-generated-data.txt (#12601), because they cannot read 184 KB whole. That exemption is needed only for this one-time add: the loader checks structure, not content, so a later large rewrite must not skip review. Once this PR is on main, remove the entry from scripts/review-generated-data.txt. The removal PR opens, held with do-not-merge until this one merges, as soon as #12601 is on main.

Related

• ci(review): count the legacy FAILs file as read through generated-data #12601: lists this file for the review lanes' generated-data input.

• perf(source-control): find pr-verify FAILs by exact-name index instead of the whole artifact listing #12302: added FAIL index mode.

• fix(source-control): refuse pre-since legacy FAILs and retry the FAIL index pre-read #12370: the legacy-fails pre-since guard.

• fix(source-control): keep a queued lanes App entry whose verdict the sweep cannot read #12472: the queue cycling this ends at its source.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T23:55:31.584387Z	2d742a4	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-review: 1 inline finding(s) on 2d742a48baf31ee089635be74bde4ab4c3ac7c79; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed 9972995106647847c047038153e0abd2ae5cfc81: read 2 of 2 changed file(s); 1 data file(s) not reviewed: plugins/source-control/skills/babysit-prs/scripts/verify-legacy-fails.json; 0 finding(s), 0 as inline comments.

claude-review (code-review lane): no findings on 9972995106647847c047038153e0abd2ae5cfc81. I read both changed files in full.

What changed

• .changes/source-control/data-verify-legacy-fails-file-1f4a8786.md is a new bump: patch changelog fragment. It says the pr-verify verdict reader now runs in FAIL index mode, using the committed verify-legacy-fails.json. That matches the PR description: 550 entries, and the same follow-up to perf(source-control): find pr-verify FAILs by exact-name index instead of the whole artifact listing #12302.
• .github/workflows/pr-require-checks.yml:1623-1624 adds plugins/source-control/skills/babysit-prs/scripts/verify-legacy-fails.json to the typos step's exclude: line, with a one-line comment giving the reason.
  • scripts/check-affected.sh:482-485 reads the first exclude: line after id: typos and passes each glob to typos with --force-exclude. The new comment line starts with #, so it can't match ^ *exclude: , and the new path does reach the local check.
  • I confirmed this by reading the code, not by running it.

Prior finding resolved: the _typos.toml exemption, discussion_r4449223436.

• The _typos.toml hunk is gone from the diff.
• The exemption now sits on the caller's exclude: line, which is the fix that finding asked for.
• The owner resolved the thread, and its lines are unchanged since then.

Not reviewed

• verify-legacy-fails.json is the generated data file the base branch declares. As instructed, I did not review its contents.
• The verification in the PR description is the author's claim and I did not check it. That covers the test pass counts, the 550 live entries, the --since guard checked by hand, and the independent security review.

Closing issues: none. The PR uses Refs: #12302 and closes no issue, so there are no acceptance criteria to check.

Path-scoped rules: no .claude/rules/ glob matches either changed file.

Manifests: no plugin.json or marketplace.json change.

Spend: none. The change adds no paid resource and doesn't change run frequency.

[github-actions]

claude-security-review reviewed 9972995106647847c047038153e0abd2ae5cfc81: read 2 of 2 changed file(s); 1 data file(s) not reviewed: plugins/source-control/skills/babysit-prs/scripts/verify-legacy-fails.json; 0 finding(s), 0 as inline comments.

Security review: PR #12558 (99729951)

No security issues found.

critical-findings: 0

Scope

I read every hunk of both files in the diff:

• .changes/source-control/data-verify-legacy-fails-file-1f4a8786.md: a new changelog fragment. It is prose only, with no instructions aimed at a lane.
• .github/workflows/pr-require-checks.yml: one comment line and one path added to the exclude: input of the Spell-check step (L1608-L1624).

The wrapper lists verify-legacy-fails.json as a generated data file the base branch declares. It is not in this diff, so this round did not review it. The prior round (8789c4de) checked its contents by pattern.

What I checked

• pr-require-checks.yml is a merge-hold path, but this hunk is not one of its ci-status hold steps.
  • The changed step is the typos Spell-check. It runs with continue-on-error: true (L1610) and only skips spelling checks on the listed path.
  • No permissions:, trigger, if:, token, or hold step changes.
  • The gitleaks secret scan is a separate step (L1626 onward). This exclude does not touch it, so the JSON file is still scanned for secrets.
• Trust controls. Nothing changes for the trusted-trigger check, lane token scope, the kill switch, the token broker, select-trusted-text, or any merge hold. Nothing widens.
• Settled thread. The _typos.toml thread (resolved by owner reply, lines unchanged) asked for exactly this fix: the exemption now sits on the caller's exclude: line. Nothing to re-raise.
• Prior findings. The one prior post (8789c4de, no line anchor) had no findings, so nothing carries forward.
• PR context. It has no directives aimed at this lane. Its "Follow-up owed after merge" (removing the file from scripts/review-generated-data.txt so a later rewrite is reviewed) is an operational note outside this diff.
• Instruction-surface lens. No standing instruction is deleted or softened.
• Plugin manifests. None changed.
• Cost. None. The PR says the reader will use fewer GITHUB_TOKEN reads in FAIL index mode.