Skip to content

perf(pr-merge): let a wake refuse on the last sweep's unmoved refusing report - #13107

Merged
kyle-sexton merged 4 commits into
mainfrom
perf/wake-skip-unmoved-report
Nov 4, 2026
Merged

kyle-sexton merged 4 commits into
mainfrom
perf/wake-skip-unmoved-report

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: part of the hub's order to cut enable-auto-merge's GITHUB_TOKEN spend. Issues are being driven to zero, so none was filed.

Summary

A PR-scoped pr-merge wake runs the keep-report step: a fresh post-merge check plus settle --idle-tick --fresh, at 64-154 calls. In 6 of 8 sampled wakes that check refused and the wake armed nothing (attribution: .work/gh-monitor/trends/eam-calls/result.md, cut #1).

Fix

  • Skip a refusal that stands (A). Before running the fresh check, a wake tests whether the last sweep's report still refuses, using what that report read:

    • Content: the report refuses on its own content (exit 1, needs-dispatch, unattributed, quarantines or excused). A refusal on time (stale-soon, dispatch-deadline), a hold, reruns, missing pins, or an unreadable lane rerun never stands.
    • Unmoved: nothing it read has moved: age within SHARED_MAX_AGE, the same agent identity, lane-merges-only, and _moved checks on head, config commit and demotion (a tripped demotion probe is read again).
    • No new runs: no verifying run was created, re-run or updated since computedAt, whatever its status (idle_runs_refusal(running=True)). Quarantine issues and pins are unchanged.

    When the refusal stands, the wake refuses as the report did. It prints a ::notice, sets output refusal-stands, and logs report=last outcome=skipped: the last sweep's report still refuses (<refusal>) and nothing it read has moved. The report is removed, so the arm step stays skipped: a wake never arms on a stale report. The sweep is unchanged (the default is running=False).

  • Change B: the arm step gets BABYSIT_POSTMERGE_ATTEMPTS, so the room's recompute reads completed run attempts from the file instead of the API. keep-report gets BABYSIT_RUN_CACHE, the arm step's path, so its processes share perf(source-control): list only open quarantine issues and share fixed-SHA compares #13096's fixed-SHA compares.

Call delta (R34), which needs measuring:

  • A wake whose refusal stands: -64 to -154.
  • A wake whose refusal does not stand: +7 to 10 for the probe (branch, config, head, demotion probe, 2 run listings, quarantine listing, pins).
  • A needs-dispatch refusal usually has a full run created after computedAt, which blocks the skip. The net saving is therefore UNVERIFIED and gets measured after merge (Verification).

Verification

  • Node pr-merge tests: 845 pass. Python babysit-prs tests: 4,103 pass. 8 new Python tests; 7 of 7 deliberate mutants were caught.
  • sync-hold-floor.py --check, check-changelog-fragments.sh --check-required origin/main, and ruff check pass. actionlint shows only the existing queue: findings.
  • After merge (R21 plus a measurement gate): the next two scheduled runs and wakes go green by step outcome. Over the first 24 h, count wakes with refusal-stands=true against wakes that paid the probe and ran the fresh check anyway. Revert if probe-only wakes x 9 > skipped wakes x 64 (net calls rise).

Related

🤖 Generated with Claude Code

kyle-sexton and others added 4 commits November 4, 2026 00:37
…ut a fresh check

A PR-scoped pr-merge wake whose last sweep's report does not stand ran
the post-merge check fresh plus `settle --idle-tick --fresh` (64-154
GITHUB_TOKEN calls), and in 6 of 8 sampled wakes that fresh check refused
and the wake armed nothing.

keep-report now asks `babysit_verify_evidence.py refusal-stands` first,
for a last report from a check that judged (exit 0, 1 or 5). It stands,
and the wake refuses with no fresh check, only when:

- the report refuses on what it found: exit 1 (readable, failing), or the
  idle refusal needs-dispatch, unattributed, quarantines or excused. Not
  stale-soon or dispatch-deadline (time), not a hold (only on an
  unreadable report), not reruns (a red run can leave the _lane_reruns
  lookback with nothing moving), and not a report missing pins or
  outstandingSince or carrying a failed lane re-run read;
- nothing it was read from moved: age within SHARED_MAX_AGE, the same
  identity and lane, main's tip, config commit and demotion (_moved, a
  tripped probe read again), the quarantine issues and pins
  (idle_marks_refusal), and no verifying run created, re-run or updated
  since computedAt whatever its status (_verifying_runs_unchanged with the
  new `running` keyword; its default, which ignores in-progress runs, is
  unchanged for the sweep).

Otherwise the fresh check runs exactly as before. The skip only refuses:
the report is removed and the arm step stays skipped, as on a refused
fresh check, so a wake never arms on a stale report. The notice names the
report (exit code, computedAt) and its refusal; the pr-merge-wake outcome
line reads "the last sweep's report still refuses (<refusal>) and nothing
it read has moved".

The arm step's env now carries BABYSIT_POSTMERGE_ATTEMPTS, pointing at the
attempts.json kept beside the report it reads (a sweep's artifact, the
last sweep's on a kept wake, and a fresh wake's check now copies its file
there), so the room's recompute answers completed attempts with no read.
keep-report also carries BABYSIT_RUN_CACHE at the arm step's path, so its
check and settle can share run-cache answers with the room.

Call delta per wake whose last report does not stand: +5 to +10 calls for
refusal-stands (branch, config, head, demotion probe, two run listings,
quarantine listing, pins), and -64 to -154 when it stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l skip

The README change under docs/conventions/pr-pipeline/ needs a pr-pipeline
fragment; it changes only the wake's GitHub calls, so no release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T06:03:37.492936Z 9c875ca PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 9c875caab75c7344120bd26adf54998ca8f03892: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #13107, no findings

I read every hunk of all 8 changed files. Three of them match trusted paths and got extra scrutiny: .github/workflows/pr-merge.yml, babysit_postmerge.py and babysit_verify_evidence.py. These are merge-hold files, so I checked both the code they change and what that code gates.

Merge hold (keep-report step and the new refusal-stands subcommand): only narrows, never widens.

  • The new elif branch at pr-merge.yml#L2557-L2564 runs only when refusal-stands exits 0. It then deletes the report (rm -rf "${last:?}") and leaves kept=false, which writes stands=false.
  • The arm step's guard at L2612 still skips arming on a PR-scoped wake unless stands == 'true'. So the new path can only make a wake arm nothing. It cannot make a wake arm on a stale report.
  • If refusal-stands exits non-zero, raises, or cannot read its input, the step falls through to the existing fresh-check path unchanged.
  • In _refusal_stands (babysit_verify_evidence.py#L4798), every read error returns unreadable, which exits 1 and leads to a fresh check.
  • The new running=True parameter on _verifying_runs_unchanged and idle_runs_refusal defaults to False. That keeps the sweep's existing behaviour, and on the wake it only adds more reasons to call the report moved.

BABYSIT_POSTMERGE_ATTEMPTS on the arm step: no change in trust.

  • The file at ${{ runner.temp }}/post-merge/attempts.json comes from one of two places. On a sweep, it is this run's own pr-merge-post-merge-report artifact (L2451-L2461). On a wake, it is the last trusted main run's artifact via last-report.mjs (L2468-L2478).
  • That is the same source the post-merge report itself comes from, and the gate's post-merge check already reads the same carried attempts (L263). Pull request authors have no way to write to it.

BABYSIT_RUN_CACHE shared with keep-report: no new trust. Only trusted base-checkout code (.base/) writes the cache file, inside this job. Token scopes are unchanged: keep-report still uses github.token, and the arm step still uses the lane token.

Untrusted text and injection: none.

  • The values the new branch writes to $GITHUB_OUTPUT and the ::notice (refusal, computedAt) come from the trusted report or from a fixed refusal set, and newlines are stripped before the output write.
  • No github context expression is interpolated into a run: block.

Other checks:

  • No trust-control widening: the trusted-actor list, lane token scope, kill switch and broker are untouched.
  • No standing instruction is deleted or softened.
  • No plugin manifest changes.
  • The PR description contains no embedded directives aimed at this lane.

Spend: this only changes GITHUB_TOKEN API calls. It cuts 64–154 calls from a wake whose refusal stands, and adds about 7–10 probe calls to a wake whose refusal does not stand. No billable resource is involved, so there is no cost hold.

critical-findings: 0

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

claude-review reviewed 9c875caab75c7344120bd26adf54998ca8f03892: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13107 (head 9c875ca)

No findings. I read every hunk of all 8 changed files in .claude-lane/pr.diff. I also read the code at head that the new path depends on: the keep-report and arm steps and the outcome line in pr-merge.yml, idle_report_refusal, idle_runs_refusal, _verifying_runs_unchanged and idle_marks_refusal in babysit_postmerge.py, and _moved, _default_branch, _default_branch_config, check_code_unchanged and SHARED_MAX_AGE in babysit_merge_policy.py.

What I checked

  • The new path can only refuse. A PR-scoped wake (a pr-merge run for one PR) now skips its fresh post-merge check when refusal-stands exits 0, deletes the report, and leaves the arm step skipped. A wake whose fresh check refused already ended that way, so the new path cannot arm anything that wasn't armed before.

    • Narrow skip condition. _refusal_stands returns 0 only when reason is None. Every other result runs the fresh check as before: a missing or unreadable report, changed check code, no content refusal, a moved input, an exception it catches, or one it doesn't (the crash exits 1).
    • What a wrong skip costs. At worst a wake misses a chance to arm until the next sweep, which runs every 30 minutes.
  • Which refusals can stand. content_refusal drops the same report shapes that idle_report_refusal would refuse for reasons other than content: reruns requested, pins not recorded, no outstandingSince, a lane re-run read that failed, or readable not true. Reports refused for being old (stale-soon) or past dispatch-deadline, and reports with a hold, fall outside CONTENT_REFUSALS.

  • Wiring between the step and the scripts.

    • refusal_moved passes a runs_unchanged callback in the (answer, default) shape _moved expects, and sets reread_demotion=True.
    • running=True is passed only from this path, so the sweep's own idle_refusal behaves as before.
    • The step only asks refusal-stands about reports that exited 0, 1 or 5. Exits 3 and 75 go straight to the fresh check, and the workflow test covers that.
    • With set -uo pipefail and no -e, standing=$(…) inside the elif still keeps the JSON when the script exits 1, so the step logs why the refusal did not stand.
  • Attempts file (change B). The sweep already keeps attempts.json beside its report (pr-merge.yml L324). The fresh path now copies the attempts file it wrote back into that folder. The arm step's BABYSIT_POSTMERGE_ATTEMPTS points at post-merge/attempts.json, and a missing file there reads as no file (ATTEMPTS_ENV contract). Pointing keep-report at the arm step's BABYSIT_RUN_CACHE keeps the cache's existing MAX_AGE limit.

  • Outcome line. The new REFUSAL_STANDS branch comes before the FRESH_STARTED and FRESH checks. This path writes neither of those, and it reports report=last, as the test asserts.

  • Tests. The new Python RefusalStands tests and the workflow tests cover:

    • content and non-content refusals;
    • moved head, runs, quarantine, pins and demotion;
    • the age and identity bounds;
    • the command's exit codes;
    • the wake's skip branch, with its notice and outcome line.

    I ran nothing. The PR description's 845 Node tests, 4,103 Python tests and mutant results are the author's claims, and I did not verify them.

Other checks

  • Closing issue: none. The PR says no issue was filed, so there are no acceptance criteria to check.
  • Changelog fragments: source-control has bump: patch, for a new internal subcommand plus a behaviour change in pr-merge's wake. pr-pipeline has bump: none. No plugin manifest changed, so there are no manifest lines.
  • Path-scoped rules: ruff-pin.md and cost-claims.md match changed files. Neither is broken: README.md states no price or saving.
  • Paved paths: no paved-path row in docs/standards/.
  • Prior findings: none (0 rounds, not past the cap).
  • Embedded text: none.
  • Spend: a wake whose refusal does not stand makes about 7–10 more GitHub API calls on the job token; one whose refusal stands saves the fresh check's calls. This uses API rate limit only and costs nothing, so there is no hold.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 55f8a62 Nov 4, 2026
47 checks passed
@kyle-sexton
kyle-sexton deleted the perf/wake-skip-unmoved-report branch November 4, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant