pr-pipeline: wake pr-merge per PR on pr-verify completion and make the cron a backstop

Owner: poteto (pr-merge). Measure before building.

Summary

The longest wait on the PR path is between a PR becoming ready and entering the merge queue. Measured over 7 days: p50 0.80 h, p90 7.41 h. Queue entry to merge is p50 0.11 h. Source: audit comment, gate #13.

pr-merge wakes only on three things (.github/workflows/pr-merge.yml:85, :93-98 at cb8429f):

A 15-minute cron. It ran 607 times in 7 days, 173 run-hours, p50 16.1 min per run.
unlabeled. It ran 1,011 times in 7 days, 206 run-hours, mostly waiting on the shared concurrency group. A 2026-11-01 sample showed a run doing 26 s of work after 21 min queued.
push.

Proposal: wake pr-merge for one PR when pr-verify completes on that PR's head, using a per-PR concurrency group. Then slow the cron to a backstop.

The stated reason this was not done

pr-merge.yml:91-95 says the lane "does not wake on each CI or review completion: those arrive faster than runs finish, which would overflow the concurrency queue below." Any design must answer this. Possible answers:

A per-PR group, so one PR's wakes collapse into each other and never queue behind other PRs.
Waking only on pr-verify's completion, not every CI or review completion.

Prior cost cuts on event wakes

pr-merge event wakes have been cut twice on cost. Rework must not bring that cost back.

Baseline 2026-10-31T08:45Z (hub baselines.log). There were 237 pull_request_target runs in 24 h. 138 were superseded and 78 gated at a median of 47 calls each, and none enqueued or merged anything. Dropping edited and ready_for_review removed about 200 runs, 3,591 github-token calls and 2,025 lanes-app calls a day.
perf(pr-merge): stop waking on edited and ready_for_review #11493.

Done when

A baseline is recorded before any change: ready -> first queue entry p50/p90, pr-merge runs per day by event, and github-token and lanes-app calls per run.
The design answers pr-merge.yml:91-95 (concurrency overflow) with a measured or reasoned bound on queued runs.
pr-merge wakes per PR on pr-verify completion, and the schedule becomes a backstop.
Ready -> queue p50 and API calls per day are re-measured against the baseline and posted here. Calls per day must not exceed the 2026-10-31 post-cut level.

Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.

Size: Medium
Complexity: High
Uncertainty: Knowable
Flags: none
Blocked by: none

Understood: wake pr-merge for a single PR when pr-verify completes on its head, using a per-PR concurrency group, and slow the 15-minute cron to a backstop. The aim is to cut the ready-to-queue wait (p50 0.80 h, p90 7.41 h) without raising API calls above the 2026-10-31 post-cut level.

The issue already names the design (per-PR group, pr-verify completion only), so I found nothing open that needs a decision. The measurement items (baseline and re-measure) are for a later step to confirm.

I found no open issue that asks for the same thing. #12284 is the referenced audit.

_{Automated triage, applied by the lanes App.}

Not admitted to the issue lane: a criterion names .github/workflows/pr-merge.yml, which the issue lane cannot push (patch-protected-path); a builder session takes it. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

BEFORE baseline (Done-when 1). Measured 2026-11-02T18:25Z from the hub's runs cache and the jobs' api-calls annotations. Windows: 7d is 2026-10-26T00Z to 11-02T00Z; 24h is 11-01T17:48:58Z to 11-02T17:48:58Z.

Measure	7d	24h
Ready to first queue entry, p50 / p90	0.51 h / 5.88 h (n=60 of 1,531 merged)	0.50 h / 2.01 h (n=30 of 277)
By author class, p50 / p90 (7d)	lanes App 0.57 / 13.87 h (n=16), sessions 0.87 / 2.23 h (n=28), other 0.19 / 0.92 h (n=16)
pr-merge runs per day	pull_request_target 154.7, push 164.0, schedule 86.7	pull_request_target 26, push 184, schedule 94
pr-merge run p50 (pull_request_target / push / schedule)	10.6 / 0.22 / 16.1 min	17.4 / 0.20 / 24.1 min
Calls per run, github-token / lanes App		schedule 162.5 / 186; pull_request_target 3.5 / 0.25 when superseded (about 42.6 when gated); push 1 / 0
Calls per day, github-token / lanes App		about 15,540 / 17,490, schedule about 15,275 / 17,484 (n=12 sampled runs, rough)
pr-verify completions	781 a day not skipped, 943 a day all runs	1,059 not skipped, 1,151 all
pr-verify burst bound		at most 31 in any 15 min repo-wide; at most 5 in 15 min for one PR; per PR p50 2 and p90 6 a day

Method:

Ready is the latest ready_for_review at or before the first added_to_merge_queue, else PR creation. The sample is spaced evenly by PR number.
The audit's 0.80 / 7.41 h covers 10-25 to 10-31 and takes the last ready event, even one after queue entry.

What this means for the design:

Wake rate. A wake on every pr-verify completion would be about 943 runs a day. At about 42.6 calls per gated run, that is far over the calls-per-day ceiling. The wake has to fire only on a pass verdict for a PR pr-merge can arm, and the slower cron has to pay for it.
PR number. workflow_run from pr-verify carries main's head SHA, not the PR's, so the wake has to come from pr-verify with the PR number.
Concurrency. The shared group drains about 35 runs an hour and caps at 100 pending (fix(pr-pipeline): stop pr-merge's workflow_run wake-ups from overflowing its queue #7634). A burst of 31 in 15 minutes would overflow it. A per-PR group sees at most 5.

Design decided (owner, in the poteto session, 2026-11-02 ~21:4xZ). Asked as Q1, which offered three options:

(a) pr-verify dispatches a single-PR, arm-only pr-merge run for a lanes App (melodic-automation-lanes) pull request when its verdict is pass or pass-notes. The cron slows from every 15 minutes to every 30, as a backstop. .github/scripts/pr-merge/who.mjs lets a run started by github-actions[bot] from pr-verify on main go on to mint the lanes App token.
(b) A separate 10-minute cron for enable-auto-merge.
(c) Do not build.

Owner's answer, verbatim: "Go with recommended." The recommended option was (a).

Evidence (BEFORE, posted above, plus):

Scheduled pr-merge runs start a median 2.9 min late, p90 10.7 min, over 1,164 runs. A faster cron keeps that delay.
enable-auto-merge needs only post-merge-dispatch (pr-merge.yml:2181). It reads carried verify-answers, not the merge job's output.

What the build must do, from the design checks:

The dispatch comes from a new pr-verify job holding only actions: write, after the verdict jobs. It sends the PR number as an input, because the workflow_run payload carries main's head SHA, not the PR's.
The PR-scoped run arms that one PR and nothing else. It keeps the shared arm concurrency group, because enable-auto-merge's lane-room check is not re-read after it arms.
post-merge-dispatch and the merge job do not run on it.
It uploads none of the sweep's carried artifacts. trustedRun accepts workflow_dispatch, so those uploads would feed the next sweep.
superseded.mjs does not count it as a superseder.
who.mjs admits only github-actions[bot] on a workflow_dispatch that carries the PR input, on main. Every other non-human run still stops.
AFTER: last-gate-to-queue p50 and p90 for lanes App pull requests, plus GITHUB_TOKEN and lanes App calls per window. The net must not rise above the BEFORE.

Built after #12494 (closes #12448) lands, since both edit pr-verify.yml.

fix(pr-pipeline): reuse the last sweep's report on a wake only under …

No description provided.

AFTER (#12550, window 02:21:57Z-06:14:41Z, 3.88 h)

Metric	BEFORE	AFTER	Basis
Wake runs (pr-merge `workflow_dispatch` from pr-verify, named `pr-merge: arm #N`)	none	18 for 15 PRs, 4.6/h; at most 5 in any 15 min, at most 2 per PR	REST `actions/workflows/pr-merge.yml/runs?event=workflow_dispatch&created>=02:21:57Z`, runs 41566007065..41583975932
Wake outcomes		armed 1 (#12719, run 41583333869); skipped 17: 16 "the last sweep's report does not stand", 1 "the who step stopped it" (run 41579071416; its log shows `check-kill-switch: proceed=false reason=kill-switch`, so the outcome text names the wrong step); 0 "a waiting run does this work"	`pr-merge-wake` notices from GraphQL check-suite annotations of all 18 runs; log of run 41579071416
Why the report did not stand		`"reason": "post-merge answer code-changed"` (run 41573426067), `"reason": "idle tick runs-changed"` (run 41583975932)	`settle --idle-tick` output in 2 of the 16 logs; the other 14 reasons were not read
github-token calls per wake, mean / p50	estimate 45-50	14.1 / 12.5 (11-14 when skipped, 52 when armed, 0 when stopped)	`pr-merge-wake` notices; `limits/gh-job-calls.jsonl` agrees on 17 of 18
lanes-app calls per wake		19 on the armed wake, 0 on every skip	`pr-merge-wake` notices
Wake calls per hour		65.5 github-token, 4.9 lanes-app (254 and 19 over 3.88 h)	sum of the notices
Scheduled pr-merge runs per hour	4	2 (`11,41 * * * *`): 7 runs, 02:43-05:44; 3 of them (03:43, 04:44, 05:44) stopped by the lanes kill switch at 0 calls	`pr-merge.yml:118` on main; REST `event=schedule`; merge-job notice "the lanes kill switch stops this run" on 41573921312, 41579821734, 41585630549
Calls per scheduled run that ran, github-token mean (p50) / lanes-app mean	162.5 / 186 (n=4, 11-01/02, `baselines.log:96`); 277.4 (237.5) / 339.6 (n=16, 21:57Z-01:57Z)	312.8 (242.5) / 287.0 (n=4)	job-total `api-calls` annotations of 24 scheduled runs 41533006484..41585630549, one GraphQL read
Projected github-token calls per day, schedule + wake	bar: 15,275 schedule, 15,540 all events (`baselines.log:96`, the 11-02 post-cut sample); 26,630 schedule on the 21:57Z-01:57Z sample	13,200-16,600 (48 runs at p50 242.5 or mean 312.8, plus 1,571 from wakes)	rows above; pull_request_target and push not re-measured
Projected lanes-app calls per day, schedule + wake	bar: 17,484 schedule, 17,490 all events (`baselines.log:96`); 32,600 on the 21:57Z-01:57Z sample	13,900 (48 x 287.0 + 118)	rows above
Pass verdict -> lane arm (auto-merge enabled by `melodic-automation-lanes`), p50	ready -> first queue entry, lanes App: 0.57 h (n=16, 7d)	50 min = 0.83 h (n=6: 1.8, 6.1, 43.3, 56.6, 56.6, 80.0 min); 5 of 6 armed by a scheduled sweep, 1 by the wake	GraphQL timelines of the 15 woken PRs. Verdict time is the first wake dispatch's `created_at`, a proxy that runs seconds after the pass artifact. The end point, auto-merge enabled, comes before queue entry, so AFTER is a lower bound against the BEFORE
PR created -> lane arm, p50	0.57 h	57 min = 0.95 h (n=6)	same timelines
Pass verdict -> first arm by anyone, p50		20.7 min (n=13; 7 are session `added_to_merge_queue` by kyle-sexton)	same timelines; #12578 excluded (armed before its wake), #12720 not armed (hold)

Three things distort the window:

From 03:28:44Z, every lanes App PR failed merge-group ci-status. This is the L84 regression; its fix, fix(ci): read a lanes App head's pass by exact name in FAIL index mode #12717, is not merged. Queue-to-merge is not reported.
The lanes kill switch stopped 3 of the 7 sweeps and 1 wake. That stretches pass -> arm: fix(discovery): give the research Effort-ceiling sentence headroom in the re-attach slice #12647 waited 80 min, across the 04:44 stopped sweep.
Sessions stopped arming lanes App PRs from about 05:00Z, so docs: replace restated upstream facts with pointers in evals, source-control #12720 is unarmed because of that hold, not because of the lane.

Verdict. The wake fires per PR as designed: 18 dispatches, about 14 github-token calls each, at most 5 in any 15 minutes, and none waited on another run. It armed 1 PR in 18. The other 16 skips came from the standing-report check, which refuses once main or a watched run has moved since the last sweep. The 2 logs read show exactly that. That this holds for most wakes, because merges land every few minutes and the sweep now runs every 30, is inference. The cron still does nearly all lane arming, so pass -> lane arm p50 is 50 min, no better than the 0.57 h BEFORE. Cost: lanes-app is below its bar; github-token, at 13,200-16,600 a day, straddles the 15,275 bar. The two BEFORE samples disagree 1.7x per run (162.5 vs 277.4), so the github-token verdict depends on which one holds.

#12445 cannot close yet.

Done-when 1: met (baseline above).
Done-when 2: met, now measured: at most 5 wakes in any 15 min against the shared group's drain of about 35/h and cap of 100 pending, and 0 waiting-run skips.
Done-when 3: met in mechanism (per-PR wake on a pass verdict, cron at 30 min).
Done-when 4: not met. Ready -> queue p50 has not improved, because the wake arms almost nothing. github-token per day is not established below the bar.

Re-measure once #12717 merges, the kill switch is off, and the wake's standing-report check stops refusing on a moved main.

Next (poteto): when the last sweep's report does not stand, the wake runs the idle-tick and post-merge check itself, fresh, instead of skipping. No report is reused and no check is skipped. Per-wake calls rise toward the original 45-50 estimate, and that rise is stated in the PR. The same change fixes the outcome text on a kill-switch stop (run 41579071416 says "the who step stopped it"). This issue stays open until done-when 4 is measured.

AFTER v2 (#12771, wakes 2026-11-03T08:17:11Z-15:20:22Z)

Source: .work/gh-monitor/wake-after-12771.sh: the executed pr-merge-wake outcome notice of each of the 25 pr-merge: arm #N runs, 41606554309..41662839689.

Outcome	Wakes	github-token calls each
armed	1 (#12786, run 41607075026)	152
refused: review evidence unbound, `rerun-not-allowed`	7 (#12814, #12812, #12795, #12665, #12636, #12604)	63-110
skipped: `settle --idle-tick` refused the fresh report (`idle tick runs-changed`)	5	64-66
skipped: a waiting run does this work	5	2
skipped: the arm step ended skipped with no outcome	3	1
skipped: the lanes kill switch stopped it	2	0
refused: other gate holds (`verify:earlier-fail`, a stopped pr-unblock lane)	2	79-96

Verdict. The wake armed 1 PR in 25, so ready -> queue p50 cannot have improved, and done-when 4 is still not met. The issue stays open.

The largest class (7) was a merge-gate defect. Each review run's attempt 1 had failed failure-class=timeout while the reviews credential was rejected at its usage limit, and that class did not let the successful attempt 2 bind. Fixed by fix(source-control): bind a review re-run after an unposted timeout #12863 (merged 17:16:43Z). Four of those PRs were queued by hand at about 16:02Z, and the other two carry holds.
Two skip classes need follow-up: the fresh idle-tick refusing on runs-changed (5), and an arm step that ends with no outcome (3; the outcome text names no step).
The armed wake cost 152 github-token calls, above the ~95 bound stated in fix(pr-pipeline): run the wake's checks fresh when the last sweep's report does not stand #12771.

Next: re-measure about 10 wakes after #12863, outside the review outage.

Navigation Menu

Repository navigation

pr-pipeline: wake pr-merge per PR on pr-verify completion and make the cron a backstop #12445

Summary

The stated reason this was not done

Prior cost cuts on event wakes

Done when

AFTER (#12550, window 02:21:57Z-06:14:41Z, 3.88 h)

AFTER v2 (#12771, wakes 2026-11-03T08:17:11Z-15:20:22Z)

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

pr-pipeline: wake pr-merge per PR on pr-verify completion and make the cron a backstop #12445

Description

Summary

The stated reason this was not done

Prior cost cuts on event wakes

Done when

Activity

github-actions commented on Nov 2, 2026

melodic-automation-lanes commented on Nov 2, 2026

melodic-automation-lanes commented on Nov 2, 2026

kyle-sexton commented on Nov 2, 2026

kyle-sexton commented on Nov 2, 2026

kyle-sexton commented on Nov 3, 2026

kyle-sexton commented on Nov 3, 2026

AFTER (#12550, window 02:21:57Z-06:14:41Z, 3.88 h)

kyle-sexton commented on Nov 3, 2026

AFTER v2 (#12771, wakes 2026-11-03T08:17:11Z-15:20:22Z)

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions