Skip to content

ci: refuse new migrations that are not newer than the base branch - #256

Merged
JOY (JOY) merged 3 commits into
mainfrom
ci/migration-order-check
Oct 16, 2026
Merged

JOY (JOY) merged 3 commits into
mainfrom
ci/migration-order-check

Conversation

@JOY

Copy link
Copy Markdown
Contributor

#249 added the "Migration order" step to the required Docs check job. It compares the migrations a PR adds with the live base branch. GitHub does not re-run PR checks when main moves, so a PR that was green when its check ran could still merge behind main (the #242 case). This PR covers the remaining acceptance criterion of #251, re-checking when the base moves.

Design

  • New workflow migration-recheck.yml. It runs on push to main, plus workflow_dispatch for a manual run. It runs tools/ci/migration-recheck.mjs, which:

    1. lists the open PRs into main (one paginated REST call);
    2. fetches their heads as git objects (refs/pull/N/head) and diffs each against its merge base with main, the same as migration-order.mjs. Renames count as added. PRs without migrations pass;
    3. for each PR whose added migration is no longer newer than main's newest, re-runs its latest Docs check workflow run. The re-run replays the old merge commit, but its Migration order step fetches the live main, so the required check fails until the author renames the file.
  • Fallback for when a re-run cannot fail the check. This covers three cases:

    • the run predates the Migration order step, since a re-run replays the original run's workflow file and would pass again;
    • GitHub refuses the re-run;
    • the run is still going after 6 minutes.

    In each of these, the script creates a failing check run named "Docs check" on the PR head (checks: write), titled "Migration order (base moved)" and listing the files. GITHUB_TOKEN creates it as the GitHub Actions app, which is the app the required "Docs check" is bound to. The next push to the PR replaces it with a real run. If the newest "Docs check" check run on the head is already that fallback, later pushes to main do not add another one.

  • Skipped cases: runs that already fail, runs waiting for a maintainer's approval (action_required), PRs with no run yet (their first run reads the live base), and heads that moved during the check (that push gets its own run).

  • Safety:

    • The script is trusted code from main only. There is no pull_request_target, and PR heads are never checked out or executed: only git fetch, merge-base, ls-tree and diff --name-only touch them.
    • All git/gh calls use execFileSync with argument arrays, so there is no shell. PR numbers and SHAs are validated, and log lines cannot start a workflow command.
    • Write scopes are job-level only: actions: write for the re-run and checks: write for the fallback. The top level stays contents: read.
    • Actions are pinned by SHA, and persist-credentials: false.
  • Why not "require branches to be up to date"? That setting is currently off (strict = false). Turning it on would also work, but every PR would have to be updated whenever main moves, and it is an owner setting. This PR changes no branch protection, and the owner does not need to change any required check.

Other changes

  • migration-order.mjs now suggests a concrete rename: the next whole hour above main's newest and above every migration the PR adds, one hour apart for several stale files. The git helpers are shared with the re-check script.
  • AGENTS.md: GitHub does not re-run checks when main moves, so the re-check does. When it fails, rename your migration to a version above main's newest and push.

Verification

Known gap: a run from before #249 that was cancelled before its job started recorded no steps. The script re-runs it, and the re-run replays the old workflow, which passes again. This can happen only to runs from before #249.

After merge (owner or agent)

Two things can only be confirmed on GitHub. Check them on the first push to main after merge:

  • that GITHUB_TOKEN may re-run a pull_request workflow run;
  • that branch protection treats the newest "Docs check" check run (the fallback) as the required result.

The first push will act on the currently stale open PRs, so watch the job summary. A failure of either shows as a red migration-recheck run on main.

Closes #251

🤖 Generated with Claude Code

JOY (JOY) and others added 3 commits October 16, 2026 14:18
GitHub does not re-run pull_request checks when the base branch moves, so a
PR whose new migration was newer than main when its Docs check ran could fall
behind while it waited and still merge green (#242).

- New migration-recheck workflow on push to main (and workflow_dispatch):
  tools/ci/migration-recheck.mjs lists open PRs into main, reads their heads
  as git objects (never checked out or executed), and re-runs the Docs check
  of every PR whose added migrations are no longer newer than main, so the
  required check fails. A re-run replays the original run's workflow file, so
  for runs that predate the Migration order step, refused re-runs, or runs
  still going after six minutes it posts a failing "Migration order (base
  moved)" commit status on the PR head instead.
- migration-order.mjs: suggest a rename to the next whole hour above main's
  newest and above every migration the PR adds; shared git helpers.
- Unit tests for the pure logic; AGENTS.md documents the rename.

Closes #251

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A commit status under its own context is not a required check, so the
fallback for runs that cannot be re-run usefully did not block anything.
Create a failing "Docs check" check run on the PR head instead (checks:
write), the name the branch protection requires. A run whose job recorded
no steps (cancelled before it started) is re-run rather than treated as
predating the Migration order step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip the fallback when the newest "Docs check" check run on the PR head is
already the failing "Migration order (base moved)" one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

@vercel

vercel Bot commented Oct 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
crove Ready Ready Preview Oct 16, 2026 7:20am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 31 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7bfc6731-63c7-4956-9fb0-0c8707dd71e9
📥 Commits

Reviewing files that changed from the base of the PR and between d656b07 and 1c61fbd.

📒 Files selected for processing: 6
  • .github/workflows/docs-check.yml
  • .github/workflows/migration-recheck.yml
  • AGENTS.md
  • tools/ci/migration-order.mjs
  • tools/ci/migration-recheck.mjs
  • tools/test/migration-order.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new migration re-check workflow (tools/ci/migration-recheck.mjs) to automatically re-evaluate open pull requests when the base branch moves, ensuring migration timestamps remain sequential. It also updates migration-order.mjs to suggest precise renames for stale migrations and adds corresponding unit tests. Feedback on the changes highlights a critical issue in tools/ci/migration-recheck.mjs where the gh api call uses bracket notation for nested parameters (such as output[title]), which is unsupported by the GitHub CLI's -f flag and will cause API failures; a fix is suggested to serialize the nested output object as JSON and pass it using the -F flag.

Comment on lines +160 to +161
const fields = [`name=${REQUIRED_CHECK}`, `head_sha=${pr.sha}`, 'status=completed', 'conclusion=failure', `output[title]=${FALLBACK_TITLE}`, `output[summary]=${staleSummary(pr)}`, ...(runUrl ? [`details_url=${runUrl}`] : [])];
const [ok, text] = post([`repos/${repo}/check-runs`, ...fields.flatMap((field) => ['-f', field])], `failing "${REQUIRED_CHECK}" check run`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The gh api command does not support nested object parameters using bracket notation (like output[title]=... and output[summary]=...) when using the -f / --field flag. Passing these will result in a flat payload with literal keys like "output[title]" instead of a nested output object, causing the GitHub API to reject the request or ignore the output.\n\nTo pass nested objects correctly with gh api, you should serialize the nested object as a JSON string and pass it using the -F / --raw-field flag, which parses the value as raw JSON.

      const outputJson = JSON.stringify({ title: FALLBACK_TITLE, summary: staleSummary(pr) });
      const fields = [
        '-f', `name=${REQUIRED_CHECK}`,
        '-f', `head_sha=${pr.sha}`,
        '-f', 'status=completed',
        '-f', 'conclusion=failure',
        '-F', `output=${outputJson}`,
        ...(runUrl ? ['-f', `details_url=${runUrl}`] : [])
      ];
      const [ok, text] = post([`repos/${repo}/check-runs`, ...fields], `failing "${REQUIRED_CHECK}" check run`);

@JOY
JOY (JOY) merged commit 8407b77 into main Oct 16, 2026
14 checks passed

This branch was successfully deployed

1 active deployment
Preview — 1c61fbd0 Deployed Oct 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: refuse a PR whose new migration is not newer than the base branch's newest

1 participant