Skip to content

fix(agent): check the GitHub token can push before the agent runs (#801) - #1053

Merged
krokoko merged 8 commits into
mainfrom
fix/801-github-token-write-preflight
Nov 20, 2026
Merged

krokoko merged 8 commits into
mainfrom
fix/801-github-token-write-preflight

Conversation

@nizar-lahlali

@nizar-lahlali nizar-lahlali commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

setup_repo now checks, right after the clone, that GitHub lets the token push to the repository, and fails the task in seconds when it does not. Before, a token that could read but not write passed clone and the whole agent run and failed only at the final push, losing the work: in the observed run, 69 turns and $1.90, most of it spent working around the failed push.

Area

  • cdk — infrastructure, handlers, constructs
  • agent — Python runtime / Docker image
  • cli — bgagent client
  • docs — guides or design sources (docs/guides/, docs/design/)
  • tooling — root mise.toml, scripts, CI workflows

CDK: error-classifier patterns for the agent's refusals (cdk/src/handlers/shared/error-classifier.ts) and their tests; no construct, stack or IAM change. The osv-scanner.toml entry of an earlier commit was removed again (see below), so the net diff has no tooling change.

Related

Closes #801. Relates to #802 (same observed run reported success with no PR): this makes that condition rare but does not fix the finalize path.

Changes

The check; a picomatch advisory entry (since removed); two rounds of fixes from /review_pr (six review agents each); two rounds of fixes from krokoko's reviews; and merges of main.

Why the existing checks miss it. The orchestrator's preflight (cdk/src/handlers/shared/preflight.ts) reads permissions.push from GET /repos/{repo}, falling back to GraphQL viewerPermission. Both describe the user's role on the repository, not what the token may do, so a classic PAT with no scopes reports push: true. Such a token also clones a public repository, since reads need no scope.

The check (_preflight_push_capability in agent/src/repo.py, called after the remote and credential helper are configured, before branch setup):

git push --dry-run --no-verify origin +<probe>:refs/heads/bgagent-push-preflight
Choice Reason
A dry-run push, not a scope check GitHub refuses the push service to a token without write access, and a dry run stops before sending a ref update. It uses the remote and credential helper of the final push, so it tests that path for classic, fine-grained and App tokens alike. GitHub returns no X-OAuth-Scopes header for a fine-grained PAT, so a scope check would have to special-case it
<probe> is a new commit of the empty tree (git commit-tree), not HEAD HEAD names no commit in a clone of an empty repository, and pushing it fails before git contacts the remote. Git knows the empty tree without storing it, so making the probe writes one unreferenced commit object and leaves HEAD, the index and the working tree alone
Forced refspec (+) Without it a ref with the probe's name already on the remote would be checked for fast-forward, and the refusal would block every task on that repository. A dry run sends nothing either way
--no-verify A dry run still runs the pre-push hook (from a global core.hooksPath or a git template; a clone copies no hooks)
Tasks that never push skip it Read-only workflows (coding/pr-review-v1); repository-bound workflows that deliver an artifact (Workflow.delivers_repo_artifact); and workflows whose ensure_pr step uses the resolve strategy, which only look up the existing PR (Workflow.ensure_pr_strategy). Both are now the rules the pipeline's post-hooks and setup share. An unloadable workflow is checked, since the post-hooks then fall back to opening a PR
The push URL is pinned to HTTPS A url.<base>.pushInsteadOf rule in the machine's git config rewrites the push URL (to SSH, say) while the fetch URL stays HTTPS, so the check and the final push would use the developer's SSH key. Setup sets an explicit push URL, which pushInsteadOf does not rewrite, reads back the effective push URL, and stops with an error when an insteadOf rule still rewrites it
Inherited github.com credential helpers are cleared Git asks every helper (system, global, local) in turn and takes the first answer, so a saved GitHub login on a developer machine would have answered before gh, and the check (and the final push) would have run as that login. Setup sets an empty local credential.https://github.com.helper before adding gh. That clears every helper read before it for github.com only, so a login for another host in the same checkout keeps working

Outcomes. Failures that look transient (5xx, 429, timeouts, DNS) are never read as refusals. Of the rest, GitHub's archive and organization-policy messages are matched as whole phrases on remote: lines only, so a user or repository name (samlee, acme/archived-notes) cannot pick the cause.

Git result Task error Classification
Exit 0 none; setup continues
Permission to … denied, Write access to repository not granted, 403, 404, Repository not found Token cannot push to <repo>, naming the token type (from its documented prefix) and the fix: public_repo for a public repository or repo for a private one (classic; repo grants every private repository the owner can reach), repository access and Contents Read and write (fine-grained), Contents write and installation (App), both otherwise; for a personal token, that its owner needs write access. A 404 counts because the clone has just fetched the repository "Insufficient GitHub token scopes", with the agent's token-specific fix as the remedy
401, Authentication failed, Invalid username or token, no credential BLOCKED[auth_failure] (resource: <repo>): replace the token. An invalid token fails earlier (orchestrator preflight and gh repo clone both send it), so this is a token revoked after the clone, or none supplied "Blocked: authentication rejected"
This repository was archived so it is read-only Repository <repo> is archived on GitHub, so it is read-only new: "Repository is archived"
… enabled or enforced SAML SSO, … IP allow list … GitHub organization policy blocks pushes to <repo>, naming the policy and its fix new: "GitHub organization policy blocks the token"
No reply within 120 s BLOCKED[dependency_unreachable], retryable, saying one attempt timed out (run_cmd_with_backoff does not retry a timeout) "Blocked: dependency unreachable"
Transient after retries / egress denial / other The existing setup classification: retryable dependency_unreachable, egress_denied, or the redacted git error

All non-retryable. Each refusal emits an agent_blocked event whose kind and detail follow its cause (auth_failure, or unknown_environmental for an archived repository, which no token change fixes). The error quotes the stderr line that matched, redacted with redact_secrets before it is shortened (so is every excerpt of git's output: a cut could remove the prefix the redactor matches), with the login of denied to <user> replaced by <token user>; the token never enters the message, the argv or .git/config. The remedies say "the repository" rather than repeat owner/repo, so the longest message (a 140-character owner/repo, a token of no known type, a full excerpt) is 992 characters and fits the 1,000 the task record keeps; the sentence the classifier matches is never cut. _fail_setup_command is annotated NoReturn, so the check cannot fall through to a passing setup. git's The requested URL returned error: 429 now counts as transient (agent/src/shell.py), which the clone benefits from too.

The classifier carries the agent's fix into error_classification.remedy (the API builds it in toTaskDetail; the web console's failure banner shows description and remedy), so the console gives the fix for the token type, not the generic fine-grained-PAT remedy. The orchestrator's preflight uses the same Token cannot push to phrase, but its message also carries the INSUFFICIENT_GITHUB_REPO_PERMISSIONS code, which classifies first as "Insufficient GitHub permissions" and keeps its remedy.

Docs: agent/README.md (the check, a table of its outcomes, the credential reset, and public_repo vs repo for a classic PAT), docs/design/SECURITY.md (a row in the policy table, after hydration), docs/guides/USER_GUIDE.md (how to tell this failure from preflight_failed: the PushCapabilityError: prefix), mirror regenerated.

picomatch (GHSA-fc5q-39xc-vjpf). An earlier commit accepted this advisory in osv-scanner.toml while it had no fixed release. GitHub withdrew it on 2026-11-18, and security:deps passes without the entry, so the entry is removed and osv-scanner.toml is unchanged from main.

Testing

On the committed tree (macOS, arm64):

  • mise run build: passes (agent 3,114 tests, cdk 7,899, cli 1,472, web 237) and rewrites no file.
  • security:sast:masking, security:sast:range (0 findings) and security:secrets: pass.
  • agent/tests/test_repo.py:
    • TestPushCapabilityPreflight: the push URL pinned to HTTPS before the check, a rewritten push URL stopping setup, a writeable resolve-strategy workflow skipped; argv, forced probe refspec and a real token absent from it; timeout value; ordering against the credential helper, branch setup and the PR-iteration and stacked-child fetches; read-only skip (no probe made) and its tie to the shipped workflows; a repository-bound artifact workflow is skipped, default/agent-v1 with a repository and an unloadable workflow are checked; the credential-helper reset runs before gh is added; passes for fine-grained and classic tokens; each refusal shape and its message and live event (403, 404, fine-grained wording, 401, archived, SAML, IP allow list); names containing saml or archived in the user or repository, on refusals and on a 502; the remedy per token type; the quoted line and the hidden login; redaction; timeout, 429, transient, egress and other failures; a failed probe commit.
    • TestPushPreflightRetries (2), through the real retry loop: a refusal takes one attempt and no sleep, a 502 three attempts.
    • TestPushPreflightAgainstRealGit (real git, local bare remote): a global pushInsteadOf leaves the push URL on HTTPS and a global insteadOf makes setup stop; the github.com helper reset leaves a second host's login working; with a saved-login helper in the global config (generic and URL-scoped) and a stand-in gh, only gh answers for github.com (without the reset the saved login answers); leaves the remote's refs, HEAD, the working tree and .git/config unchanged; reaches the remote (fails when it is missing); passes on an empty repository; passes when the remote has a divergent ref with the probe's name; does not run a pre-push hook that a plain dry run does run.
    • New tests fail against the earlier commits with the defects each review predicted (empty repository, existing ref, then names picking the cause, events, timeout wording).
  • agent/tests/test_shell.py: git's 429 is transient.
  • TestPushPreflightRedaction: a 519-character App token through the timeout warning stays redacted (fails against 8e20a76). TestPushRefusalFitsTheTaskRecord: the longest refusal is 992 characters.
  • error-classifier.test.ts: the longest refusal, cut at 1,000 as stored, still gets the agent's fix (the message at 8e20a76 was 1,373 characters and lost it); the persisted messages, printed from the agent code, classify as in the table, all non-retryable; for classic, fine-grained, App and unknown tokens and a repository with a dot in its name, the remedy is the agent's fix; the orchestrator's message keeps its own; toTaskDetail gives the console the App token's fix.
  • By hand: git push --dry-run to a public GitHub repository the account may not write was refused with 403 Permission to … denied, while git ls-remote on it succeeded.

Not verified: a task on a deployed stack; the dry run against GitHub with a real scope-less classic PAT, a fine-grained PAT, an App token, an archived repository or an SSO or IP-allow-list organization (those stderr shapes follow GitHub's documented messages, not live runs).

Follow-ups worth filing: make the orchestrator's preflight test the token rather than the user's role, which would catch this before compute starts; count a timeout as a transient attempt in run_cmd_with_backoff, which the clone needs too.

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

A token can authenticate and read a repository while GitHub refuses
every write: a classic PAT with no scopes still reads public
repositories. Such a token passed clone and the whole agent run and
failed only at the final push, so the work was lost (69 turns, $1.90 in
the observed run). The orchestrator's preflight does not catch it
because the REST API's permissions.push describes the user's role on the
repository, not what the token may do.

setup_repo now runs `git push --dry-run --no-verify` to a ref that does
not exist, right after the clone and credential-helper setup. The dry run
asks GitHub for the push service through the same credential helper as
the final push, so it works for classic, fine-grained and App tokens
without parsing scopes, and sends no ref update. --no-verify keeps a
pre-push hook from the cloned repository from running.

A refusal fails the task in seconds with PushCapabilityError, whose
message starts "Token cannot push to <repo>" (the phrase the CDK error
classifier maps to "Insufficient GitHub token scopes", non-retryable)
and names the token type and the permission to add. Transient failures
and egress blocks go through the existing setup classification.
Read-only workflows (pr-review) never push and skip the check.
bgagent added 3 commits November 18, 2026 08:52
GHSA-fc5q-39xc-vjpf (updated 2026-11-17) covers every picomatch release
through 4.0.7, the latest, so no version fixes it and security:deps
fails on every branch. The locked copies (2.3.2, 4.0.5, 4.0.7) serve
build, test and lint tooling only; no runtime dependency of cdk, cli or
web reaches picomatch, so no Lambda bundle, the CLI or the console ships
it, and the tooling matches only patterns from this repository's own
configuration. #1052 accepts the same advisory as part of its dependency
upgrade; whichever merges second keeps one entry.
Review of #1053 found cases where the first version failed a task that
worked before, or named the wrong fix:

- An empty repository: HEAD names no commit there, so pushing it failed
  before git contacted the remote. The probe is now a commit of the
  empty tree (git commit-tree), which every clone has; it writes one
  unreferenced object and leaves HEAD, the index and the tree alone.
- A ref with the probe's name already on the remote made the dry run a
  non-fast-forward refusal and blocked every task on that repository.
  The refspec is now forced; a dry run sends nothing either way.
- A 120 s timeout escaped as TimeoutExpired, classified "Build/tests
  didn't finish in time". It is now a retryable dependency_unreachable.
- A rejected credential (401) got the scope remedy and a message saying
  the token can read the repository. It is now BLOCKED[auth_failure]:
  replace the token. An archived repository and SAML SSO or IP allow
  list refusals get their own message instead of the scope remedy.
- The error quotes the stderr line that matched, not a warning git
  prints first, and each refusal emits an agent_blocked event.
- _fail_setup_command is annotated NoReturn.

Comments that misdescribed the classifier, the final push and the
pre-push hook are corrected. Tests cover each case, four of them with
real git (empty repository, existing ref, pre-push hook, nothing
changed). SECURITY.md and the user guide name the check.
… names (#801)

The second /review_pr found that the archived and SAML checks searched
all of git's stderr, which carries the user and repository names: a
scope-less token refused for user `samlee`, or on `acme/archived-notes`,
got the SSO or archive message, and a 502 on such a repository became a
permanent refusal. Transient failures now never reach the refusal
checks, and GitHub's archive, SAML SSO and IP allow list messages are
matched as whole phrases on remote: lines only.

- The archived and organization-policy refusals no longer carry the
  "Token cannot push to" phrase, which the classifier titles
  "Insufficient GitHub token scopes" with a scope remedy. Two classifier
  patterns give them their own titles and remedies.
- PushCapabilityError carries its cause, and the agent_blocked event
  follows it: an archived repository is not reported as the token's
  fault.
- A timeout says one attempt timed out instead of claiming retries, and
  logs what git printed before it.
- git's "returned error: 429" counts as transient (the clone too).
- The quoted line hides the login of "denied to <user>"; "could not read
  username" and "Invalid username or token" count as a rejected
  credential; the fine-grained remedy names the token's repository
  access.
- Comments corrected: an invalid token fails before the check, the probe
  commit is made, not found, in each clone, and the hook and 404
  rationales. The user guide's way to tell this failure from
  preflight_failed works, the SECURITY.md row follows the pipeline
  order, and the picomatch reason names the docs dependency and the
  agent image's own copy.

Tests cover each case, through the real retry loop and real git as well
(the dry run reaches the remote; the hook test pins core.hooksPath).
@krokoko

krokoko commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

I’d request changes before merging. Checking immediately after clone makes sense, and the 114 repository and shell tests passed for me on 4919245. I found three things to fix.

  1. Make sure the dry run checks the task’s token. Setup adds gh to Git’s credential helpers. On a developer machine, Git can try a saved GitHub login first. If that login can push, the check passes even when the task’s token can only read. Please clear inherited helpers for github.com in this clone before adding gh, and add a test with another helper already configured. Repository setup · How Git chooses a helper

  2. Skip the check for workflows that only deliver a file. The pipeline supports workflows that clone a repository for context, write a document locally, and send it to S3. With read_only: false, setup now stops those tasks unless the token can push. Please use the same delivery rule that already skips the PR step. The existing artifact test mocks setup_repo, so it misses this case. Delivery rule · Existing test

  3. Show the permission the operator actually needs. For the public repository in fix(agent): preflight GitHub token write capability in setup_repo — a scope-less PAT fails only at push, after all work is done #801, a classic token with public_repo can push if its owner has write access. The suggested repo scope also grants access to private repositories. Please explain both cases and carry the token-specific advice through to the web console. Its failure banner currently shows fine-grained PAT instructions even for a GitHub App token. GitHub scope reference · Failure banner

GitHub also withdrew the picomatch advisory after this PR opened. I ran mise run security:deps with the same lockfiles and without the new exception, and it passed. The osv-scanner.toml change can come out. Withdrawn advisory

@krokoko krokoko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment above

… scope (#801)

From krokoko's review of #1053:

- Setup added gh to git's credential helpers, but git asks every helper
  (system, global, local) in turn and takes the first answer: a saved
  GitHub login on a developer machine answered first, so the push check,
  and the final push, ran as that login. An empty local
  credential.helper now clears every inherited helper, URL-scoped ones
  included, before gh is added.
- A repository-bound workflow that delivers an artifact clones for
  context and never pushes, but was checked. The rule the post-hooks
  use is now Workflow.delivers_repo_artifact, and setup skips the check
  by it; an unloadable workflow is still checked, as the post-hooks
  then open a PR.
- A classic PAT needs `public_repo` for a public repository; `repo`,
  which grants every private repository its owner can reach, only for a
  private one, and either way the owner needs write access. The
  classifier carries the agent's token-specific fix into the remedy, so
  the console no longer shows fine-grained PAT advice for an App token.
- GitHub withdrew GHSA-fc5q-39xc-vjpf, and security:deps passes without
  the picomatch entry: osv-scanner.toml is back to main's.
@nizar-lahlali

Copy link
Copy Markdown
Contributor Author

@krokoko thanks for the review. All four points are addressed in 2ac5526.

  1. Make sure the dry run checks the task's token.

Reproduced: with a saved-login helper in the global git config, git credential fill in the clone answered with that login rather than gh. Setup now sets an empty credential.helper in the clone's local config before adding gh (_configure_git_credentials in agent/src/repo.py). That clears every helper read before it, URL-scoped ones included, so the check and the final push both use the task's token. TestPushPreflightAgainstRealGit::test_only_the_tasks_token_answers_for_github sets up a saved-login helper (generic and https://github.com-scoped) and a stand-in gh, and asserts only gh answers. Without the reset, the saved login answers.

  1. Skip the check for workflows that only deliver a file.

The delivery rule is now one property, Workflow.delivers_repo_artifact (agent/src/workflow/models.py). The pipeline uses it to skip the PR step, and setup uses it to skip the check (_task_pushes). New tests call setup_repo itself, without mocking it:

  • a repository-bound artifact workflow (built from coding/new-task-v1 the same way the pipeline test builds it) is not checked;
  • default/agent-v1 run with a repository is checked, since it still opens a PR;
  • a workflow that fails to load is checked, since the post-hooks then fall back to opening a PR.
  1. Show the permission the operator actually needs.

For a classic PAT, the refusal now says public_repo if the repository is public, or repo if it is private, and notes that repo grants every private repository the owner can reach. For classic and fine-grained PATs it adds that the token's owner needs write access. The agent's README scopes table says the same.

For the console: the classifier now passes the agent's token-specific fix through as error_classification.remedy, instead of the generic remedy the Token cannot push to pattern shares with the orchestrator's preflight. The banner shows description and remedy, so it needed no change. error-classifier.test.ts checks the remedy for each case, using the persisted messages printed from the agent code:

  • classic, fine-grained, App and unknown tokens;
  • a repository with a dot in its name.

A toTaskDetail test checks that an App token gets the App fix and no fine-grained or PAT-scope advice. The orchestrator's own message keeps its classification.

picomatch

Confirmed the withdrawal, and security:deps passes without the entry, so osv-scanner.toml is back to main's.

Locally on 2ac5526: mise run build passes and rewrites no file (agent 2,095 tests, cdk 6,495, cli 1,258, web 235); security:deps, security:sast:masking and security:secrets pass. CI is green.

Not verified: none of this has run against GitHub with real classic, fine-grained or App tokens.

bgagent and others added 2 commits November 19, 2026 15:10
Conflicts:
- agent/src/pipeline.py: main (#1052) made the repo-bound artifact rule
  use resolved_requires_repo; this branch moved the rule into
  Workflow.delivers_repo_artifact, which the push preflight shares. Kept
  the shared property and gave it main's semantics.
- cdk/src/handlers/shared/error-classifier.ts: kept main's PR_FROM_FORK
  pattern and this branch's organization-policy and archived patterns.
- agent/tests/test_repo.py: kept both new imports.
@krokoko

krokoko commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Thanks for the fixes, Nizar. I checked 8e20a76. Tasks that only deliver a file now skip the push check, and the console gives the right advice for classic PATs, fine-grained PATs, and GitHub App tokens in the cases I tested. The 257 agent tests and 164 classifier tests I ran all passed. I’d still fix three things before merging.

  1. On a machine with a Git pushInsteadOf rule, the clone showed an HTTPS fetch URL but git remote get-url --push origin showed SSH. The dry run and final push can then use the developer’s SSH key even if the task token is read-only. Please make sure both use the intended HTTPS push URL. Please also scope the helper reset to github.com. In my test it disabled a working login for a second host in the same checkout. Credential setup
  2. Please redact Git’s error output before shortening it. I tested the timeout path with a fake 519-character App token. The warning printed the last 500 characters because the cut removed the prefix the redactor looks for. The 200-character excerpt in the refusal message uses the same order. Timeout warning · Refusal message
  3. A custom workflow can set read_only: false and ensure_pr.strategy: resolve. That passes validation. Its final step only looks up the existing PR, but setup now requires a token that can push. Could setup use the same PR strategy as the final step here? Setup decision · PR lookup

I also found a smaller console issue. With a long owner/repo and an unrecognized token prefix, the 1,000-character limit cuts off the sentence the classifier looks for. The banner then tells the operator to grant PR and Issues permissions for a push failure. A test that passes the stored error through the classifier would catch this. Storage limit · Classifier

…PR strategy (#801)

From krokoko's second review of #1053:

- A url.<base>.pushInsteadOf rule rewrote the push URL (to SSH) while the
  fetch URL stayed HTTPS, so the push preflight and the final push could
  use the developer's SSH key, not the task's token. Setup now sets an
  explicit HTTPS push URL, which pushInsteadOf does not rewrite, reads
  back the effective push URL and stops when an insteadOf rule still
  rewrites it.
- The credential-helper reset cleared helpers for every host. It is now
  credential.https://github.com.helper, so a login for another host in
  the same checkout keeps working.
- Git's output is redacted before it is shortened (the timeout warning,
  the refusal excerpt, the setup failure snippet): keeping the last 500
  characters of a 519-character App token removed the prefix the
  redactor matches and left the rest in the warning.
- A writeable workflow whose ensure_pr step uses the resolve strategy
  only looks up its PR. Workflow.ensure_pr_strategy is now the rule both
  the post-hooks and setup read, and setup skips the check for resolve.
- The task record keeps 1,000 characters of the error. With a long
  owner/repo and a token of no known type the message was 1,373, so the
  sentence the classifier matches was cut and the console showed the
  generic PR and Issues remedy. The remedies now say "the repository"
  instead of repeating its name: the longest message is 992 characters.

Tests: pushInsteadOf and insteadOf with real git, the scoped reset
against a second host, the 519-character token through the timeout
path, a resolve-strategy workflow, and the longest refusal both measured
in Python and classified, cut at 1,000, in the CDK suite. The new tests
fail against 8e20a76 where they cover a defect.
@krokoko
krokoko enabled auto-merge November 20, 2026 17:52
@krokoko
krokoko added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit 0392b94 Nov 20, 2026
9 of 10 checks passed
@krokoko
krokoko deleted the fix/801-github-token-write-preflight branch November 20, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(agent): preflight GitHub token write capability in setup_repo — a scope-less PAT fails only at push, after all work is done

2 participants