Cleanup concurrency problems. - #2914
Open
jpascucci-nv wants to merge 3 commits into
Open
jpascucci-nv wants to merge 3 commits into
jpascucci-nv wants to merge 3 commits into
Conversation
The `gh-pages` branch has accumulated thousands of generated-site commits, substantially increasing the size of full repository clones. This PR keeps `gh-pages` at a single commit while preserving the existing documentation, PR previews, and coverage pages. It: - Enables `single-commit` publishing for documentation, PR previews, preview removal, and coverage. - Serializes every automated `gh-pages` writer using the shared `cuda-python-gh-pages-publish` concurrency group. - Separates expensive docs and coverage generation from the serialized deployment jobs, so builds can still run in parallel. - Passes generated output to the deployment jobs through workflow artifacts. - Updates stale-preview cleanup to: - Work from a detached snapshot without modifying a local `gh-pages` branch. - Create a parentless commit containing the updated complete tree. - Push using an explicit `--force-with-lease`. - Permit remote pushes only from the serialized cleanup workflow. - Adds the required targeted `actionlint` suppressions for GitHub's `concurrency.queue` extension. Preview and coverage pages remain intact because every deployment begins with the existing `gh-pages` tree and updates only its assigned target directory. This change intentionally performs a non-fast-forward update on every deployment. The active `Protect gh-pages` ruleset currently blocks these updates. Before exercising this workflow—including PR preview deployment—the authenticated publishing actor must have an `always` bypass for the ruleset, or the force-push restriction must be changed. `git-config-name: cuda-python-bot` is only commit metadata; as written, the authenticated actor is the GitHub Actions integration. The `Restrict deletions` rule is not otherwise a problem: the workflows delete files within commits, not the `gh-pages` branch ref. Granting the GitHub Actions integration a bypass is relatively broad. If that is undesirable, deployment should instead use a dedicated GitHub App with narrowly scoped repository permissions. - Drain any docs, coverage, and cleanup jobs started with the previous workflow before the first compacting deployment. Concurrency groups do not apply retroactively. - After the first deployment, verify that: - `gh-pages` contains one commit. - Latest and versioned documentation still exist. - Existing PR previews still exist. - Coverage pages still exist. - If a deployment-only job fails, use **Re-run all jobs** so its prerequisite job recreates the deployment artifact. - [x] New or existing tests cover these changes. - [x] The documentation is up to date with these changes.
Contributor
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
jpascucci-nv
changed the title
leofang
reviewed
Sep 18, 2026
Member
There was a problem hiding this comment.
btw one other thing: I think we have two workflows that have a race condition in cleaning up docs. I suggested @Andy-Jost that we should let the nightly job (cleanup-pr-reviews) be the sole owner, and don't do cleanup in this workflow.
@Andy-Jost do I recall correctly?
Contributor
Author
There was a problem hiding this comment.
I've removed the place where cleanup-pr-reviews was called during builds of main and release, so it will only cleanup nightly.
Contributor
There was a problem hiding this comment.
Yes, isolating cleanup in a nightly job makes sense to me.
…ve single-commit.
jpascucci-nv
changed the title
jpascucci-nv
marked this pull request as ready for review
September 21, 2026 15:51
Contributor
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2197 - addressed by Leo Fang as not to fix now.
Description
Incorporates the force-free deployment approach from #2883.
The original version of this PR compacted
gh-pageson every deployment. That required non-fast-forward pushes, conflicted with the activeProtect gh-pagesruleset, and introduced additional operational risk around preserving the complete published site.This revision does not rewrite
gh-pageshistory. Instead, it addresses the concurrency problems identified while investigating #2197 and keeps the existing branch-protection rules intact.Non-goal
This PR no longer compacts the existing
gh-pageshistory and therefore doesnot resolve the clone-size problem tracked by #2197. History compaction will
require a separate, ruleset-compatible design.
Changes
Serialize every automated
gh-pageswriter through the sharedcuda-python-gh-pages-publishconcurrency group.queue: max.cancel-in-progress: false.Keep expensive documentation and coverage generation outside the serialized
section.
Set
force: falseon all remaininggithub-pages-deploy-actioninvocations:Let the deploy action fetch, rebase, and retry if
gh-pagesadvances beforeits normal push completes.
Make the scheduled nightly workflow the sole owner of stale PR-preview
deletion.
workflow_dispatchentry point.publishing lock.
Update the cleanup script to create a normal descendant commit and push
without force.
gh-pagesbranch.Add targeted
actionlintsuppressions for GitHub'sconcurrency.queueextension.
New or existing tests cover these changes.
The documentation is up to date with these changes.