Repository navigation
ci: skip packages already on NuGet.org instead of re-pushing them (fixes the 403 Quota Exceeded runs) - #476
Merged
Conversation
MPCoreDeveloper
force-pushed
the
chore/publish-skip-existing
branch
from
October 25, 2026 11:04
1319abf to
859fd5e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
MPCoreDeveloper
force-pushed
the
chore/publish-skip-existing
branch
from
October 25, 2026 11:08
859fd5e to
c797777
Compare
… them A master push packs every project and pushes every artifact. `--skip-duplicate` does not avoid the PUT: a duplicate is still a request, and NuGet.org counts it, so a burst of merges exhausts the push quota and the run dies with `403 Quota Exceeded` (retry after 822s) long after the release itself shipped. That is what turned the four dependabot merges after #475 red: 2.1.1 was on the feed, the publish job was not. push_layer now reads id and version out of each nupkg nuspec, asks the flatcontainer index, and only pushes what is missing. Anything already published is skipped without touching the push endpoint, so an unchanged tree costs one GET per package and zero pushes, and the 30s indexing wait only happens for a layer that actually pushed something. The step summary now reports pushed vs skipped and says plainly when a run shipped nothing. `ls *.nupkg | wc -l` never did: it read "Published 28 package(s)" on runs that pushed none. The catch-all `push_layer "Remaining packages" "*.nupkg"` also re-pushed the whole set after the layered passes - 50 already-exists conflicts in a single run - so every release paid for 28 extra requests. It still catches packages the layers do not name; the feed check makes it free. The same skip logic goes into publish-manual.yml, so a re-dispatch after a partial publish costs one request per package instead of a full re-push. Validated: both workflows parse as YAML; the two helpers run against the live feed (published version skipped, unpublished pushed, unknown package id pushed, 404 index pushed, nuspec 2.1.0-RC.3 matching the feed lowercase 2.1.0-rc.3); nuspec extraction returns id/version from a nuspec fragment.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
2.1.1shipped fine: the run started by #475 (40185386962) published all 27 packages between 10:43 and 10:46 UTC. Four dependabot merges landed minutes later, each started its own master run, and each of those re-pushed the full set. NuGet.org answered403 Quota Exceeded(retry after 822s) and four runs went red after the release was already out.--skip-duplicateis not protection here: a duplicate is still a PUT and NuGet.org counts it. The old step also pushed everything twice by design — the layered passes, thenpush_layer "Remaining packages" "*.nupkg"over the same artifacts, which produced 50already existsconflicts inside that single green run.What
push_layernow asks the feed first:<id>/<version>out of the nupkg nuspec (unzip -p … | grep -oP, so a pre-release2.1.0-RC.3comes from the package instead of being parsed out of the file name),GET https://api.nuget.org/v3-flatcontainer/{id}/index.jsonand match the version case-insensitively (the feed lowercases pre-release labels:2.1.0-rc.3),/tmp/seen.txt), so the catch-all layer costs nothing.An unchanged tree therefore costs one GET per package and zero pushes, and the 30s indexing wait only runs for a layer that actually pushed something. The step summary now reports pushed vs skipped and says plainly when a run shipped nothing —
ls *.nupkg | wc -lread "Published 28 package(s)" on runs that pushed none, which is the same false green that hid the original 2.1.0 no-op.publish-manual.ymlgets the same logic, so a re-dispatch after a partial publish costs one request per package instead of a full re-push.Validation
Both files parse as YAML. Helpers run against the live feed:
SharpCoreDB 2.1.1SharpCoreDB 9.9.9SharpCoreDB.Server 2.1.1(no such package, index 404)2.1.0-RC.3vs feed2.1.0-rc.3SharpCoreDB.VectorSearch/2.1.1Dedupe simulation over three fake artifacts and three layers (core, dependents, catch-all): 3 visits, 3 skips, 0 pushes.
End-to-end, on this branch (
gh workflow run publish-manual.yml --ref chore/publish-skip-existing), where every package is already published at 2.1.1: