From dbc351671ebcad5d58d46e3d41284dc96af4393b Mon Sep 17 00:00:00 2001 From: Warren Gifford Date: Fri, 23 Oct 2026 21:51:44 -0700 Subject: [PATCH] release: Run promote finalize once, and make tag creation retry-safe (#497) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v8.1.1335 promotion ran finalize twice in this repo, and the build the release worker monitors lost: | Build | Commit | Started by | Promote finalize | |---|---|---|---| | [#1197](https://buildkite.com/sourcegraph/deploy-sourcegraph-k8s/builds/1197) | `e84e57b` "release: v8.1.1335" (internal config commit) | creating `promote/release-v8.1.1335` | **passed** at 07:09: tagged the promote branch head `6a0aebb`, pushed `release-v8.1.1335`, promoted on the release registry | | [#1201](https://buildkite.com/sourcegraph/deploy-sourcegraph-k8s/builds/1201) | `6a0aebb` "promote-release: v8.1.1335" | push | **failed** 4×: `fatal: tag 'v8.1.1335' already exists` | Creating the promote branch from the internal branch builds the internal config commit, and `--config-from-commit` accepts its config, so that build runs the promote jobs too. Its finalize tags whatever the promote branch head is when it gets there. Build #1201 is the one the release worker monitors, so the promote workflow failed and never closed the release PRs or opened the "Update main" PR. Those were done by hand: #494 and #495 are closed, and #496 is the "Update main" PR. Helm failed the same way (sourcegraph/deploy-sourcegraph-helm#979). Docker avoided it only because the extra build was cancelled by a newer push. This is the same change as sourcegraph/deploy-sourcegraph-helm#979 and sourcegraph/deploy-sourcegraph-docker#1449: - **Promote jobs run only for the promote config commit.** `(promote) Release: test` and `finalize` now also require `build.message =~ /^promote-release:/`. Both the release worker and this repo's `release.yaml` manual flow write that message. - **`git:tag` is retry-safe.** If `vX` already points at the promote branch head, it skips creating the tag and continues. If it points anywhere else, it fails. The release registry's promote endpoint already treats re-promoting as a no-op (200), so a retried finalize now succeeds end to end. ## How this was tested - **Job conditions.** I checked the old and new conditions against this repo's real v8.1.1335 promote-branch commits. Before, every commit ran the promote jobs. After, only `6a0aebb` "promote-release: v8.1.1335" runs them; `e84e57b` "release: v8.1.1335" (build #1197) and the three image-update commits skip them. This PR's Buildkite build validates that the pipeline, with its new conditionals, uploads. - **`git:tag`.** I extracted the step from this repo's `release.yaml`, stubbed `gh` and `buildkite-agent`, and ran it in fresh clones against a scratch bare remote: - **First finalize:** exits 0; the tag and `release-vX` are on the promote head. - **Retry or duplicate:** exits 0 with "already points at … not recreating it". - **Tag on another commit:** exits 1 with a message naming both commits. - **Control:** `main`'s step on the retry case exits 128 with `fatal: tag 'v9.9.9' already exists`. ## Rollout Buildkite reads `pipeline.yaml` and `release.yaml` from the commit it builds, so patch releases need this on their `X.Y.x` branch. It's labelled `backport 8.1.x`; both files are identical on `8.1.x`, so the backport applies cleanly. (cherry picked from commit ff33932fbe142929251968c15a6c06ff189b4fed) --- .buildkite/pipeline.yaml | 7 +++++-- release.yaml | 11 ++++++++++- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/.buildkite/pipeline.yaml b/.buildkite/pipeline.yaml index ed717ef6..655e1fe2 100644 --- a/.buildkite/pipeline.yaml +++ b/.buildkite/pipeline.yaml @@ -33,8 +33,11 @@ steps: command: | sg release run test --workdir=. --config-from-commit + # Promote jobs run only for the promote config commit. Creating the promote + # branch also builds its first commit, the internal config commit, which + # --config-from-commit would otherwise accept, running finalize twice. - label: "(promote) Release: test" - if: build.branch =~ /^promote\/release-.*/ && build.branch !~ /-update-main$/ + if: build.branch =~ /^promote\/release-.*/ && build.branch !~ /-update-main$/ && build.message =~ /^promote-release:/ plugins: - ssh://git@github.com/sourcegraph/sg-buildkite-plugin.git#main: ~ command: | @@ -57,7 +60,7 @@ steps: sg release run internal finalize --workdir=. --config-from-commit - label: "(promote) Release: finalize" - if: build.branch =~ /^promote\/release-.*/ && build.branch !~ /-update-main$/ + if: build.branch =~ /^promote\/release-.*/ && build.branch !~ /-update-main$/ && build.message =~ /^promote-release:/ plugins: - ssh://git@github.com/sourcegraph/sg-buildkite-plugin.git#main: ~ command: | diff --git a/release.yaml b/release.yaml index 8aebbbb8..ded54771 100644 --- a/release.yaml +++ b/release.yaml @@ -349,7 +349,16 @@ promoteToPublic: # Create the final branch holding the tagged commit git checkout "${promote_branch}" git switch -c "${release_branch}" - git tag {{version}} + # A retried finalize finds its own tag; only a tag elsewhere is an error + if tagged=$(git rev-parse -q --verify "refs/tags/{{version}}^{commit}"); then + if [ "${tagged}" != "$(git rev-parse HEAD)" ]; then + echo "Tag {{version}} already exists at ${tagged}, not at ${promote_branch} ($(git rev-parse HEAD))" >&2 + exit 1 + fi + echo "Tag {{version}} already points at ${promote_branch}, not recreating it" + else + git tag {{version}} + fi git push origin ${release_branch} --tags # Web URL to the tag