Skip to content

[Backport 8.1.x] release: Run promote finalize once, and make tag creation retry-safe - #498

Open
DaedalusG wants to merge 1 commit into
8.1.xfrom
backport-497-to-8.1.x
Open

DaedalusG wants to merge 1 commit into
8.1.xfrom
backport-497-to-8.1.x

Conversation

@DaedalusG

Copy link
Copy Markdown
Contributor

Backport ff33932 from #497

The v8.1.1335 promotion ran finalize twice in this repo, and the build the release worker monitors lost:

Build Commit Started by Promote finalize
#1197 e84e57b "release: v8.1.1335" (internal config commit) creating promote/release-v8.1.1335 passed at 07:09: tagged the promote branch head 6a0aebb, pushed release-v8.1.1335, promoted on the release registry
#1201 6a0aebb "promote-release: v8.1.1335" push failed 4×: fatal: tag 'v8.1.1335' already exists

Creating the promote branch from the internal branch builds the internal config commit, and --config-from-commit accepts its config, so that build runs the promote jobs too. Its finalize tags whatever the promote branch head is when it gets there. Build #1201 is the one the release worker monitors, so the promote workflow failed and never closed the release PRs or opened the "Update main" PR. Those were done by hand: #494 and #495 are closed, and #496 is the "Update main" PR. Helm failed the same way (sourcegraph/deploy-sourcegraph-helm#979). Docker avoided it only because the extra build was cancelled by a newer push.

This is the same change as sourcegraph/deploy-sourcegraph-helm#979 and sourcegraph/deploy-sourcegraph-docker#1449:

  • Promote jobs run only for the promote config commit. (promote) Release: test and finalize now also require build.message =~ /^promote-release:/. Both the release worker and this repo's release.yaml manual flow write that message.
  • git:tag is retry-safe. If vX already points at the promote branch head, it skips creating the tag and continues. If it points anywhere else, it fails. The release registry's promote endpoint already treats re-promoting as a no-op (200), so a retried finalize now succeeds end to end.

How this was tested

  • Job conditions. I checked the old and new conditions against this repo's real v8.1.1335 promote-branch commits. Before, every commit ran the promote jobs. After, only 6a0aebb "promote-release: v8.1.1335" runs them; e84e57b "release: v8.1.1335" (build #1197) and the three image-update commits skip them. This PR's Buildkite build validates that the pipeline, with its new conditionals, uploads.
  • git:tag. I extracted the step from this repo's release.yaml, stubbed gh and buildkite-agent, and ran it in fresh clones against a scratch bare remote:
    • First finalize: exits 0; the tag and release-vX are on the promote head.
    • Retry or duplicate: exits 0 with "already points at … not recreating it".
    • Tag on another commit: exits 1 with a message naming both commits.
    • Control: main's step on the retry case exits 128 with fatal: tag 'v9.9.9' already exists.

Rollout

Buildkite reads pipeline.yaml and release.yaml from the commit it builds, so patch releases need this on their X.Y.x branch. It's labelled backport 8.1.x; both files are identical on 8.1.x, so the backport applies cleanly.

…497)

The v8.1.1335 promotion ran finalize twice in this repo, and the build
the release worker monitors lost:

| Build | Commit | Started by | Promote finalize |
|---|---|---|---|
|
[#1197](https://buildkite.com/sourcegraph/deploy-sourcegraph-k8s/builds/1197)
| `e84e57b` "release: v8.1.1335" (internal config commit) | creating
`promote/release-v8.1.1335` | **passed** at 07:09: tagged the promote
branch head `6a0aebb`, pushed `release-v8.1.1335`, promoted on the
release registry |
|
[#1201](https://buildkite.com/sourcegraph/deploy-sourcegraph-k8s/builds/1201)
| `6a0aebb` "promote-release: v8.1.1335" | push | **failed** 4×: `fatal:
tag 'v8.1.1335' already exists` |

Creating the promote branch from the internal branch builds the internal
config commit, and `--config-from-commit` accepts its config, so that
build runs the promote jobs too. Its finalize tags whatever the promote
branch head is when it gets there. Build #1201 is the one the release
worker monitors, so the promote workflow failed and never closed the
release PRs or opened the "Update main" PR. Those were done by hand:
#494 and #495 are closed, and #496 is the "Update main" PR. Helm failed
the same way (sourcegraph/deploy-sourcegraph-helm#979). Docker avoided
it only because the extra build was cancelled by a newer push.

This is the same change as sourcegraph/deploy-sourcegraph-helm#979 and
sourcegraph/deploy-sourcegraph-docker#1449:

- **Promote jobs run only for the promote config commit.** `(promote)
Release: test` and `finalize` now also require `build.message =~
/^promote-release:/`. Both the release worker and this repo's
`release.yaml` manual flow write that message.
- **`git:tag` is retry-safe.** If `vX` already points at the promote
branch head, it skips creating the tag and continues. If it points
anywhere else, it fails. The release registry's promote endpoint already
treats re-promoting as a no-op (200), so a retried finalize now succeeds
end to end.

## How this was tested

- **Job conditions.** I checked the old and new conditions against this
repo's real v8.1.1335 promote-branch commits. Before, every commit ran
the promote jobs. After, only `6a0aebb` "promote-release: v8.1.1335"
runs them; `e84e57b` "release: v8.1.1335" (build #1197) and the three
image-update commits skip them. This PR's Buildkite build validates that
the pipeline, with its new conditionals, uploads.
- **`git:tag`.** I extracted the step from this repo's `release.yaml`,
stubbed `gh` and `buildkite-agent`, and ran it in fresh clones against a
scratch bare remote:
- **First finalize:** exits 0; the tag and `release-vX` are on the
promote head.
- **Retry or duplicate:** exits 0 with "already points at … not
recreating it".
- **Tag on another commit:** exits 1 with a message naming both commits.
- **Control:** `main`'s step on the retry case exits 128 with `fatal:
tag 'v9.9.9' already exists`.

## Rollout

Buildkite reads `pipeline.yaml` and `release.yaml` from the commit it
builds, so patch releases need this on their `X.Y.x` branch. It's
labelled `backport 8.1.x`; both files are identical on `8.1.x`, so the
backport applies cleanly.

(cherry picked from commit ff33932)
@github-actions

Copy link
Copy Markdown

@DaedalusG
DaedalusG enabled auto-merge (squash) October 24, 2026 19:04
@DaedalusG
DaedalusG requested a review from a team October 24, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant