ci: add post-publish install channel verification workflow#5605
ci: add post-publish install channel verification workflow#5605avallete wants to merge 1 commit into
Conversation
Add a post-publish verify-install-channels reusable workflow that does a real brew install, scoop install, and curl|bash install-script install against the just-published channels, then asserts supabase --version matches. brew, scoop, and the install script each verify the declared sha256/hash against the downloaded tarball, so this reproduces the exact CLI v2.107.0 failure where the brew/scoop manifests shipped checksums that did not match the release tarballs. Wire it into release-shared.yml alongside setup-cli-smoke (non-gating, runs last) and only for beta/stable, which are the channels that publish brew/scoop. Document it in the release-process playbook. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNp9yTyRoDYJTs5xsWRbAr
Supabase CLI previewnpx --yes https://pkg.pr.new/supabase/cli/supabase@5194a53db461416701b6e8af6e9ec90d0dee35a7Preview package for commit |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5194a53db4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # The install script downloads the GitHub Release tarball plus | ||
| # checksums.txt and aborts on a sha256 mismatch (verify_checksum). | ||
| # --no-modify-path keeps it from editing the runner's shell rc files. | ||
| ./install --version "${VERSION#v}" --no-modify-path |
There was a problem hiding this comment.
For beta releases this job runs from develop, but the documented curl|bash install channel is the public script from main (or the uploaded release asset), so this can pass while users running the published install command still fail when main's script differs from the release branch or the release asset is missing/stale. Fetch and execute the actual public install script URL/asset here instead of ./install from the workflow checkout.
Useful? React with 👍 / 👎.
| # while release tags / dispatch inputs may include it, so strip it | ||
| # before comparing. | ||
| expected="${VERSION#v}" | ||
| actual="$(supabase --version | tr -d '\r' | head -n1)" |
There was a problem hiding this comment.
For beta/stable releases the shipped CLI is the legacy shell, and --version is handled by the Bun wrapper without spawning the colocated supabase-go sidecar. A Homebrew/Scoop/install-script package that omits or misplaces supabase-go would pass this check while proxied legacy commands fail for users; the existing release-process doc calls out supabase projects list --help as the required sidecar smoke, so add that (or an equivalent proxied command) before considering the channel verified.
Useful? React with 👍 / 👎.
| jobs: | ||
| homebrew: | ||
| name: Homebrew (${{ inputs.brew_name }}) | ||
| runs-on: macos-latest |
There was a problem hiding this comment.
When the generated Homebrew formula's Linux stanza has a bad URL or sha256, this workflow still passes because the tap is only installed on macOS. update-homebrew.ts emits separate on_linux URLs/checksums for Linux users, so add an Ubuntu/Homebrew leg (or another check that actually exercises the Linux stanza) before treating the tap as verified.
Useful? React with 👍 / 👎.
3 participants
Add automated end-to-end verification that published install channels (Homebrew, Scoop, and curl|bash install script) successfully install the released CLI and serve artifacts with matching checksums.
Summary
This adds a new
verify-install-channels.ymlworkflow that runs realbrew install,scoop install, and install-script installs against the just-published channels, then verifies the installed version matches. The workflow is triggered automatically after successful Homebrew and Scoop publishes, and can also be manually dispatched for debugging install regressions.Key Changes
New workflow:
.github/workflows/verify-install-channels.yml./installon Linux and macOS, verifies versionsupabase --versionmatches the released versionversion,brew_name, andscoop_nameas workflow inputsIntegration into release pipeline: Modified
.github/workflows/release-shared.ymlverify-install-channelsjob that runs after successfulpublish-homebrewandpublish-scoopDocumentation: Updated
apps/cli/docs/release-process.mdverify-install-channelsto the release flowchartImplementation Details
The workflow catches checksum mismatches that would cause real user installs to fail, since brew, scoop, and the install script all verify published checksums against downloaded tarballs before installation. By running actual installs against the live channels immediately after publish, this provides the signal that would have caught the v2.107.0 regression where every
brew install/scoop installfailed with "Formula reports different checksum".Closes: CLI-1642
https://claude.ai/code/session_01RNp9yTyRoDYJTs5xsWRbAr