Skip to content

ci: build each published platform on a native runner - #549

Merged
pmakanju-e2b merged 2 commits into
mainfrom
ci/native-arm64-publish
Nov 6, 2026
Merged

pmakanju-e2b merged 2 commits into
mainfrom
ci/native-arm64-publish

Conversation

@pmakanju-e2b

@pmakanju-e2b pmakanju-e2b commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Publishing a release takes far longer than the build itself needs. The last five Publish runs took 15–29 minutes. In run 42075381880 (v0.5.1), next build took 150s on amd64 and 1449s on arm64, because the arm64 half of the manifest list was built under QEMU on one x64 runner.

Change

.github/workflows/publish.yml is now three jobs:

  1. preflight (ubuntu-latest): the existing checks, unchanged. It checks the credentials are configured, resolves the tag and commit, checks the tag names the commit, and refuses to overwrite an existing tag. It exposes tag and sha to the later jobs.
  2. build: a matrix of linux/amd64 on ubuntu-24.04 and linux/arm64 on ubuntu-24.04-arm, with no QEMU. Both legs check out the exact commit preflight resolved and build with the same flags, build-arg and OCI labels as before. Each leg pushes its image by digest only (untagged) and uploads that digest as an artifact. Native runners also avoid cross-building, which would ship amd64-only native modules (sharp) in the arm64 image.
  3. merge: joins the two digests with docker buildx imagetools create. It first does a --dry-run and checks that the planned manifest list holds exactly linux/amd64 and linux/arm64 with this run's digests. Only then does it write the tag, because tags are immutable and a wrong list could never be replaced. The step summary is the same as before.

A dry run builds both platforms natively, loads them into the runner's local Docker and pushes nothing.

Verification

Dry run from this branch: run 42202046094 passed in 3m17s, using ref=ci/native-arm64-publish, tag=v0.0.0 and dry_run=true. The baseline is the v0.5.1 publish, run 42075381880, which took ~26 min.

Baseline (QEMU, one runner) This PR (native runners)
Whole run ~26 min 3m17s
preflight job – 12s
Build job / step 1569s job (1510s build step, both platforms) amd64 177s job (146s step) · arm64 162s job (141s step)
merge job – skipped (dry run)
next build amd64: "Compiled in" / step DONE 89s / 150.7s 61s / 105.9s
next build arm64: "Compiled in" / step DONE 15.5 min / 1449.1s 62s / 109.4s
bun install amd64 / arm64 10.5s / 16.3s 6.7s / 2.8s

The arm64 next build went from 1449s to 109s. Both legs ran together on ubuntu-24.04 and ubuntu-24.04-arm.

Push path rehearsed locally: I ran the push-and-merge path against a throwaway registry:2 with a small Dockerfile and a separate Buildx builder:

  • Each platform was pushed by digest, and its digest was read from --metadata-file.
  • imagetools create --dry-run with the check from merge matched exactly linux/amd64 <digest> and linux/arm64 <digest>.
  • Passing the amd64 digest twice was rejected.
  • imagetools create --tag and the summary's imagetools inspect both worked.

actionlint 1.7.12 (with shellcheck) reports no errors.

Not yet exercised against Artifact Registry: the real push to Artifact Registry and the merge job first run on the next release. Things to watch on that run:

  • Build legs: GCP auth and setup-gcloud succeed on the arm64 runner, and each leg pushes by digest and uploads digest-amd64 / digest-arm64.
  • merge: the manifest-list check passes, and the step summary shows the index digest.
  • Result: docker buildx imagetools inspect us-docker.pkg.dev/e2b-artifacts/dashboard/dashboard:<tag> lists exactly linux/amd64 and linux/arm64.

The arm64 half of the image was built under QEMU on one x64 runner and
took about 24 of a publish run's 25 minutes. Each platform now builds
natively on its own runner and pushes by digest, and a final job joins
the two into the tagged manifest list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread .github/workflows/publish.yml Resolved
The publish workflow test still required the QEMU action and a single
multi-platform build. It now checks for native per-architecture runners,
push by digest, a manifest list verified before it is tagged, and a dry
run that never reaches the merge job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pmakanju-e2b
pmakanju-e2b merged commit 0de94d9 into main Nov 6, 2026
9 checks passed
@pmakanju-e2b
pmakanju-e2b deleted the ci/native-arm64-publish branch November 6, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants