Skip to content

feat(aws-lambda): attach the arm64 video image to releases and pin it - #6070

Merged
miguel-heygen merged 2 commits into
mainfrom
feat/lambda-arm64-image-release
Oct 23, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
feat/lambda-arm64-image-release

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

Every release attaches the arm64 Lambda video function image as an OCI layout tarball, hyperframes-lambda-video-arm64.oci.tar with its .sha256, and @hyperframes/aws-lambda pins it (URL, sha256, bytes, version) in prebuilt/video-image.json. A new SDK function, prebuiltVideoImage(), reads that pin. Stacked on #6064.

Nothing consumes the pin yet: the CLI option that pushes the image and deploys it comes next. The x86 ZIP deploy is untouched.

Why

A released CLI must be able to deploy the arm64 video function without a repo checkout or a Docker build on the user's machine, the same way it deploys the handler ZIP today. That needs a prebuilt image whose bytes the npm package pins.

How

  • ARM job. publish.yml gains lambda-arm64-ffmpeg on ubuntu-24.04-arm. It has the same release if: and the same immutable checkout as publish, and only contents: read. It builds build(aws-lambda): pinned native ffmpeg build for an arm64 Lambda image #6062's ffmpeg artifact and uploads the tarball as a workflow artifact. publish now needs it, so a failed ARM build stops the release before anything is tagged or published.
  • Cache. The ffmpeg tarball is cached under a key that hashes packages/aws-lambda/image/**.
    • GitHub only lets a run restore caches from its own ref, its PR's base branch, or the default branch. A release run (a merged release PR or a prerelease tag) saves into a scope no later release can read.
    • So lambda-arm64-image.yml now also runs on pushes to main that touch image/**, and copy a saves the same key there. Releases then compile only when the recipe changed and that push's run has not finished yet. A re-run of the same release also hits its own first attempt's save.
    • What a cache hit trusts: the tarball some earlier run wrote under that key on main (or this release's first attempt). On main that is lambda-arm64-image.yml, which built it from the same recipe and ran the codec check. A hit skips the compile and its codec check. stage-arm64-image.ts still refuses ffmpeg or ffprobe binaries that are not arm64 ELF.
  • Image build. In publish, right after the handler ZIP is pinned, the image is staged from that ZIP (dist/hyperframes-lambda-handler.zip) and the ARM tarball with stage-arm64-image.ts.
    • The Dockerfile's copy-only video stage is built for linux/arm64 on the x86 runner, with no emulation.
    • The output is --output type=oci from a docker-container buildx builder, with --provenance=false --sbom=false, so index.json lists exactly one image manifest.
  • Retries. A run whose release already has the image downloads it instead of rebuilding and skips the attach step. This is the same pattern the ZIP uses. A retried run that downloads the existing ZIP still builds a matching image, because the image is built from that same ZIP.
  • One pin helper. pin-handler-zip.ts's hashing is now pinReleaseAsset(repoRoot, asset, manifestFile), which writes the .sha256 sidecar and the manifest. The ZIP keeps its move-and-template step on top of it, and --video-image pins the image. HandlerZipManifest is renamed ReleaseAssetManifest, since both pins share that shape.
  • Package size. The npm package carries only the small JSON pin. The image stays a release asset.
  • publish's timeout goes from 20 to 30 minutes for the base image pull, the staging (Chrome arm64 pack download) and the image upload.

Test plan

  • bun test scripts/pin-handler-zip.test.ts: 7 pass, 0 fail, exit 0, three runs in a row
  • CLI tests that load the adapter SDK (prebuiltHandler, deploy, stackCheck): 11 passed, exit 0, three runs in a row
  • One mutation per new behaviour, each failing only its own tests:
    • prebuiltVideoImage reads the handler manifest: the image pin test and the "no image when only the zip is pinned" test fail
    • the .sha256 sidecar name drops -arm64: the image pin test fails
    • the pin URL names the handler ZIP instead of the asset: the image pin test fails
  • tsc --noEmit for the package, actionlint on both workflows: clean
  • Dry run of the real publish workflow with act, cut before "Create GitHub Release" (no release, no upload, no npm publish)
  • Comments follow CONTRIBUTING.md "Comments"

somanshreddy
somanshreddy previously approved these changes Oct 22, 2026

@somanshreddy somanshreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 2fc197a2 (delta over feat/lambda-arm64-image). No blockers.

Checked

  • Gate wiring: lambda-arm64-ffmpeg uses the same if: and the same EXPECTED_RELEASE_SHA checkout as publish, so needs: can't skip publish on a trigger the ffmpeg job doesn't run on. The job has contents: read only, and an ARM failure stops the release before the tag/release/npm steps.
  • Artifact name: build-ffmpeg.sh writes /out/hf-media-linux-$(uname -m).tar.gz, which is aarch64 on ubuntu-24.04-arm. That matches the upload path and the --ffmpeg= path in publish.
  • No-emulation build: the video stage is FROM ${BASE} plus a single COPY --from=task, with no RUN, so building --platform linux/arm64 on the x86 runner holds. --provenance=false --sbom=false keeps index.json to one manifest.
  • Cache trust: the save happens only when github.ref == 'refs/heads/main' (copy a). PR runs of lambda-arm64-image.yml can't write into main's scope, and release runs (PR-closed or prerelease tag) restore from main. On a hit, the codec check is skipped, but stage-arm64-image.ts still enforces the arm64 ELF check.
  • Retry: an existing release asset is downloaded and re-pinned, and the attach step is skipped. On a fresh build, the image comes from the same dist/hyperframes-lambda-handler.zip the ZIP pin used.
  • Pin refactor: the ${asset.split(".")[0]}.sha256 sidecar gives hyperframes-lambda-handler.sha256, so the ZIP sidecar name is unchanged, and hyperframes-lambda-video-arm64.sha256 for the image. files already ships prebuilt/, so video-image.json goes into the package and the tarball doesn't.

Non-blocking

  • On a push to main, copy a saves the cache in its own job, before anything checks that copies a and b match. The codec check has still run (artifact depends on check), so the only thing a release could inherit is an unreproduced-but-checked tarball. Fine as is; to be stricter, save only after the a == b comparison.
  • The act dry run is still unchecked in the test plan. The first real release is the end-to-end test of the OCI export and upload, so it's worth watching the release job's timing against the new 30-min timeout.

— Somu

@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-image branch from 003729a to b1f62bb Compare October 22, 2026 16:21
@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-image-release branch from 2fc197a to 6053c59 Compare October 22, 2026 16:21
@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-image branch from b1f62bb to 325339f Compare October 23, 2026 01:14
Base automatically changed from feat/lambda-arm64-image to main October 23, 2026 02:59
@miguel-heygen
miguel-heygen dismissed somanshreddy’s stale review October 23, 2026 02:59

The base branch was changed.

The release builds the arm64 video function image from the handler ZIP it
attaches, attaches it as an OCI layout tarball with its sha256, and pins it
in prebuilt/video-image.json, read through prebuiltVideoImage(). A retried
release reuses the image already attached.
@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-image-release branch 2 times, most recently from 0859e96 to fe4b485 Compare October 23, 2026 03:13

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head 2008aeab

  • blocker — CI Build fails at this head: verify:packed-manifests reports @hyperframes/aws-lambda packed 71201 B > budget 70874, unpacked 259303 B > 258166 (scripts/package-size-budgets.json:3-4). Main at the base f6cbf87c passes Build, so the overage comes from this PR's SDK additions. publish.yml runs the same step before tagging, so the release would fail there too. Fix: raise the aws-lambda budget and say why in the PR body, as the check's message asks.
Scope

Code is the same as the dismissed approval at 2fc197a2. I compared the patches after the rebase and only context lines changed. I re-read all 6 files: the job gate and checkout match publish, the cache key and path match across both workflows, the stage-script args and video target are copy-only, the retry reuse path still pins the attached image, .sha256 naming works for both assets, and prebuilt/ is in files. Other workflows at this head are green; Windows render verification was still running when I posted.

…kage

pin-handler-zip.ts is run from a checkout by publish.yml; nothing in dist or the CLI imports it. Leaving it out keeps aws-lambda under its size budget with the video image reader added.
@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-image-release branch from fe4b485 to e409d84 Compare October 23, 2026 03:15

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 1 resolved, 0 open, 0 new · 2008aeab → e409d84f

✅ The Build size-budget blocker is fixed in e409d84f. !scripts/pin-handler-zip.ts keeps the script out of the tarball. Only publish.yml and tests use it, from the repo checkout, and no shipped file imports it. All workflows at this head are green, including CI/Build.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 8d562a3 Oct 23, 2026
67 checks passed
@miguel-heygen
miguel-heygen deleted the feat/lambda-arm64-image-release branch October 23, 2026 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants