Skip to content

feat(aws-lambda): run chunks on an optional arm64 image function - #6065

Merged
miguel-heygen merged 6 commits into
mainfrom
feat/lambda-arm64-video-function
Oct 23, 2026
Merged

miguel-heygen merged 6 commits into
mainfrom
feat/lambda-arm64-video-function

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

The SAM template (examples/aws-lambda/template.yaml, also the template hyperframes lambda deploy uses) gets an optional VideoImageUri parameter. When it is set, an arm64 container-image function, <project>-video, runs the plan and renders the chunks. The existing x86 ZIP function keeps audio and assemble.

Empty, the default, deploys exactly as before: one x86 function does everything. Nothing passes the parameter yet; the CLI option comes next (#6073).

Why

Chunk rendering is the bulk of a render's compute and runs well on arm64. The plan must run there too: a chunk refuses a plan recorded with a different ffmpeg build (renderChunk.ts), so plan and chunks share one function. Audio and assemble stay on x86, so an MP4's audio, which the separate audio task encodes, is byte-identical to an x86-only deploy.

How

  • The video function lives in examples/aws-lambda/video-function.yaml, a nested stack (VideoFunctionStack) created only when VideoImageUri is set.
    • It is nested because sam deploy refuses a template with any top-level image function unless it is given an image repository, even when that function's Condition is false. Inline, it broke every deploy.
    • The child is plain CloudFormation, so callers need no extra capability: an AWS::Lambda::Function (PackageType: Image, arm64) and its role.
    • It gets the render function's memory, timeout, ephemeral storage, tracing, tags, environment and S3 access. It also gets lambda:GetAccountSettings, which the plan calls to size the render.
  • Both plan tasks and both RenderChunk tasks (plan v1 and v2) invoke !If [HasVideoFunction, <video function>, RenderFunction]. Every other task is unchanged, and the state machine role may invoke both functions.
  • Runtime and Architectures move from Globals onto the ZIP function.
  • Reserved concurrency follows the chunks: with a video function, the reservation sits on it and the render function is unreserved. The plan still reads the value from its own environment to size the render.
  • The chunk-invocation alarm watches whichever function renders the chunks. The video function gets its own errors alarm and a VideoFunctionArn output.
  • The release writes video-function.yaml into the package beside template.yaml, and lambda deploy without a checkout stages both.
  • New workflow, Lambda SAM template: examples/aws-lambda/scripts/check-sam-deploy.sh runs sam validate --lint on both templates. It then runs the CLI's sam deploy argv with fake credentials, with and without a video image, and requires each run to get past validation and stop at the first upload.

Test plan

  • bash examples/aws-lambda/scripts/check-sam-deploy.sh with SAM CLI 1.169.0: both templates lint clean, and both deploys get past validation. The same script on this PR's previous template fails with Missing option '--image-repositories', '--image-repository', '--resolve-image-repos'.
  • sam package with real credentials uploads the nested template and leaves the child's ImageUri as the parameter reference
  • bun test src/cdk/HyperframesRenderStack.snapshot.test.ts scripts/pin-handler-zip.test.ts: 29 pass, three runs in a row. CLI vitest run src/commands/lambda: 65 pass, three runs in a row. Typecheck, oxlint, oxfmt and fallow are clean.
  • Each of these fails only its own test: chunks routed back to the render function, an environment variable missing from the child, the release not writing the child, the CLI staging only template.yaml. The contract test pins the plan and the chunks to the same function: routing either one back to x86 fails it. (Earlier: v2 chunks left on x86; role unable to invoke the video function.)
  • Comments follow CONTRIBUTING.md "Comments"
  • Real deploys with and without an image (with feat(cli): lambda deploy renders on arm64 by default; --video-arch x86_64 keeps x86 #6073), heavy 4K renders on both, then teardown with nothing left behind

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at f3dbc6be (one commit on main 2ef60f3f). The head then moved to 3502b082, which only rewrites the Globals tags comment in template.yaml; the test file's blob is unchanged.

What holds

  • Audio parity holds even where chunks carry audio. renderChunk does mux audio into chunks when encoder.chunksCarryAudio is set (renderChunk.ts:1011-1019). But muxChunkWithAudio stream-copies (chunkAudio.ts:234, -c copy), so the audio packets in an arm64 chunk are the plan's x86 encodes, byte for byte. The only encodes are in plan, audio and assemble, and all of them stay on RenderFunction.
  • Every chunk renders on one architecture. Both RenderChunk tasks (v1 :464, v2 Distributed Map :583) take the same !If, and nothing else in the handler invokes renderChunk. So a render never concats x86-encoded chunks with arm64 ones.
  • The concurrency sizing still lines up. The plan reads HYPERFRAMES_LAMBDA_RESERVED_CONCURRENCY from its own environment (handler.ts:376), which equals the video function's reservation. With no reservation, it sizes from GetAccountSettings, which only RenderFunction's role may call. That's correct, because only the plan calls it.
  • Moving Runtime and Architectures off Globals is required, not cosmetic. An Image function rejects Runtime. MemorySize, Timeout, Tracing, Tags and the global Environment still reach both functions.
  • The role and alarm wiring holds. The !If … AWS::NoValue list element on the role drops cleanly when the parameter is empty, and the chunk-invocation alarm follows the chunks.
  • Locally: cfn-lint 1.58.0 passes on this template, as it does on main's.

Nits (non-blocking)

  • ChromeSource reaches the video function through Globals. HYPERFRAMES_LAMBDA_CHROME_SOURCE: !Ref ChromeSource merges into the image function's environment. With ChromeSource=chrome-headless-shell and a VideoImageUri, every chunk would look for a shell binary the image never has (#6064's staging refuses shell ZIPs). Setting HYPERFRAMES_LAMBDA_CHROME_SOURCE: sparticuz on VideoFunction (function-level keys win), or adding a Rules assertion, would close that.
  • For the CLI-option PR: packages/cli/src/commands/lambda/policies.ts has no ecr: actions. Deploying an image function needs ecr:BatchGetImage / ecr:GetDownloadUrlForLayer on the repository (plus ecr:SetRepositoryPolicy when Lambda adds its own pull policy) for whoever runs sam deploy.
  • The CDK stack has no video-function option yet (HyperframesRenderStack.ts:3 says it mirrors this template). That's fine as a follow-up.

CI: green at 3502b082. Seven of the 8 required checks show success (Build, Typecheck, Test: runtime contract, Tests on windows-latest, Render on windows-latest, Semantic PR title, regression), and Comments passes too. The Test aggregate check-run still reads "in_progress", but its job's steps all completed and the CI run concluded success at 15:33Z. Its shards (Test (runtime), (studio), (support)) are green.

Verdict: APPROVE
Reasoning: An empty parameter leaves the deploy unchanged except for where Runtime and Architectures are declared. With the parameter set, only renderChunk moves to arm64, and chunk audio is stream-copied, so the audio-bytes claim holds.

— Via

A new VideoImageUri parameter adds an arm64 container-image function that only
renders chunks. Plan, audio and assemble stay on the x86 ZIP function, so audio
encodes are unchanged. Empty (the default) deploys exactly as before. The
reserved concurrency moves to whichever function renders the chunks.
@miguel-heygen
miguel-heygen force-pushed the feat/lambda-arm64-video-function branch from 3502b08 to f901661 Compare October 22, 2026 16:21

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing my approval at f901661f to request changes. I missed this in my earlier review, and it was found while reviewing #6073, the CLI PR that deploys this template.

Blocker: sam deploy refuses this template, so hyperframes lambda deploy stops working for everyone

VideoFunction (examples/aws-lambda/template.yaml:193-201) is an AWS::Serverless::Function with PackageType: Image. SAM CLI checks every deploy for image functions before it evaluates any Condition. image_repository_validation sets required whenever any resource with an ImageUri has PackageType: Image. So when --image-repository, --image-repositories and --resolve-image-repos are all absent, sam deploy exits 2. That holds even when VideoImageUri is empty and HasVideoFunction is false.

I reproduced this with SAM CLI 1.169.0, running the argv the CLI's samDeploy builds:

sam deploy --stack-name s --region us-east-1 --resolve-s3 --capabilities CAPABILITY_IAM \
  --no-confirm-changeset --no-fail-on-empty-changeset --parameter-overrides ChromeSource=sparticuz
Error: Missing option '--image-repositories', '--image-repository', '--resolve-image-repos'

The same command against main's template gets past validation and only fails at CreateChangeSet on my fake credentials. The released CLI deploys this template, because prebuilt-template.mjs copies it into the package. So if this merges and ships, the default x86 lambda deploy breaks too. cfn-lint, which I ran last time, doesn't check this; only SAM CLI does.

Everything else in my earlier review still holds. The fix may live in #6073, since the CLI is what calls sam deploy. Either way, this template shouldn't ship without a deploy path that passes SAM's check.

Verdict: REQUEST CHANGES
Reasoning: With this template, sam deploy fails SAM's image-repository check for every stack, which breaks the CLI's default deploy. I reproduced it on SAM CLI 1.169.0.

— Via

… image repository

sam deploy refuses a template with any top-level image function unless it is given an image
repository, even when that function's Condition is false, so every deploy of the template failed.
The video function now lives in video-function.yaml, a nested stack created only when
VideoImageUri is set. The release ships it beside template.yaml and the CLI stages both.

A new workflow runs the CLI's sam deploy argv against the template with fake credentials and
requires it to reach the first upload.
@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Thanks, this was a real blocker, and the live deploy would have hit it. Fixed in f9bfde8, at the template rather than in one caller, because the CLI, smoke.sh, eval.sh and the docs' sam deploy example all run non-guided deploys of this file.

  • Root cause: SAM's image_repository_validation counts every packageable resource whose ImageUri or Code.ImageUri is set and whose PackageType is Image, at the top level of the template, before any Condition. Switching to AWS::Lambda::Function alone doesn't help: Code.ImageUri is a packageable path too.
  • Fix: the video function moved to examples/aws-lambda/video-function.yaml, a nested stack created only when VideoImageUri is set. The child is plain CloudFormation, so there's no transform and callers need no CAPABILITY_AUTO_EXPAND. It carries the render function's memory, timeout, storage, tracing, tags, environment and S3 access. A new contract test pins its environment to the render function's.
  • Packaging: the release writes the child beside template.yaml, and lambda deploy without a checkout stages both.
  • Proof, SAM CLI 1.169.0: the CLI's argv with fake credentials now gets past validation to the first S3 upload, both with an empty VideoImageUri and with a digest-pinned URI. Main's template behaves the same way. The previous template still fails with your Missing option error. With real credentials, sam package uploads the nested template and leaves the child's ImageUri as the parameter reference.
  • CI: a new Lambda SAM template workflow runs exactly that check (examples/aws-lambda/scripts/check-sam-deploy.sh). It fails on the previous template, so it would have caught this.

The live deploy and render with #6073 come next.

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at ede453d1: two new commits, f901661f..ede453d1. f9bfde86 nests the video function, and ede453d1 runs the plan on it. This resolves my change request (5617532846).

What's fixed

  • sam deploy gets past SAM's image-repository check again. The image function now lives in video-function.yaml as a plain AWS::Lambda::Function, nested through a conditional AWS::CloudFormation::Stack. get_template_artifacts_format reads only the parent's resources, so nothing there needs an image repository.

  • I ran the full packaging step locally, not just validation. I ran SAM CLI 1.169.0 against a local S3 and CloudFormation mock (moto), with the CLI's argv and --s3-bucket, for both VideoImageUri="" and a digest-pinned ECR URI.

    • Both runs packaged and reached Changeset created successfully. Execution then failed only inside the mock.
    • The nested template was uploaded and TemplateURL rewritten to its S3 URL.
    • Code.ImageUri stayed a Ref, so SAM didn't try to push an image.
  • The new sam deploy validation job is a real check. I ran check-sam-deploy.sh with f901661f's template, and it exits 1 on Missing option '--image-repositories'…. It's green at this head.

  • The nested function keeps what Globals used to give it. Globals is unchanged, and the nested function sets each of its keys itself: memory, timeout, active tracing, the two tags, NODE_OPTIONS and HYPERFRAMES_LAMBDA_CHROME_SOURCE. Its role spells out S3CrudPolicy's actions plus AbortMultipartUpload, as before.

  • Every reference moved to the nested stack's outputs. That covers both RenderChunk tasks, the state-machine invoke policy, both alarms and the VideoFunctionArn output.

  • The packaged CLI ships the nested template. prebuilt-template.mjs copies video-function.yaml, and stagePrebuiltHandler stages every .yaml beside the ZIP.

  • The deploy policy already covers the nested stack. The actions it needs, cloudformation:CreateStack/UpdateStack/DeleteStack and iam:CreateRole/PutRolePolicy/AttachRolePolicy/TagRole, are all in policies.ts. CAPABILITY_IAM covers the unnamed role.

  • Running the plan on the video function (ede453d1) is needed, not optional. renderChunk.ts:554 refuses a chunk whose ffmpeg -version first line differs from the one the plan recorded. x86's ffmpeg-static says ffmpeg version 7.0.2-static https://johnvansickle.com/ffmpeg/ …, and the arm64 image says ffmpeg version 7.0.2 Copyright …. So with an x86 plan, every arm64 chunk fails with FFMPEG_VERSION_MISMATCH. I missed this at f901661f. Both Plan tasks (v1, v2) now take the same !If as the chunks. The nested role gains lambda:GetAccountSettings, which is the only AWS call the plan makes beyond S3. Audio (v2) and both Assemble tasks stay on RenderFunction.

Important (please fix, not blocking):

  • The PR body's audio claim is stale. "Plan, audio and assemble … Keeping them on the x86 function means a mixed deploy produces the same audio bytes as an x86-only one" no longer holds. The plan's mix (mixPlanAudio, plan.ts:1337) now runs on the arm64 build, and v1 chunks stream-copy slices of it. It's the same ffmpeg version, but a different build and SIMD path, so byte-identical audio across deploy types isn't guaranteed. I don't see a user-facing defect, but the body (and the squash message, if it carries the body) should say what actually ships.

Nits (non-blocking)

  • The RenderChunkInvocationAlarm now also counts plan invocations, one per render, because it watches the video function.
  • The CI check never reaches the nested stack. It stops at the first upload, which is RenderFunction's ZIP and comes before the nested stack in the template. So a regression in packaging the nested stack would still pass. The script also hard-codes the CLI's argv, and the workflow only triggers on examples/aws-lambda/**, so a change to sam.ts won't re-run it.

#6073's restack onto this head (3cfaadd4) already prices Plan/PlanV2 at the video function's architecture and keeps the stage-every-.yaml loop, so I've dropped those two cross-PR notes from my last comment.

CI: green at ede453d1, with all 8 required checks passing (Build, Typecheck, Test, Test: runtime contract, Tests on windows-latest, Render on windows-latest, Semantic PR title, regression). The new sam deploy validation job is green. Comments (not required) is red on the comment-share ratchet.

Verdict: APPROVE
Reasoning: Nesting the image function clears SAM's check for both the x86 and arm64 values, which I confirmed through packaging and change-set creation. Running the plan where the chunks run fixes an ffmpeg-version mismatch that would have failed every arm64 chunk. The audio-parity sentence in the body needs updating, but it doesn't block.

— Via

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 3ba71a33, two commits on ede453d1, which I approved (5618919196). I've reviewed both commits.

What changed

  • ebea3594 is a rename only. In stagePrebuiltHandler (prebuiltHandler.ts:94), templates becomes templateDir. It still stages every .yaml beside the pinned template.
  • 3ba71a33 is docs only. VideoImageUri's description now says the image runs the plan and the chunks, which matches the !If on both Plan tasks (template.yaml:278, :342).
  • The PR body now matches the code. It covers the plan on the video function, lambda:GetAccountSettings on its role (video-function.yaml:63), and the contract test that pins plan and chunks to one function (HyperframesRenderStack.snapshot.test.ts:348). That resolves the important note from my last review.

Nit (non-blocking): the audio sentence is a little broad. "Audio and assemble stay on x86, so an MP4's audio, which the separate audio task encodes, is byte-identical to an x86-only deploy". The separate audio task only runs for plan v2 with workerConcurrency > 1, MP4 and non-CFR (handler.ts:513, plan.ts:1094). A CFR MP4, a single-worker render, or a v1 plan still mixes its audio in the plan (plan.ts:1536), which now runs on arm64. Scoping it to "the default v2 render of a VFR MP4" would make it exact.

Nits from earlier that still stand: the chunk-invocation alarm counts plan invocations, and the sam deploy check stops before packaging the nested stack.

CI: green at 3ba71a33, with all 8 required checks passing. Build, Typecheck, Test and Test: runtime contract passed in CI run 39845787362; a second CI run on the same head skipped those jobs. Tests on windows-latest first failed on its engine-cli shard: chunkEncoder.unreadableFrame.test.ts took 7.8s against a 5s limit, and parallelWorkerBrowsers.browser.test.ts timed out at 45s. This PR touches neither, and the re-run passed. sam deploy validation and Comments are green.

Verdict: APPROVE
Reasoning: Since my approval at ede453d1, the only changes are a local-variable rename and a parameter description. The PR body now describes what ships, apart from one sentence that is a little broad.

— Via

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 23, 2026
@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 03ed146 Oct 23, 2026
109 of 111 checks passed
@miguel-heygen
miguel-heygen deleted the feat/lambda-arm64-video-function branch October 23, 2026 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants