Repository navigation
feat(cli): lambda deploy renders on arm64 by default; --video-arch x86_64 keeps x86 - #6073
miguel-heygen wants to merge 6 commits into
Conversation
4b36353 to
db3f19e
Compare
2fc197a to
6053c59
Compare
vanceingalls
left a comment
There was a problem hiding this comment.
Reviewed at db3f19eb (two commits on feat/lambda-arm64-deps ba5e7a23). I reviewed only this PR's delta, not #6063, #6065 or #6070.
Blocker: sam deploy refuses this stack's template, for x86 as well as arm64
The template (from #6065, examples/aws-lambda/template.yaml:193-201) declares VideoFunction as AWS::Serverless::Function with PackageType: Image. SAM CLI checks every deploy for image functions before it evaluates any Condition. image_repository_validation sets required = any(artifact == IMAGE for … in get_template_artifacts_format(template)), and get_template_artifacts_format reads PackageType off each resource that has an ImageUri. So when --image-repository, --image-repositories and --resolve-image-repos are all absent, the deploy stops. samDeploy (sam.ts:96-121) passes none of the three.
I reproduced this with SAM CLI 1.169.0 (latest), running the exact argv samDeploy builds, from a copy of this head's template:
sam deploy --stack-name s --region us-east-1 --resolve-s3 --capabilities CAPABILITY_IAM \
--no-confirm-changeset --no-fail-on-empty-changeset \
--parameter-overrides ChromeSource=sparticuz 'VideoImageUri=""'
Error: Missing option '--image-repositories', '--image-repository', '--resolve-image-repos' (exit 2)
VideoImageUri=<digest-pinned ECR URI>fails the same way.- Main's template passes the same check. The same command against
main's template gets past validation and only fails atCreateChangeSeton my fake credentials. - Every user's
lambda deploybreaks, including the default x86 one. The released CLI deploys the templateprebuilt-template.mjscopies fromexamples/aws-lambda/template.yaml, and a checkout deploys that file directly. - Adding
--image-repository <uri>gets past validation for x86. I haven't checked what SAM's packaging step then does with an image function whoseImageUriis a!Ref, so I'm not proposing it as the fix. - Another option to check: declare the video function in a way SAM's packager doesn't count as an image artifact.
Whichever fix you choose, please run real sam deploy validation in a test or CI. The unit tests mock spawnSync, so they can't catch this. The unchecked live-deploy item in the test plan would have caught it.
(The parameter itself is fine. SAM 1.169.0's override parser turns VideoImageUri="" into an empty string, and a non-empty URI passes through unchanged. I ran its CfnParameterOverridesType regexes on both.)
What holds
- The ECR actions now cover the push and Lambda's pull. The push uses only ECR API calls (
CreateRepository,DescribeRepositories,BatchCheckLayerAvailability,InitiateLayerUpload,UploadLayerPart,CompleteLayerUpload,PutImage), and every one is inpolicies.ts. NoGetAuthorizationTokenis needed, and no tags are passed, soTagResourceisn't needed either.BatchGetImage,GetDownloadUrlForLayerandGet/SetRepositoryPolicycover Lambda's image pull. That closes my #6065 nit. - arm64 with
chrome-headless-shellis refused before anything is pushed (deploy.ts). That closes theChromeSource-via-Globalsnit from #6065, at the CLI level. fetchPinnedreuses the handler ZIP's verify-then-rename path, and it now streams the hash, so a large OCI tarball isn't read into memory. The arch check runs on the pushed manifest, not on a label.- Cost: only
RenderChunk/RenderChunkV2invocations are priced at the arm64 rate. The running-Map path adds counts, not invocations, so nothing there is mispriced.
Nits (non-blocking)
destroycan delete a repository it didn't create. If<stack>-videoalready exists,ensureEcrRepositoryadopts it, anddestroythen runsdelete-repository --forceon it, removing every image. Tag the repository on create and delete only a tagged one, or refuse to adopt an existing one.- A failed first deploy can orphan the repository. The push happens before
sam deploy, so if that first deploy fails, no state file records the repository. A retry reuses it, so this only matters if the user gives up. - The arm64 rate follows the stack's current state, not the state at render time.
progresson an old render after a switch to x86 prices its chunks at x86.
CI: Comments (not required) fails on the comment ratchet. deploy.ts goes from 7.0% to 7.3% comment lines and state.ts from 15.7% to 17.1%. The PR's ffmpeg a/ffmpeg b jobs were still running when I looked.
Verdict: REQUEST CHANGES
Reasoning: With an image function in the template, sam deploy stops at SAM's image-repository check for every arch, so hyperframes lambda deploy can't deploy any stack. I reproduced that with the CLI's own argv on SAM 1.169.0.
— Via
db3f19e to
9e2f7b8
Compare
|
Thanks. Your blocker was real, and the live deploy would have hit it. Blocker ( Nits
Checks at 9e2f7b8, on Linux: typecheck in both packages, 237 aws-lambda tests, 74 CLI lambda tests three runs in a row, the SAM check script, oxlint, oxfmt and fallow. Removing the tag from the CLI's |
vanceingalls
left a comment
There was a problem hiding this comment.
Re-reviewed at 9e2f7b8f. It now sits on feat/lambda-arm64-deps 8f440b6d, which contains #6065's nested-template fix f9bfde86. This resolves my change request (5617533229).
What's fixed
- The blocker is gone, because the template is #6065's. The image function now lives in a nested
video-function.yaml. I verified that template separately on SAM CLI 1.169.0 plus a local AWS mock (moto): with the CLI's argv, bothVideoImageUri=""and a digest-pinned ECR URI package the nested stack and create the change set. This PR adds nothing to the template.fetchStackOutputsstill readsVideoFunctionArn, which the parent now takes from the nested stack's outputs. - The restack kept both halves of
prebuiltHandler.ts.git range-diff: the first commit is unchanged, and the second differs only there.fetchPinnedstages the ZIP, then every.yamlbeside the pinned template is copied next to it, sovideo-function.yamlships with released installs. - The new commit closes my
destroynit.pushVideoImagepasses ahyperframes:stack=<stack>tag.ensureEcrRepository(8e7f3c27in the base) sets that tag on create, and adopts an existing repository only when it already carries it, refusing otherwise. Sodestroy --forcecan only hit a repository this stack created.ecr:TagResourceandecr:ListTagsForResourceare added topolicies.ts.
Locally at this head: bunx vitest run src/commands/lambda (cli) passes 74/74, bun test src/sdk/ (aws-lambda) passes 107/107, and tsc --noEmit is clean in both packages.
Still open (nits, unchanged):
- A failed first deploy can leave the repository orphaned. A retry by the same stack now adopts it, because it carries the tag.
- The arm64 price follows the stack's current state, not the state at render time.
CI: as a stacked PR (base feat/lambda-arm64-deps), ci.yml's required checks don't run here, and no path-triggered Lambda workflow fired. regression, Preview parity and preview-regression are green. Comments (not required) is red only on the comment ratchet: prebuiltHandler.ts is at 6.9% against 6.7% at the merge-base.
Verdict: APPROVE
Reasoning: The sam deploy blocker was in the template, and the base now carries #6065's nested-stack fix, which I verified through packaging for both archs. The new tag gate closes the one risky nit, and the lambda tests and typecheck pass at this head.
— Via
9e2f7b8 to
3cfaadd
Compare
vanceingalls
left a comment
There was a problem hiding this comment.
Re-reviewed at f5321f02. That's the pinned 6f2b735b plus a merge of #6065's 3ba71a33, which only rewords VideoImageUri's description in template.yaml. Everything since my approval at 9e2f7b8f (5618033598): 3cfaadd4 (the plan priced at the video function's rate), #6065's ede453d1/ebea3594/3ba71a33 merged through, and 6f2b735b (arm64 by default).
What holds
- The default deploy outside a checkout gets an image.
publish.ymlbuilds the arm64 video image from the pinned handler ZIP. It pins the image into the adapter (pin-handler-zip.ts --video-image) and attaches it to the release, all beforenpm publish. A matching CLI and adapter therefore always have an image to push. An older adapter with no pinned image fails with "pins no arm64 video image … or pass --video-arch x86_64" (videoImage.ts:83), not a broken stack. - Every arm64 refusal names the way out:
--chrome-source chrome-headless-shellrefuses before any work (deploy.ts:49), whether the arch is defaulted or explicit.it.each([undefined, "arm64"])covers both.- In a checkout with no
HYPERFRAMES_LAMBDA_VIDEO_IMAGE, the error says "or pass --video-arch x86_64".
parseVideoArchnow returnsundefinedfor an omitted flag (lambda.ts:494), so the default lives in one place,runDeploy.- Nothing in CI deploys with the old default. No workflow or script calls
lambda deploy. - The default push needs no new permissions beyond the policy.
pushImageToEcruploads through the ECR API (InitiateLayerUpload/UploadLayerPart/CompleteLayerUpload/PutImage), not a registry login. All of those are inpolicies.ts, so noecr:GetAuthorizationTokenis needed. - Pricing follows the plan.
stateArchitecture(getRenderProgress.ts:504) now pricesPlan/PlanV2and the chunks at the video function's architecture, which matches the template. - Locally: CLI
vitest run src/commands/lambdapasses 76/76, and aws-lambdabun test src/sdkpasses 107/107.tsc --noEmitis clean for both packages.
Nits (non-blocking)
- The upgrade note doesn't mention the deploy policy. Arm64 by default makes a plain redeploy create and push to an ECR repository. A deploy credential set up from an older
lambda policiesoutput has noecr:*, so it fails atCreateRepository. One line in "Upgrading from a version that deployed x86 only" (aws-lambda.mdx:105) would cover it, for example "re-applyhyperframes lambda policies user|role". - The architecture doesn't carry over to a redeploy, unlike
--concurrency. A stack deployed with--video-arch x86_64moves to arm64 on the next flaglesslambda deploy. The docs say so ("pass--video-arch x86_64on every deploy"), but--concurrencykeeps the stack's current value. Keeping the stack's recorded architecture would be more consistent. - In a checkout, the missing-image error comes after the ZIP build.
prepareSamDirbuilds the handler ZIP (deploy.ts:56) beforepushVideoImagefinds noHYPERFRAMES_LAMBDA_VIDEO_IMAGE. Checking the env var first would fail in seconds. - Two comments still assume opt-in:
policies.ts:84("deploy --video-arch arm64tags and pushes…") andvideoImage.ts:1.
CI: this PR is stacked on feat/lambda-arm64-deps, so ci.yml's required checks don't run here. That's why I ran the suites locally. regression is green. Comments (not required) is red on the comment-share ratchet: prebuiltHandler.ts went from 6.0% to 6.2% against the stack-internal base a72b497c.
Verdict: APPROVE
Reasoning: The arm64 default is safe for both deploy paths: the release always ships a pinned image, and every combination arm64 can't serve fails before deploying, naming --video-arch x86_64. The remaining notes are about upgrade docs and consistency.
— Via
getRenderProgress takes chunkArchitecture, the architecture of the function that renders chunks. RenderChunk invocations are priced at its per GB-second rate, both inline and from a Distributed Map's output; plan, audio and assemble stay at the x86 rate.
…deo-arch --video-arch arm64 pushes the arm64 video function image to the stack's own ECR repository and deploys the template with VideoImageUri set to its digest. Outside a checkout the image is the one the installed adapter pins; inside a checkout it must come from HYPERFRAMES_LAMBDA_VIDEO_IMAGE, built from that checkout. The default stays x86_64, which always sends an empty VideoImageUri so a redeploy drops a video function rather than keeping it. destroy deletes the repository, render and progress price chunks at the arm64 rate, and policies lists the ECR actions.
eb0f8e0 to
b6f069f
Compare
The base branch was changed.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
There was a problem hiding this comment.
Re-reviewed at b6f069f3, rebased onto main 0df08892 (it now contains #6065 and #5866). The last approval was at f5321f02 (5620212563, dismissed by the base change).
This is not purely a rebase. I compared the PR's net patch at f5321f02 against feat/lambda-arm64-deps a72b497c with its net patch at b6f069f3 against main. Apart from index lines and hunk offsets, the only difference is the new commit b6f069f3. It drops two doc comments in packages/cli/src/commands/lambda/prebuiltHandler.ts:
/** A release asset@hyperframes/aws-lambdapins by URL, sha256 and size. *//** A verified copy ofpinnedin the cache, downloaded unless one is already there. */
That's comment-only and doesn't change behaviour. The other 18 files match the approved patch line for line.
No blockers. Approving. When I looked, CI and Windows render verification were still running; regression and PR captures had passed.
What
hyperframes lambda deploynow deploys the stack with an arm64 container-image function that runs the plan and renders the chunks. Audio and assemble stay on the x86 ZIP function.--video-arch x86_64deploys everything on x86, as before this PR.Redeploying an existing stack without the flag moves it to arm64 and adds its
<stack>-videoECR repository. To keep a stack on x86, pass--video-arch x86_64on every deploy. The Lambda guide has an upgrade note.Depends on #6070 (the pinned image), #6063 (ECR push) and #6065 (the template's
VideoImageUri). Its base,feat/lambda-arm64-deps, is #6070's branch with #6063 and #6065 merged in and nothing else, so this diff is only the CLI option and default, cost accounting and progress.Why
The plan and the chunks are most of a render's compute, and arm64 Lambda costs 20% less per GB-second. Users of the released CLI have no Docker, so the CLI pushes the release's image itself through the ECR API.
How
videoImage.ts: the image is the OCI tarball the installed@hyperframes/aws-lambdapins. The samefetchPinnedthe handler ZIP now uses downloads it and checks its sha256. The CLI then:<stack>-videoin ECR and pushes it;linux/arm64;HYPERFRAMES_LAMBDA_VIDEO_IMAGEpoints at a locally built tarball instead.--video-arch x86_64as the way out:--chrome-source chrome-headless-shell, since the image bundles sparticuz Chrome only.sam.ts:VideoImageUriis always sent, and is empty for x86. If it were omitted, SAM would sendUsePreviousValueand keep a video function the user dropped.fetchStackOutputsreadsVideoFunctionArn.destroydeletes the repository after the stack; a repository already gone counts as deleted. Switching back to x86 keeps the repository name sodestroystill finds it.getRenderProgresstakesvideoArchitectureand prices the plan andRenderChunkinvocations at the arm64 rate, inline and from the Distributed Map's output.render --waitandprogresspass it from state.hyperframes:stack=<stack>. An existing one is adopted only if it carries that tag (feat(aws-lambda): push a prebuilt image to ECR without Docker #6063), sodestroyonly ever deletes this stack's own repository.policieslists the ECR actions the push, the tag check and Lambda's image pull need.Test plan
bunx vitest run src/commands/lambda(cli): 76 pass, three runs in a row.bun test(aws-lambda) passes.examples/aws-lambda/scripts/check-sam-deploy.shpasses.tsc --noEmit,oxlint,oxfmt,fallow audit,check:cli-process-ownershipandcomment-ratchetare clean.VideoImageUriomitted for x86, no arch check, release image used in a checkout, repository name not lowercased, image URI or repository dropped by deploy, a missing repository treated as a failure,VideoFunctionArnignored, chunks priced x86, the stack tag dropped from the repository call.destroywith nothing left behind.