Skip to content

feat(sandbox-adapter): add AWS Lambda MicroVMs provider - #395

Draft
NathanFlurry wants to merge 1 commit into
mainfrom
task-20261114-130155-96f64422-aws
Draft

NathanFlurry wants to merge 1 commit into
mainfrom
task-20261114-130155-96f64422-aws

Conversation

@NathanFlurry

@NathanFlurry NathanFlurry commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Adds an AWS Lambda MicroVMs sandbox provider at @rivet-dev/sandbox-adapter/lambda-microvms. AWS launched MicroVMs on 2026-06-22: each MicroVM is a Firecracker VM with suspend, resume, and terminate APIs and its own HTTPS endpoint.

sandbox: lambdaMicrovmsProvider({
	create: {
		imageIdentifier: "arn:aws:lambda:us-east-1:123456789012:microvm-image:sandbox-agent",
	},
}),

Lifecycle

Event Lambda MicroVMs call
First tool use RunMicrovm(create) with defaults for the HTTP_INGRESS connector, an 8 hour max lifetime, and an idle policy (auto-suspend after 15 idle minutes, auto-resume on the next request). create can be a function of the Actor context.
Wake GetMicrovm. A SUSPENDED MicroVM gets ResumeMicrovm, and a SUSPENDING one is waited on first. A 404, TERMINATING, or TERMINATED returns undefined, so the Actor gets a new MicroVM. The provider then polls /v1/health through the endpoint, because AWS says state is eventually consistent.
Sleep SuspendMicrovm. Conflict and 404 are ignored.
Destroy TerminateMicrovm. 404 is ignored.

Commands and files

Lambda MicroVMs has no exec or file API. The only ways in are the image itself, the /run hook, or the endpoint. Its shell WebSocket is undocumented and AWS says not to use it in production. So the image runs Sandbox Agent, and the provider calls Sandbox Agent's process and filesystem API through the endpoint:

exec(command)
 ├─ CreateMicrovmAuthToken(port)            cached for 60 min, refreshed 5 min early
 ├─ POST /v1/processes  /bin/sh -c command  X-aws-proxy-auth, X-aws-proxy-port
 ├─ poll GET /v1/processes/{id}, then /logs?since=N  → onData  (runRemoteProcess)
 ├─ kill/timeout: find descendants by parent pid in /proc/*/stat, SIGKILL the tree, then /kill
 └─ DELETE /v1/processes/{id}               frees the retained output

The Sandbox Agent HTTP client lives in its own internal module (sandbox-agent-http.ts, not exported from the package) so the AWS-specific code stays small.

The docs give this Dockerfile. It is zipped with the aarch64 musl binary (MicroVMs default to ARM64), uploaded to S3, and built with create-microvm-image. The docs list each step.

FROM public.ecr.aws/lambda/microvms:al2023-minimal
COPY sandbox-agent /usr/local/bin/sandbox-agent
RUN chmod 755 /usr/local/bin/sandbox-agent
CMD ["/usr/local/bin/sandbox-agent", "server", "--no-token", "--host", "0.0.0.0", "--port", "8080"]

Tests

tests/lambda-microvms.test.ts runs the provider end to end against a real Sandbox Agent server and an in-memory fake of the MicroVMs control plane. A local proxy stands in for the endpoint: it returns 403 without the right X-aws-proxy-auth and X-aws-proxy-port headers and 502 before the server is up. The suite covers:

  • create defaults
  • the token's port scope
  • suspend, resume, and destroy
  • a missing MicroVM
  • separate stdout and stderr, exit code, cwd, and env
  • streaming plus abort, which must kill a backgrounded child
  • timeouts
  • every file operation

The tests run only when SANDBOX_AGENT_BIN points to a sandbox-agent binary, so CI skips them. Locally, all 38 package tests pass with the 0.5.6 release binary. The abort test fails if the child walk is removed.

On-demand live test

cd packages/sandbox-adapter
pnpm test:live lambda-microvms

This runs tests/lambda-microvms.live.test.ts against a real MicroVM with the AWS credentials and Region from the environment. It first builds a temporary MicroVM image the way the docs describe: it downloads the aarch64 Sandbox Agent binary (SANDBOX_AGENT_VERSION, default latest), zips it with the Dockerfile, uploads the zip to a new S3 bucket, and creates a build role. All of that is deleted afterwards. Set LAMBDA_MICROVMS_TEST_IMAGE to an image ARN to skip the build. A full run takes about 3 minutes.

CI never runs it. vitest.config.ts excludes *.live.test.ts, and only vitest.live.config.ts, used by pnpm test:live, includes it. @aws-sdk/client-s3 and @aws-sdk/client-iam are new dev dependencies for the image build.

Live run (us-east-1)

All 6 live tests pass on an image built from the Dockerfile above. The image build took about 3 minutes. The tests cover:

  • exec with separate stdout and stderr, the exit code, cwd, and env
  • streaming plus abort, which must kill a backgrounded child
  • timeouts
  • file operations, including a 1 MiB write
  • suspend, then connect resuming the MicroVM with its files kept (about 11 s)
  • destroy

Fixed from the live run:

  1. Tree kill. The MicroVM kernel (6.1, aarch64) has no /proc/<pid>/task/<tid>/children, which the old tree kill read. It now finds children by the parent pid in /proc/*/stat, which works on any Linux kernel.
  2. Connect after terminate. GetMicrovm still reports RUNNING for about 3 s after TerminateMicrovm, so connect waited the full readyTimeoutMs on 502s. The readiness wait now rechecks GetMicrovm every 5 s and returns undefined once the MicroVM is gone. It has a new fake test.
  3. Docs.
    • imageIdentifier must be an ARN. A bare name fails with Malformed ARN.
    • lambda:PassNetworkConnector is not a documented action, so it is gone from the permissions list.
    • The Dockerfile copies a bundled binary instead of running curl during the build.

Known: Sandbox Agent runs as pid 1 in the MicroVM and does not reap orphans, so killed children stay zombies. They are dead and hold only a process table entry. Reaping belongs in Sandbox Agent, as a follow-up.

Still unverified:

  • The minimal IAM policy. The test identity had broad permissions.
  • The upper bound for suspendedDurationSeconds: 28800.

Package and docs

  • @aws-sdk/client-lambda-microvms@^3.1169.0 is an optional peer dependency. Adding it moves the lockfile's shared @aws-sdk/credential-provider-node peer from 3.972.84, which is deprecated, to 3.972.97.
  • scripts/test-packed.mjs checks that dist/lambda-microvms.js is packed.
  • The Sandboxes guide has a providers table row and an AWS Lambda MicroVMs section with options, the Dockerfile, permissions, and the 8 hour lifetime. The Sandbox Adapter guide's table and the docs index card list it. The example is examples/docs/sandboxes/lambda-microvms.ts. The quickstart is unchanged, because this provider needs an image built first.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

📟 Pip-Boy · d7708f7

📦 Packages · fresh npm install resolve, no lockfile · this PR vs base

No notable install size, dependency, or entrypoint changes across 10 packages.

All 10 packages · linux-x64-gnu
package (linux-x64-gnu) @latest base this PR Δ vs base packages
@rivet-dev/pi 392.5 MB 392.5 MB 392.6 MB +30.6 KB (+0.0%) 220 (0)
@rivet-dev/sandbox-adapter 114.5 KB 114.5 KB 145.1 KB +30.6 KB (+26.7%) 1 (0)
@sandbox-agent/cli-shared 27.2 KB 27.2 KB 27.2 KB 0 1 (0)
@sandbox-agent/persist-indexeddb 12.4 KB 12.4 KB 12.4 KB 0 1 (0)
@sandbox-agent/persist-postgres 12.3 KB 12.3 KB 12.3 KB 0 1 (0)
@sandbox-agent/persist-rivet 12.2 KB 12.2 KB 12.2 KB 0 1 (0)
@sandbox-agent/persist-sqlite 12.3 KB 12.3 KB 12.3 KB 0 1 (0)
@sandbox-agent/react 41.6 MB 41.9 MB 41.9 MB 0 15 (0)
acp-http-client 6.7 MB 6.7 MB 6.7 MB 0 3 (0)
sandbox-agent 31.0 MB 31.0 MB 31.0 MB 0 8 (0)

🧹 Rust dependencies

Skipped: this PR doesn't change any Rust code or dependencies.

🦀 Binaries

Skipped: this PR doesn't change any Rust code.

⏱ Timing

job wall time slowest step
zizmor 16s checkout · 9s
changes 21s checkout · 15s
footprint 3m09s Run pnpm --dir sandbox-agent install --frozen-lockfile · 1m01s
⏭ self-test — skipped
⏭ hygiene — skipped
⏭ native · ${{ matrix.platform }} — skipped
report 27s checkout · 10s
🔭 Full CI trace for this commit

One OpenTelemetry trace of CI and Pip-Boy for d7708f7: every job and step, with the sizes above attached. Run after CI finishes.

pnpm install --frozen-lockfile --filter 'pipboy...'
pnpm --filter pipboy pipboy trace --sha d7708f77d13bbf73e48b594d4decc333a007f11d

Add --otlp <url> to send it to any OTLP/HTTP collector.

Agent instruction: From a rivet-dev/agents checkout, run the commands above, then read the trace file the last command prints. List the slowest jobs and steps and explain where CI time went.

@NathanFlurry
NathanFlurry force-pushed the task-20261114-130155-96f64422-aws branch 2 times, most recently from b08d21c to 3963eb4 Compare November 15, 2026 19:53
Add @rivet-dev/sandbox-adapter/lambda-microvms. Each Actor gets one
Lambda MicroVM, suspended while the Actor sleeps and terminated when it
is destroyed.

Lambda MicroVMs has no exec or file API, so the image runs Sandbox Agent
and the provider drives its process and filesystem API through the
MicroVM's HTTPS endpoint, with a cached auth token scoped to that port.
A kill or timeout ends the command's whole process tree.

`pnpm test:live` runs tests/lambda-microvms.live.test.ts against a real
MicroVM. It builds a temporary MicroVM image first and deletes it
afterwards, unless LAMBDA_MICROVMS_TEST_IMAGE names an existing one. CI
never runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NathanFlurry
NathanFlurry force-pushed the task-20261114-130155-96f64422-aws branch from 3963eb4 to d7708f7 Compare November 15, 2026 19:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant