feat(sandbox): add a Sail Sailbox VM sandbox provider

[psinghal20]

Summary

Adds sail_vm, a built-in VM sandbox provider for Sail Research Sailboxes (type = "sail_vm", mode = "vm"), alongside modal_vm and e2b. Each Sailbox boots from Sail's devbox image, which ships Docker and Compose v2 and runs as root. That means the existing docker-in-VM flows work unchanged through VmSandbox: gateway docker-compose, agent docker run, deploy_sandbox, image builds and artifact collection. The gateway routes it to _deploy_via_vm with no provider-specific branch.

[sandbox.providers.sail_vm.config]
api_key = "secret:sail_api_key"   # required; env:SAIL_API_KEY for local dev
app = "agent-env"                 # optional: the Sail App every Sailbox belongs to
min_size = "s"                    # optional: smallest size to pick (s|m|l)
auto_sleep = false                # optional: default false (never sleep)
# auto_sleep_min_idle_seconds = 600   # optional, 1-3600; turns autosleep on
# runtime_threads = 16                # optional, sets SAIL_RUNTIME_THREADS (1-256)
inject_model_key = true           # optional: default true, Sail injects the agent's model key

Install the optional extra with pip install 'agentenv-framework[sail]', then select it with --sandbox sail_vm or [sandbox] default = "sail_vm". Usage is documented in sandbox_providers/sail_vm/README.md.

What's in it

• sail_vm/_sdk.py: imports the Sail SDK (the sail extra) and installs the API key, described in the call-outs below. The package's __init__ imports nothing and the registry points at sail_vm.provider, so core never loads the SDK.
• sail_vm/sandbox.py (SailVmSandbox):
  • Exec: separate stdout and stderr, exact bytes, async-iterable so collect_artifacts streams. Output is queued with bounded backpressure, and output nobody reads is drained by wait(). A leading sudo is dropped, a lost host or transport gives exit -1 so exec_script retries, and other stream errors are raised rather than reported as success.
  • Docker readiness: polls docker info and starts dockerd once if it isn't running. Compose v2 is required.
  • Host file writes: go through Sail's native filesystem API instead of base64 chunks.
  • Downloads under an allowlist: before an image or object download, the signed-URL hosts are added to the box's egress policy. This happens under a per-Sailbox lock with a fresh read of the applied policy, and is skipped when the cached policy already admits the hosts. It fails closed on an unknown policy and refuses to exceed Sail's 128 entries.
• sail_vm/provider.py (SailVmSandboxProvider):
  • Sizing: picks the smallest size covering the CPU request (s/m/l = 1/4/8 vCPU). Memory and disk become ceilings, rounded up into that size's range. A request no size fits is refused before provisioning.
  • Lifetime: timeout becomes the box's max_lifetime_seconds.
  • Ports: public *.sail.box listeners. Create waits until every exposed port is routed.
  • Egress policies: allow-all → {}; an allowlist → {"allowlist": hosts + IPv4 CIDRs}. IPv6 entries and more than 128 entries are refused before provisioning. EGRESS_HOSTS = ("*.sail.box",).
  • Reconnect: get_sandbox restores the tunnel URLs from listeners and the applied egress policy. An unrepresentable policy becomes None (fail closed).
  • Create failures: a failed status, or a failure during setup or port routing, terminates the box.
  • create_container: reuses the inherited login → pull → run, then deletes /root/.docker/config.json.
• Registration: one line in _BUILTIN_SANDBOX_PROVIDERS, a sail extra in pyproject.toml (sail~=0.12.8, also pulled in by dev; the lock adds sail and cloudpickle), THIRD_PARTY_NOTICES.md, and the AGENTS.md provider list.

The agent's model key never enters the Sailbox

Sail's credential injection adds the key to the agent's model requests as they leave the Sailbox. The Sailbox disk (which Sail checkpoints, see below) and its memory hold only a placeholder. This lives entirely in sail_vm/model_key.py and the sail_vm provider; no core code changes.

• Secret named by the key. create_sandbox reads LITELLM_API_KEY and LITELLM_BASE_URL from the agent's env. It stores the key as a Sail secret named AGENTENV_LITELLM_<sha256(key)[:32]>, so one key maps to one secret and different keys never share one. The secret is never deleted, so no launch can remove one that a concurrent launch (in this process or another) still needs. It holds only a key Sail already stores; sweep AGENTENV_LITELLM_* if you rotate keys.
• Saved policy applied at create. A saved egress policy is attached when the Sailbox is created, so it's in force before the agent starts. Its rules set authorization: Bearer ${secrets.…} and x-api-key: ${secrets.…} on requests to the endpoint's host. A restricted allowlist also gets that host.
• Placeholder only inside. Every command and file write from that SailVmSandbox replaces the key with sail-injected-model-key, in every form agent-env writes it (raw, shell-quoted, escaped inside single quotes). So docker run -e and docker exec -e (agent, judge, unit-test verifier) carry only the placeholder.
• Refused rather than sent. With injection on, any Sailbox refuses a command or file write that assigns LITELLM_API_KEY/ANTHROPIC_API_KEY a value other than the placeholder. In shell scripts each word is decoded as the shell reads it, quoted or escaped names included, and only a value that is wholly one $NAME reference passes. Files are checked literally, KEY: value (YAML/JSON) included. Keys under 16 characters are refused, since scrubbing replaces every occurrence. That covers an agent deployed into an existing Sail sandbox, and a reconnected handle that can't recover the key. Shell scripts are read as the shell reads them ($VAR references pass); other arguments and file contents are taken literally.
• Containers trust Sail's CA. Sail terminates TLS for rule hosts with its own CA, which is in the VM's trust store but not in containers'. A docker shim at /usr/local/bin/docker gives every container it runs or creates the VM's CA bundle, plus SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE.
• Lifecycle.
  • Widening the allowlist for downloads replaces the saved policy and keeps the rules; inline documents can't name secrets.
  • Every saved policy made for a Sailbox shares a name prefix, which a reconnected handle reads from the applied policy. Terminate, from any handle in any process, deletes the whole family. Policy changes and terminate are serialized per Sailbox in a process.
  • A failed or cancelled create deletes its policy. A policy is named before it's created, so one whose create response was lost is found by name and deleted, with retries. A failed policy swap deletes its replacement. One whose outcome Sail can't report keeps both ids until it can, and one that applied is settled even when cancelled. A create whose response was lost terminates the Sailbox it made (found by its name, with retries) before releasing the policy.
  • Keys are held per launch and per Sailbox in process memory (never persisted) and forgotten once nothing needs them. A handle reconnected in the deploying process recovers the agent's key from there; in another process it recovers the configured [model] key, if that is the one injected. A key it can't recover is refused, not sent.
• Limits.
  • The endpoint must be HTTPS; an http one is refused before anything is created, unless inject_model_key = false.
  • A different model key sent into a Sail box (for example an agent deployed into an existing sandbox) is refused; use the agent's own Sail sandbox, or inject_model_key = false.
  • The shim covers docker run / docker create on the VM. Containers started by an agent's own Docker-in-Docker don't get the CA.
  • Gateway, deploy_sandbox and other create_vm boxes carry no model key and don't inject.

Call-outs for reviewers

• The Sail API key is written to os.environ. This is the first core provider to do that. The Sail Python SDK has no client that takes a key explicitly; it reads SAIL_API_KEY once, when it builds its process-wide client. _sdk.connect:

  • sets the configured key under a lock;
  • makes the one synchronous App.find call that builds the client;
  • restores the previous value straight away, about 0.4s later.

  A different key in the same process raises ConfigError (one Sail key per process). An operator's own SAIL_API_KEY is overridden only for that build, then restored. The key is never passed to workloads, exec env, CONTAINER_ENV, REQUEST_HEADERS, logs or repr. A local-sandbox subprocess started during that window could inherit it, but the local provider already runs as the same user with the full host environment, so this adds no exposure. SAIL_API_KEY and SAIL_RUNTIME_THREADS are added to the env-var guard's not-configuration set: agent-env only writes them for the SDK and never reads them as configuration.

• Sail checkpoints every Sailbox automatically. A box with autosleep off was checkpointed about 2.5 minutes after creation with nothing on our side triggering it, and there is no setting to turn this off. The model key is kept out (above), and registry credentials are deleted after create_container's pull. Other values deployments put in container env still land on disk and in checkpoints:

  • object-store credentials when the S3 store has share_credentials = true: frozen host AWS credentials, so keep this off on Sail or use a scoped, short-lived role;
  • MCP server extra_env_vars;
  • an external env-state database URL.

  Retention, whether checkpoints include memory, and deletion on terminate are open questions for Sail.

• A cancelled create is cleaned up. If the caller is cancelled while a create is in flight (for example by ChainedSandboxProvider's 180s deadline), the box it produces is terminated: three attempts with backoff, then an error log naming the box. Otherwise it would leak a billed Sailbox.

• Attribution is not on the Sailbox. Sail has no labels or tags. Attribution goes in the box name (ae-<random>-<slugged values in key order>, ≤128 chars) and in a structured agent_env.sail_vm_sandbox_started log event keyed by sailbox_id, the same pattern as Modal's modal_sandbox_started. Per-box cost comes from Sail's spend API joined on sailbox_id.

• Autosleep is off by default (AutoSleep.never()), so idle MCP servers and gateways don't sleep or come back cold. Config can turn it on.

• Resource requests round up. Requests are ceilings on Sail (billing is for observed usage), so CPU rounds up to a size and memory and disk round up into the size's range. This raises a cap, not the bill.

• SAIL_MODE is not supported. Core must not read a stage variable, so the provider uses Sail's production endpoint.

• Optional and self-contained. sail is an optional extra like gcp. Without it, selecting sail_vm fails with load_impl's standard ConfigError naming the extra, and the contribution helpers (egress hosts, container env) skip the provider. The source change is confined to sandbox_providers/. The gateway exec-transport hardening (retrying read_trajectory / container lookups that lose their exec connection) was split out for a separate PR.

Testing

• Unit: make unit-test gives 6602 passed, 13 skipped, with the SDK faked and nothing offline touching the network. New tests are sail_vm_sandbox_provider_test.py, sail_vm_sandbox_test.py and sail_vm_model_key_test.py (secret naming, the rules, the shim's argument rewriting, create, scrubbing, policy replacement, cleanup order and reconnect). sail_vm is also added to the network-policy stamping test, the egress-host floor, config interpolation, the capability gate and the gateway VM-path routing test.
• Integration (int_test_slow):
  • New sail_vm_sandbox_smoke_test.py: Docker in the box, nginx served through a *.sail.box URL, an exact binary write round-trip, reconnect restoring ports and policy, and an allowlist enforced for containers. A third test checks model-key injection end to end: a container's request reaches a header-echo endpoint with the real key, while the container's env and docker inspect show only the placeholder, and the policy is gone after terminate.
  • sail_vm is added to the sandbox_provider parametrisations in gateway_test.py and task_steps_test.py.
  • With no Sail config, these skip with agentenv-capability-missing: remote_sandbox.
• Run live against Sail (sail==0.12.8, a local key, on the final code):
  • smoke tests: 3 passed;
  • agent-env run hello --sandbox sail_vm: passed;
  • gateway_test.py::test_gateway_with_multi_mcp_server[sail_vm]: passed in about 1m40s on the Sailbox, covering two MCP servers, a universe load, MCP tool calls and /step;
  • no Sailboxes were left running afterwards.
• Not run live: task_steps_test[sail_vm] (needs a model endpoint), autosleep staying off through a long idle period, and how promptly a box expires at its max lifetime.

🤖 Generated with Claude Code

Confidence Score: 5/5

The changes since the previous review appear safe to merge.

What we checked:

• Setup command fails: The CLI accepts --sandbox, and the package registers the bundled hello task.

Summary

Adds sail_vm, a built-in VM sandbox provider for Sail Research Sailboxes, so agent-env workloads can run on Sail’s Docker-ready VMs. It also keeps agent model keys out of those VMs by having Sail add them to outgoing requests.

• Sailboxes can now run agent-env workloads as Docker-ready VMs.
• Sail adds an agent’s model key to requests outside the Sailbox.

_{Reviews (13) · Last reviewed commit: "docs(sail_vm): walk through setup with c..." · Reviewed by Greptile}

[psinghal20]

@greptile-apps re-review

[psinghal20]

@greptile-apps re-review the pr

[psinghal20]

Follow-up (separate PR): gateway reads that lose their exec connection

Greptile's remaining concern is a core gateway issue, which this PR leaves alone to keep its source changes inside sandbox_providers/.

EnvironmentGatewayProvider ignores the exit code of its in-VM reads. When a provider's exec connection drops (exit -1), _get_container_id reads the empty result as "no container", and read_trajectory parses whatever came back. An agent check can then record an empty or partial tool-call history as real. This affects every provider; sail_vm makes it more visible because its exec_with_output returns no stdout for a cut stream.

The fix will come in its own PR from main:

• read_trajectory and _get_container_id retry a read that exited -1 and raise after three attempts, instead of reading it as no container or no history. A gateway with no history still returns [].
• The deploy-time log dumps become best-effort, so a failed lookup or docker logs call never aborts a healthy deploy.
• Unit tests for each. It was verified with the live gateway multi-MCP test on Sail before being split out of this PR.