feat(run_code): give a script on an env's host the env's address on any provider, and stage each run in a dir of its own

[earakely-scale]

Summary

A run_code script on an env's host had no way to find the env. Scripts assumed 127.0.0.1:18765, where the VM providers publish a gateway. That breaks in three cases:

• Local provider. It maps that port into 20000–30000, because deployments share one host, so on a laptop those scripts got connection refused. A CoopBench task materialized as a bundle stopped at its warmup step this way, on Linux and macOS.
• No gateway. An env deployed by the server provider, or by a plugin's provider, has no gateway address to assume.
• No host. An env in a container sandbox (Modal containers, the sdk's modal_gpu) has no VM host, so the step failed with 'ModalSandbox' object has no attribute 'exec_script'.

What changes:

• The script's input names the env in protocol terms:

  input["env"] == {"id": "...", "url": "http://127.0.0.1:<port>", "mcp_url": "http://127.0.0.1:<port>/mcp"}

  • url is the env's address. Its card is at url + "/.well-known/agent-env.json", and every path the card declares joins onto url.
  • mcp_url is the MCP endpoint the card declares.
  • Both come from the base DeployedEnv record, so this holds for a gateway, a lone server or a plugin's env, with no check on the record's type. A record from before the card has url set to None.

• The sandbox says how its host reaches its own URLs. New Sandbox.url_on_host(url), beside url_from_sandbox, which is untouched.

  • A URL under one of tunnel_urls (longest match first, path kept) becomes http://<address>:<host_port(port)><path>. The address is loopback unless the sandbox publishes only on other addresses (host_ips). Any other URL is returned unchanged.
  • It uses only the base contract, so every core and sdk provider gets it as is. A provider whose host reaches its services some other way can override it.
  • VM providers give 127.0.0.1:18765, the address scripts already use. beta_scale's path-prefixed proxy tunnel, which a host can't hairpin through, maps there too. The local provider gives the mapped port.

• An env in a container sandbox runs the script in that container, as a container-mode deploy_sandbox on Modal already does. The env is at loopback on its own port there. The image needs python3.

• Only env_id steps get env. Agent and deploy_sandbox scripts are unchanged, and scripts that don't read the key see no difference.

Not done: having the local provider publish 18765 itself when it's free. With two local runs at once, one run's script would silently reach the other run's env.

Also fixed (found in chaos, and on the old release too):

• The race. On the local provider, concurrent runs of a step with the same id staged into one shared /tmp/agent_env_run_code/<step>, for example the same task run twice at once. One run's reset deleted another's runner.py, or replaced its input and result, so one script read another run's env.
• The fix. A host target now stages in sandbox.scoped_name(_RUN_DIR)/<step>. On the local provider that ends in the sandbox's id, so its teardown removes it. Elsewhere scoped_name keeps the name, so nothing changes on VM providers.

Tests

• Unit, url_on_host (test_url_rewrites.py):
  • a Modal-style tunnel, with and without a path;
  • a path-prefixed proxy with another tunnel above it, where the longest match wins;
  • a host that only starts like the tunnel's, and a URL elsewhere, both unchanged;
  • host_ips empty, with loopback, non-loopback only, and IPv6;
  • a real LocalSandbox port map.
• Unit, input["env"] from four kinds of record: a gateway env, a lone server, a plugin's env whose card puts MCP at /v2/mcp, and a card-less older record.
• Unit, through execute:
  • an env-host script's staged input.json;
  • an agent script gets no env;
  • a container env's script really runs in the container, with no sudo, docker or host-file writes, and gets loopback URLs.
  • The VM sandbox mock is now specced on VmSandbox, since the target is chosen by class.
• Unit, the run directory: a step run on two real local sandboxes stages into a directory of each sandbox's own.
• Slow tier: the reach test now runs against both a gateway env and a lone-server env (env.toml env_provider_type) on the local provider. It checks that url is loopback at the mapped port, fetches the card at url, and opens an MCP session at mcp_url. Both pass on macOS.
• Unit tier: 8,339 passed.

Rollout

bump-version: this is on the hot path of every run_code step on the worker. The chaos results are in one comment below. They cover Linux and macOS VMs plus live modal_vm, beta_scale and Modal-container runs, with gateway and server envs, old and new code.

For a stored task's scripts to use this, they read input["env"]["url"], keeping their current default as a fallback. CoopBench's stored scripts are left as they are, so a materialized copy still needs that one-line edit to run on the local provider.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue was found.

Summary

Gives env-host scripts the env's address and MCP endpoint, rewritten for where the script runs.

• Env scripts receive addresses they can use to reach the env.
• Each host run stages its files in a sandbox-specific folder.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["run_code with env_id"] --> B["Load deployed env and sandbox"]
  B --> C{"VmSandbox?"}
  C -->|Yes| D["Run on host"]
  C -->|No| E["Run inside container"]
  B --> F["Rewrite env URLs through url_on_host"]
  F --> G["Stage input.json with args, results, and env"]
  D --> G
  E --> G
  G --> H["Run script and read result"]

Loading

_{Reviews (4) · Last reviewed commit: "feat(run_code): give an env-host script ..." · Reviewed by Greptile}

[earakely-scale]

Chaos and live verification

The same battery ran on the old release (agentenv-framework 0.9.1332, from PyPI) and on this PR (7de72ab; 1bbbbc9 only rewraps a docstring).

• Hosts: an Ubuntu 24.04 VM (m5.2xlarge) and a stock macOS 15.8 VM (mac-m4.metal), both on the local provider. Live modal_vm, beta_scale and Modal-container runs were driven from a laptop with the sdk's dev config.
• The probe: the echo env, deployed by the gateway provider and by the server provider (one MCP server, no gateway), plus a run_code step on its host. The script records its input, fetches the card at input["env"]["url"], opens an MCP session at mcp_url, and also tries the old 127.0.0.1:18765 default.
• Remote VMs: modal_vm and beta_scale run a server's image by name. They don't build a bundle's Dockerfile; that's existing behaviour. So their server runs used a stored env whose image was in the dev registry, imported with bundle init --from-task. The test image and repository have since been deleted.

Local provider:

	Old	New
Gateway env, ×1 and 4 at once ×3, Linux + Mac (26 runs)	no env; 127.0.0.1:18765 refused in every run that reached the script	26/26 completed; each url is its own deployment's mapped port (13 distinct per host); card and MCP reached
Server env, same (26 runs)	no env; 127.0.0.1:18765 refused in every run that reached the script	26/26 completed; same as above
4 at once, old (16 runs)	6 failed on the shared run dir (runner.py, input.json, script.py or output.json gone)	0 of 52

Live:

	Old	New
modal_vm: gateway (bundle and stored), server (stored)	127.0.0.1:18765 works	url = http://127.0.0.1:18765; card, MCP and the old default all work
beta_scale: gateway (bundle and stored), server (stored); stored URL https://sandbox.internal.scale.com/sandbox/sandbox-vm-…-18765	127.0.0.1:18765 works	http://127.0.0.1:18765; card, MCP and the old default all work
Modal containers: gateway (bundle), server (bundle and stored)	all 3 fail: 'ModalSandbox' object has no attribute 'exec_script'	all 3 complete: the script runs in the env's container and reaches it at 127.0.0.1:18765

CoopBench on Linux and the stock Mac:

• Its scripts as stored (hard-coded 127.0.0.1:18765): old and new both fail at world-warmup with the same Connection refused, so nothing changes for them.
• A local copy edited to read input["env"]["url"] first (the stored artifacts are untouched): 18/18 steps on new, on both hosts. world-warmup took 5.3 s and 6.7 s, collect-evidence passed, and no forwarder was needed.

Unchanged: #221's run_code battery on the new code, VM and container mode, on both hosts. Success, a 20 KB crash, exit 3, a timeout with children, and a script that ignores SIGTERM (7.1–7.9 s) give the same results, error sizes and timings as the last round, with nothing left running.

Clean:

• No scoped run dir was left after any run.
• The run key appears in no log.
• The VMs, their network and the test registry repository are torn down.

smol_vm and sail_vm

These ran on an Ubuntu VM (m8i.2xlarge with nested virtualization, so smol gets /dev/kvm) and on the stock Mac, both driving each provider. Old is 0.9.1332; new is 7de72ab.

url_on_host on real sandboxes, both hosts. Each provider created a VM exposing two ports (18765 and 18080), each serving a marker of its own from inside the VM. The sandbox was then re-fetched by id, as run_code does:

	smol_vm	sail_vm
Tunnel form	http://127.0.0.1:<random>, a port forwarded on the driver	https://sb-<id>-<port>.sail.box
Tunnels after re-fetch	same as at creation	same as at creation
Mapped URL, each port with and without a path, from inside the VM	8/8 reach their own port's marker at 127.0.0.1:<port>	8/8
Unmapped tunnel URL, from inside the VM	8/8 Connection refused, so the mapping is needed	8/8 work: Sail's public URL is also reachable from inside the Sailbox
Tunnel from the driver	8/8	8/8
A URL under no tunnel	unchanged	unchanged

The reach task:

	Old	New
smol_vm, gateway and server, alone and 2 at once, both hosts (12 new, 8 old)	7/8 via 127.0.0.1:18765 inside the guest; the 8th is a smol VM that didn't start (krun_start_enter … EINVAL) during a concurrent start on Linux, before any step	12/12: each record's forwarded tunnel maps to the guest's 127.0.0.1:18765; card and MCP reached
sail_vm, gateway, alone and 2 at once, both hosts (6 new, 2 old)	2/2 via 127.0.0.1:18765	6/6: each https://sb-…-18765.sail.box maps to 127.0.0.1:18765; card and MCP reached
sail_vm, server env from a bundle	refused before any step: Sail runs a server's image by name and doesn't build a bundle's Dockerfile (existing behaviour)	same