Skip to content

fix(gateway): never read a gateway result whose exec transport failed - #105

Merged
psinghal20 merged 2 commits into
mainfrom
fix/gateway-exec-transport
Oct 8, 2026
Merged

psinghal20 merged 2 commits into
mainfrom
fix/gateway-exec-transport

Conversation

@psinghal20

@psinghal20 psinghal20 commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A VM provider reports an exec whose transport failed (a lost host, a dropped connection) as exit -1, with partial or no output; VmSandbox.exec_script already retries on it. EnvironmentGatewayProvider ignored that exit code on its in-VM reads:

  • _get_container_id read the empty result as "no container".
  • read_trajectory parsed whatever stdout came back. An agent check (for example verify_a2a_agent_mcp) could then record a partial or empty tool-call history as the real one.

This was raised in review on #83 and split out of it, because it's a gateway issue, not a Sail one.

Which providers it affects

The code is the shared gateway path for every VM-mode provider. Behaviour changes only where a lost transport comes back as -1:

  • modal_vm: its exec returns -1 when the websocket closes without an exit frame, possibly with partial stdout. Today that partial history is parsed.
  • sail_vm: a cut stream returns -1 with no stdout. Today that reads as no container, so no history.
  • e2b: a transport failure raises an SDK exception, so the gateway already fails loudly. Unchanged.
  • local: execs are local subprocesses. Unchanged in practice.

Changes

  • Retry, then raise. read_trajectory and _get_container_id retry a read that exited -1, three attempts in all. If it keeps failing they raise RuntimeError, rather than returning a partial history or reading as "no container". Other exit codes behave as before, so a gateway with no history yet still reads as [].
  • Best-effort log dumps. The deploy-time gateway and sidecar log dumps now go through _log_service. A lookup or docker logs call that fails is logged (Skipping <service> logs: …) and never aborts a deploy whose gateway is healthy. Before this change those dumps used the lookup directly, so making it raise would have let a diagnostic stop a deploy.

Testing

  • Unit: make unit-test gives 6722 passed, 13 skipped. New tests in env_gateway_provider_test.py:
    • retry, then succeed;
    • raise after three failed reads, for both trajectory reads and container lookups;
    • a gateway with no history still reads as empty;
    • a log lookup or docker logs failure never stops a deploy, and a docker logs that exits non-zero is reported as a warning;
    • a service without a container is still reported as missing.
  • Live: gateway_test.py::test_gateway_with_multi_mcp_server[sail_vm] passed against Sail in 2m12s, covering the deploy, log dumps, MCP tool calls and /step.

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The PR appears safe to merge.

Summary

The gateway retries VM reads that end with exit -1 and raises if three attempts fail. Deploy-time log dumps now warn and continue when diagnostic reads fail.

  • Gateway history reads retry lost connections before using results.
  • Gateway deployments continue when diagnostic log reads fail.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Read gateway data] --> B{Exec returned -1?}
  B -->|Yes| C{Attempts left?}
  C -->|Yes| A
  C -->|No| D[Raise an error]
  B -->|No| E[Use the result]
  F[Dump deploy logs] --> G{Lookup or logs failed?}
  G -->|Yes| H[Warn and continue]
  G -->|No| I[Print logs and continue]
Loading

Reviews (2) · Last reviewed commit: "fix(gateway): address greptile review fi..." · Reviewed by Greptile

A VM provider reports an exec whose transport failed (a lost host, a
dropped connection) as exit -1, with partial or no output. The gateway
ignored that exit code: _get_container_id read the empty result as no
container and read_trajectory parsed whatever came back, so an agent
check could record a partial or empty tool-call history as real. It
affects every VM provider that reports a lost transport as -1, notably
modal_vm and sail_vm.

- read_trajectory and _get_container_id retry a read that exited -1 and
  raise after three attempts. Other exit codes keep their behaviour, so
  a gateway with no history still reads as [].
- The deploy-time gateway and sidecar log dumps go through a best-effort
  _log_service, so a lookup or docker logs call that fails is logged and
  never aborts a healthy deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@psinghal20
psinghal20 requested a review from a team as a code owner October 7, 2026 21:44
Comment thread src/agent_env/providers/env_providers/env_gateway_provider.py
- _log_service reports a docker logs call that exits nonzero as a
  "Skipping <service> logs" warning with its stderr, instead of printing
  its output as the service's logs; the deploy still goes on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@psinghal20
psinghal20 merged commit c7c6ff6 into main Oct 8, 2026
13 of 14 checks passed
@psinghal20
psinghal20 deleted the fix/gateway-exec-transport branch October 8, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants