feat(task-steps): per-run step_overrides for the solver agent, model and effort

[earakely-scale]

What

DeployAgentTaskStep now reads a2a_agent_id / a2a_agent_version, and PromptAgentTaskStep reads model / effort, from the run's step_overrides entry for their own step id. These win over the run-wide a2a_agent_id / agent_model / agent_effort overrides and the stored values. An overridden agent id takes the override's version (latest when unset), never the stored agent's pin.

Both steps use the existing TaskStep.step_param_overrides seam; the hub and the explorer already forward step_overrides into user_overrides.step_params, so no caller changes are needed.

Why

A task that deploys several agents (a solver, a user simulator, judges) could so far be run under a different solver only by registering a new task version per harness, because the run-wide a2a_agent_id override swaps every deploy_agent step at once. With a per-step override, one task version runs under any solver harness, model and effort, and the judges keep their stored agents.

Example run body:

{"version": 17,
 "step_overrides": {"my-task-deploy-solver": {"a2a_agent_id": "codex-cli"},
                    "my-task-prompt": {"model": "gpt-6-astra", "effort": "xhigh"}}}

Tests

tst/unit/task_step/test_step_overrides_agent_and_model.py (12 cases: stored values, step override with and without a version, version-only, step over run-wide, run-wide alone, other steps' overrides ignored; model and effort precedence, partial override, default-model fallback). The deploy-agent, prompt-agent and overrides unit tests in tst/unit/task_step pass (272).

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

Adds per-step agent, version, model, and effort overrides without changing stored task definitions.

• Runs can pick the solver agent and prompt settings by step id.

No actionable issues were found. Tests were inspected, not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Run["Run step_overrides"] --> Params["user_overrides.step_params"]
  Params --> Step["Select entry by step id"]
  Step --> Deploy["Resolve agent id and version"]
  Step --> Prompt["Resolve model and effort"]
  Deploy --> Lookup["A2AAgent.get"]
  Lookup --> Agent["Deploy selected agent"]
  Prompt --> Config["Send supported agent settings"]
  Prompt --> Response["Save selected model in PromptResponse"]

Loading

_{Reviews (1) · Last reviewed commit: "feat(task-steps): per-run step_overrides..." · Reviewed by Greptile}

Chaos

Timeboxed battery on three fresh Ubuntu VMs (24.04 amd64, 24.04 arm64, 22.04 amd64; Docker 29.9, Python 3.12), each with this branch and main installed side by side, local stores and the local sandbox, two registered echo agents (echo, echo2) and a task of two deploy_agent steps (solver, judge) plus one prompt_agent per agent. 114 runs in total, 38 per VM.

scenario	result (all three VMs)
no overrides; run-wide a2a_agent_id; run-wide agent_model + agent_effort	stored behaviour unchanged
step_overrides on the solver's deploy step	solver on echo2, judge stays on echo
step_overrides model/effort on the solver's prompt step, with a run-wide agent_model set	solver prompt on the step model (confirmed on the agent's own config read-back), judge prompt on the run-wide model
step override wins over a run-wide a2a_agent_id; version-only override; override for a step id not in the task	as specified / as specified / ignored
unknown agent id, unknown agent version in a step override	fails at the deploy step in <1 s with the store's not-found error, no sandbox created
four runs with different overrides concurrently	no cross-talk
deploys and prompts run as separate Task.run halves with the context serialised in between (the worker's shape)	overrides still applied in the second half
runner SIGKILLed mid-deploy; agent containers removed under a live run; Docker restarted under a live run	the live run fails cleanly (the existing 60 s liveness window), no hang; the next run on the same state passes; no containers left behind
soak, 6 × (agent swap + model swap) per VM	36/36
main baseline, same harness	per-step overrides ignored, everything else identical

Unit tier tst/unit/task_step on the VMs: 1,642 pass on the branch, 1,630 on main (the 12 new tests), with the same 3 environment-dependent failures on both (a local-Docker zip test and two MCP CLI codegen tests).