Pass attachments to subagents directly and set subagent mode

[hanna-paasivirta]

Short Description

Attachments (run logs, dataclips) now reach subagents as the exact text the user sent. Before, only the planner's own message got through, so a subagent saw a summary of a log, or nothing. Attachments also no longer stick to the conversation history, and oversized ones are refused rather than quietly cut down.

Fixes #643

Implementation Details

The router used to splice attachments into the user's message as text. Nothing carried them any further if the planner was called. So if a user attached a run log and asked "why did the last two steps fail?", the job_chat call got the planner's summary of the log instead of the log. The workflow YAML already travels as a payload field, and now attachments do as well.

Each agent receives them the way it already receives that kind of context.

• job_chat is unchanged — byte-identical to main. It already had context.log / context.input / context.output, which render as <run_logs> / <input> / <output>. Lightning already uses them. Nothing to regress in a production service.
• workflow_chat and the planner have no such fields. They take an attachments payload field and render it where their other context goes. For workflow_chat that is the end of the system message, next to existing_yaml, and only when attachments are present. With none, its prompt is byte-identical to before.
• There are five attachment types but three context fields. input_dataclip and run_input both mean input; the two output types likewise. When two attachments want the same field they are joined, not overwritten, so neither is lost. This clash comes from the mapping. It does not exist on main, where each field has one source.
• An attachment type with no matching field is logged and not passed on. This matches how the service already works: job_chat checks a fixed list of context keys rather than reading the whole dict, so an unknown key is ignored today too. A new type needs one line in the mapping.

Who decides what a subagent gets depends on the route.

• The router forwards everything to whichever single subagent it picks. There is no one else to judge.
• The planner names, per call, which attachments that subagent must read itself. A subagent sees only what it is handed. So a step told "change state.patients to state.cases" is not billed for a log the planner has already read. Both tools take a required attachments list of types.
• Named or not, what travels is the original content. The planner is told to hand over the reading that needs the subagent's expertise, and keep what it has already turned into an instruction.

Attachments no longer stick to the history. The enriched string used to be what got saved, so an attached log became a permanent turn. It was re-sent on every later turn with nothing to show it belonged to an earlier run. Attachments now belong to the turn they arrived on. That makes re-sending them the client's job, which is a contract change for Lightning and is documented in PAYLOAD_SPEC.md.

Planner-invoked subagents were running in production mode.

• call_job_agent never set subagent: True. It also put the YAML in context["workflow_yaml"], which Payload.from_dict does not read. So job_chat ran under the production scope prompt ("You ONLY help with job code... tell them to navigate to the workflow overview"), with no <workflow_structure> block and no inspect_job_code tool. The router's direct route set both correctly.
• call_workflow_agent had the same defect. It kept its "save your workflow and go to the Inspector" instruction. Both now run in subagent mode.
• One consequence. Subagent mode gives job_chat the edit_workflow escalation tool, and its target is the planner — which is now the caller. A handover returns an empty response, so format_subagent_result_for_llm turns it into the reason that agent could not finish, for the planner to act on with a different tool.

Attachments are never trimmed. Oversized ones are refused. Shortening an attachment would mean answering from evidence the user thinks we read in full. Context Apollo injects itself, like adaptor docs, is a different case and is still truncated.

• A turn totalling more than 250,000 characters is rejected with 400 ATTACHMENT_TOO_LARGE before any model is called. The error names the largest attachment so the client can offer a shorter selection.
• The limit is what a subagent prompt can hold once the context window, the max_tokens reserve, the static prompt and adaptor docs are accounted for. The arithmetic sits next to the constant so it can be re-derived if the model or window changes.
• It applies to the total across all attachments, because the prompt carries them together.

Adds unit tests for the mapping, the selection, the size guard and the subagent payloads. Also adds the repo's first service-tier tests: the whole router → planner → job_chat chain runs with every LLM call scripted. The attachment carries an improbable canary, so "did this arrive" is a check on bytes rather than a judgement on wording. No tokens, no human review. One acceptance spec covers the planner reasoning from a log across steps. The attachment types in the existing specs now match PAYLOAD_SPEC.md. The four test tiers are described in services/testing/README.md; they previously existed only in pyproject.toml markers and the root conftest.py.

Left alone on purpose: a genuinely too-long prompt still surfaces differently per route. job_chat gives a clean PROMPT_TOO_LONG, workflow_chat a raw BAD_REQUEST, the planner a 500 "Tool execution error". Attachments can no longer cause it, but a long conversation still can, so it wants its own change.

AI Usage

Please disclose whether you've used AI in this work (it's cool, we just want to
know!):

• Yes, I have used AI
• No, I have not used AI

You can read more details in our
Responsible AI Policy