feat(workflows): add opt-in output capture to command and prompt steps

[mehrdadbozorgmehr]

Description

Closes #4613.

command and prompt steps stream the integration CLI's output to the terminal, so steps.<id>.output.stdout and stderr are always empty. A later step cannot parse or branch on agent output, even though the workflow reference shows steps.<id>.output.stdout for a prompt step.

This adds opt-in, bounded capture to both steps, within the scope agreed in the issue: streaming stays the default, and shell, structured output and output mirroring are unchanged.

- id: verdict
  type: prompt
  prompt: "Reply with exactly READY or BLOCKED"
  capture: true
  capture_max_bytes: 4096   # optional; default 1048576, -1 = no limit
  timeout: 120              # optional

Contract

Condition from the issue	Implementation
Limits	capture_max_bytes defaults to 1 MiB per stream. It accepts a positive integer or -1, and rejects 0, values below -1, and booleans. With -1 nothing is truncated and both flags stay false. The docs warn about memory and run-state growth.
Encoding and bounded capture	The limit counts raw bytes before decoding. Invalid UTF-8 becomes U+FFFD. A character split by the limit, a timeout or an interruption is dropped rather than replaced. Both streams are drained by reader threads after the cap, so the process cannot block on a full pipe; memory stays bounded by the limit.
Truncation	stdout_truncated and stderr_truncated report the size limit only. timed_out and interrupted report a stopped process, whose partial output is kept even with no limit. The docs say from_json does not read these flags and show how to check completeness first.
Retention	Captured text is persisted in state.json and is not redacted; the docs carry a prominent warning. The step error is an explicit summary (Command exited with code 2, Prompt timed out after 120 seconds., Interrupted by user), so captured stderr never reaches log.jsonl or the terminal.
Dispatch	dispatch_command() gains output_json and capture_max_bytes keyword arguments in IntegrationBase and in Copilot's override. Their defaults keep existing callers byte-for-byte unchanged: stream=False alone still requests JSON and still raises TimeoutExpired. If an installed integration overrides dispatch_command() with the older signature, a captured command step fails with a clear error instead of a TypeError.
Timeouts	prompt keeps its existing timeout. A captured command step defaults to 600 seconds; streaming command steps keep no timeout, and their timeout value stays unvalidated as before. A timeout or interruption fails the step cleanly and keeps the bounded partial output.

Captured text is the integration's normal textual stdout. It is not guaranteed to be identical across integrations, to contain only the final answer, or to be valid JSON.

Testing

Minimal reproduction of the composition gap

I ran this through specify workflow run --json with a fake claude executable first on PATH (#!/bin/sh + echo READY), so the result does not depend on a live agent:

schema_version: "1.0"
workflow: {id: capture-gap, name: Capture gap, version: "1.0.0", integration: claude}
steps:
  - id: verdict
    type: prompt
    prompt: "Reply with exactly READY or BLOCKED"
    capture: true
  - id: plan
    command: speckit.plan
    capture: true
  - id: ready
    type: if
    condition: "{{ (steps.verdict.output.stdout | contains('READY')) and (steps.plan.output.stdout | contains('READY')) }}"
    then:
      - id: report
        type: shell
        run: "echo both steps captured READY"

	verdict.stdout	plan.stdout	report ran
main (96774bc)	'' (printed to the terminal)	'' (printed to the terminal)	no
this branch	'READY\n'	'READY\n'	yes

Automated tests

tests/integrations/test_capture.py covers the capture helper and the dispatch contract, including Copilot. tests/workflows/test_step_capture.py covers both steps. Together they cover:

• finite limits, unlimited capture, output exactly at the limit, and draining a 3 MB stream past the cap;

• a multibyte character split by the limit, invalid UTF-8, and line endings;

• non-zero exits, timeouts, and interruption with partial output;

• validation, both through validate() and through execute() without prior validation;

• an integration whose dispatch_command() has the older signature;

• streaming behavior unchanged, including the terminal output and the absence of the new fields;

• existing dispatch_command(stream=False) callers unchanged;

• downstream consumption before and after capture;

• resume across a gate, with no second dispatch;

• captured stderr present in state.json but absent from log.jsonl and the terminal;

• continue_on_error letting a later step inspect a timed-out capture.

• Tested locally with uv run specify --help

• Ran existing tests with uv sync && uv run pytest: run through .venv/bin/python -m pytest tests, as CONTRIBUTING recommends. 11,999 passed and 28 failed. All 28 are PowerShell tests that fail identically on main in this environment (WSL execution policy blocks powershell.exe scripts). ruff@0.15.0 check src tests and markdownlint on the changed docs pass.

• Tested with a sample project (if applicable): the reproduction above, through the CLI with a stub agent. I also ran a captured prompt step against the live Claude Code CLI (claude -p). It completed with stdout: 'READY\n', nothing printed to the terminal, and the downstream if branch ran. The CLI's own stdin notice (Warning: no stdin data received in 3s…) landed in output.stderr as designed. Streaming steps inherit stdin the same way, so that notice is not specific to capture. I did not run a live command step, because that would execute a real /speckit-plan; the command path is covered by the stub reproduction and the tests.

AI Disclosure

• I did not use AI assistance for this contribution
• I did use AI assistance (fill in the disclosure below)

AI disclosure: Implemented with Claude Code (model: Claude Opus 5.5, default settings) in autonomous agent mode. The agent did the code generation, tests, documentation, the reproduction and this PR description. I, @mehrdadbozorgmehr, directed the work, and the scope and contract follow the maintainer's conditions in #4613.

[Copilot]

🟡 Changes recommended

Timeout and interruption handling leaves spawned descendant processes and reader threads running.

1 open finding

• Terminate subprocess trees and reader threads on timeout or cancellation · New

What changed in this PR

Adds bounded, opt-in output capture for workflow command and prompt steps.

Changes:

• Introduces bounded subprocess capture with timeout and truncation metadata.
• Integrates capture into command and prompt execution.
• Adds documentation and comprehensive tests.

File	Description
src/​specify_cli/​integrations/​_capture.py	Implements bounded process capture.
src/​specify_cli/​integrations/​base.py	Extends dispatch capture options.
src/​specify_cli/​integrations/​copilot/​__init__.py	Supports capture in Copilot dispatch.
src/​specify_cli/​workflows/​step/​_capture.py	Shares capture validation and result handling.
src/​specify_cli/​workflows/​step/​command/​__init__.py	Adds command-step capture.
src/​specify_cli/​workflows/​step/​prompt/​__init__.py	Adds prompt-step capture.
tests/​integrations/​test_capture.py	Tests capture and dispatch behavior.
tests/​workflows/​test_step_capture.py	Tests workflow capture behavior.
docs/​reference/​workflows.md	Documents the capture contract and risks.
workflows/​README.md	Adds a concise usage example.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mnriem]

Please address Copilot feedback and address test & lint errors

[mehrdadbozorgmehr]

@mnriem Thanks. Both are addressed in dec5943:

• Copilot finding: a timeout or Ctrl+C now kills the CLI's whole process tree, not just the CLI, and the readers stop before the step returns. I've replied with the details on the inline comment.
• Test failure: pytest (windows-latest, 3.14) failed 5 tests, and windows-latest, 3.13 was cancelled after it. All 5 compared streamed terminal output with \n, but Windows writes \r\n, so the tests now normalise it. Ruff passed on the previous run. Locally, ruff 0.15.0 and markdownlint pass, and tests/workflows, tests/integrations and tests/test_workflow*.py pass (4,517 passed). I couldn't run the Windows job-object path locally, so CI is the first Windows check of it.

AI disclosure: this comment was drafted with Claude Code (Claude Opus 5.5, autonomous mode).

[Copilot]

🟡 Changes recommended

Windows process startup has a race that can leave descendants running after timeout or interruption.

1 open finding

• Start Windows CLI suspended before assigning it to the job object · New

1 resolved since last review

• Terminate subprocess trees and reader threads on timeout or cancellation

🧠 Review effort: Balanced

[mnriem]

Please address Copilot feedback

[mehrdadbozorgmehr]

@mnriem Addressed in 34ca95a. Copilot's second finding was that on Windows the CLI ran briefly before it joined its job object, so anything it started in that window survived a timeout. The CLI now starts suspended and is only resumed once it is in the job. If that setup fails, the CLI is killed before it runs. Details are on the inline thread. On the previous commit, ruff, markdownlint, CodeQL and the Ubuntu and macOS pytest jobs all passed.

AI disclosure: this comment was drafted with Claude Code (Claude Opus 5.5, autonomous mode).

[Copilot]

🔵 Needs a closer look

The documentation overstates descendant termination guarantees beyond the implemented process-group and job boundaries.

0 open findings

1 resolved since last review

• Start Windows CLI suspended before assigning it to the job object

Previously missed (2)

In code that hasn't changed since last review

Document process-group kill boundary and detachment caveat

docs/​reference/​workflows.md:1084

This overstates the timeout guarantee. The capture helper explicitly allows a descendant that leaves the POSIX session/process group (or Windows job) to survive the kill boundary, so users should not rely on every spawned process being stopped—especially for CLIs that daemonize helpers. Document the actual process-group/job boundary and its detachment caveat.

Clarify descendant-kill contract and process isolation boundary

src/​specify_cli/​integrations/​_capture.py:132

The class contract makes the same absolute descendant-kill claim as the reference, but the implementation later acknowledges that a descendant can leave the session/job and survive. Describe the actual isolation boundary here so future callers do not treat this helper as a complete process-tree guarantee.

This issue also appears on line 323 of the same file.

🧠 Review effort: Balanced

[mnriem]

Please address the previously missed Copilot findings and fix test & lint errors

[mehrdadbozorgmehr]

@mnriem Addressed in e0346e0.

Previously missed Copilot findings (2): both said the docs claimed every process the CLI starts is killed on a timeout. The real boundary is the CLI's process group (POSIX) or job object (Windows), and a helper that detaches from it survives. The timeout row in docs/reference/workflows.md, the _ProcessTree docstring and the run_captured docstring now describe that boundary and the detachment caveat.

Test failure: pytest (windows-latest, 3.14) failed one test, test_windows_cli_is_in_the_job_before_its_first_instruction, and windows-latest, 3.13 was then cancelled. The bug was in the test. It called GetCurrentProcess() through ctypes without a declared return type, so the 64-bit pseudo-handle was truncated and IsProcessInJob always reported False. A CI runner can also already run inside a job, so that check could not prove the race was closed anyway. It is replaced by test_windows_cli_does_not_run_before_it_is_in_the_job. That test checks that the CLI has not created its marker file before job setup, then that it runs normally afterwards.

The same Windows run passed the tests that exercise the fix itself: the timeout and interrupt descendant-kill tests, the CLI-exits-but-child-holds-the-pipes test, and the fail-closed test (11,885 passed). Ruff and the Lint workflow passed, so I couldn't find a lint error. If there's one I've missed, please point me to it. Locally, ruff@0.15.0 check src tests and markdownlint pass. The replacement test is Windows-only and could not run here, so this CI run is its first run.

AI disclosure: this comment and the changes were drafted with Claude Code (Claude Opus 5.5, autonomous mode).

[Copilot]

🔵 Needs a closer look

Interruptions during process setup can leave the CLI or its descendants running.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Terminate job before closing it after partial thread resume

src/​specify_cli/​integrations/​_capture.py:246

After ResumeThread succeeds, _resume_threads() can still raise KeyboardInterrupt before returning. This handler then only closes the job handle; because the job has no kill-on-close limit, the now-running CLI can spawn descendants that survive the outer self.proc.kill(). Terminate the job before closing it so setup still fails closed after any partial resume.

This issue also appears on line 338 of the same file.

🧠 Review effort: Balanced

[mnriem]

Please address the previously missed Copilot feedback

[mehrdadbozorgmehr]

Addressed Copilot's latest "previously missed" finding (Ctrl+C during process setup) in baa78b4. Both places it flagged are fixed:

• _capture.py:246, _start_in_job: if the failure or interruption comes after the CLI has been resumed, the job is now terminated before its handle is closed. Closing the handle alone did not kill anything, because the job has no kill-on-close limit.
• _capture.py:338, run_captured: the reader threads now start inside the try block, so an interruption at that point is handled like any other. The process group or job is killed and the result reports interrupted. Pipes whose reader never started are closed.

New tests in tests/integrations/test_capture.py:

• test_interrupt_while_starting_the_readers_kills_descendants: interrupts the reader startup once the CLI has started a descendant, then checks that the descendant is killed and no reader thread is left. Against the previous code, the KeyboardInterrupt escapes run_captured and the tree keeps running. With the fix, the test passes.
• test_windows_interrupt_after_resume_kills_the_job (Windows only): interrupts after the real resume, once a descendant exists, then checks that the descendant is killed. It could not run locally, so this CI run is its first run.

Locally, tests/workflows, tests/integrations and tests/test_workflow*.py pass (4,518 passed). ruff@0.15.0 check src tests and markdownlint pass.

AI disclosure: this comment and the changes were drafted with Claude Code (Claude Opus 5.5, autonomous mode).

[Copilot]

🔵 Needs a closer look

An interrupt during process setup escapes capture handling, contradicting the documented step-result contract.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Handle KeyboardInterrupt during process setup in run_captured

src/​specify_cli/​integrations/​_capture.py:342

_ProcessTree is constructed before the KeyboardInterrupt handler, so an interrupt during process/job setup escapes run_captured() instead of returning interrupted=True. The Windows test at tests/integrations/test_capture.py:265-282 explicitly demonstrates this by expecting KeyboardInterrupt; in a workflow the engine then pauses the run and never records the captured step's exit_code: 130 or partial output, contrary to the documented contract that interruption fails the step and preserves capture. Handle interruption around construction as well, while retaining the constructor's fail-closed cleanup, and update that test to assert a CapturedProcess result.

🧠 Review effort: Balanced

[mnriem]

Almost there. Please address the previously missed Copilot finding. If not applicable please explain why

[mehrdadbozorgmehr]

@mnriem Thanks. The remaining previously missed Copilot finding, "Handle KeyboardInterrupt during process setup in run_captured" (_capture.py:342), applies and is fixed in 44f0a1a.

An interrupt while _ProcessTree was being set up escaped run_captured() as KeyboardInterrupt. The workflow then paused instead of failing the step with exit_code: 130 and interrupted: true, as the capture contract documents. run_captured() now catches the interrupt around construction and returns that interrupted result with nothing captured. Setup still fails closed: on Windows the constructor terminates the job and kills the CLI before the interrupt reaches run_captured().

Tests:

• test_interrupt_during_process_setup_is_reported_not_raised: against baa78b4 the KeyboardInterrupt escapes, and with the fix the test passes.
• test_windows_interrupt_after_resume_kills_the_job (Windows only) now expects the interrupted result instead of the raise that Copilot pointed out. It still checks that the descendant is killed.

Locally, tests/workflows, tests/integrations and tests/test_workflow*.py pass (4,519 passed). ruff@0.15.0 check src tests passes.

AI disclosure: this comment and the change were drafted with Claude Code (Claude Opus 5.5, autonomous mode).