[Copilot]

gh-aw resolves the main model:, but not Copilot sub-agent models. An alias or a retired id (gpt-5.4-mini) can silently run on the main model in SDK mode. In CLI mode, AWF rewrites it at request time or the request fails mid-run. Nothing records which model the sub-agent was meant to use.

This PR makes gh-aw resolve sub-agent models before Copilot starts, record the result, and fail closed when a model can't be resolved.

Resolution (copilot_harness.cjs)

• New function resolveCopilotSubagentModels. For each sub-agent that declares model:, it calls resolveEngineModel with engine: copilot, role: <agent name>, Copilot provider scope, the AWF apiProxy.models alias map, and normalizeForCopilotCLI.
• Live-catalogue check. It then checks the model against the live /reflect catalogue, the same check the main SDK model uses. One /reflect refresh is shared across all sub-agents.
• Coverage. It runs in both CLI and SDK mode, including routed runs, and covers inline ## agent: blocks and .github/agents/*.agent.md.
• Records. Each sub-agent gets a model.resolution record. When several sub-agents share a dead model, each gets its own failed record.
• Fail closed.
  • The record gets status: failed, the cause (model_unavailable or the resolver's cause) and resolved_model: null.
  • The log shows the shared ModelAliasResolutionError message, and the harness exits 1 before Copilot starts.
  • Every failed sub-agent is recorded before the harness exits.
• No model:. The sub-agent inherits the main model, as before, and gets no record.

CLI mode

• The resolved id is written into $GITHUB_WORKSPACE/.github/agents/<file> as model: "<id>". Only that line changes, and only when the resolved id differs from the declared one.
• Git-tracked agent files are marked git update-index --skip-worktree, so the rewrite can't reach the agent's commits or create-pull-request / push-to-pull-request-branch patches.
• If the file can't be hidden, the harness restores it and exits 1.
• Inline sub-agents are untracked files that the restore step copies in, so they need no git step.

SDK mode

• The resolved id must qualify against the SDK provider models (qualifyModelForMultiProvider). For example, small becomes copilot-<wire>/claude-haiku-5.5.
• The harness passes the resolved models to the driver as GH_AW_COPILOT_SDK_SUBAGENT_MODELS, a JSON map keyed by agent name.
• copilot_sdk_session.cjs:
  • Removed the hard-coded small/medium/large → session-model mapping.
  • Removed the silent fallback to the session model and the subagent.model_unavailable driver event. Go audit still reads that event from older runs.
  • A sub-agent model that doesn't qualify now throws instead of falling back.
• medium is not an alias in gh-aw or AWF, so it is treated as a literal model id and fails unless that model is live.

Unchanged

• The main-model paths: applyCopilotModelAliasResolution and resolveCopilotSDKMultiProvider.
• Routing, fallback-models and allowed-models.
• Pi, Claude and Codex sub-agents.
• extract_inline_sub_agents.cjs and experiment interpolation.
• Workflow model pins.

Behaviour changes for reviewers

These SDK sub-agents ran on the main model. They now run on the alias's model, as resolved against the catalogue of run 39304573724:

Workflow	Sub-agent	model:	Before	After
daily-firewall-report	firewall-chart-generator, firewall-data-aggregator	small	main (default)	claude-haiku-5.5
jsweep	file-triage	small	main (default)	claude-haiku-5.5
pr-code-quality-reviewer	grumpy-coder	small	copilot/gpt-5	claude-haiku-5.5
uk-ai-operational-resilience	asset-tier-classifier, control-verifier, ai-risk-scorer	small	main (default)	claude-haiku-5.5
evoskill-evolver	executor	small	main (default)	claude-haiku-5.5
evoskill-evolver	proposer, skill-builder, validator	large	main (default)	claude-sonnet-5.5
pr-description-caveman	pr-description-synthesizer	large	main (default)	claude-sonnet-5.5

Now fail at startup with model_unavailable. These sub-agents are pinned to the dead gpt-5.4-mini; the pins are tracked in #70367:

• daily-model-resolution: run-analyzer
• smoke-copilot-sdk-routed-claude: mini-whoami
• smoke-copilot-sub-agents: mini54-whoami

smoke-copilot / #70387. Until #70387 lands, file-summarizer still declares the unexpanded ${{ experiments.subagent_model }}. It now fails at startup instead of failing mid-run with HTTP 400.

Follow-up. Claude and Codex sub-agents have the same gap: gh-aw doesn't resolve their models. They need a separate issue.

Tests and fixtures

• Fixtures from run 39304573724, kept verbatim:
  • fixtures/awf-reflect-copilot-sub-agents-39304573724.json: 38 Copilot models, no gpt-5.4-mini
  • fixtures/awf-config-models-copilot-sub-agents-39304573724.json: the run's apiProxy.models
• Named acceptance tests for cases 1–7, both in-process and with the harness spawned. They also cover the tracked-file skip-worktree protection.

Needs maintainer dispatch on this branch

The agent can't trigger workflow runs, so these smokes haven't run yet:

• smoke-copilot-sub-agents and smoke-copilot-sdk-routed-claude: expected to fail at startup with model_unavailable.
• One SDK smoke and one CLI smoke with a live or alias sub-agent model: expected to pass with resolved or exact sub-agent records.

• Fixes Copilot sub-agent models are never resolved: aliases and retired ids silently run on the main model (SDK) or are rewritten/fail at request time (CLI) #70388

[github-actions]

🔬 Test Quality Sentinel is analyzing test quality on this pull request...

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🔎 Code quality review by PR Code Quality Reviewer

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · 11.2 AIC · ◇ 14.1 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

[github-actions]

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 24.4 AIC · ⌖ 5.39 AIC · ⊞ 14.7K · ◷

Generated by Ponytail Reviewer for #70389

[github-actions]

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Request changes

The preflight introduces two correctness regressions: CLI discovery is broader than the actual custom-agent loader and can reject unrelated .md files, and model resolution runs before the noop short-circuit, so completed/no-op workflows can fail on an agent model they would never use. Both should be fixed before merge.

Findings

• Restrict CLI discovery to .agent.md files, matching the SDK loader.
• Move CLI sub-agent resolution after the noop safe-output check.

The advisory first-pass reviewer did not return parseable findings; this review is based on independent inspection of the changed lines.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 4.17 AIC · ⊞ 1.6K
_{Comment /review to run again}

[github-actions]

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Skills-Based Review 🧠

Applied /codebase-design and /tdd to the new sub-agent model resolution logic. The overall approach (collect all failures before throwing, fail-closed instead of silent fallback) is solid and well-tested with ~15 new test cases, but two issues should be addressed before merge.

📋 Key Themes & Highlights

Key Findings

• Inconsistent agent-file filter (/codebase-design): listCopilotAgentFiles (CLI mode, copilot_harness.cjs:677) matches any *.md, while loadCopilotSDKCustomAgents (SDK mode) and the docs both treat *.agent.md as the canonical extension. This repo's own .github/agents/ directory already contains non-agent .md files (developer.instructions.md, agentic-workflows.md) that would be swept up by the CLI-mode path but ignored by SDK mode — a real behavioral divergence introduced by this PR.
• Untested fail-closed branch (/tdd): hideTrackedAgentFileRewrite's failure path (git command errors) drives the restore-and-exit(1) safety guarantee in applyCopilotCLISubagentModels, but only the success path is covered by tests ("case 6, CLI, tracked file"). Given this is explicitly called out in the PR description as a safety guarantee, the failure branch deserves direct coverage.

Positive Highlights

• ✅ resolveCopilotSubagentModels correctly collects every failing sub-agent's model.resolution record before throwing the first failure — matches the PR's stated goal of full diagnostics on fail-closed exit.
• ✅ The SDK-mode throw-on-unresolvable-model change is properly caught by runWithCopilotSDK's existing try/catch and converted to exitCode: 1, verified against the new test's expectations.
• ✅ Strong test coverage overall: alias resolution, dead models, refresh behavior, SDK qualification, and git skip-worktree protection are all exercised.
• ✅ Docs (engines.md) accurately describe the new fail-closed behavior.

Inline comments posted on both findings with suggested fixes.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 174.3 AIC · ⊞ 10.2K
_{Comment /matt to run again}

[Copilot]

🟡 Changes recommended

Unresolved correctness defects affect model selection, startup validation, and preservation of repository changes.

9 open issues

• Replace complete multiline YAML model scalar · New
• Preserve inherited model sentinel semantics · New
• Preserve provider scope during model qualification · New
• Record qualification exceptions as resolution failures · New
• Refresh SDK provider configuration after catalogue updates · New
• Discover agents from the effective engine working directory · New
• Preserve tracked agent edits after skip-worktree use · New
• Distinguish Git tracking failures from untracked files · New
• Verify skip-worktree was successfully applied · New

What changed in this PR

Moves Copilot sub-agent model resolution into the gh-aw harness, enabling startup validation and recorded model decisions in CLI and SDK modes.

Changes:

• Resolves aliases and checks live model availability, including routed runs.
• Rewrites CLI model fields and passes resolved SDK models without silent fallback.
• Adds catalogue fixtures, regression coverage, and documentation.

File	Description
docs/​src/​content/​docs/​reference/​engines.md	Documents resolution and fail-closed behavior.
actions/​setup/​js/​fixtures/​awf-reflect-copilot-sub-agents-39304573724.json	Adds recorded model catalogue.
actions/​setup/​js/​fixtures/​awf-config-models-copilot-sub-agents-39304573724.json	Adds recorded alias configuration.
actions/​setup/​js/​copilot_sdk_session.test.cjs	Supplies harness-resolved test models.
actions/​setup/​js/​copilot_sdk_session.cjs	Consumes resolved models and removes fallback.
actions/​setup/​js/​copilot_sdk_driver.test.cjs	Tests resolved models and startup rejection.
actions/​setup/​js/​copilot_sdk_driver.cjs	Documents the model-handoff environment variable.
actions/​setup/​js/​copilot_harness.test.cjs	Covers resolution, rewriting, and startup behavior.
actions/​setup/​js/​copilot_harness.cjs	Implements sub-agent validation and model handoff.

_{Review effort: Max}

[gh-aw-bot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
2. Review (actions/setup/js/copilot_harness.cjs:676): This scans every .md file, so a repository file such as .github/agents/README.md or another non-.agent.md document with a model: key can now cause startup to fail even though the SDK loader (and the custom-agent convention) only loads .agent.md files. That makes CLI and SDK behavior inconsistent and introduces false blocking failures from files Copilot will ignore. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
3. Review (actions/setup/js/copilot_harness.cjs:1689): This resolves and potentially rejects every CLI sub-agent before the existing noop short-circuit below runs. A workflow that has already emitted a noop can now fail with model_unavailable (or even a reflect refresh error) instead of exiting successfully without starting Copilot, contradicting the preflight contract documented immediately below and changing the no-op path into a model-dependent path. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
4. Review (actions/setup/js/copilot_harness.cjs:677): [/codebase-design] This CLI-mode agent-file lister accepts any *.md file, but the SDK-mode loader (loadCopilotSDKCustomAgents in extract_inline_sub_agents.cjs) only accepts *.agent.md, and the docs (engines.md line 609) say Copilot "treats .github/agents/*.agent.md files as custom agents." This repo already has non-agent .md files sitting in .github/agents/ (developer.instructions.md, agentic-workflows.md). - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
5. Review (actions/setup/js/copilot_harness.cjs:871): [/tdd] hideTrackedAgentFileRewrite is the safety net that keeps a rewritten tracked agent file out of git status/commits, but no test exercises the branch where git ls-files/git update-index --skip-worktree fails (non-zero exit or spawn error), which is what triggers the restore-original-content-and-exit(1) path in applyCopilotCLISubagentModels. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
6. Review (actions/setup/js/copilot_harness.cjs:711): [P2] Replace the complete YAML model scalar, not only its first line. The loader accepts model: >- followed by an indented small, but this rewrite leaves that continuation beneath model: "claude-haiku-5.5", producing invalid frontmatter. Copilot then starts with a corrupted agent file despite a successful resolution record. Replace the scalar's full source range while preserving other fields and the body, or reject unsupported multiline forms before modifying the file. Add a multiline-model CLI regression case. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
7. Review (actions/setup/js/copilot_harness.cjs:742): Preserve the existing inherited sentinel. The Copilot SDK workflow .github/workflows/dependabot-burner.md:405 declares it for dependabot-remediator, and model_pin_check.cjs:681 explicitly treats it as using the parent's model. This filter now sends it through literal-id validation, where it fails with model_unavailable and aborts the workflow at startup. Either remove that declaration from the workflow and recompile, or treat inherited as no model in both the harness and SDK session so it inherits the main model without a resolution record. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
8. Review (actions/setup/js/copilot_harness.cjs:801): [P1] Preserve provider scope through the catalogue check and SDK qualification. normalizeForCopilotCLI removes copilot/, so copilot/gpt-5.4-mini passes this check when only OpenAI advertises that id, and the SDK then selects openai/gpt-5.4-mini. The resolution record still reports Copilot. If both providers advertise the id, qualification instead throws an ambiguity error despite the explicit prefix. Use the resolver's provider identity for validation and SDK qualification; keep the bare id for CLI dispatch. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
9. Review (actions/setup/js/copilot_harness.cjs:801): [P2] Handle qualification exceptions through the failure-recording path. qualifyModelForMultiProvider throws rather than returning null when two configured Copilot endpoints advertise the same sub-agent id. That exception bypasses fail(): this agent keeps the successful record already written by resolveEngineModel, and later agents get no record. Catch the qualification error, record a failed resolution, and continue collecting results before throwing the aggregated failure. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
10. Review (actions/setup/js/copilot_harness.cjs:801): [P2] Refresh the SDK provider configuration along with /reflect. If the main model is present initially but a sub-agent model appears only after refresh, the liveness check succeeds while this lookup rejects the model against the old providerConfig.models. The driver JSON was also serialized before sub-agent resolution. Rebuild and propagate the provider configuration from the refreshed catalogue before qualification and driver startup. Extend case 3c to SDK mode; it currently omits providerConfig. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
11. Review (actions/setup/js/copilot_harness.cjs:836): [P2] Match CLI agent discovery to the effective engine working directory. process_runner.cjs:162 launches Copilot in GH_AW_ENGINE_CWD, but this code always scans GITHUB_WORKSPACE. When engine.cwd points at another checkout, that checkout's .github/agents models bypass resolution and recording, while unrelated workspace agents can abort startup. Use the CLI's actual discovery paths and precedence, including the restored inline-agent location. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
12. Review (actions/setup/js/copilot_harness.cjs:851): [P2] Skip-worktree hides the agent's edits as well as the harness rewrite. In a non-sparse checkout, once a tracked agent file's model is rewritten, later prompt or tool changes to that file are omitted by git add -A, git commit -a, and working-tree patch collection. Nothing clears the flag or reconciles the file after Copilot exits. A workflow tasked with updating .github/agents can therefore silently publish an incomplete change. Keep runtime agent definitions separate from tracked sources, or preserve agent edits while removing only the harness substitution and its index flag before changes are captured. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
13. Review (actions/setup/js/copilot_harness.cjs:876): [P2] Do not treat a failed tracking check as proof that the file is untracked. A spawn error or git ls-files failure, such as exit 128 for unsafe ownership or an unreadable index, returns success here. The caller then leaves a rewritten tracked file without skip-worktree protection. A later process with working repository access can include it in commits or patches. Distinguish confirmed untracked/non-checkout cases from unexpected Git errors; return false for indeterminate checks so the caller restores the file and aborts. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
14. Review (actions/setup/js/copilot_harness.cjs:882): [P2] Verify that skip-worktree actually took effect. With cone-mode sparse checkout that materializes .github, Git can clear this bit for a file present in the working tree even though update-index --skip-worktree exits 0. Returning true then allows the rewritten model into subsequent commits and safe-output patches. Sparse checkout is supported by checkout.sparse-checkout. Check the resulting index flag and return false if it is absent, so the caller restores the original file and stops. - Resolve Copilot sub-agent models in the harness (CLI and SDK) and fail closed #70389 (comment)
15. Fix failing check agent (FAILURE): https://github.com/github/gh-aw/actions/runs/39347479634/job/118384702986.
16. Fix failing check conclusion (FAILURE): https://github.com/github/gh-aw/actions/runs/39347479634/job/118385883190.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef state: a78ac41f377c4917ec6ac2c621f496cdb22cd3fb707d550007467f7a31bd0d12

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 12.4 AIC · ⌖ 7.06 AIC · ⊞ 1.1K · ◷
_{Comment /souschef to run again}

[SivaKesava1]

Review of b81266d7fc: three design changes before end-to-end runs

I reviewed the head commit, ran the five affected suites in a clean worktree (628 tests pass), and ran the new functions against the recorded /reflect capture and alias map from run 39304573724. The resolution logic itself works: small → claude-haiku-5.5, large → claude-sonnet-5.5, copilot/opus → claude-opus-5.5, and the dead gpt-5.4-mini fails closed. Three design problems remain. The first comes from the issue we wrote (#70388, step 1), so it's on us, not the implementation.

1. Only workflow-declared sub-agents should fail closed (blocking)

The PR resolves every .github/agents/*.agent.md in the checkout and stops the run when any one of them doesn't resolve, even if the workflow never uses that agent. Those files are shared with VS Code and the Copilot coding agent, and VS Code pins models by display name:

• GitHub code search finds about 7,900 .github/agents Markdown files containing (copilot). Examples: model: GPT-5.4 (copilot) in adamgell/cmtraceopen planner.agent.md, and model: GPT-5.2-Codex (copilot) in PEZ/epupp commit.agent.md.
• Against run 39304573724's catalogue, Claude Sonnet 4.5 (copilot), GPT-5 (copilot) and Auto all fail with model_unavailable.

So after this PR, every gh-aw Copilot workflow (CLI or SDK) in such a repository would fail at startup, even though nothing in the workflow changed. gh-aw's own .github/agents files declare no model:, which is why our CI doesn't show this.

Requested behaviour:

• Declared sub-agents (inline ## agent: blocks, staged under <tmp>/gh-aw/.github/agents): keep this PR's behaviour. Resolve, record, and fail closed.
• Other repository agent files: resolve when possible and use the resolved id. When resolution fails, log a warning, record model.resolution with status: failed and the cause, and leave that agent as it is today: CLI gets the file unchanged, and SDK inherits the session model as before. Do not stop the run.

@pelikhan, does that split match how you want repository agent files treated? The alternative is to fail closed on every agent file in the checkout, as the PR does now.

2. Skip sub-agent resolution in threat detection (blocking)

The detection job runs "Checkout repository for patch context" and then the same harness in CLI mode (smoke-copilot.lock.yml around L3179 and L3378). Detection never uses sub-agents, but with this PR it resolves (and in CLI mode rewrites) every repository agent file, so problem 1 also breaks threat detection. The harness already has isDetectionPhase(); sub-agent resolution should not run in that phase.

3. CLI mode: write resolved copies to Copilot's personal agents directory instead of rewriting tracked files (blocking)

The in-place rewrite plus git update-index --skip-worktree plus restore is about 420 lines and produced 13 review threads of git edge cases. Two problems are still there at this head:

• The agent's own edits to a rewritten agent file are lost. While Copilot runs, skip-worktree hides the file from git add -A / git commit -a. create_pull_request builds its patch with git format-patch from the agent's commits during the run, so an edit to that agent file never reaches the PR. Clearing the flag after the run is too late. The test "clears skip-worktree after CLI and preserves edits to a tracked agent file" asserts this: it expects git status --porcelain to be empty after the agent's edit.
• Restore is not byte-identical for block scalars. A tracked file with model: >- followed by an indented small comes back with an extra blank line after the scalar, so the file stays modified after the run. CRLF, BOM, quoted values and trailing comments do round-trip.

Copilot CLI documents a simpler mechanism (Creating custom agents for Copilot CLI): "If personal and repository custom agents have the same ID, the personal agent is used." For an agent file, the ID is the file name without .agent.md. gh-aw already writes $HOME/.copilot/settings.json and mcp-config.json before Copilot starts. So in CLI mode the harness can write each resolved agent as a copy with the same file name in ${COPILOT_HOME:-$HOME/.copilot}/agents/, with only model: changed, and never touch the checkout. That removes hideTrackedAgentFileRewrite, restoreCopilotCLIAgentFiles, the skip-worktree handling and their tests. The block-scalar bug and the lost-edit bug both go away.

The end-to-end CLI run has to confirm that Copilot CLI 1.0.98 in offline BYOK mode under AWF loads the personal copy over the repository file. Its subagent.started event should show the resolved model, and git status should show no change under .github/agents.

Merge order with #70387

With this PR, smoke-copilot's file-summarizer (whose unexpanded ${{ experiments.subagent_model }} is fixed by #70387) fails at startup instead of mid-run. Today AWF resolves the expanded small/large at request time, so #70387 is safe on its own. Merge #70387 first, then this PR.

Non-blocking

• When the round is done, update the "Behaviour changes" section and engines.md to describe the declared/repository split and the personal-agents copy.
• The two github-actions CHANGES_REQUESTED reviews are from 7f35145678; I'll dismiss them once their points are verified at the new head.

[Copilot]

Review of b81266d7fc: three design changes before end-to-end runs

I reviewed the head commit, ran the five affected suites in a clean worktree (628 tests pass), and ran the new functions again...

Addressed in b9f7f48 (after make merge-main):

1. Declared vs repository agents. listCopilotSubagents marks inline ## agent: blocks under <os.tmpdir()>/gh-aw/.github/agents as declared. An inline agent with the same name as a repository file overrides it. Declared sub-agents still resolve, record and fail closed in CLI and SDK mode, routed runs included. Every other .github/agents/*.agent.md in the engine working directory goes through the same resolver and live-catalogue check, and uses the resolved id when that succeeds. When it fails, the harness logs a warning naming the agent, file and model, and records failed with the cause and resolved_model: null. The run continues: CLI leaves that agent unchanged, and in SDK mode it gets the session model with the existing warning and subagent.model_unavailable event, as before this PR.
2. Detection. Sub-agent resolution is skipped when isDetectionPhase(GH_AW_PHASE) is true. Nothing is read, resolved, recorded or written in that phase.
3. CLI personal-agent copies. When the resolved id differs from the declared value, the harness writes a copy with the same file name to ${COPILOT_HOME:-$HOME/.copilot}/agents/. Only the full model: scalar changes. The checkout is never modified. hideTrackedAgentFileRewrite, restoreCopilotCLIAgentFiles, the skip-worktree logic, cliAgentFileRestorations and the finally restore have been removed, along with their tests. If a copy can't be written, a declared sub-agent exits 1 before Copilot starts; a repository agent logs a warning and the run continues.

New tests use the run 39304573724 fixtures:

• A repository agent with GPT-5.4 (copilot) records failed/model_unavailable, logs the warning and doesn't stop the run, in both CLI and SDK. In the SDK session it gets the session model.
• An inline gpt-5.4-mini sub-agent in the same setup still fails closed.
• A repository agent with small resolves to claude-haiku-5.5.
• CLI personal copies keep the file name and the resolved model, including a block scalar. The workspace files stay byte-identical and git status is clean.
• In the detection phase nothing is resolved, recorded or written, even with dead models in both repository and inline agents.

Checks run: make fmt, make lint, tsc, the harness, SDK session, SDK driver, inline sub-agent, model-pin and resolver suites (670 tests passing), and make agent-report-progress. make recompile produced no lock-file diffs. I updated engines.md and the PR body ("Behaviour changes", "CLI mode", and the note that #70387 should merge first).

The end-to-end check that Copilot CLI 1.0.98 under AWF loads the personal copy instead of the repository file still needs a maintainer to dispatch a CLI smoke run on this branch.

Superseded: the points in this review were fixed in b81266d, and the skip-worktree design it reviewed was replaced in b9f7f48 (declared sub-agents only fail closed, detection skipped, CLI personal-agent copies). Re-verified at b9f7f48.

[SivaKesava1]

Verified at b9f7f48a3b

All three requested changes are in: only declared sub-agents fail closed, detection is skipped, and CLI mode writes personal-agent copies. The skip-worktree and restore code is gone. All 648 tests in the six affected JS suites pass locally, CI is green, and the PR touches only its 9 files.

End to end

1. SDK mode, dead pins: smoke-copilot-sub-agents 39499460077

The harness stopped before the SDK driver started (0 subagent.started). usage/aw_session.jsonl has:

role	configured	resolved	status
main	gpt-5.3-codex	gpt-5.3-codex	exact
haiku-whoami	claude-haiku-4.5	claude-haiku-4.5	exact
mini-whoami	gpt-5-mini	null	failed / model_unavailable
mini54-whoami	gpt-5.4-mini	null	failed / model_unavailable

Each dead pin gets its own failed record and the shared "The requested model is not supported" message. gpt-5-mini has also left the catalogue since 19 Oct, so it now fails as well (tracked with the other pins in #70367).

2. SDK mode, routed: smoke-copilot-sdk-routed-claude 39499464895

It stopped before the driver started. mini-whoami (gpt-5.4-mini) is recorded as failed / model_unavailable. There is no main record, because the main model comes from the router, as intended.

3. CLI mode: private sandbox run, compiled with --gh-aw-ref b9f7f48a3b

The workflow had:

• an inline quick-checker with model: small;
• an inline file-summarizer with model: claude-haiku-4.5;
• a repository .github/agents/conventions-helper.agent.md with model: GPT-5.4 (copilot), a VS Code display name.

The harness log shows:

warning: repository agent 'conventions-helper' (.../.github/agents/conventions-helper.agent.md) declares model 'GPT-5.4 (copilot)', which could not be resolved (cause: model_unavailable); Copilot CLI uses the agent file unchanged.
copilot sub-agent 'file-summarizer': model 'claude-haiku-4.5' -> 'claude-haiku-4.5'
copilot sub-agent 'quick-checker': model 'small' -> 'claude-haiku-5.5'
copilot sub-agent 'quick-checker': wrote model 'claude-haiku-5.5' to personal agent /home/runner/.copilot/agents/quick-checker.agent.md

The records are quick-checker small → claude-haiku-5.5 resolved, file-summarizer exact, conventions-helper failed / model_unavailable and main exact. The run succeeded.

• The personal-agent override works in Copilot CLI 1.0.98. The session's subagent.started for quick-checker shows model: claude-haiku-5.5 with taskModelSource: custom_agent_definition. Copilot CLI loaded the resolved copy, not the workspace file with small. The checkout was not modified.
• The repository agent no longer blocks the run. Before this round it would have stopped startup.

Unrelated failure in run 3. Both Claude sub-agents then failed with AWF's 400 Cannot translate Copilot request feature 'include' between Responses and Chat Completions. The main model gpt-5.6-luna is on /responses, and Copilot CLI sends one wire API for the whole session. This is the existing cross-family sub-agent limitation of Copilot CLI's BYOK mode, not caused by this PR: file-summarizer, whose model this PR doesn't change, fails the same way.

Review state

• The 13 earlier review threads are resolved. Each point is either still fixed at this head or no longer applies after the redesign.
• The two github-actions CHANGES_REQUESTED reviews from 7f35145678 are dismissed.

Before merge

• Merge Resolve experiment references in inline sub-agent frontmatter before writing the agent file; fail smoke-copilot when file-summarizer doesn't run #70387 first. On this branch, smoke-copilot's file-summarizer still declares the unexpanded ${{ experiments.subagent_model }}, so it now stops at startup.
• Six production SDK workflows change behaviour. The PR body lists their sub-agents moving from the main model to claude-haiku-5.5 (small) or claude-sonnet-5.5 (large).
• Scope question. @pelikhan, please confirm the declared/repository split from the earlier review comment: only workflow-declared sub-agents fail closed, and other repository agent files warn and record failed.