Skip to content

Register Claude inline sub-agents and resolve Claude/Codex sub-agent models before engine startup - #70671

Merged
SivaKesava1 merged 7 commits into
mainfrom
copilot/resolve-claude-codex-models
Oct 21, 2026
Merged

SivaKesava1 merged 7 commits into
mainfrom
copilot/resolve-claude-codex-models

Conversation

Copilot AI commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Claude Code skipped every inline sub-agent gh-aw staged, because the generated agent file had no name:. Typist failed with Agent type 'duplicate-type-finder' not found and fell back to general-purpose. Claude and Codex also resolved only the main model: declared sub-agent models were never checked or recorded, and never reached aw_info. This PR applies the Copilot pre-start contract from #70389 to Claude, and a declare-and-warn version of it to Codex. Copilot and Pi are unchanged.

Claude: registration

  • extract_inline_sub_agents.cjs adds name: "<block-name>" to the generated .claude/agents/<name>.md copy when the author left it out. This applies to Claude only; other engines' staging is unchanged.

Claude: model resolution (engine_subagent_models.cjs, called from claude_harness.cjs before spawn)

  • Each declared selector goes through the shared resolve_model_alias.cjs, scoped to the engine's provider. The result is checked against the live /reflect catalogue, with at most one shared refresh per batch.
  • Each agent gets one model.resolution record, with role set to the agent name.
  • Workflow-declared inline agents fail closed. On failure the harness records failed with model_unavailable or catalog_unavailable, emits the shared ModelAliasResolutionError and exits before the CLI starts. On success it writes the concrete ID into the generated workspace copy only.
  • Repository agents are resolved and recorded, and failures only warn. Their files are never modified. Claude Code ranks project agents above user agents, and the session-local --agents override has not been verified to preserve the agent's other settings. Instead, a warning says the resolved ID was not applied.
  • inherit keeps the main model and gets no record. haiku, sonnet and opus resolve through gh-aw's existing alias definitions, which are unchanged.
 ---
+name: "duplicate-type-finder"
 description: ...
-model: small
+model: "claude-haiku-5-5"
 ---

Codex: declare and warn only (codex_harness.cjs)

  • Models of the staged .codex/agents/*.md files are resolved and recorded.
  • A warning explains that Codex 0.163.0 discovers custom agents only from TOML role files, so these agents and their models are not applied.
  • The run never fails over them. A native TOML handoff is left for a follow-up issue.

aw_info declarations

  • writeSubagentModelDeclarations now emits GH_AW_INFO_SUB_AGENT_MODELS for Claude and Codex, with patterns limited to the engine's provider. Claude inherit gets no patterns.
  • Existing lock files gain only this one environment-variable line.

Threat detection

  • The whole pass is skipped when isDetectionPhase is true.

smoke-claude-sub-agents (new, read-only, workflow_dispatch, bare: false)

  • The first action must delegate to the declared model: small worker marker-worker, which returns a fixed marker.
  • Post-steps check:
    • the worker's model.resolution record;
    • name and the resolved ID in the generated definition;
    • that native initialization lists the worker;
    • that the first Agent call succeeds;
    • that subagent.started names the worker, and no general-purpose fallback ran;
    • that subagent.configured.data.model matches the resolved ID;
    • completion;
    • that the worker's own messages and the proxy's served requests used that model.
  • A new workflow can't run on its own PR before merge. It still needs verifying in a sandbox copy compiled against this PR's head; no branch run has been made.
  • It uses the standard Claude secrets and pinned actions/* steps, like the other Claude smoke workflows.

Tests

  • The issue's named tests use reduced real artifacts from Typist run 39415087125 and arXiv run 39394965127: staged agents, the /reflect catalogue, the initialization record, the failed Agent call and session records.
  • For Codex, they cover the declare-and-warn branch.
  • Deliberate fixture mutations, such as a removed catalogue ID or an unknown selector, are labelled in the tests.

Out of scope / follow-ups

Copilot AI and others added 2 commits October 21, 2026 00:22
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
@SivaKesava1
SivaKesava1 marked this pull request as ready for review October 21, 2026 00:30
Copilot AI max review requested due to automatic review settings October 21, 2026 00:30
Copilot AI changed the title [WIP] Resolve and record Claude/Codex sub-agent models before engine startup Register Claude inline sub-agents and resolve Claude/Codex sub-agent models before engine startup Oct 21, 2026
Copilot AI requested a review from SivaKesava1 October 21, 2026 00:31
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 12.1 AIC · ⌖ 5.41 AIC · ⊞ 14.7K · ◷

Generated by Ponytail Reviewer for #70671

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · 14.1 AIC · ⌖ 15.1 AIC · ◇ 6.59 AIC · ⊞ 12.6K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR adds 150 new lines of business logic (more than the 100-line threshold, mostly in actions/setup/js/engine_subagent_models.cjs, the Claude and Codex harnesses and pkg/workflow/compiler_yaml_step_lifecycle.go), but the PR body does not link an Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/70671-resolve-claude-codex-sub-agent-models-before-engine-startup.md. Review and complete it before merging.

The draft records this decision: resolve and record every Claude and Codex sub-agent model before the engine CLI starts. How strict it is depends on who owns the agent:

  • Claude workflow-declared inline agents: fail closed, and the resolved ID is written into the generated copy.
  • Claude repository agents: resolved and recorded, but only a warning on failure; their files are never changed.
  • Codex: declare and warn only, never fatal.

It also covers adding the missing name: to generated Claude agent files and emitting provider-scoped GH_AW_INFO_SUB_AGENT_MODELS declarations.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to your branch. It was generated from the PR diff and description.
  2. Complete the missing sections. Fill in Deciders (currently [TODO: verify]) and refine the rejected alternatives: the --agents session override, failing closed everywhere, and generating native Codex TOML now.
  3. Commit the finalized ADR to docs/adr/ on your branch.
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-70671: Resolve Claude and Codex Sub-Agent Models Before Engine Startup

Once the PR body links an ADR, this gate re-runs and checks that the implementation matches the decision.

❓ Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context: what is the problem, and what forces are at play?
  • Decision: what did you decide, and why?
  • Alternatives Considered: what else could have been done?
  • Consequences: what are the trade-offs, positive and negative?

ADRs are stored in docs/adr/ as Markdown files named by PR number (for example, 70671-*.md for PR #70671).

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · opus55 · 36.3 AIC · ⌖ 40.7 AIC · ⊞ 1.9K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: comment

No actionable blocking issues were identified in the changed lines. The review was static; repository tests and builds were not run.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 7.65 AIC · ⌖ 0.55 AIC · ⊞ 1.6K
Comment /review to run again

const name = typeof frontmatter.name === "string" && frontmatter.name.trim() ? frontmatter.name.trim() : null;
agents.push({ fileName: entry.name, path: agentPath, name, ...(typeof frontmatter.model === "string" ? { model: frontmatter.model.trim() } : {}) });
} catch (error) {
logger(`warning: cannot read custom agent ${agentPath}: ${getErrorMessage(error)}`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Malformed workflow-declared agent files are silently dropped here, so the fail-closed path never runs. listMarkdownAgentFiles logs the parse/read error and returns no entry; applyClaudeSubagentModels then reports success and Claude starts with the unresolved staged agent, reproducing the registration/model failure this change is intended to prevent. Treat parse failures under the workflow staging directory as fatal (while retaining lenient handling for repository agents), and add a regression test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1f97c55. A staged workflow sub-agent file that can't be read or parsed (or a non-ENOENT read error on the staging directory) now stops Claude before it starts. listMarkdownAgentFiles takes an onInvalid callback, and applyClaudeSubagentModels returns ok: false. Repository agent files are still only warned about. Regressions are in engine_subagent_models.test.cjs (staged: fails closed; repository: lenient) and claude_harness.test.cjs (exit 1, no spawn).

const resolvedNote = model !== configuredModel ? ` (resolved to '${model}')` : "";
if (liveCheck && !isModelAvailableInReflectData(model, reflectData, providers)) {
const initialReason = classifyUnlistedModel(reflectData, providers);
const refreshed = await refreshOnce();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The /reflect refresh promise can reject, and this await refreshOnce() is outside the surrounding resolution try/catch. Because the promise is memoized, one transient fetch failure aborts the entire batch; the Claude harness does not catch applyClaudeSubagentModels, so the run exits via an uncaught rejection without recording the remaining agents or emitting the intended catalog-unavailable diagnostic. Catch refresh failures inside refreshOnce and return null so the existing failure classification remains reliable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1f97c55. refreshOnce now catches refresh failures, including synchronous throws from refetchReflectData. It logs a warning and returns null, so the batch still resolves and records every agent, and an incomplete catalogue is classified as catalog_unavailable. A regression test checks that a rejected refresh is attempted once and both agents are recorded as catalog_unavailable.

env,
logger,
});
if (failures.some(failure => failure.fatal)) return { ok: false, resolutions };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes an incomplete /reflect catalog a hard startup failure for every fatal Claude workflow agent, even when the provider may still serve the configured model. catalog_unavailable covers discovery being incomplete or absent, not evidence that the model is invalid; exiting before the workflow starts introduces a new availability dependency and can turn transient discovery failures into total run outages. Keep catalog-unavailable non-fatal (or make this policy explicit), and reserve fail-closed behavior for a complete catalog proving model_unavailable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping this as is, on purpose. For a workflow-declared Claude sub-agent, catalog_unavailable fails closed. That's the same policy #70389 uses for Copilot sub-agents and #70320 uses for the Copilot SDK main model: gh-aw doesn't start an agent whose declared model it can't confirm. A transient failure gets one shared refresh first. Repository agents and Codex agents only record and warn. The policy is now documented in the PR body under "catalog_unavailable policy (intentional)".

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd, /codebase-design, and /grill-with-docs (triage via pr-triage returned new_feature). Overall this is a well-tested, carefully scoped change (623 touched JS tests, new Go declaration tests, a realistic smoke workflow built from real failure fixtures). No blocking correctness issues found; comments below are refinement suggestions.

📋 Key Themes & Highlights

Key Themes

  • Duplicated/hard-to-verify branch logic in classifyUnlistedModel's two call sites in engine_subagent_models.cjs — the short-circuit ternary deserves a one-line comment explaining the invariant it relies on.
  • No surrounding try/catch around the new applyClaudeSubagentModels call in claude_harness.cjs; worth confirming the shared resolver truly never throws, matching the documented "never thrown" contract, or add a safety net consistent with the adjacent fetchAWFReflect/buildClaudeChildEnv try/catch.
  • Single string-equality guard (engineId.toLowerCase() === "claude") is the only thing preventing the regression this PR fixes (Agent type '<name>' not found); the lone call site (interpolate_prompt.cjs) does pass the expected literal, but it's worth a defensive test for unexpected casings/aliases given how costly a silent skip would be.
  • Provider-string duplication between the new claudeCodexSubagentProvider helper and the existing llmProviderAliases map in llm_provider.go — a provider == "copilot" branch looks unreachable given the normalized LLMProvider type returned upstream.
  • Docs gap: inline-sub-agents.md's "Runtime behavior" section doesn't describe the new fail-closed-for-Claude / declare-and-warn-for-Codex model resolution semantics, which is a meaningful behavior change for anyone declaring a model: on an inline sub-agent.

Positive Highlights

  • ✅ Fail-closed vs. declare-and-warn split between Claude and Codex is clearly reasoned about and documented in code comments and the PR body.
  • ✅ Extensive fixture-driven tests sourced from real run failures (Typist 39415087125, arXiv 39394965127), with each fixture mutation explicitly labelled.
  • ✅ New smoke-claude-sub-agents.md workflow is a genuinely useful end-to-end check (native init, delegation, proxy-served model) rather than a superficial smoke test.
  • ✅ providerScopedSubagentPatterns correctly special-cases Claude's inherit as "no patterns", matching the semantic that it isn't a real model.

Note: the diff is large (38 files, ~3.5k additions); review focused on the pr-triage-identified high-impact files (engine_subagent_models.cjs, claude_harness.cjs, codex_harness.cjs, extract_inline_sub_agents.cjs, compiler_yaml_step_lifecycle.go, inline-sub-agents.md) rather than the 14 mechanically-updated lock files or the fixture data.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 143.4 AIC · ⌖ 15.5 AIC · ⊞ 10.2K
Comment /matt to run again


module.exports = {
applyClaudeSubagentModels,
declareCodexSubagentModels,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] classifyUnlistedModel is computed before refreshOnce() as initialReason, then the post-refresh branch re-derives it with a terse ternary: initialReason === "model_unavailable" || !refreshed ? initialReason : classifyUnlistedModel(refreshed, providers). This is hard to verify at a glance without tracing both call sites.

💡 Suggested simplification
const reason = !refreshed ? initialReason : classifyUnlistedModel(refreshed, providers);

If initialReason === "model_unavailable" always implies the refreshed classification would also be model_unavailable (since a complete catalog without the model can't become incomplete), make that invariant explicit with a short comment so the short-circuit isn't mistaken for dead code or an optimization-only branch.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1f97c55. The line is now initialReason === "catalog_unavailable" && refreshed ? classifyUnlistedModel(refreshed, providers) : initialReason. Behaviour is unchanged. A comment states the invariant: a complete catalogue without the model already proves model_unavailable, so only an incomplete one is reclassified from refreshed data.

@@ -559,6 +560,23 @@ async function main() {
process.exit(0);
}

// Sub-agent models are resolved on routed runs too, but a pre-existing noop must not

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] applyClaudeSubagentModels runs after fetchAWFReflect() and the pre-flight noop/safeInitialArgs checks, but there's no unit/integration test asserting the exact ordering relative to removeClaudePlugin(pluginDir) cleanup on a mid-stream failure of resolveClaudePromptFileArgs (the earlier try/catch at line ~517). If applyClaudeSubagentModels throws instead of returning { ok: false } (e.g. an unexpected error inside resolveEngineSubagentModels's outer try/catch), this call site has no surrounding try/catch, so pluginDir cleanup would be skipped on an unhandled rejection.

💡 Suggested check

Confirm resolveEngineSubagentModels truly never throws (the PR description says "never thrown"), or wrap this call in a try/catch that calls removeClaudePlugin(pluginDir) before re-throwing, matching the pattern used for fetchAWFReflect/buildClaudeChildEnv just above.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1f97c55. The call is now wrapped in try/catch. An unexpected throw logs fatal: unexpected error while resolving Claude sub-agent models: …, then goes through the existing ok: false path: removeClaudePlugin(pluginDir), then process.exit(1). A harness test injects a throw and checks the exit code, that nothing was spawned, the message, and that cleanup ran.

@@ -383,7 +402,8 @@ function writeInlineSubAgents(content, workspaceDir, agentsBaseDir, engineId) {

for (const agent of agents) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] ensureClaudeSubAgentName is only applied when engineId.toLowerCase() === "claude", string-compared against the raw engineId parameter. If an upstream caller ever passes a differently-cased or aliased Claude engine id (e.g. "Claude-Code"), this silently skips the name injection and reintroduces the exact Agent type '<name>' not found bug this PR fixes, with no test coverage for that mismatch.

💡 Suggested test
it('still ensures name for an unexpected engineId casing or alias', () => {
  // document whatever ids this function is actually called with across the codebase
});

Worth confirming (via grep) every writeInlineSubAgents call site passes exactly the literal "claude" string, since this is the only guard against regressing the bug.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1f97c55. writeInlineSubAgents now normalises the engine id once (trimmed and lower-cased) and uses it for both the target directory and the name: injection, so every id that stages into .claude/agents gets the name. The only production caller is interpolate_prompt.cjs, which passes GH_AW_ENGINE_ID; the compiler emits that as the canonical claude. A test covers Claude and CLAUDE.

if provider == string(LLMProviderGitHub) || provider == "copilot" {
return "github-copilot"
}
return provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] claudeCodexSubagentProvider collapses LLMProviderGitHub and the literal "copilot" string into "github-copilot", duplicating the normalization already expressed via llmProviderAliases in llm_provider.go (which maps copilot/github/github-copilot/github_models all to LLMProviderGitHub). Since resolveEngineLLMProviderFromModel already returns a normalized LLMProvider, the provider == "copilot" branch here looks unreachable (it would always be the already-normalized LLMProviderGitHub string "github"), which is subtly different from the alias map's source strings.

💡 Suggested check

Add a short comment explaining why the "copilot" string branch exists here if it's reachable, or remove it if resolveEngineLLMProviderFromModel never returns that literal. Consider having this function call into the shared llmProviderProfileFor/alias logic instead of re-deriving the github-copilot provider-id string, to avoid the two call paths drifting apart.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in 1f97c55. resolveEngineLLMProviderFromModel always returns a normalised LLMProvider, because normalizeLLMProvider and fallbackModelProvider both map copilot through llmProviderAliases. So the literal "copilot" branch was unreachable. A comment now explains the github → github-copilot mapping. The existing "claude on the copilot provider" declaration test still passes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Production startup changes still lack live end-to-end validation, and blocking correctness feedback remains unresolved.

6 open issues
What changed in this PR

Extends gh-aw’s pre-start sub-agent model resolution to Claude and Codex, following the Copilot contract, and fixes Claude inline-agent registration.

Changes:

  • Adds Claude agent names and resolves declared models before startup.
  • Records provider-scoped declarations in aw_info.
  • Adds artifact-based tests and a Claude delegation smoke workflow.
File Description
pkg/​workflow/​compiler_yaml_step_lifecycle.go Emits provider-scoped declarations.
pkg/​workflow/​claude_codex_subagent_declarations_test.go Tests compiled declarations.
actions/​setup/​js/​smoke_claude_sub_agents.test.cjs Tests smoke evidence assertions.
actions/​setup/​js/​generate_aw_info.test.cjs Tests declaration export.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​typist-39415087125/​.claude/​agents/​duplicate-type-finder.md Preserves Claude registration fixture.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​arxiv-39394965127/​.codex/​agents/​relevance-ranker.md Adds recorded ranker definition.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​arxiv-39394965127/​.codex/​agents/​preliminary-investigator.md Adds recorded investigator definition.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​arxiv-39394965127/​.codex/​agents/​paper-screener.md Adds recorded screener definition.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​arxiv-39394965127/​.codex/​agents/​opportunity-extractor.md Adds recorded extractor definition.
actions/​setup/​js/​fixtures/​claude_codex_subagent_runs.cjs Supplies recorded events and provenance.
actions/​setup/​js/​fixtures/​awf-reflect-codex-arxiv-39394965127.json Supplies OpenAI catalogue evidence.
actions/​setup/​js/​fixtures/​awf-reflect-claude-typist-39415087125.json Supplies Anthropic catalogue evidence.
actions/​setup/​js/​extract_inline_sub_agents.test.cjs Tests Claude name staging.
actions/​setup/​js/​extract_inline_sub_agents.cjs Adds missing Claude names.
actions/​setup/​js/​engine_subagent_models.test.cjs Tests resolution and failure behavior.
actions/​setup/​js/​engine_subagent_models.cjs Implements shared sub-agent resolution.
actions/​setup/​js/​codex_harness.test.cjs Tests Codex pre-start declarations.
actions/​setup/​js/​codex_harness.cjs Records declarations before spawn.
actions/​setup/​js/​claude_harness.test.cjs Tests Claude startup enforcement.
actions/​setup/​js/​claude_harness.cjs Applies resolution before spawn.
.github/​workflows/​typist.lock.yml Adds worker metadata.
.github/​workflows/​spec-enforcer.lock.yml Adds worker metadata.
.github/​workflows/​smoke-claude-sub-agents.md Adds delegation smoke and assertions.
.github/​workflows/​smoke-claude-sub-agents.lock.yml Adds compiled smoke workflow.
.github/​workflows/​purelock.lock.yml Adds worker metadata.
.github/​workflows/​github-mcp-tools-report.lock.yml Adds worker metadata.
.github/​workflows/​developer-docs-consolidator.lock.yml Adds worker metadata.
.github/​workflows/​detection-analysis-report.lock.yml Adds worker metadata.
.github/​workflows/​daily-safeoutputs-git-simulator.lock.yml Adds worker metadata.
.github/​workflows/​daily-awf-spec-compiler-surfacing.lock.yml Adds worker metadata.
.github/​workflows/​daily-arxiv-researcher.lock.yml Adds Codex declarations.
.github/​workflows/​daily-agentrx-trace-optimizer.lock.yml Adds worker metadata.
.github/​workflows/​claude-code-user-docs-review.lock.yml Adds worker metadata.
.github/​workflows/​aw-failure-investigator.lock.yml Adds worker metadata.
.github/​workflows/​api-consumption-report.lock.yml Adds worker metadata.
.github/​workflows/​agent-persona-explorer.lock.yml Adds inheritance declaration.
.github/​workflows/​ab-testing-advisor.lock.yml Adds worker metadata.
.changeset/​claude-codex-subagent-model-resolution.md Documents runtime behavior changes.

Review effort: Max

sandbox:
agent:
id: awf
post-steps:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and this smoke workflow safe-outputs change isn't one of them. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. The smoke workflow is now read-only:

  • Disabled: the default create-issue, failure and failed-jobs issue reporting, issue creation for missing-tool, missing-data and report-incomplete, and threat detection.
  • It finishes with a noop that doesn't publish (report-as-issue: false).

The regenerated lock grants no issues: write and has no create_issue. smoke_claude_sub_agents.test.cjs checks this.

try {
result = await resolveEngineModel({
engine,
role: agent.name,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and reserving the main role for sub-agent records isn't one of them. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. main is now a reserved sub-agent name for Claude and Codex:

  • A workflow-declared main stops the run with a message asking to rename the agent.
  • A repository agent named main logs a warning and is skipped.

New tests check that both the main and worker model.resolution records survive, and that the harness stops before spawning when a declared main is present.

Comment on lines +343 to +346
const existing = fs.existsSync(agent.workspacePath) ? fs.readFileSync(agent.workspacePath, "utf8") : null;
if (existing !== content) {
fs.mkdirSync(path.dirname(agent.workspacePath), { recursive: true });
fs.writeFileSync(agent.workspacePath, content);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and the bare-mode plugin copy isn't one of them. The PR also lists bare-mode behaviour as out of scope. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. With bare: true, the plugin's agents link is replaced by a run-local copy of the staged definitions, with resolved models written in. The staged and repository files are not changed.

claude_harness.test.cjs reads the definition through the real --plugin-dir/agents path, and engine_subagent_models.test.cjs uses the real claudeBareCapabilities.

Comment on lines +148 to +150
if (/^name[ \t]*:/m.test(frontmatter)) return content;
const start = content.indexOf("\n") + 1;
return content.slice(0, start) + nameLine + "\n" + content.slice(start);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and parser-based name: detection and flow-mapping insertion aren't among them. If such a file did come out unparseable, it now fails closed before Claude starts (1f97c55) rather than being skipped. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. ensureClaudeSubAgentName now uses the YAML parser:

  • An existing name key is detected, including a quoted "name":.
  • For flow-mapping frontmatter, the name goes inside the {...}, so the YAML stays valid.

Staging tests cover a quoted key, {description: ..., model: small} and { }.

Comment on lines +286 to +289
patterns := expandSubagentModelPatterns(agent.Model, data, provider)
if engineID == "claude" || engineID == "codex" {
patterns = providerScopedSubagentPatterns(agent.Model, patterns, provider)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and expanding provider-qualified aliases in declarations isn't one of them. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. Claude and Codex declaration patterns now strip a provider prefix that matches the engine's provider before alias expansion. For example, anthropic/fast with fast: [anthropic/*haiku*] expands to the Haiku pattern.

Prefixes for a different provider are left as they are. Declaration tests cover anthropic/fast, anthropic/small, copilot/small, openai/small, and a mismatched prefix.

Comment on lines +321 to +322
if strings.EqualFold(strings.TrimSpace(model), "inherit") {
return scoped

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed in this round. The maintainer limited the review to items 1–4 in #70671 (comment), and matching inherited declarations in the audit consumer isn't one of them. It's left for the maintainer to schedule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f55f661. declaredModelMatches now matches inherit and inherited declarations against the parent model. That model comes from the main agent's observed models (requested and served); if none were observed, it falls back to the model in aw_info.

As a result, a successful inherited invocation is no longer reported as "Declared Sub-agent Model Not Observed". TestDeclaredInheritedSubagentModelMatchesParentModel covers this.

@SivaKesava1

Copy link
Copy Markdown
Collaborator

Review round: fix three of the review threads, keep the fail-closed policy

CI: the only failure, impacted-go-tests (TestDryRunWorkQueueAndCallWorkflow), also fails on main (latest CGO run on 7f78411cc1).

Fix:

  1. Malformed staged agent files (engine_subagent_models.cjs:100): a workflow-declared inline agent under the staging directory that can't be read or parsed must fail closed before the engine starts. Repository agents stay lenient. Add a regression.
  2. /reflect refresh rejection (:248): catch refresh failures inside refreshOnce and return null, so the batch still records every agent and the catalog-unavailable classification applies. Add a regression.
  3. Unhandled throw at the Claude call site (claude_harness.cjs:563): make sure an unexpected error from applyClaudeSubagentModels still runs the plugin-dir cleanup and exits with a clear message.
  4. Small items:
    • simplify the post-refresh reason expression (:403);
    • normalise the engine id before the Claude name: injection (extract_inline_sub_agents.cjs:403), or document why the caller always passes claude;
    • comment or remove the unreachable "copilot" branch (compiler_yaml_step_lifecycle.go:314).

Keep (:327): an incomplete catalogue (catalog_unavailable) stops a workflow-declared sub-agent. This is the same policy #70389 uses for Copilot sub-agents and #70320 uses for the Copilot SDK main model. Document it in the PR body and reply on that thread. Don't change it.

…refresh failures, guard harness call site

Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
…, bare plugin agents, YAML name key, provider-prefixed aliases, inherited audit match

Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
@SivaKesava1
SivaKesava1 merged commit 15d31e5 into main Oct 21, 2026
7 of 8 checks passed
@SivaKesava1
SivaKesava1 deleted the copilot/resolve-claude-codex-models branch October 21, 2026 03:07
SivaKesava1 added a commit that referenced this pull request Oct 21, 2026
`safe-outputs.create-issue: false` is valid frontmatter: it disables the
handler and stops default issue creation. smoke-claude-sub-agents (#70671)
uses it to stay read-only. The allowed-labels rule passed the value to
policyMapping, which accepts only a mapping, so the repository lint test
TestRepositoryAgenticWorkflows/smoke-claude-sub-agents.lock.yml fails on
main ("AST policy lint requires a mapping at safe-outputs.create-issue").

A boolean handler has no allowed-labels to check, so skip it. Other
non-mapping values (strings, numbers) are still reported as malformed.

Co-authored-by: Siva Kakarla <SivaKesava1@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resolve and record Claude/Codex sub-agent models before engine startup

3 participants