Resolve model: aliases the same way on every engine #69484

Problem

gh-aw resolves a workflow's model: differently on each engine, so a model alias or a family pin works on one engine and breaks on another.

On 16 Oct 2026 Copilot removed claude-opus-5 from /models (still in AWF's live list at 05:03 UTC, gone by 18:15; only claude-opus-5.5 remains). gh-aw's Design Decision Gate (pi, model: copilot/claude-opus-5) failed every run from 18:08 UTC until #69469 moved it to 5.5. For example, run 38945917145:

[gh-aw/pi-models-json] warning: awf-reflect model availability check failed (provider=copilot, model=claude-opus-5)
[gh-aw/pi-models-json] fatal: Pi model "claude-opus-5" is not advertised by the configured copilot proxy endpoint; choose a model listed by AWF /reflect

A family pin such as the built-in opus alias would have survived on the Copilot engine, but not on pi. Peli asked (Slack, 16 Oct) that all agentic engines apply the same resolution rules.

Evidence

Private sandbox (described here because it can't be linked). Variant routing-test-pi-opus-alias is a pi workflow with model: copilot/opus, compiled at gh-aw main f1621a88b3. It compiled with no warning, and the lock passes GH_AW_PI_MODEL: copilot/opus through. Run 38958912807 (16 Oct) failed in the agent step before any model request:

[gh-aw/pi-models-json] warning: awf-reflect model availability check failed (provider=copilot, model=opus)
[gh-aw/pi-models-json] fatal: Pi model "opus" is not advertised by the configured copilot proxy endpoint; choose a model listed by AWF /reflect

/reflect listed claude-opus-5.5 at that time. The same model: opus on the Copilot engine resolves to claude-opus-5.5.

Rechecked on main 6738f9ca21. f1621a88b3 is an ancestor of main, and none of the resolution files have changed since then. Compiling the same workflow on main gives the same lock: no warning, GH_AW_PI_MODEL: copilot/opus, GH_AW_PI_MODEL_ID: opus, and GH_AW_PI_MODEL_ALIASES containing "opus": ["copilot/*opus*", "anthropic/*opus*"]. The alias map reaches the pi harness, but the main-model path doesn't use it.

Public DDG run 38945917145. Its /reflect (agent artifact, sandbox/firewall/awf-reflect.json) has models_fetch_complete: true, 40 Copilot models with claude-opus-5.5 as the only Opus, and the merged alias map as model_aliases. With that file, resolve_model_alias.cjs gives opus → claude-opus-5.5, haiku → claude-haiku-5.5, opus?effort=high → claude-opus-5.5?effort=high. The run's usage/aw_session.jsonl has an agent.execution event with no categories or error codes (exit code 1), and the conclusion reported Model not supported error: false. The DDG failure issue #69472 says "The runtime did not record a specific cause". On Copilot, an unavailable model gives [aw] <workflow> has an unavailable configured model with a Model Not Supported section (for example #69339).

Current behaviour on main (verified at 6738f9ca21)

• Copilot (CLI and SDK). copilot_harness.cjs (applyCopilotModelAliasResolution) calls resolveConfiguredCopilotModel in resolve_model_alias.cjs. It uses the merged alias map from the AWF config (apiProxy.models; built-ins from pkg/workflow/data/model_aliases.json), resolves globs inside alias entries to the newest live match (selectLatestGlobMatch), refetches /reflect once, and stops before the CLI with model_unavailable or catalog_unavailable (fix(copilot): reject silent configured-model substitution #68819). Exact ids go to the CLI unchanged. If one is unavailable, it fails at the first request, and detect_agent_errors turns that into the unavailable-model failure issue. The selection is recorded in aw_info.json (model_selection, copilot_model_selection.cjs), not in aw_session.jsonl. Gap: only a bare alias key is looked up, so copilot/opus isn't resolved and reaches the CLI as opus.
• Pi. pi_models_json.cjs (validatePiModelAvailability) requires the main model (GH_AW_PI_MODEL_ID, the model without its provider prefix, set in pi_engine.go) to match a /reflect entry exactly. The alias map (GH_AW_PI_MODEL_ALIASES) is used only for sub-agents (pi_subagent_config.cjs, which runs resolveModelAlias against a catalogue limited to the provider). Pi has no error-detection step (its GetErrorDetectionScriptId returns empty), and its message doesn't match MODEL_NOT_SUPPORTED_PATTERN, so its failure issue doesn't say that the model is unavailable.
• Claude. In model_fallback.cjs, normalizeClaudeModel strips copilot/, and resolveClaudeAutoModel resolves only auto (Copilot provider: newest Sonnet, then Opus, then Haiku; Anthropic provider: Claude Code's sonnet). Every other value goes to Claude Code as given, apart from the dotted-id model overrides from Claude on Copilot: pass Claude Code a modelOverrides map for Copilot's dotted Claude model ids #68839. model: copilot/opus with the Copilot provider compiles with no warning to ANTHROPIC_MODEL: opus, which is Claude Code's own built-in alias, not gh-aw's map or the live list.
• Codex. normalizeCodexModel only checks and strips the provider prefix. model: copilot/gpt-5.6 reaches Codex as gpt-5.6, although gpt-5.6 is a built-in alias.
• Gemini. There's no harness. gemini_engine.go sets GEMINI_MODEL to the configured value.
• Agy. agy_harness.cjs requires a native slug (no provider/), reads /reflect only to find the Gemini endpoint, and passes --model unchanged. There's no agy_harness.test.cjs.

Incomplete /reflect discovery today:

• Copilot: exact ids pass through; aliases fail with catalog_unavailable after one refetch.
• Pi main model: the check is skipped when /reflect is missing, models_fetch_complete isn't true, there's no configured endpoint, or there's no model list. Exact ids and aliases both pass through unchecked.
• Pi sub-agents: aliases resolve against the Pi SDK's static model list instead of failing.
• Claude: auto on the Copilot provider fails with Claude's own message; other values pass through.
• Codex, Gemini, Agy: the model isn't checked against /reflect.

Other gaps found while checking:

• Workflow-defined aliases can't be written today. The models: schema in main_workflow_schema.json allows only allowed, blocked, providers and default-ai-credits-pricing. models: { opus: [...] } fails to compile with "Unknown property: opus", both in a main workflow and in an imported one. The main workflow's map is also never merged (workflow_builder.go passes nil to MergeImportedModelAliases), although spec §10.1 says it always wins. Meanwhile, reference/model-tables.md and reference/glossary.md say custom aliases can be defined in frontmatter.
• AWF model fallback. On providers other than Copilot, AWF's proxy may rewrite an unrecognised model id (sandbox.agent.model-fallback, on by default). On Copilot it is suppressed (copilot_standard_authoritative in the DDG /reflect). This issue doesn't change that.

Proposed solution

• One shared "resolve configured model" step. Generalise resolveConfiguredCopilotModel (and the Copilot harness's bounded refetch around it) in resolve_model_alias.cjs instead of adding a new resolver. Every harness runs this step before it launches its CLI. It covers the main model, and sub-agent models where the engine has them, so pi_subagent_config.cjs uses the same code.
• Inputs: the configured model (an exact id, a built-in alias, or a workflow-defined alias, with an optional provider/ prefix and ?effort= parameters), the merged alias map (already in the AWF config and in /reflect model_aliases; pi also gets GH_AW_PI_MODEL_ALIASES), and the /reflect catalogue for the engine's configured provider.
• Rules, the same on every engine:
  • Aliases, and globs inside alias entries, resolve to the newest live match. The result is the one Copilot gets today for the same input.
  • A provider prefix limits the lookup to that provider. copilot/opus looks up opus and considers only Copilot models, as the pi sub-agent path already does. This also makes copilot/opus work on the Copilot engine.
  • Exact ids are never rewritten, so a typo can't resolve to a different model. Where a harness already checks exact ids before launch (pi), an id that isn't listed fails with model_unavailable. Copilot, Claude and Codex keep failing at the first request, as they do today.
  • ?effort= and other parameters carry through as today.
  • auto keeps its current meaning on each engine, even though it is also a built-in alias: a Copilot gateway sentinel on pi and Codex, Claude's own rule on Claude, and today's handling on Copilot.
  • Failure causes and messages are the ones Copilot uses today: model_unavailable and catalog_unavailable, with the fix(copilot): reject silent configured-model substitution #68819 wording. The sentence about the agentic-workflows integration appears only when the provider is Copilot. Pi gets the detect_agent_errors step, so its failure issue matches Copilot's.
• Incomplete discovery (decided): every engine follows Copilot's current rule. Exact ids pass through unchanged, and aliases fail with catalog_unavailable after one bounded refetch. This changes pi: aliases no longer pass through, and sub-agents no longer fall back to the Pi SDK's static list. It also changes Claude and Codex on native providers: sonnet, opus, haiku and gpt-5.6 now go through gh-aw's map instead of the CLI's own meaning, and they fail with catalog_unavailable when that provider's discovery is incomplete.
• Engine-specific handling stays after resolution: Copilot's copilot/ prefix handling, Claude Code's model-id overrides (Claude on Copilot: pass Claude Code a modelOverrides map for Copilot's dotted Claude model ids #68839) and its auto rule, Codex's provider-prefix check, and Agy's native-slug check.
• Record the result. The harness logs the configured and resolved model, as Copilot does today. It writes a small record under agent/, like agent/model-endpoint-mismatch.json, and that path is added to the agent artifact. unified_session.cjs then projects the record into usage/aw_session.jsonl as a runner-written event, added to §4.7 of docs/src/content/docs/specs/unified-agent-session-specification.md. The event holds the engine, the role (main or sub-agent name), the configured model, resolved model, provider, status and cause. Audit can then show when an alias moved to a different, possibly more expensive, model. Copilot's aw_info.json model_selection stays.
• Compiler warning until supported. Agy runs the shared step against the Gemini catalogue and passes the native slug. Gemini (no harness) and the declarative engines don't get the shared step in this PR. The compiler warns when one of them is given an alias, naming the engine and the alias.
• Workflow-defined aliases. Accept alias keys under models: and merge the main workflow's map last, as spec §10 describes. The reserved keys keep their meaning, and the V-MAF key rules apply. Cases 5 and 7 need this, so it's part of this fix.

Acceptance cases

Recorded /reflect shapes: use the DDG run 38945917145 file above, stored once as a shared fixture. For case 8, use a recorded run with incomplete discovery if one exists. Otherwise, derive it from that file (models_fetch_complete: false, Copilot models: null) and say so in the test.

1. Pi, model: copilot/opus, with claude-opus-5.5 in /reflect. Result: the agent's model requests go to claude-opus-5.5. The harness log has one line with configured copilot/opus and resolved claude-opus-5.5. aw_session.jsonl has one model-resolution event recording both. Tests: pi_models_json.test.cjs "resolves copilot/opus to the newest live Opus"; unified_session.test.cjs "projects the model resolution record".
2. Pi, model: copilot/claude-opus-5 (retired). Result: fails before any model request with cause model_unavailable and the fix(copilot): reject silent configured-model substitution #68819 message. The failure issue is titled [aw] <workflow> has an unavailable configured model and has the Model Not Supported section, as on Copilot. Tests: pi_models_json.test.cjs "fails closed with model_unavailable for a retired exact id"; detect_agent_errors.test.cjs "reports model_unavailable from the pi harness"; Go TestPiEngineUsesAgentErrorDetection in pkg/workflow/pi_engine_test.go.
3. Copilot engine, model: opus and model: claude-opus-5. Result: unchanged from today. opus runs on claude-opus-5.5. claude-opus-5 reaches the CLI, fails as not supported, and produces the same failure issue as today. The only difference: model: copilot/opus now also resolves to claude-opus-5.5. Test: copilot_harness.test.cjs "keeps alias and exact-id behaviour with the recorded /reflect".
4. Claude engine on the Copilot provider, model: copilot/opus. Result: runs on the newest live Opus (claude-opus-5.5 with the fixture). model: copilot/auto still picks the newest Sonnet, then Opus, then Haiku. Tests: claude_harness.test.cjs "resolves copilot/opus to the newest live Opus" and "auto keeps its Sonnet, Opus, Haiku order".
5. Codex engine on the Copilot provider, with a workflow-defined alias whose entry is copilot/gpt-5.6*. Result: Codex runs on the newest live match, the same model Copilot's resolver returns (gpt-5.6-luna with the fixture). An exact id such as copilot/gpt-5.6-sol is passed through as gpt-5.6-sol. Tests: codex_harness.test.cjs "resolves a workflow-defined glob alias to the newest live match" and "passes an exact id through unchanged"; Go TestMainWorkflowModelAliasesMerged in pkg/workflow/model_aliases_test.go, plus a schema test that accepts alias keys under models:.
6. Gemini and Agy. Result: an exact id is unchanged on both. On Agy, an alias resolves against the Gemini provider's /reflect catalogue, and Agy gets the native slug. On Gemini, an alias produces a compile-time warning naming the engine and the alias, and the lock is unchanged. Tests: new agy_harness.test.cjs "passes an exact slug unchanged" and "resolves an alias to a native Gemini slug"; Go TestWarnsWhenEngineCannotResolveAlias in pkg/workflow/configured_model_validation_test.go (Gemini and one declarative engine warn; Copilot, Claude, Codex, Pi and Agy don't).
7. A workflow-defined alias overriding a built-in one, for example opus pinned to copilot/claude-opus-5*. Use the fixture plus an added claude-opus-6. Result: model: opus runs on claude-opus-5.5 on every engine that runs the shared step. The built-in opus would pick claude-opus-6. Tests: "workflow-defined alias overrides built-in" in each harness test file; Go TestMainWorkflowAliasOverridesBuiltin in pkg/workflow/model_aliases_test.go.
8. Incomplete /reflect discovery. Result, on every engine that runs the shared step: exact ids pass through unchanged, and aliases (including pi sub-agent aliases) fail with catalog_unavailable after one refetch. Tests: "incomplete discovery" in copilot_harness, pi_models_json, pi_subagent_config, claude_harness, codex_harness and agy_harness test files.

End-to-end checks

Dispatch these on the PR branch, link the runs in the PR, and check each run's usage/aw_session.jsonl for the new event:

• Copilot: smoke-copilot-small (model: small).
• Claude: smoke-claude-on-copilot with its model switched to an alias, for example copilot/haiku.
• Codex: smoke-github-codex (copilot/auto must keep working).
• Pi: smoke-pi-sub-agents with its main model switched to an alias (this covers main and sub-agent models), and smoke-pi-routed-claude to confirm that model routing is unchanged.
• Gemini and Agy have no smoke on Copilot (smoke-gemini and smoke-agy use Gemini directly), so unit tests cover them.

Docs

• Describe the shared rule in docs/src/content/docs/reference/engines.md and the engine pages docs/src/content/docs/engines/{copilot,claude,codex,pi,gemini,agy}.md, including that "exact stays exact" covers gh-aw's step and that sandbox.agent.model-fallback: false turns off AWF's own rewrite.
• Correct reference/model-tables.md and reference/glossary.md: custom aliases now work, and resolution happens at runtime, not at compile time.
• Update model-alias-specification.md for provider-prefixed aliases. Its §14 also maps V-MAF-020 to claude_engine.go and to codex_tools.go, a file that doesn't exist.

Sequencing

This touches the pi harness and pkg/workflow/pi_engine.go, which #69429 also changed. #69429 merged on 16 Oct (6738f9ca21), so start from a main that includes it.

Scope

• Keep the PR to this one fix. Do not change files or behavior unrelated to it.
• Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR; merge main once main is fixed, and note in the PR which checks are known failures on main.
• This applies even if a bot comment asks to fix a failing check: fix only failures caused by this PR's changes.

Separately (not in this PR): the session collector rejected the DDG run's /reflect as invalid_model_inventory, because AWF reports routing_models: null for unconfigured endpoints. normalizeModelInventoryData accepts models: null but not routing_models: null, so that run's session has no model.inventory event.