Copilot CLI model-selection persistence aborts routed attribution and token usage #69965

Problem

Routed Copilot CLI runs execute on the router-selected model, but the runner's post-step rejects their model-selection log and skips both routing attribution and token-usage accounting. This is a runtime metadata regression, not a failure to select or enforce a route.

The affected path is shared by routed Copilot CLI workflows, not just the smoke tests: an initially unset COPILOT_MODEL becomes a concrete model after routing, but the observer retains the earlier empty configured value. The non-routed environment-fallback path has the same defect.

Verified run evidence

The six supplied runs were inspected: three failing CLI runs on main, the failing CLI run for #69848, and two passing controls. Dates below are UTC. The four October 18 main runs use eb42b26807ac37c20f0a43ba3beba68375ab43e1.

In each failing CLI run, the agent's Parse token usage for step summary step reports ERR_PARSE: Cannot record Copilot model selection: Invalid Copilot model selection. The later smoke assertion reports runner-written routing status is missing, expected selected. The parse step is configured with continue-on-error; the smoke assertion is the job-failing boundary.

The downloaded agent artifact from run 39147795806 contains:

• sandbox/agent/logs/copilot-model-selection.json: status is observed, configured_model is the empty string, and both resolved_model and effective_model are gpt-5.6-luna. The other two failing main artifacts have the same empty configured value, with their selected model in the resolved/effective fields.
• Both agent/aw_info.json and runtime/aw_info.json: engine_id is copilot, model remains auto, and neither model_routing nor model_selection is present.
• sandbox/firewall/logs/api-proxy-logs/model-routing.jsonl: real model-routing/v0.28.62 selection and successful request records corroborate the selected model and endpoint. The token-usage log also exists, but the agent's aggregated agent_usage.json is absent.

The workflow.info event in usage/aw_session.jsonl has data.model and data.requestedModel equal to auto, but no data.modelRouting or data.modelSelection. Its workflow provenance points to runtime/aw_info.json. In the same-commit SDK control, the corresponding event has model equal to gpt-5.6-luna and modelRouting.status equal to selected; its runner metadata and token-usage aggregate are populated.

Root cause and regression

The relevant source was checked against the failing revision, not inferred only from the smoke assertion.

1. In copilot_harness.cjs, main(), configuredCopilotModel captures COPILOT_MODEL before routing and before applyModelFallback. The routed agent job has no COPILOT_MODEL, so the captured value is empty.
2. resolveCopilotModelRouting calls applyCopilotRoutingSelection, which sets COPILOT_MODEL to the selected wire model. The later verifyModel check and observer construction therefore enable CLI observation, but pass the stale empty configured value to createCopilotModelObserver. Its observed record has correct resolved/effective models and an empty configured_model.
3. persistCopilotModelSelection correctly rejects an empty configured identifier. In parse_token_usage.cjs, main(), persistence runs first in the same error boundary as recordModelRoutingFromArtifacts and token accounting. Its throw skips both operations. The final snapshot still runs, but copies metadata without the missing attribution.
4. SDK mode does not enable this CLI observer, so it produces no such invalid selection log and the post-step proceeds normally.

The new rejecting persistence function and its position before routing were introduced by commit c29fcddb1aa4b0161ebecbc7e7efec86da01e5a0, "Keep Copilot model metadata writes on the runner", within #69162. The PR's merged title is "Harden Copilot CLI and Cloud Hypervisor compatibility"; it merged on October 17, 2026, at 19:33:35 UTC. Previously, the post-step began with routing attribution rather than this new failure point.

Fallback is also confirmed locally, not merely suspected. With routing disabled, no COPILOT_MODEL or model argument, and GH_AW_MODEL_FALLBACK set to gpt-5.6-luna, an offline invocation of the real harness with a local CLI test double applies the fallback and records the same empty configured value. The real runner persistence function then rejects it. No hosted fallback run was inspected.

An additional offline replay used the unchanged selection, workflow metadata, routing, outcome, and token-usage records from run 39147795806. The real post-step reported the same error, wrote no token summary or aggregate, and left model as auto. Calling recordModelRoutingFromArtifacts independently on those records succeeded with status selected and wire model gpt-5.6-luna. No workflows were dispatched for this investigation.

Intended behavior and possible solution

For these routed runs, define configured_model as the requested dynamic-routing input, auto, consistent with the original workflow metadata. resolved_model is the router-selected concrete model; effective_model is the model observed for the main CLI invocation. For the representative run those latter fields are both gpt-5.6-luna. This is a metadata meaning for auto, not an instruction to use Copilot's native auto picker: actual CLI requests must continue using the router-selected model.

For non-routed CLI runs, retain the explicit configured model or alias before resolution. When only the environment fallback supplies the model, configured_model must instead be that fallback value, never the earlier empty environment value.

The focused fix should:

1. Resolve the source of the requested/configured model before freezing the value used by the observer. Preserve the distinction between the requested identifier and the resolved wire model; do not blindly recapture the environment after alias/routing resolution.
2. Isolate model-selection persistence errors from routing attribution and token accounting. Retain a visible warning or step failure with the original selection error, but still attempt recordModelRoutingFromArtifacts, the token-usage summary/aggregate, outputs, and the final runner metadata snapshot. Invalid selection evidence must remain invalid, not become a synthetic successful selection.
3. Preserve runner-only workflow metadata writes, validated selection fields, exact-session observation, and proxy-corroborated routing attribution. Do not restore sandbox metadata writes, relax validation to accept an empty configured model, or weaken R1 to trust the agent-written routing outcome.

Acceptance cases and covering tests

The test names below are proposed regressions to add or existing coverage to retain, not claims that the missing cases already exist.

Fixture provenance is required: build the regression fixtures from sandbox/agent/logs/copilot-model-selection.json and agent/aw_info.json / runtime/aw_info.json in the real agent artifact of run 39147795806. Use that artifact's real routing and token-usage records for the post-step test as needed. Preserve the actual field names, types, absent fields, and empty configured value; do not substitute invented record shapes or older routing-schema examples. Document any redaction of unrelated metadata.

The implementation PR must carry the smoke-routing label.

Scope

• Keep the PR to this one fix. Do not change files or behavior unrelated to this model-selection/runner-accounting regression and its directly related tests or documentation.
• Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR; merge main once main is fixed, and note in the PR which checks are known failures on main.
• This applies even if a bot comment asks to fix a failing check: fix only failures caused by this PR's changes.