Preserve AWF candidate rankings and request/failure metadata in unified sessions (G-2/G-11) #69725

Problem

The unified session collector drops metadata that AWF already persists about routing candidates, request endpoints and HTTP results, and failure phase. Consequently, usage/aw_session.jsonl cannot tell a reader which candidates the router ranked or provide the recorded endpoint/status context for deviations and failures. A consumer that reads only unified sessions cannot recover that information.

This is G-2, step 1, plus G-11 of the analytics design in #69507. The Model routing analytics implementation, githubnext/gh-cao#18475, reads unified sessions and documents candidate ranking as an outstanding gap. This issue fixes the collector contract, not the analytics page.

Verified against main

Checked on October 17, 2026, against:

• github/gh-aw main at 55f40737985235e50c1515d0f4481018803ba8f4.
• github/gh-aw-firewall main at 5c08caf3840608cd4f3902f4ca9f724684b3a831.

Two source-shape corrections are important for implementation and tests:

• Selection endpoint: current selection records use endpoint, not selected_endpoint. Current request records carry selected_endpoint. Keep the existing selection endpoint; preserve selectedEndpoint wherever AWF actually supplies it, but do not fabricate it on a selection that lacks it.
• Meaning of unavailable: current AWF uses this array for unobserved request-telemetry dimensions, initially model and effort when the body has not been observed, and an empty array when the body is observed. It is not a list of unavailable model IDs. Preserve the supplied array without reinterpreting it. The G-11 description in the design currently misstates this meaning.

Recorded reproduction

The successful smoke-copilot-routed main run 38984582061, created October 17, 2026, at 00:03:42 UTC, ran commit 354b5c3423ea95046f4db235f9fea7bb2885aaa8. Its existing usage artifact contains both agent/model-routing.jsonl and aw_session.jsonl, as well as agent/token_usage.jsonl; no new run is needed.

The raw routing log has schema model-routing/v0.28.59, one classification record, one selection record, and four request records. Its selection has 12 ranked choices, eligible_choices 12, and catalogue_overlap 12. Its requests have all three endpoint fields, HTTP status 200, and explicitly supplied empty unavailable arrays. The corresponding published firewall.model_routing events omit those fields.

The committed goldens corroborate both the recorded metadata and compatibility boundary:

• codex-routed-provider-unavailable, from run 37728959715: the older session retains raw snake_case AWF model-routing/v0.28.44 records, including 23 ranked choices, counts, request endpoints/status, and failure phase.
• copilot-budget-exhausted-steering, from run 38174216426: the raw AWF model-routing/v0.28.51 log retains six ranked choices, both counts, endpoints/status, and failure phase. Its committed session is already projected and lacks these fields. Do not describe that session as retaining the raw record.

Possible solution

Extend the existing firewall.model_routing metadata projection rather than adding another event or telemetry source. Accept the camelCase name and its snake_case source spelling, following the table's existing alias conventions, and emit:

This remains metadata-only. Keep excluding conversation_sha256, detail, method, pathname, their camelCase aliases where applicable, and other free-form or payload-bearing fields. Project candidate entries explicitly to rank/optional ID/model/supplied effort, not an opaque copy of arbitrary nested properties.

Missing fields remain missing: no inferred counts, manufactured endpoints, zero values, empty arrays, or null placeholders for absent data. Explicitly supplied zero, null effort, or empty arrays remain supplied values, not missing values. Both raw snake_case records and already-projected camelCase events must normalize deterministically and idempotently without mutating the input.

No per-candidate predictions of any kind. Routing tables are ordered lists; retain the ranking, the observed pick, and observed results only. Step 2 of G-2 in the early design drafts was dropped in #69507's 0.2.0 revision. Predicted success, predicted credits/AIC, new routing-table formats, exclusion explanations not already recorded, AWF/router changes, and other analytics gaps are out of scope.

Update directly dependent writer declarations in actions/setup/js/types/unified_session.d.ts and regenerate the session schemas using the existing generator as needed for the added contract. Do not turn this into a schema/type cleanup.

Specification and ordering dependency

Update docs/src/content/docs/specs/unified-agent-session-specification.md:

• Extend T-UAS-070, "Model routing attribution", its essential firewall.model_routing field list, and the model-routing payload summary with the fields and semantics above, including rank order, omission behavior, and privacy exclusions.
• Add a change-log entry and update the document version/date and version references consistently. The specification's own rule at the end of Section 13 requires a minor document-version increment for backward-compatible additions. Keep numeric session.format version 1 unchanged.
• Start implementation from a main that includes Resolve model aliases the same way on every engine #69487. That PR is open as of this investigation, adds T-UAS-074, and changes the same specification from 1.10.0 to 1.11.0. Preserve its additions and select the next available minor version after merging it, rather than independently claiming 1.11.0 or reusing T-UAS-074.

Audit decision

Leave gh aw audit output unchanged in this fix. The existing renderConsoleModelRoutingSelection already displays top_choices from the raw routing analysis. Do not add a new full-ranked-list or eligible-count display, or refactor Go routing readers, as a prerequisite to preserving session metadata.

The new fields make that evidence available to unified-session consumers; expanding audit or the analytics page is separate work. Under this choice, no new Go rendering tests or audit-output changes are expected. Continue running the existing TestModelRoutingGoldenAudit and golden verification.

Acceptance cases

Use minimized, sanitized copies of the recorded shapes above, with provenance identifying the run, artifact, source path, and schema. Do not invent production fields or describe compatibility/privacy variants as new recorded runs. Name each case in actions/setup/js/unified_session_payload.test.cjs as follows.

1. Selection ranking and pool metadata

Test: preserves recorded AWF selection rankings and pool counts in source order

Input: the selection at 2026-10-17T00:05:52.673Z from run 38984582061's usage/agent/model-routing.jsonl, plus the recorded selection records in both named golden fixtures.

Exact observable result: the smoke selection's projected firewall.model_routing.data.rankedChoices contains exactly these entries in this order, with numeric ranks and no invented IDs:

eligibleChoices and catalogueOverlap both equal 12. Existing endpoint remains /responses. This recorded selection has no selected_endpoint, so selectedEndpoint is absent, not inferred. Whenever a source actually supplies selected_endpoint or selectedEndpoint, retain it exactly; case 2 exercises the recorded producer shape that supplies it.

For the Codex and budget selections, the complete model/effort sequences equal their raw AWF sequences element-for-element, with ranks 1..23 and 1..6, respectively; counts are 23/23 and 6/6. Do not sort, deduplicate equal models with different efforts, truncate, or infer counts from array length.

2. Request endpoints, status, and availability

Test: preserves recorded AWF request endpoints HTTP status and unavailable metadata

Input: the request with ID 3e8b8c51-1a07-46a5-a21f-c8ecea381bf4 from the same smoke run; also the Codex request ff510188-891a-4ad1-97eb-a12c47c2d7fc and budget request 2698b73b-55b7-4c97-8d6d-8d7c88b69461 from the golden raw logs.

Exact observable result: all three projected requests have selectedEndpoint, requestedEndpoint, and upstreamEndpoint equal to /responses, and unavailable equal to the explicitly recorded empty array. Their status/existing outcome pairs remain 200/completed, 400/failed, and 403/rejected, respectively. The request ID and existing deviation/model/effort fields are unchanged. If unavailable is absent in an input, it is absent in the output, not synthesized as an empty array.

3. Failure phase

Test: preserves recorded AWF failure phase independently of collection provenance

Input: the Codex failure at 2026-10-08T04:47:41.353Z and budget failure at 2026-10-11T20:59:12.712Z from their raw golden routing logs.

Exact observable result: both projected events have data stage equal to failure and data phase equal to primary. Existing code values stay provider_unavailable and ai_credits_limit_exceeded, respectively. Preserve source timestamps and collection provenance unchanged; do not replace data phase with provenance phase or manufacture a phase on failure records that do not supply one.

4. Older or missing-field input

Test: leaves absent AWF routing metadata absent for older session records

Input: the already-projected model-routing/v0.28.51 selection event at 2026-10-11T20:58:04.898Z from copilot-budget-exhausted-steering/usage/aw_session.jsonl. It lacks all nine newly retained fields even though its raw AWF counterpart supplies several of them. Also exercise explicitly labelled compatibility variants of the real selection/request/failure inputs with the new fields omitted.

Exact observable result: none of rankedChoices, eligibleChoices, catalogueOverlap, selectedEndpoint, requestedEndpoint, upstreamEndpoint, unavailable, status, or phase is an own property of the projected event data when absent from that input. Existing projected data is unchanged. No zero, empty array/object/string, null placeholder, reconstructed ranking, or invented default appears. Older files and older AWF shapes remain loadable.

5. Privacy and nested allowlisting

Test: excludes private AWF routing fields while retaining only candidate metadata

Input: the recorded selection and request shapes, plus the AWF controller failure shape, with clearly labelled privacy-test sentinels in the real AWF fields conversation_sha256, detail, method, and pathname. Exercise their applicable aliases and unexpected nested candidate properties as privacy variants, not as claimed recorded telemetry.

Exact observable result: none of those four fields or applicable aliases appears anywhere in the projected routing event; none of their sentinel values survives serialization. Candidate entries contain only their one-based rank, model, optional supplied ID, and supplied effort. All allowed metadata in cases 1-3 remains intact; no arbitrary nested payload or free-form prediction/explanation is copied through.

6. Golden regression boundary

Test: projects recorded model-routing goldens with only the intended metadata additions

Input: routing records under pkg/cli/testdata/model_routing_golden/, including both named cases' raw logs and their existing raw/already-projected session variants.

Exact observable result: projected routing events retain only the new allowlisted metadata and its canonical names/ranks, with ordered candidate deep equality against the raw sources. Normalizing an already-normalized event again yields the same event; JSON round trips preserve the supported metadata; the input remains deeply equal to its original copy.

Review any affected session fixture changes: only changes attributable to this projection are permitted. Preserve historical raw-file compatibility coverage. Run make update-model-routing-golden, review every generated diff, and then run make verify-model-routing-golden. Under the no-audit-output-change decision, existing audit expected*.json output should remain unchanged, including routing status/failure attribution, top choices, costs, token totals, deviations, sub-agent attribution, and documented known discrepancies. Do not recapture or churn unrelated fixtures to make checks pass.

Parameterize cases 1-5 over snake_case raw metadata and its camelCase equivalent, with the same semantic result, and check idempotence and input immutability. For optional candidate IDs, use an explicitly labelled contract-compatibility variant with an ID from a real AWF route-choice shape; do not pretend current persistence emits it. Test additions must stay in the existing payload suite and directly related fixtures. Go tests are needed only if an explicitly justified audit behavior change is introduced; that is not the chosen scope here.

Scope

• Keep the PR to this one fix. Do not change files or behavior unrelated to it. Necessary collector tests, directly dependent declarations/generated schemas, specification changes, and narrowly affected fixtures belong with the fix; unrelated analytics, inventory, alias resolution, routing behavior, or cleanup does not.
• Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR. Merge main once main is fixed, and note in the PR which checks are known failures on main.
• This applies even if a bot comment asks to fix a failing check: fix only failures caused by this PR's changes.