Defect (data-correctness, caught during a live gateway regeneration)
integrations/providers/usai/scripts/build-catalog.mjs's models.dev fuzzy matcher is too permissive: when it cannot find a correct match for a gateway model id, it falls through to a different model's data instead of leaving the fields absent.
Reproduced: with only claude-opus-5 / claude-sonnet-5 present under the anthropic provider in the enrichment subset, claude_4_5_haiku was enriched with Opus numbers (context 1,000,000, $5/$25 — correct is 200,000, $1/$5) and claude_4_5_sonnet picked up Sonnet-5 pricing ($2/$10 vs the correct $3/$15).
This is worse than missing data: the fields look populated and plausible, so a completeness check ("every model has cost/context") passes while the values are wrong. Shipping that would tell users Haiku costs $5/$25.
Fix
Add a precision guard: a candidate is accepted only when it agrees on both
- family/tier —
haiku must not match opus/sonnet; flash must not match pro; flash-lite must not match plain flash; mini/nano/pro variants must not match the base model; and
- major.minor version —
4.5 must not match 5 or 4.8 (handle 4_5, 4.5, -4-5, bare 5).
If no candidate satisfies both, return no match and emit the model with absent context/output/cost. Absent-but-honest beats populated-but-wrong.
Keep the legitimate normalizations that were verified working: gateway-suffix strips (guardrails.*, latest.*, -default-?v\d+) so gpt_5_5_default_v2 still matches gpt-5.5, and the llama4→llama-4 glue-split for the Bedrock meta.llama4-maverick-… form.
Tests to add
Hermetic (inline models.dev-shaped fixtures, not the committed subset):
claude_4_5_haiku must not match an opus/sonnet entry; claude_4_5_sonnet must not match claude-sonnet-5.
gemini-2.5-flash must not match gemini-2.5-pro; gemini-2.5-flash-lite must not match plain gemini-2.5-flash.
- Positive controls that must still match:
gpt_5_5_default_v2→gpt-5.5, llama_4_maverick→bedrock llama4-maverick, claude-opus-5→claude-opus-5, gemini-3.5-flash→gemini-3.5-flash.
Context
Deferred from the urgent model-refresh PR (which shipped correct values by carrying forward the reviewed numbers for the unchanged ids rather than trusting the matcher). The guard is what makes future live regenerations safe to trust.
AI-assisted (OpenCode).
Defect (data-correctness, caught during a live gateway regeneration)
integrations/providers/usai/scripts/build-catalog.mjs's models.dev fuzzy matcher is too permissive: when it cannot find a correct match for a gateway model id, it falls through to a different model's data instead of leaving the fields absent.Reproduced: with only
claude-opus-5/claude-sonnet-5present under theanthropicprovider in the enrichment subset,claude_4_5_haikuwas enriched with Opus numbers (context 1,000,000, $5/$25 — correct is 200,000, $1/$5) andclaude_4_5_sonnetpicked up Sonnet-5 pricing ($2/$10 vs the correct $3/$15).This is worse than missing data: the fields look populated and plausible, so a completeness check ("every model has cost/context") passes while the values are wrong. Shipping that would tell users Haiku costs $5/$25.
Fix
Add a precision guard: a candidate is accepted only when it agrees on both
haikumust not matchopus/sonnet;flashmust not matchpro;flash-litemust not match plainflash;mini/nano/provariants must not match the base model; and4.5must not match5or4.8(handle4_5,4.5,-4-5, bare5).If no candidate satisfies both, return no match and emit the model with absent context/output/cost. Absent-but-honest beats populated-but-wrong.
Keep the legitimate normalizations that were verified working: gateway-suffix strips (
guardrails.*,latest.*,-default-?v\d+) sogpt_5_5_default_v2still matchesgpt-5.5, and thellama4→llama-4glue-split for the Bedrockmeta.llama4-maverick-…form.Tests to add
Hermetic (inline models.dev-shaped fixtures, not the committed subset):
claude_4_5_haikumust not match an opus/sonnet entry;claude_4_5_sonnetmust not matchclaude-sonnet-5.gemini-2.5-flashmust not matchgemini-2.5-pro;gemini-2.5-flash-litemust not match plaingemini-2.5-flash.gpt_5_5_default_v2→gpt-5.5,llama_4_maverick→bedrock llama4-maverick,claude-opus-5→claude-opus-5,gemini-3.5-flash→gemini-3.5-flash.Context
Deferred from the urgent model-refresh PR (which shipped correct values by carrying forward the reviewed numbers for the unchanged ids rather than trusting the matcher). The guard is what makes future live regenerations safe to trust.
AI-assisted (OpenCode).