Repository navigation
Fix international model catalog aggregation - #110
Merged
Merged
Conversation
Use the personal model table as the primary catalog and supplement missing picker and account-root entries from /v3/config. Fix WebUI regional filtering to use published profile metadata for automatic routes. Fixes #109
Reviewer's GuideThe PR changes personal catalog loading to merge authoritative personal picker/account-root data with missing entries from Sequence diagram for personal model catalog aggregationsequenceDiagram
participant Caller
participant Catalog as fetch_model_scopes
participant Personal as fetch_personal_model_scopes
participant Config as _fetch_config_model_scopes
Caller->>Catalog: fetch_model_scopes(prefer_personal=true)
Catalog->>Personal: fetch_personal_model_scopes()
alt personal catalog available
Personal-->>Catalog: personal picker and account scopes
Catalog->>Config: _fetch_config_model_scopes()
alt config catalog available
Config-->>Catalog: legacy picker and account scopes
Catalog->>Catalog: _merge_model_scopes(personal, legacy)
Note over Catalog: Personal declarations win for duplicate IDs
else config catalog unavailable
Config-->>Catalog: RuntimeError or ValueError
Catalog-->>Caller: personal scopes
end
else personal endpoint unavailable
Personal-->>Catalog: _PersonalCatalogUnavailable
Catalog->>Config: _fetch_config_model_scopes()
Config-->>Catalog: legacy scopes
Catalog-->>Caller: legacy scopes
end
Catalog-->>Caller: merged model scopes
Flow diagram for WebUI regional model filteringflowchart LR
Model[ModelRule] --> Explicit{region present?}
Explicit -->|yes| Match[Compare with selected region]
Explicit -->|no| Profile{profile present?}
Profile -->|yes| Prefix[Use profile region prefix]
Profile -->|no| Metadata[Inspect metadata_by_profile]
Metadata --> Prefix
Prefix --> Match
Match --> Result[Include or exclude model]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 5 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="app/credits.py" line_range="800" />
<code_context>
+ try:
+ legacy = _fetch_config_model_scopes(access_token, user_agent, domain=domain,
+ uid=uid, enterprise_id=enterprise_id)
+ except (RuntimeError, ValueError):
+ # Personal data is already trusted; a supplemental catalog failure must not erase it.
+ return personal
</code_context>
<issue_to_address>
**Valid catalogs disappear on supplemental 401**
When the personal endpoint returns a valid catalog and `/v3/config` responds with HTTP 401, `fetch_model_scopes` does not catch `AuthExpiredError`, so `_sync_one_account_catalog` never publishes the valid personal catalog and a new account has no usable catalog.
Catch `AuthExpiredError` as a supplemental catalog failure and publish the trusted personal catalog.
</issue_to_address>
### Comment 2
<location path="app/credits.py" line_range="801-802" />
<code_context>
+ legacy = _fetch_config_model_scopes(access_token, user_agent, domain=domain,
+ uid=uid, enterprise_id=enterprise_id)
+ except (RuntimeError, ValueError):
+ # Personal data is already trusted; a supplemental catalog failure must not erase it.
+ return personal
+ return _merge_model_scopes(personal, legacy)
+ return _fetch_config_model_scopes(access_token, user_agent, domain=domain,
</code_context>
<issue_to_address>
**Supplemental outages hide legacy models**
When the personal endpoint succeeds but the supplemental v3 request fails with a caught error during refresh, `fetch_model_scopes` returns personal-only scopes as a success, so `_sync_one_account_catalog` logs a successful refresh, replaces any merged catalog, and resets its freshness timestamp; operators miss the failure and legacy-only models remain unavailable until the next refresh, up to six hours later.
Surface supplemental failures and prevent partial scopes from being treated as fresh or replacing an existing merged catalog.
</issue_to_address>
### Comment 3
<location path="app/credits.py" line_range="759" />
<code_context>
+ rows = list(primary.get(scope) or [])
+ seen = {item.get("id") for item in rows if isinstance(item, dict) and item.get("id")}
+ rows.extend(item for item in supplemental.get(scope) or []
+ if isinstance(item, dict) and item.get("id") not in seen)
+ merged[scope] = rows
+ return merged
</code_context>
<issue_to_address>
**Disabled models become discoverable**
When the personal catalog disables a model while `/v3/config` returns that ID as enabled, `_merge_model_scopes` builds `seen` after disabled personal rows have been removed, then appends the supplemental row, making the disabled model discoverable in the picker or account-root catalog.
Preserve disabled personal model IDs when filtering supplemental rows during the merge.
</issue_to_address>
### Comment 4
<location path="app/credits.py" line_range="798-803" />
<code_context>
+ personal = None
+ if personal is not None:
+ try:
+ legacy = _fetch_config_model_scopes(access_token, user_agent, domain=domain,
+ uid=uid, enterprise_id=enterprise_id)
+ except (RuntimeError, ValueError):
+ # Personal data is already trusted; a supplemental catalog failure must not erase it.
+ return personal
+ return _merge_model_scopes(personal, legacy)
+ return _fetch_config_model_scopes(access_token, user_agent, domain=domain,
+ uid=uid, enterprise_id=enterprise_id)
</code_context>
<issue_to_address>
**Older refresh overwrites newer catalog**
When a background account refresh and a catalog scan overlap for the same account, and the older refresh completes last, the older request can finish its personal and v3 fetches after a newer refresh has stored its result, then return this older merged snapshot to the caller for storage; the cache replaces the newer catalog and resets its freshness time, so callers keep seeing stale models until the next refresh.
Track a per-account refresh sequence or fetched-at version and reject a result if a newer refresh has already been stored.
</issue_to_address>
### Comment 5
<location path="tests/test_personal_models.py" line_range="105" />
<code_context>
+ client = self._client(self.response(200, personal_payload()), self.response(503, {"code": 503}))
+ scopes = credits.fetch_model_scopes(
+ jwt("https://www.codebuddy.cn/auth/realms/copilot"), prefer_personal=True)
+ self.assertEqual([item["id"] for item in scopes["picker"]], ["auto", "hy4-preview"])
+ self.assertEqual(client.get.call_count, 2)
+
</code_context>
<issue_to_address>
**Personal root models escape the failure check**
When the supplemental v3 request fails, `test_personal_table_survives_supplemental_v3_failure` checks only `scopes["picker"]` on line 105, so it passes if the failure path preserves the picker but returns an empty `account` scope; callers then lose personal root models without this regression test failing.
Assert that the personal account scope is also preserved after the supplemental request fails.
</issue_to_address>Linked issues
#109 webui中更新1.5.0后，国际版账号都搜不到模型了。 · 1 of 1 met
| # | Requirement | Status | Notes |
|---|---|---|---|
| 1 | 国际版账号能搜到模型 | ✅ Met | Personal catalogs are supplemented with missing /v3/config models, and the WebUI’s international filter recognizes intl-* profile metadata. |
Sourcery assessment
Needs a human reviewer. 5 findings to address first, and v3-only model entries can be added to the personal account’s cached picker and routing catalogs. Reverting stops future merges, but already cached entries may remain until the catalog is refreshed; that cache is recomputable.
Blocking findings: app/credits.py:800, app/credits.py:802, app/credits.py:759, app/credits.py:803, tests/test_personal_models.py:105
| rows = list(primary.get(scope) or []) | ||
| seen = {item.get("id") for item in rows if isinstance(item, dict) and item.get("id")} | ||
| rows.extend(item for item in supplemental.get(scope) or [] | ||
| if isinstance(item, dict) and item.get("id") not in seen) |
There was a problem hiding this comment.
🟡 Medium · Disabled models become discoverable
When the personal catalog disables a model while /v3/config returns that ID as enabled, _merge_model_scopes builds seen after disabled personal rows have been removed, then appends the supplemental row, making the disabled model discoverable in the picker or account-root catalog.
Preserve disabled personal model IDs when filtering supplemental rows during the merge.
Prompt for AI agents
In `app/credits.py` at line 759:
**Disabled models become discoverable**
When the personal catalog disables a model while `/v3/config` returns that ID as enabled, `_merge_model_scopes` builds `seen` after disabled personal rows have been removed, then appends the supplemental row, making the disabled model discoverable in the picker or account-root catalog.
Preserve disabled personal model IDs when filtering supplemental rows during the merge.| client = self._client(self.response(200, personal_payload()), self.response(503, {"code": 503})) | ||
| scopes = credits.fetch_model_scopes( | ||
| jwt("https://www.codebuddy.cn/auth/realms/copilot"), prefer_personal=True) | ||
| self.assertEqual([item["id"] for item in scopes["picker"]], ["auto", "hy4-preview"]) |
There was a problem hiding this comment.
🟡 Medium · Personal root models escape the failure check
When the supplemental v3 request fails, test_personal_table_survives_supplemental_v3_failure checks only scopes["picker"] on line 105, so it passes if the failure path preserves the picker but returns an empty account scope; callers then lose personal root models without this regression test failing.
Assert that the personal account scope is also preserved after the supplemental request fails.
Prompt for AI agents
In `tests/test_personal_models.py` at line 105:
**Personal root models escape the failure check**
When the supplemental v3 request fails, `test_personal_table_survives_supplemental_v3_failure` checks only `scopes["picker"]` on line 105, so it passes if the failure path preserves the picker but returns an empty `account` scope; callers then lose personal root models without this regression test failing.
Assert that the personal account scope is also preserved after the supplemental request fails.Keep an already fetched personal catalog when the supplemental /v3/config request fails or rejects the token, record the result as incomplete so the refresh is retried, and never replace a complete cached catalog with an incomplete one. Reject a catalog write that finishes after a newer refresh of the same account, and report supplemental failures in the manual scan and representative probe. Cover the account scope, partial cache and stale sequence cases.
Remove unused imports and variables, drop needless f-strings, and rename ambiguous loop variables flagged by ruff across the gateway, tests, scripts and preview. Keep the re-exported trial_rewards module and mark the deliberate sys.path-before-import test files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/v2/enterprises/personal/modelstable as the primary catalog and supplement missing picker and account-root models from/v3/config.Verification
uv run --locked --no-build python -B tests/test_personal_models.pyfor s in tests/test_*.py; do uv run --locked --no-build python -B "$s"; donecd web && vp test run src/modelMetadata.test.tsx src/models.test.tsx src/components.test.tsxcd web && vp checkcd web && vp buildgit diff --checkFixes #109
Summary by Sourcery
Improve international model catalog aggregation by combining personal and legacy catalogs, preserving partial results safely, and applying regional filters from published profile metadata.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests: