refactor(core): move the models.dev catalog cache from disk to KV - #41649
Merged
Conversation
kitlangton
marked this pull request as ready for review
August 11, 2026 02:24
Contributor
Author
|
Simplify pass complete and pushed in Applied
Rejected
Verification remains green:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Move the implicit models.dev provider/model catalog cache from
Global.cachefiles into the coreKVservice.The catalog still fetches on a cache miss, refreshes stale entries on the existing five-minute cadence, recovers from corrupt cached blobs, and serves an existing cached catalog when a refresh fails.
Why
Core services should not use the filesystem to acquire the server's own state. The disk cache breaks or silently becomes ineffective in filesystem-less runtimes, while the catalog cache is naturally a keyed blob.
Using the database-backed KV service makes the cache available across every core runtime, including DO-SQLite/workerd. This continues the filesystem-acquisition direction from #41618, #41622, and #41629.
How
models-dev:catalogand custom endpoints undermodels-dev:catalog:<source-hash>.{ updatedAt, body }, retaining the raw JSON blob for corruption detection and replacing file mtime with an explicit freshness timestamp.updatedAtonly controls whether non-forced refresh can skip the network.models.file/OPENCODE_MODELS_PATHinput.FSUtilremains only for that configured file source; the implicit cache no longer uses it.GlobalandFlockare removed from the service graph.Testing
cd packages/core && bunx tsgo --noEmitcd packages/core && bun run test test/models.test.ts test/plugin/models-dev.test.ts test/catalog.test.ts(28 passed)cd packages/core && bun run test(1,634 passed, 16 skipped, 0 failed across 172 files)bun turbo typecheck --concurrency=3(33 tasks successful)Findings
<Global.cache>/models.json; custom source URLs used<Global.cache>/models-<Hash.fast(source)>.json. Each file contained the raw/api.jsonresponse.get()accepted a readable cache regardless of age and memoized the normalized result indefinitely in-process. On a miss,get()fetched and atomically wrote via a temporary file plus rename. A five-minute file-mtime TTL only controlledrefresh(false): refresh ran once at service startup and then five minutes after each completion, rechecked freshness underFlock, overwrote the file on success, invalidated the in-memory value, and publishedmodels-dev.refreshed. Forced refresh bypassed freshness. Refresh errors were logged and ignored, leaving the stale disk and in-memory values available. A malformed implicit cache file was removed and treated as a miss.packages/core/test/models.test.ts, which is ported to an in-memory KV service.OPENCODE_MODELS_PATHis a separate explicit catalog file input used by the CLI/test preload and remains supported. Othermodels-*.jsonreferences belong to unrelated fixtures or console exports.models-dev:cataloggives the canonical catalog a stable, discoverable key;models-dev:catalog:<source-hash>preserves isolation for custom source URLs without embedding arbitrary URLs in database keys. The value remains one blob plusupdatedAt, matching the cache's storage shape and preserving the old raw-response/corruption behavior while making the five-minute timestamp portable.