Repository navigation
revert: stop putting the endpoint in the embedding index identity - #126
Merged
Merged
Conversation
The endpoint was added to `indexId` so a switch between two deployments of the same model would mark stored vectors stale instead of mixing two spaces. The risk it guarded against was speculative - the same model name from the same provider - while the cost was measured: every existing user's next write re-embedded their whole store, 32 documents per batch, and on a stalled provider that turned one Add into five to twelve embedding calls and 53-76 seconds. Reverted to the previous identity (`text-embedding-v4@5980a18a2d69`), so the vectors stored before the change are valid again and the backfill stops happening. The test that asserted the separation went with it. Not reverted, because they do not touch storage: the 25 s document timeout, the 45 s write budget, the per-text query cache, and the memory trace.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this reverts, and why
indexIdgained the provider endpoint so that switching between two deployments of the same model would mark stored vectors stale instead of mixing two spaces. The risk it guarded against was speculative - the same model name from the same provider - and the cost is now measured:every existing user's next write re-embedded their whole store, 32 documents per batch, inside
mutateUser's retry loop. Under contention each retry re-opened a newer partition, found new pending rows and embedded again. Deployed logs show an Add taking 53-76 s withlast_embedding.ms3.8-12.3 s per call and only 8 s of partition work at the end.Reverted to the identity that was in use before (
text-embedding-v4@5980a18a2d69), so vectors stored earlier are valid again and the backfill stops.What stays
Nothing here touches storage: the 25 s document timeout, the 45 s write budget, the per-text query cache (one embedding call per search instead of up to twenty-four), and the memory trace.
Verification
indexIdfor the same model/dimensions istext-embedding-v4@5980a18a2d69, identical to the value before the change.tests/core/openai-embedding.test.ts: 11 pass (the test that asserted endpoint separation was removed with the behaviour it pinned).tsc --noEmitand eslint clean.Follow-up worth doing later
Embedding should happen before
mutateUser's retry loop, keyed by the request's own documents, so a CAS retry can never re-embed. That removes this amplification class by construction rather than by keeping the index stable.