ASI Cloud embeddings support - #333
Conversation
`openai_embed_batch` hardcoded the `/openai/` gateway location and `text-embedding-3-large`, so remote embeddings could only reach api.openai.com. Build the base URL from `embeddingprovider` instead — the same `<gateway>/<provider>/` convention `lib_llm_ext.AIProvider` already uses for chat, so the existing Nginx locations supply each provider's key — and take the model from a new `embedding_model`. `initKnowledge` tested for `OpenAI` and fell through to `Local`, which loaded the SentenceTransformer model even for a remote provider. Test for `Local` instead, mirroring the dispatch in `embed`. `scripts/omegaclaw` pinned the embedding backend to `Local` for every provider but `OpenAI` and never passed a model, so the launcher could not express this. `EMBEDDING_PROVIDER` and `EMBEDDING_MODEL` now override it; with neither set the arguments it emits are unchanged. Defaults are unchanged. Verified against ASI Cloud with `embeddingprovider=ASICloud embedding_model=WhereIsAI/UAE-Large-V1`: remember and query round-trip through the gateway, recall is correct, and the local model is never loaded — agent RSS 155 MiB instead of 1.7 GiB.
Select the embedding provider and model by configuration
| echo | ||
| echo -e "Environment:" | ||
| echo -e "\tEMBEDDING_PROVIDER override the embedding backend (default depends on provider)" | ||
| echo -e "\tEMBEDDING_MODEL set the model asked of a non-Local EMBEDDING_PROVIDER" |
There was a problem hiding this comment.
Why this is passed via environment? Does ASI:Create start instances via ./scripts/omegaclaw script? I would recommend starting using Docker command line instead and don't touch omegaclaw script at all because it will probably be removed in nearest future.
vsbogd
left a comment
vsbogd
left a comment
There was a problem hiding this comment.
My understanding the main idea is to introduce new configuration parameter to set embeddings api endpoint and model. I am not sure how this should work. PR allows replacing "prefix" in the endpoint URL, but URL itself points to the local Nginx proxy which doesn't pass through some arbitrary prefix.
|
The prefix is not an arbitrary string, it's a provider id, and Nginx already has a location for each provider. Chat uses the same scheme today: with GATEWAY_URL set, AIProvider._create_client builds <GATEWAY_URL>//, so provider=ASICloud chat already goes through location /asicloud/. This PR sends embeddings to the same location with the same key. That's why the PR doesn't touch the proxy config. True, a wrong value is not validated. But I think validation belongs to a future registerEmbeddingProvider in the plugin API, like #222 did for LLM providers. This PR only reuses the existing URL convention. About the launcher: agreed, I'll drop the script changes and rebase. embeddingprovider= and embedding_model= are normal container arguments, so I would propose just adding a few parameters for Create team |
Thanks @Eljaja I got it finally. |
2 participants
Description
rag.openai_embed_batchhardcoded the/openai/gateway location andtext-embedding-3-large, so remote embeddings could only reach api.openai.com. It now builds the base URL fromembeddingprovider— the<GATEWAY_URL>/<provider>/conventionlib_llm_ext.AIProvideralready uses for chat, so the existing Nginx locations supply the key — and takes the model from a newembedding_model.initKnowledgetested forOpenAIand fell through toLocal, loading the SentenceTransformer model even for a remote provider. It now tests forLocal, mirroringembed.scripts/omegaclawpinned the embedding backend toLocalfor every provider butOpenAIand never passed a model, so the launcher could not express this.EMBEDDING_PROVIDERandEMBEDDING_MODELnow override it.Defaults unchanged. 8 files, +48/−14.
Embeddings are not a plugin: #222 moved channels and LLM providers to the plugin API, but there is no
registerEmbeddingProvider, so the choice is still the branch insrc/memory.metta. This PR does not add a parallel registry — it only putsrag.pyon the convention #222 established, and stays a subset of any future embedding plugin API.How Has This Been Tested?
Autotests/test_openai_runtime_embeddings.py+tests/— 15 passed. The existing proxy test passes with unchanged assertions; one new test covers provider/model routing.-p Test -t test) — 6 passed:test_memory_chromadb_mock,test_transition_metta_to_remember_mock,test_transition_pin_to_remember_mock,test_transition_episodes_after_eviction_mock,test_skill_query_mock,test_memory_episode_mock.embeddingprovider=ASICloud embedding_model=WhereIsAI/UAE-Large-V1) —remember/queryround-trip through the gateway, recall correct, 0 errors, agent RSS 155 MiB against 1.7 GiB onLocal.embeddingprovider=Localregression — boots clean, model loaded as before.scripts/omegaclaw start -p ASICloudwith neither env var set emits byte-identical arguments; with them set it emitsembeddingprovider=ASICloud embedding_model=..., and the agent resolves both.how to run
Checklist