Skip to content

feat(onlion): the private status says which endpoint and vector space are live - #127

Merged
wefio merged 1 commit into
mainfrom
feat/admin-status-embedding
Oct 29, 2026
Merged

wefio merged 1 commit into
mainfrom
feat/admin-status-embedding

Conversation

@wefio

@wefio wefio commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Why

Three incidents had the same unanswered question - is this build talking to the endpoint I think it is - and every time it was answered by inferring from latency. /admin/status now answers it directly.

What it reports

field why
embedding.endpoint scheme, host and path - never the key. Read from the client the requests use, so it cannot drift from what is in effect
embedding.{provider, model, dimensions, profile, index_id} the vector space actually in use
embedding.{timeout_ms, document_timeout_ms, query_budget_ms, write_budget_ms} the four numbers that bound a request
revision which build is answering
memory already present; now next to the rest
failures.summary counts by code and by route. Reading twenty entries to answer "what is failing" is what made those incidents slow

Failure entries also carry embedding_host and embedding_model, so an old incident can be told apart by configuration instead of by timestamp.

Cost change: shared failure history is opt-in

It cost two bucket round trips per failure - a read and a CAS write on one shared key - paid exactly when the service is already in trouble. The platform log carries the same entry for free and is where incidents are actually read, so persistence now requires NMG_ERROR_HISTORY=1. /admin/status reports which scope it is looking at (isolate_only by default).

Verification

  • onlion npm run verify: 36 PASS.
  • New assertions: /admin/status is owner-only (401 without the owner identity), reports endpoint/model/index identity/timings/memory shape, never echoes the key, and reports the configured-endpoint case where a provider is stubbed.
  • eslint clean.

… are live

Three incidents had the same unanswered question - is this build talking to the endpoint I
think it is - and it was answered by inferring from latency. `/admin/status` now reports it
directly, from the client the requests themselves use so it cannot drift from what is in
effect:

- `embedding.endpoint` as scheme, host and path (never the key), plus provider, model,
  dimensions, profile, `index_id`, and the four timings that bound a request
- `revision`, so the answer names the build
- `memory`, already present, now alongside the rest
- `failures.summary`: counts by code and by route over the retained entries, because reading
  twenty entries to answer "what is failing" is what made those incidents slow

Failure entries also carry `embedding_host` and `embedding_model`, so an old incident can be
told apart by configuration instead of by timestamp.

**Shared failure history is now opt-in** (`NMG_ERROR_HISTORY=1`). It cost two bucket round
trips per failure - a read and a CAS write on one shared key - paid exactly when the service
is already in trouble, and the platform log carries the same entry for free. `/admin/status`
reports which scope it is looking at.

Tests: `/admin/status` is owner-only (401 without the owner identity), reports the endpoint,
model, index identity, timings and memory shape, never echoes the key, and reports the
configured-endpoint case where a provider is stubbed.

Verified: onlion npm run verify 36 PASS; eslint clean.
@wefio
wefio merged commit 8d5d91d into main Oct 29, 2026
8 checks passed
@wefio
wefio deleted the feat/admin-status-embedding branch October 29, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant