Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions TRACKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@

| | |
|---|---|
| **Last updated** | 2026-06-11 |
| **Last updated** | 2026-06-19 |
| **Current phase** | Phase 7 — Pilot, Harden, GA (**10 / 10 steps ✅**) — **all phases complete** |
| **Overall** | **84 / 84 steps ✅ — v1.0.0 GA** |
| **Next action** | **🎉 GA reached.** Engineering scope (Phases 0–7) complete; `v1.0.0` cut. Remaining items are external/process (external pentest, SOC 2 audit, registry first-publish, signed referenceable customers, on-call staffing + status page, launch assets) — tracked in the [GA readiness checklist](docs/ga/ga-readiness-checklist.md). Post-GA: cut the **v1.0.1** patch tag (serve-path fix below; note: GitHub Actions runs are currently blocked by an account **billing/spending-limit** failure — every `docker.yml` run since GA failed at job start, so the GHCR image was never actually published), then the V1.1 backlog. **V1.1 underway:** Step 3.10 corpus admin CRUD — gateway write surface + console wiring ([#189](https://github.com/officialCodeWork/AgentContextOS/pull/189), first post-GA feature). |
| **Next action** | **🎉 GA reached.** Engineering scope (Phases 0–7) complete; `v1.0.0` cut. Remaining items are external/process (external pentest, SOC 2 audit, registry first-publish, signed referenceable customers, on-call staffing + status page, launch assets) — tracked in the [GA readiness checklist](docs/ga/ga-readiness-checklist.md). Post-GA: cut the **v1.0.1** patch tag (serve-path fix below; note: GitHub Actions runs are currently blocked by an account **billing/spending-limit** failure — every `docker.yml` run since GA failed at job start, so the GHCR image was never actually published), then the V1.1 backlog. **V1.1 underway:** Step 3.10 corpus admin CRUD landed ([#189](https://github.com/officialCodeWork/AgentContextOS/pull/189)); a follow-on post-GA batch in [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194) adds corpus/webhook **PATCH** (completes the 3.10 *update* path), admin-UI bug fixes, real-backend (OpenAI LLM + content hydration) wiring, and the local Docker/demo stack. |

**Recently shipped**

- **3.10** ✅ *(V1.1)* Corpus admin CRUD — the console can now **create / delete corpora against the gateway** (was local-only React state with no write route — `POST /v1/corpora` 405'd, so "created" corpora never persisted). `CorpusStore` SPI gains tenant-scoped `create` / `delete` (Noop + Pg; same cross-tenant invisibility rule as `get`); gateway `POST /v1/corpora` (201, **server-minted `corpus_…` id**, tenant from `ctx`) + `DELETE /v1/corpora/{id}` (204/404) + a `CreateCorpusRequest` wire type; admin-ui `createCorpus`/`deleteCorpus` wired into the Corpora page (live→local-demo gate, mirrors Webhooks). Update (`PATCH`) still deferred (the *Edit* action stays a placeholder). Verified locally (CI billing-blocked): full create→list→delete→404 + tenant isolation + CORS preflight/POST from `:3100`; `pytest` / `ruff` / `mypy --strict` / admin-ui `tsc` + `vitest` + OpenAPI·schema drift all green. [#189](https://github.com/officialCodeWork/AgentContextOS/pull/189)
- **post-GA batch** ✅ *(V1.1, [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194))* — corpus/webhook **PATCH** completes the Step 3.10 CRUD *update* path (`CorpusStore` / `SubscriptionStore.update` — SPI + Noop + Pg; `PATCH /v1/corpora/{id}` + `PATCH /v1/webhooks/subscriptions/{id}`; embedding model/dim immutable, signing secret stays masked) + admin-UI **bug fixes** (BUG-002/003/004/007/008/009 — Edit dialogs, per-source demo badges, metrics hydration, stale copy) + **real-backend wiring** (config-driven `OpenAILLM` + content hydration so ingested docs are retrievable end-to-end) + a local **Docker app stack** (pg + gateway + admin-ui), branding, and GTM demo assets. Verified locally (CI billing-blocked): `ruff` / `mypy --strict` (333 files) / full `pytest` / proto·openapi·schema drift + admin-ui `tsc`·`eslint`·**46 vitest**·`next build` all green. Testing-findings tracker added at [BUGS.md](BUGS.md) (BUG-005 audit-coverage + BUG-006 ingest doc-count remain open, root-caused). [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194)
- **3.10** ✅ *(V1.1)* Corpus admin CRUD — the console can now **create / delete corpora against the gateway** (was local-only React state with no write route — `POST /v1/corpora` 405'd, so "created" corpora never persisted). `CorpusStore` SPI gains tenant-scoped `create` / `delete` (Noop + Pg; same cross-tenant invisibility rule as `get`); gateway `POST /v1/corpora` (201, **server-minted `corpus_…` id**, tenant from `ctx`) + `DELETE /v1/corpora/{id}` (204/404) + a `CreateCorpusRequest` wire type; admin-ui `createCorpus`/`deleteCorpus` wired into the Corpora page (live→local-demo gate, mirrors Webhooks). Update (`PATCH`) followed post-GA in [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194) — the *Edit* action is now wired end-to-end (corpus + webhook). Verified locally (CI billing-blocked): full create→list→delete→404 + tenant isolation + CORS preflight/POST from `:3100`; `pytest` / `ruff` / `mypy --strict` / admin-ui `tsc` + `vitest` + OpenAPI·schema drift all green. [#189](https://github.com/officialCodeWork/AgentContextOS/pull/189)
- **7.9b** ✅ Admin-ui **Operate pages** — the console can now *drive* the request path, not just observe it (closes the three High-priority gaps from the [v2 API-coverage analysis](apps/admin-ui/design/claude-design-prompt-v2-api-coverage.md); built to the [Operate design brief](apps/admin-ui/design/claude-design-prompt-operate.md), 1:1 on the real wire types in `rag_core.gateway_types` / `rag_core.openai_types`). New **Operate** nav group + three pages: **Query Playground** (`/playground` — full `QueryRequest` control rail: corpus multiselect / `top_k` / rerank·pack·generate switches / advanced `pack_budget`·`generate_max_tokens`·`generate_temperature`·`filters`; **Run query** `POST /v1/query` vs **Retrieve only** `POST /v1/retrieve`; results render the decision strip (shape · bm25-fallback · corpus strategy · packed tokens · experiment tag), the answer with clickable `[n]` citation chips + guard verdict, citations, the ranked-chunks table (trust badges, detail sheet), a stacked per-stage timings bar with skipped-stage markers, and a **feedback bar** (👍/👎/1–5★/copy/regenerate/PII-redacted comment → `POST /v1/feedback`, exact signal enum); **View full trace** deep-links to `/trace?rid=…` — the seeded run reuses `DEMO_REQUEST_ID` so the demo resolves end-to-end); **Ingest** (`/ingest`, Knowledge — drag-and-drop multipart `POST /v1/ingest/document`, staged queue with progress → `IngestResult` chunk/embedding badges or error+Retry, the *parse → chunk → enrich → PII-redact → embed → index* explainer); **Chat** (`/chat` — OpenAI-compatible: streaming SSE thread with the non-standard `rag` annotations surfaced per reply (shape/chunks/citations/guard/finish_reason/usage; RAG **on by default**), an **Embeddings** tab (one input per line, float/base64, dim + L2 norm + sparkline), and a **Models** tab (`GET /v1/models`)). All follow the live→seed hybrid: action surfaces *simulate* without a gateway (deterministic worked query, token-streamed replies honouring `prefers-reduced-motion`). New `Slider`/`Tabs` primitives + `CorpusMultiSelect`/`GuardBadge`/`CitedText` shared components; identity body-carried on `/v1/query`·`/v1/retrieve`·`/v1/feedback`, header-driven elsewhere. Gates: tsc / eslint / **46 vitest** (12 new) / `next build` (17 static routes); browser-verified light+dark, all three pages + the trace deep-link. [reference/admin-ui.md](docs/reference/admin-ui.md)
- **v1.0.1** ✅ *(patch)* — **production serve path fixed end-to-end.** The image's default CMD (a Phase-0 placeholder that printed the version and exited) now serves via the new config-driven **`rag_gateway.serve:create_app`** uvicorn-factory entrypoint: `RAG_CONFIG_PATH` set → `build_app_from_config(load(path))`, unset → noop `build_app()`, set-but-broken → **fail fast** (`ConfigError`), never a silent noop fallback. The image **build** itself was also broken — the full `uv sync` needs every workspace member on disk but `sdks/python` was never copied (`Distribution not found`); both stages now copy it. New public **`GET /readyz`** backs the Helm chart's readiness probe (liveness stays `/healthz`). The chart's ConfigMap previously rendered an *invalid* `rag.yaml` (`env:/log_level:/service:` against the strict `extra='forbid'` schema → guaranteed CrashLoop); it now renders a minimal valid document with a **`config.ragYaml`** passthrough for full platform configs (`${VAR}` env interpolation keeps secrets out of the ConfigMap), chart `0.2.0` / `appVersion` `1.0.1` (the old `0.1.0` appVersion pointed the default image tag at a tag that never existed), `values.dev.yaml` `env: development` → `local`. Docs: image tags are published *without* the `v` prefix — `:v1.0.0`-style pulls in README/release-notes never matched. [release notes](docs/release-notes/v1.0.1.md) [#185](https://github.com/officialCodeWork/AgentContextOS/pull/185)
- **7.10** ✅ **GA cutover — v1.0.0** 🎉 — the close of an 8-phase, 84-step build. Cut **`v1.0.0`** with **Semantic Versioning** over the drift-gated public contracts (REST/OpenAPI · gRPC proto · `rag.yaml` · SDKs · `ragctl`) + a deprecation policy ([ADR-0052](docs/adr/ADR-0052-ga-cutover.md)); the `rag-platform` meta bumped to 1.0.0. GA **release notes** ([v1.0.0](docs/release-notes/v1.0.0.md)) + a **GA readiness checklist** ([ga-readiness-checklist](docs/ga/ga-readiness-checklist.md)) that maps every Phase 7 exit gate to a CI gate / in-repo artifact, with inherently-external items (external pentest, SOC 2 audit, registry first-publish, signed customers, on-call staffing, launch assets) marked **(external)** — the same in-repo-machine vs external-deliverable split used across Phase 7. The `v1.0.0` tag fans out to every channel via `docker.yml` + `release.yml` + `release-airgap.yml`.
Expand Down
20 changes: 5 additions & 15 deletions apps/gateway/tests/test_corpora.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ def _client() -> TestClient:
return TestClient(build_app())


def _create(
client: TestClient, headers: dict[str, str] = HEADERS, **body: Any
) -> dict[str, Any]:
def _create(client: TestClient, headers: dict[str, str] = HEADERS, **body: Any) -> dict[str, Any]:
payload = {"display_name": "Support KB", **body}
resp = client.post("/v1/corpora", json=payload, headers=headers)
assert resp.status_code == 201, resp.text
Expand Down Expand Up @@ -72,9 +70,7 @@ def test_create_requires_auth() -> None:
def test_delete_removes_then_404s() -> None:
client = _client()
created = _create(client)
assert (
client.delete(f"/v1/corpora/{created['id']}", headers=HEADERS).status_code == 204
)
assert client.delete(f"/v1/corpora/{created['id']}", headers=HEADERS).status_code == 204
# gone from the list...
assert client.get("/v1/corpora", headers=HEADERS).json()["total"] == 0
# ...and a second delete is a 404 with the canonical error code.
Expand Down Expand Up @@ -102,9 +98,7 @@ def test_tenant_isolation_on_get_and_delete() -> None:
# ---- update (PATCH /v1/corpora/{id}, Step 3.10 / BUG-002) -----------------
def test_update_patches_fields_and_persists() -> None:
client = _client()
created = _create(
client, embedding_model="text-embedding-3-small", embedding_dimension=1536
)
created = _create(client, embedding_model="text-embedding-3-small", embedding_dimension=1536)
resp = client.patch(
f"/v1/corpora/{created['id']}",
json={"display_name": "Renamed KB", "description": "updated"},
Expand Down Expand Up @@ -135,19 +129,15 @@ def test_update_only_patches_provided_fields() -> None:

def test_update_unknown_returns_404() -> None:
client = _client()
resp = client.patch(
"/v1/corpora/corpus_nope", json={"display_name": "x"}, headers=HEADERS
)
resp = client.patch("/v1/corpora/corpus_nope", json={"display_name": "x"}, headers=HEADERS)
assert resp.status_code == 404
assert resp.json()["error"]["code"] == "corpus_not_found"


def test_update_rejects_blank_display_name() -> None:
client = _client()
created = _create(client)
resp = client.patch(
f"/v1/corpora/{created['id']}", json={"display_name": ""}, headers=HEADERS
)
resp = client.patch(f"/v1/corpora/{created['id']}", json={"display_name": ""}, headers=HEADERS)
assert resp.status_code == 422 # pydantic min_length


Expand Down
13 changes: 13 additions & 0 deletions docs/reference/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
| `GET /v1/corpora` | List corpora visible to the calling tenant. |
| `GET /v1/corpora/{id}` | Describe one corpus. |
| `POST /v1/corpora` | Create a corpus for the calling tenant (admin surface, Step 3.10). |
| `PATCH /v1/corpora/{id}` | Update a corpus's mutable fields (admin surface). |
| `DELETE /v1/corpora/{id}` | Delete a corpus owned by the calling tenant (admin surface, Step 3.10). |
| `GET /openapi.json` | Auto-generated OpenAPI 3.1 spec. |
| `GET /docs` | Swagger UI. |
Expand Down Expand Up @@ -170,6 +171,18 @@ curl -s -X POST http://localhost:8000/v1/corpora \

A blank `display_name` is rejected with `422`; a request without tenant / principal credentials is rejected with `401`.

## `PATCH /v1/corpora/{corpus_id}`

Update a corpus's mutable fields — `display_name`, `description`, `metadata`. **PATCH semantics: only the fields present in the body change**; omitted fields are left untouched. `embedding_model` / `embedding_dimension` are **immutable** (they define the corpus's vector space — changing them would invalidate every embedding already indexed under it), as are `id`, `tenant_id`, and `document_count`. Returns `200` with the updated `Corpus`. **Cross-tenant or unknown ids return `404`** — the same invisibility rule as `GET`, so a caller can neither mutate nor probe for another tenant's corpora.

```bash
curl -s -X PATCH http://localhost:8000/v1/corpora/corpus_8f… \
-H 'content-type: application/json' \
-H 'x-tenant-id: demo' \
-H 'x-principal-id: curl' \
-d '{"display_name": "Support KB (EMEA)"}' | jq
```

## `DELETE /v1/corpora/{corpus_id}`

Delete a corpus owned by the calling tenant. Returns `204` on success. **Cross-tenant or unknown ids return `404`** — the same invisibility rule as `GET`, so a caller can neither delete nor probe for another tenant's corpora.
Expand Down
1 change: 1 addition & 0 deletions docs/reference/webhooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ same as the other header-identity routes.
| `POST /v1/webhooks/subscriptions` | Create a subscription (secret returned **once**) |
| `GET /v1/webhooks/subscriptions` | List subscriptions (secrets masked) |
| `GET /v1/webhooks/subscriptions/{id}` | Describe one (secret masked) |
| `PATCH /v1/webhooks/subscriptions/{id}` | Update url / event types / description / active toggle (secret stays masked) |
| `DELETE /v1/webhooks/subscriptions/{id}` | Delete one |
| `POST /v1/webhooks/subscriptions/{id}/test` | Send a synthetic test event |

Expand Down
8 changes: 3 additions & 5 deletions packages/backends/src/rag_backends/corpus/pg_corpus_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
from __future__ import annotations

import json
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

import asyncpg
Expand Down Expand Up @@ -97,9 +97,7 @@

# ``RETURNING`` lets us report existed/not-existed without a prior SELECT,
# while the ``tenant_id = $1`` predicate keeps the delete tenant-scoped.
_SQL_DELETE = (
"DELETE FROM {table} WHERE tenant_id = $1 AND corpus_id = $2 RETURNING corpus_id"
)
_SQL_DELETE = "DELETE FROM {table} WHERE tenant_id = $1 AND corpus_id = $2 RETURNING corpus_id"


def _as_dict(value: Any) -> dict[str, Any]:
Expand Down Expand Up @@ -238,7 +236,7 @@ async def update(
current = await self.get(ctx, corpus_id)
if current is None:
return None
patch: dict[str, Any] = {"updated_at": datetime.now(timezone.utc)}
patch: dict[str, Any] = {"updated_at": datetime.now(UTC)}
if display_name is not None:
patch["display_name"] = display_name
if description is not None:
Expand Down
4 changes: 1 addition & 3 deletions packages/backends/src/rag_backends/vector/pgvector.py
Original file line number Diff line number Diff line change
Expand Up @@ -370,9 +370,7 @@ async def put_content(self, ctx: RequestContext, chunks: list[Chunk]) -> None:
async with pool.acquire() as conn:
await conn.executemany(_SQL_CONTENT_UPSERT.format(table=self._content_table), rows)

async def hydrate_content(
self, ctx: RequestContext, chunk_refs: list[ChunkRef]
) -> list[Chunk]:
async def hydrate_content(self, ctx: RequestContext, chunk_refs: list[ChunkRef]) -> list[Chunk]:
"""Hydrate ``chunk_refs`` into full :class:`Chunk`\\s with text.

Satisfies the reranker ``Hydrator`` protocol
Expand Down
4 changes: 2 additions & 2 deletions packages/core/src/rag_core/spi/noop/corpus_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

from __future__ import annotations

from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

from rag_core.gateway_types import Corpus
Expand Down Expand Up @@ -70,7 +70,7 @@ async def update(
current = self._by_key.get(key)
if current is None:
return None
patch: dict[str, Any] = {"updated_at": datetime.now(timezone.utc)}
patch: dict[str, Any] = {"updated_at": datetime.now(UTC)}
if display_name is not None:
patch["display_name"] = display_name
if description is not None:
Expand Down
4 changes: 2 additions & 2 deletions packages/core/src/rag_core/spi/noop/subscription_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from __future__ import annotations

from datetime import datetime, timezone
from datetime import UTC, datetime

from rag_core.spi.subscription_store import SubscriptionStore
from rag_core.types import RequestContext
Expand Down Expand Up @@ -56,7 +56,7 @@ async def update(
current = bucket.get(subscription_id)
if current is None:
return None
patch: dict[str, object] = {"updated_at": datetime.now(timezone.utc)}
patch: dict[str, object] = {"updated_at": datetime.now(UTC)}
if url is not None:
patch["url"] = url
if event_types is not None:
Expand Down
7 changes: 6 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,12 @@ line-length = 100
target-version = "py312"
# Generated gRPC stubs are committed (for the drift gate) but excluded from
# lint/format — protoc/mypy-protobuf own that output format, not us.
extend-exclude = ["apps/gateway/src/rag_gateway/_grpc_gen"]
extend-exclude = [
"apps/gateway/src/rag_gateway/_grpc_gen",
# GTM demo tooling (slide builders + adversarial safety demos) — not shipped
# code; held to a lighter bar than the workspace packages.
"scripts/demo",
]

[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "S", "ANN", "T"]
Expand Down
4 changes: 4 additions & 0 deletions tests/policy/coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,10 @@
Path("packages/policy/src/rag_policy/writer.py"),
Path("packages/policy/src/rag_policy/engine.py"),
Path("packages/policy/src/rag_policy/noop.py"),
# GTM demo tooling — the adversarial safety demo deliberately seeds a
# poisoned chunk via bulk_index to show the injection guard catching it;
# it's a demonstration, not a production call site.
Path("scripts/demo"), # whole directory
# Tests — fixtures and conformance suites exercise the SPI directly.
# (The coverage rule is about production call sites.)
Path("tests"), # whole directory; pruned by has_test_parent below
Expand Down
Loading