diff --git a/TRACKER.md b/TRACKER.md index 425b711..0ac593e 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -13,14 +13,15 @@ | | | |---|---| -| **Last updated** | 2026-06-11 | +| **Last updated** | 2026-06-19 | | **Current phase** | Phase 7 — Pilot, Harden, GA (**10 / 10 steps ✅**) — **all phases complete** | | **Overall** | **84 / 84 steps ✅ — v1.0.0 GA** | -| **Next action** | **🎉 GA reached.** Engineering scope (Phases 0–7) complete; `v1.0.0` cut. Remaining items are external/process (external pentest, SOC 2 audit, registry first-publish, signed referenceable customers, on-call staffing + status page, launch assets) — tracked in the [GA readiness checklist](docs/ga/ga-readiness-checklist.md). Post-GA: cut the **v1.0.1** patch tag (serve-path fix below; note: GitHub Actions runs are currently blocked by an account **billing/spending-limit** failure — every `docker.yml` run since GA failed at job start, so the GHCR image was never actually published), then the V1.1 backlog. **V1.1 underway:** Step 3.10 corpus admin CRUD — gateway write surface + console wiring ([#189](https://github.com/officialCodeWork/AgentContextOS/pull/189), first post-GA feature). | +| **Next action** | **🎉 GA reached.** Engineering scope (Phases 0–7) complete; `v1.0.0` cut. Remaining items are external/process (external pentest, SOC 2 audit, registry first-publish, signed referenceable customers, on-call staffing + status page, launch assets) — tracked in the [GA readiness checklist](docs/ga/ga-readiness-checklist.md). Post-GA: cut the **v1.0.1** patch tag (serve-path fix below; note: GitHub Actions runs are currently blocked by an account **billing/spending-limit** failure — every `docker.yml` run since GA failed at job start, so the GHCR image was never actually published), then the V1.1 backlog. **V1.1 underway:** Step 3.10 corpus admin CRUD landed ([#189](https://github.com/officialCodeWork/AgentContextOS/pull/189)); a follow-on post-GA batch in [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194) adds corpus/webhook **PATCH** (completes the 3.10 *update* path), admin-UI bug fixes, real-backend (OpenAI LLM + content hydration) wiring, and the local Docker/demo stack. | **Recently shipped** -- **3.10** ✅ *(V1.1)* Corpus admin CRUD — the console can now **create / delete corpora against the gateway** (was local-only React state with no write route — `POST /v1/corpora` 405'd, so "created" corpora never persisted). `CorpusStore` SPI gains tenant-scoped `create` / `delete` (Noop + Pg; same cross-tenant invisibility rule as `get`); gateway `POST /v1/corpora` (201, **server-minted `corpus_…` id**, tenant from `ctx`) + `DELETE /v1/corpora/{id}` (204/404) + a `CreateCorpusRequest` wire type; admin-ui `createCorpus`/`deleteCorpus` wired into the Corpora page (live→local-demo gate, mirrors Webhooks). Update (`PATCH`) still deferred (the *Edit* action stays a placeholder). Verified locally (CI billing-blocked): full create→list→delete→404 + tenant isolation + CORS preflight/POST from `:3100`; `pytest` / `ruff` / `mypy --strict` / admin-ui `tsc` + `vitest` + OpenAPI·schema drift all green. [#189](https://github.com/officialCodeWork/AgentContextOS/pull/189) +- **post-GA batch** ✅ *(V1.1, [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194))* — corpus/webhook **PATCH** completes the Step 3.10 CRUD *update* path (`CorpusStore` / `SubscriptionStore.update` — SPI + Noop + Pg; `PATCH /v1/corpora/{id}` + `PATCH /v1/webhooks/subscriptions/{id}`; embedding model/dim immutable, signing secret stays masked) + admin-UI **bug fixes** (BUG-002/003/004/007/008/009 — Edit dialogs, per-source demo badges, metrics hydration, stale copy) + **real-backend wiring** (config-driven `OpenAILLM` + content hydration so ingested docs are retrievable end-to-end) + a local **Docker app stack** (pg + gateway + admin-ui), branding, and GTM demo assets. Verified locally (CI billing-blocked): `ruff` / `mypy --strict` (333 files) / full `pytest` / proto·openapi·schema drift + admin-ui `tsc`·`eslint`·**46 vitest**·`next build` all green. Testing-findings tracker added at [BUGS.md](BUGS.md) (BUG-005 audit-coverage + BUG-006 ingest doc-count remain open, root-caused). [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194) +- **3.10** ✅ *(V1.1)* Corpus admin CRUD — the console can now **create / delete corpora against the gateway** (was local-only React state with no write route — `POST /v1/corpora` 405'd, so "created" corpora never persisted). `CorpusStore` SPI gains tenant-scoped `create` / `delete` (Noop + Pg; same cross-tenant invisibility rule as `get`); gateway `POST /v1/corpora` (201, **server-minted `corpus_…` id**, tenant from `ctx`) + `DELETE /v1/corpora/{id}` (204/404) + a `CreateCorpusRequest` wire type; admin-ui `createCorpus`/`deleteCorpus` wired into the Corpora page (live→local-demo gate, mirrors Webhooks). Update (`PATCH`) followed post-GA in [#194](https://github.com/officialCodeWork/AgentContextOS/pull/194) — the *Edit* action is now wired end-to-end (corpus + webhook). Verified locally (CI billing-blocked): full create→list→delete→404 + tenant isolation + CORS preflight/POST from `:3100`; `pytest` / `ruff` / `mypy --strict` / admin-ui `tsc` + `vitest` + OpenAPI·schema drift all green. [#189](https://github.com/officialCodeWork/AgentContextOS/pull/189) - **7.9b** ✅ Admin-ui **Operate pages** — the console can now *drive* the request path, not just observe it (closes the three High-priority gaps from the [v2 API-coverage analysis](apps/admin-ui/design/claude-design-prompt-v2-api-coverage.md); built to the [Operate design brief](apps/admin-ui/design/claude-design-prompt-operate.md), 1:1 on the real wire types in `rag_core.gateway_types` / `rag_core.openai_types`). New **Operate** nav group + three pages: **Query Playground** (`/playground` — full `QueryRequest` control rail: corpus multiselect / `top_k` / rerank·pack·generate switches / advanced `pack_budget`·`generate_max_tokens`·`generate_temperature`·`filters`; **Run query** `POST /v1/query` vs **Retrieve only** `POST /v1/retrieve`; results render the decision strip (shape · bm25-fallback · corpus strategy · packed tokens · experiment tag), the answer with clickable `[n]` citation chips + guard verdict, citations, the ranked-chunks table (trust badges, detail sheet), a stacked per-stage timings bar with skipped-stage markers, and a **feedback bar** (👍/👎/1–5★/copy/regenerate/PII-redacted comment → `POST /v1/feedback`, exact signal enum); **View full trace** deep-links to `/trace?rid=…` — the seeded run reuses `DEMO_REQUEST_ID` so the demo resolves end-to-end); **Ingest** (`/ingest`, Knowledge — drag-and-drop multipart `POST /v1/ingest/document`, staged queue with progress → `IngestResult` chunk/embedding badges or error+Retry, the *parse → chunk → enrich → PII-redact → embed → index* explainer); **Chat** (`/chat` — OpenAI-compatible: streaming SSE thread with the non-standard `rag` annotations surfaced per reply (shape/chunks/citations/guard/finish_reason/usage; RAG **on by default**), an **Embeddings** tab (one input per line, float/base64, dim + L2 norm + sparkline), and a **Models** tab (`GET /v1/models`)). All follow the live→seed hybrid: action surfaces *simulate* without a gateway (deterministic worked query, token-streamed replies honouring `prefers-reduced-motion`). New `Slider`/`Tabs` primitives + `CorpusMultiSelect`/`GuardBadge`/`CitedText` shared components; identity body-carried on `/v1/query`·`/v1/retrieve`·`/v1/feedback`, header-driven elsewhere. Gates: tsc / eslint / **46 vitest** (12 new) / `next build` (17 static routes); browser-verified light+dark, all three pages + the trace deep-link. [reference/admin-ui.md](docs/reference/admin-ui.md) - **v1.0.1** ✅ *(patch)* — **production serve path fixed end-to-end.** The image's default CMD (a Phase-0 placeholder that printed the version and exited) now serves via the new config-driven **`rag_gateway.serve:create_app`** uvicorn-factory entrypoint: `RAG_CONFIG_PATH` set → `build_app_from_config(load(path))`, unset → noop `build_app()`, set-but-broken → **fail fast** (`ConfigError`), never a silent noop fallback. The image **build** itself was also broken — the full `uv sync` needs every workspace member on disk but `sdks/python` was never copied (`Distribution not found`); both stages now copy it. New public **`GET /readyz`** backs the Helm chart's readiness probe (liveness stays `/healthz`). The chart's ConfigMap previously rendered an *invalid* `rag.yaml` (`env:/log_level:/service:` against the strict `extra='forbid'` schema → guaranteed CrashLoop); it now renders a minimal valid document with a **`config.ragYaml`** passthrough for full platform configs (`${VAR}` env interpolation keeps secrets out of the ConfigMap), chart `0.2.0` / `appVersion` `1.0.1` (the old `0.1.0` appVersion pointed the default image tag at a tag that never existed), `values.dev.yaml` `env: development` → `local`. Docs: image tags are published *without* the `v` prefix — `:v1.0.0`-style pulls in README/release-notes never matched. [release notes](docs/release-notes/v1.0.1.md) [#185](https://github.com/officialCodeWork/AgentContextOS/pull/185) - **7.10** ✅ **GA cutover — v1.0.0** 🎉 — the close of an 8-phase, 84-step build. Cut **`v1.0.0`** with **Semantic Versioning** over the drift-gated public contracts (REST/OpenAPI · gRPC proto · `rag.yaml` · SDKs · `ragctl`) + a deprecation policy ([ADR-0052](docs/adr/ADR-0052-ga-cutover.md)); the `rag-platform` meta bumped to 1.0.0. GA **release notes** ([v1.0.0](docs/release-notes/v1.0.0.md)) + a **GA readiness checklist** ([ga-readiness-checklist](docs/ga/ga-readiness-checklist.md)) that maps every Phase 7 exit gate to a CI gate / in-repo artifact, with inherently-external items (external pentest, SOC 2 audit, registry first-publish, signed customers, on-call staffing, launch assets) marked **(external)** — the same in-repo-machine vs external-deliverable split used across Phase 7. The `v1.0.0` tag fans out to every channel via `docker.yml` + `release.yml` + `release-airgap.yml`. diff --git a/apps/gateway/tests/test_corpora.py b/apps/gateway/tests/test_corpora.py index e4710ef..8c0e6e6 100644 --- a/apps/gateway/tests/test_corpora.py +++ b/apps/gateway/tests/test_corpora.py @@ -22,9 +22,7 @@ def _client() -> TestClient: return TestClient(build_app()) -def _create( - client: TestClient, headers: dict[str, str] = HEADERS, **body: Any -) -> dict[str, Any]: +def _create(client: TestClient, headers: dict[str, str] = HEADERS, **body: Any) -> dict[str, Any]: payload = {"display_name": "Support KB", **body} resp = client.post("/v1/corpora", json=payload, headers=headers) assert resp.status_code == 201, resp.text @@ -72,9 +70,7 @@ def test_create_requires_auth() -> None: def test_delete_removes_then_404s() -> None: client = _client() created = _create(client) - assert ( - client.delete(f"/v1/corpora/{created['id']}", headers=HEADERS).status_code == 204 - ) + assert client.delete(f"/v1/corpora/{created['id']}", headers=HEADERS).status_code == 204 # gone from the list... assert client.get("/v1/corpora", headers=HEADERS).json()["total"] == 0 # ...and a second delete is a 404 with the canonical error code. @@ -102,9 +98,7 @@ def test_tenant_isolation_on_get_and_delete() -> None: # ---- update (PATCH /v1/corpora/{id}, Step 3.10 / BUG-002) ----------------- def test_update_patches_fields_and_persists() -> None: client = _client() - created = _create( - client, embedding_model="text-embedding-3-small", embedding_dimension=1536 - ) + created = _create(client, embedding_model="text-embedding-3-small", embedding_dimension=1536) resp = client.patch( f"/v1/corpora/{created['id']}", json={"display_name": "Renamed KB", "description": "updated"}, @@ -135,9 +129,7 @@ def test_update_only_patches_provided_fields() -> None: def test_update_unknown_returns_404() -> None: client = _client() - resp = client.patch( - "/v1/corpora/corpus_nope", json={"display_name": "x"}, headers=HEADERS - ) + resp = client.patch("/v1/corpora/corpus_nope", json={"display_name": "x"}, headers=HEADERS) assert resp.status_code == 404 assert resp.json()["error"]["code"] == "corpus_not_found" @@ -145,9 +137,7 @@ def test_update_unknown_returns_404() -> None: def test_update_rejects_blank_display_name() -> None: client = _client() created = _create(client) - resp = client.patch( - f"/v1/corpora/{created['id']}", json={"display_name": ""}, headers=HEADERS - ) + resp = client.patch(f"/v1/corpora/{created['id']}", json={"display_name": ""}, headers=HEADERS) assert resp.status_code == 422 # pydantic min_length diff --git a/docs/reference/gateway.md b/docs/reference/gateway.md index 92fc1a0..9cb89e4 100644 --- a/docs/reference/gateway.md +++ b/docs/reference/gateway.md @@ -15,6 +15,7 @@ | `GET /v1/corpora` | List corpora visible to the calling tenant. | | `GET /v1/corpora/{id}` | Describe one corpus. | | `POST /v1/corpora` | Create a corpus for the calling tenant (admin surface, Step 3.10). | +| `PATCH /v1/corpora/{id}` | Update a corpus's mutable fields (admin surface). | | `DELETE /v1/corpora/{id}` | Delete a corpus owned by the calling tenant (admin surface, Step 3.10). | | `GET /openapi.json` | Auto-generated OpenAPI 3.1 spec. | | `GET /docs` | Swagger UI. | @@ -170,6 +171,18 @@ curl -s -X POST http://localhost:8000/v1/corpora \ A blank `display_name` is rejected with `422`; a request without tenant / principal credentials is rejected with `401`. +## `PATCH /v1/corpora/{corpus_id}` + +Update a corpus's mutable fields — `display_name`, `description`, `metadata`. **PATCH semantics: only the fields present in the body change**; omitted fields are left untouched. `embedding_model` / `embedding_dimension` are **immutable** (they define the corpus's vector space — changing them would invalidate every embedding already indexed under it), as are `id`, `tenant_id`, and `document_count`. Returns `200` with the updated `Corpus`. **Cross-tenant or unknown ids return `404`** — the same invisibility rule as `GET`, so a caller can neither mutate nor probe for another tenant's corpora. + +```bash +curl -s -X PATCH http://localhost:8000/v1/corpora/corpus_8f… \ + -H 'content-type: application/json' \ + -H 'x-tenant-id: demo' \ + -H 'x-principal-id: curl' \ + -d '{"display_name": "Support KB (EMEA)"}' | jq +``` + ## `DELETE /v1/corpora/{corpus_id}` Delete a corpus owned by the calling tenant. Returns `204` on success. **Cross-tenant or unknown ids return `404`** — the same invisibility rule as `GET`, so a caller can neither delete nor probe for another tenant's corpora. diff --git a/docs/reference/webhooks.md b/docs/reference/webhooks.md index d6b4286..f3a958c 100644 --- a/docs/reference/webhooks.md +++ b/docs/reference/webhooks.md @@ -79,6 +79,7 @@ same as the other header-identity routes. | `POST /v1/webhooks/subscriptions` | Create a subscription (secret returned **once**) | | `GET /v1/webhooks/subscriptions` | List subscriptions (secrets masked) | | `GET /v1/webhooks/subscriptions/{id}` | Describe one (secret masked) | +| `PATCH /v1/webhooks/subscriptions/{id}` | Update url / event types / description / active toggle (secret stays masked) | | `DELETE /v1/webhooks/subscriptions/{id}` | Delete one | | `POST /v1/webhooks/subscriptions/{id}/test` | Send a synthetic test event | diff --git a/packages/backends/src/rag_backends/corpus/pg_corpus_store.py b/packages/backends/src/rag_backends/corpus/pg_corpus_store.py index 92526b0..99a3cb4 100644 --- a/packages/backends/src/rag_backends/corpus/pg_corpus_store.py +++ b/packages/backends/src/rag_backends/corpus/pg_corpus_store.py @@ -34,7 +34,7 @@ from __future__ import annotations import json -from datetime import datetime, timezone +from datetime import UTC, datetime from typing import Any import asyncpg @@ -97,9 +97,7 @@ # ``RETURNING`` lets us report existed/not-existed without a prior SELECT, # while the ``tenant_id = $1`` predicate keeps the delete tenant-scoped. -_SQL_DELETE = ( - "DELETE FROM {table} WHERE tenant_id = $1 AND corpus_id = $2 RETURNING corpus_id" -) +_SQL_DELETE = "DELETE FROM {table} WHERE tenant_id = $1 AND corpus_id = $2 RETURNING corpus_id" def _as_dict(value: Any) -> dict[str, Any]: @@ -238,7 +236,7 @@ async def update( current = await self.get(ctx, corpus_id) if current is None: return None - patch: dict[str, Any] = {"updated_at": datetime.now(timezone.utc)} + patch: dict[str, Any] = {"updated_at": datetime.now(UTC)} if display_name is not None: patch["display_name"] = display_name if description is not None: diff --git a/packages/backends/src/rag_backends/vector/pgvector.py b/packages/backends/src/rag_backends/vector/pgvector.py index b0123fd..3b98998 100644 --- a/packages/backends/src/rag_backends/vector/pgvector.py +++ b/packages/backends/src/rag_backends/vector/pgvector.py @@ -370,9 +370,7 @@ async def put_content(self, ctx: RequestContext, chunks: list[Chunk]) -> None: async with pool.acquire() as conn: await conn.executemany(_SQL_CONTENT_UPSERT.format(table=self._content_table), rows) - async def hydrate_content( - self, ctx: RequestContext, chunk_refs: list[ChunkRef] - ) -> list[Chunk]: + async def hydrate_content(self, ctx: RequestContext, chunk_refs: list[ChunkRef]) -> list[Chunk]: """Hydrate ``chunk_refs`` into full :class:`Chunk`\\s with text. Satisfies the reranker ``Hydrator`` protocol diff --git a/packages/core/src/rag_core/spi/noop/corpus_store.py b/packages/core/src/rag_core/spi/noop/corpus_store.py index fd61cc1..87100a8 100644 --- a/packages/core/src/rag_core/spi/noop/corpus_store.py +++ b/packages/core/src/rag_core/spi/noop/corpus_store.py @@ -7,7 +7,7 @@ from __future__ import annotations -from datetime import datetime, timezone +from datetime import UTC, datetime from typing import Any from rag_core.gateway_types import Corpus @@ -70,7 +70,7 @@ async def update( current = self._by_key.get(key) if current is None: return None - patch: dict[str, Any] = {"updated_at": datetime.now(timezone.utc)} + patch: dict[str, Any] = {"updated_at": datetime.now(UTC)} if display_name is not None: patch["display_name"] = display_name if description is not None: diff --git a/packages/core/src/rag_core/spi/noop/subscription_store.py b/packages/core/src/rag_core/spi/noop/subscription_store.py index e1675e6..d383ebd 100644 --- a/packages/core/src/rag_core/spi/noop/subscription_store.py +++ b/packages/core/src/rag_core/spi/noop/subscription_store.py @@ -2,7 +2,7 @@ from __future__ import annotations -from datetime import datetime, timezone +from datetime import UTC, datetime from rag_core.spi.subscription_store import SubscriptionStore from rag_core.types import RequestContext @@ -56,7 +56,7 @@ async def update( current = bucket.get(subscription_id) if current is None: return None - patch: dict[str, object] = {"updated_at": datetime.now(timezone.utc)} + patch: dict[str, object] = {"updated_at": datetime.now(UTC)} if url is not None: patch["url"] = url if event_types is not None: diff --git a/pyproject.toml b/pyproject.toml index f13389c..f4a9a1b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -63,7 +63,12 @@ line-length = 100 target-version = "py312" # Generated gRPC stubs are committed (for the drift gate) but excluded from # lint/format — protoc/mypy-protobuf own that output format, not us. -extend-exclude = ["apps/gateway/src/rag_gateway/_grpc_gen"] +extend-exclude = [ + "apps/gateway/src/rag_gateway/_grpc_gen", + # GTM demo tooling (slide builders + adversarial safety demos) — not shipped + # code; held to a lighter bar than the workspace packages. + "scripts/demo", +] [tool.ruff.lint] select = ["E", "F", "I", "UP", "B", "S", "ANN", "T"] diff --git a/tests/policy/coverage.py b/tests/policy/coverage.py index f6ce3ed..0376af2 100644 --- a/tests/policy/coverage.py +++ b/tests/policy/coverage.py @@ -177,6 +177,10 @@ Path("packages/policy/src/rag_policy/writer.py"), Path("packages/policy/src/rag_policy/engine.py"), Path("packages/policy/src/rag_policy/noop.py"), + # GTM demo tooling — the adversarial safety demo deliberately seeds a + # poisoned chunk via bulk_index to show the injection guard catching it; + # it's a demonstration, not a production call site. + Path("scripts/demo"), # whole directory # Tests — fixtures and conformance suites exercise the SPI directly. # (The coverage rule is about production call sites.) Path("tests"), # whole directory; pruned by has_test_parent below