| File | Description |
|---|---|
| RAG-Platform-HLD.md | High-Level Design: problem statement, layered architecture, pluggable backends, rag.yaml contract, deployment topologies, KPIs, risks, glossary |
| high-level-architecture.svg | Layered architecture diagram (SVG) |
| request-context.md | RequestContext — the per-request envelope threaded through every SPI (tenant, principal, namespace, ACLs, PII policy, trace, budget) |
| multi-tenancy.md | Logical multi-tenancy (Step 6.1): make per-tenant rag.yaml config drive requests. TenantResolver (rag_config.tenancy) maps a tenant id → frozen TenantSettings (namespace / pii_policy / acl_labels), applied once at the gateway boundary (namespace + pii_policy onto the RequestContext, acl_labels unioned into the principal); unknown tenants resolve to safe defaults (namespace = id, default PII, no labels — isolated not privileged); RequestContext.namespace defaults to tenant_id (Pinecone partitions on it; filter_pushdown unchanged); resolves+threads only — ACL push-down is 6.3, PII egress 6.5, physical tenancy 6.2; GET /v1/status/tenant; inert in build_app |
| policy-engine.md | PolicyEngine (PDP) — single decision point for ACL, PII, quotas, redaction; replaces scattered governance checks |
| audit-log.md | Immutable audit log (Step 6.6): tamper-evidence (SHA-256 hash chain) vs immutability-at-rest (WORM export, 6.6b); one shared AuditWriter/store on app.state; read-path tenant scoping (tenant-scoped list vs whole-log verify); why the read API defaults on |
| byok.md | BYOK / envelope encryption (Step 6.7): what's encrypted (chunk content at rest) vs not (embedding vectors — search needs plaintext); DEK+KEK envelope (client-side AES-GCM DEK, provider wraps the DEK); per-tenant isolation via KEK + tenant_id AAD; sealing as a typed error; rag-core/rag-backends split; slicing (6.7a library, 6.7b config + factory + AWS KMS, 6.7c GCP/Azure/Vault, 6.7d rotation) |
| sso-scim.md | SSO / SCIM (Step 6.8): OIDC + SAML federation + SCIM 2.0 provisioning + per-tenant IdP config. FederatedAuth is an Auth SPI backend (the authenticate(token, tenant_id) → Principal seam — no middleware change); group claims → acl_labels so Step 6.3/6.5 govern federated users; dependency-free defaults (stdlib HS256 JWT, defusedxml SAML) with asymmetric OIDC / XML-DSig behind [oidc] / [saml] extras; algorithm-allowlist (alg:none/downgrade defense); per-tenant IdP on tenants[].sso; SCIM is a separate surface with its own per-tenant bearer token + tenant-scoped ScimStore; PII-free sso.*/scim.* events (hashed subject); deferred (JWKS rotation, SP-initiated SAML, directory-backed deprovisioning) |
| airgap-bundle.md | Air-gapped install bundle (Step 6.9): one signed .tar.gz of all runtime images (docker save) + the packaged Helm chart + rag.yaml + a standalone installer, for networks with no registry / internet. Integrity = a standard SHA256SUMS (verifiable with sha256sum -c, no cosign/network) whose hash is pinned as manifest.content_hash, plus an optional cosign signature over it (the 6.6b content-hash + optional-signature pattern); the same SHA256SUMS drives the Python verifier and the standalone shell installer. Logic in ragctl.airgap (typed/tested, subprocess seam stubbable; --dry-run = verifiable bundle minus blobs); standalone install.{sh,ps1} need only docker+helm; digest-pinned manifest-driven image set; key-based cosign for air-gap, keyless for connected releases; deferred (ctr/podman load, registry re-tag/push, backend charts) |
| compliance.md | Compliance posture (Step 6.10, the Phase-6 capstone): data retention + GDPR erasure/residency + the live control posture that backs the SOC 2 / GDPR mapping. New rag-compliance package (RetentionEnforcer + compliance_posture / residency_ok, config-free); retention is a non-abstract purge_before / purge_tenant capability on the Feedback/Provenance stores (dry_run in the SPI so a preview counts-without-deleting; audit chain never purged in place — its retention is the 6.6b WORM export); POST /v1/compliance/erase (tenant-scoped right-to-erasure, dry-run default + two-flag delete); per-tenant data_region enforced at ingest (ResidencyViolationError → 403); GET /v1/status/compliance posture computed from the control flags; PII-free compliance.* events; deferred (subject-level erasure, multi-region routing) |
| caching.md | Three-cache split: EmbeddingCache, RetrievalCache, AnswerCache — distinct invalidation rules |
| performance.md | Hot-path discipline, per-SPI p99 budgets, async telemetry, reviewer checklist |
| pipeline-batcher.md | Pipeline (async DAG, bounded queues, per-stage workers) + Batcher (DataLoader-pattern coalescing) primitives — Step 1.1d |
| eval-skeleton.md | Eval framework architecture: golden-set schema, metric functions, RAGAS spike, ragctl eval CLI, extension points |
| iac.md | IaC overview: Terraform module design, Helm chart structure, dev/prod environments, extension points |
| storage-backends.md | Storage backend architecture: PgVector, Qdrant, Redis, S3/MinIO, tenant isolation, integration test strategy |
| connectors.md | Connectors framework: SPI shape, ConnectorState watermark, Crawler base class, policy boundary, CDC interaction |
| parsers.md | Document parsers: Block taxonomy, ParsedDocument return shape, MIME detection, policy boundary, library choices |
| ocr.md | OCR pipeline: region-aware SPI evolution, Tesseract vs. PaddleOCR output normalisation, pipeline placement, policy boundary |
| chunker.md | Structure-aware chunker: heading stack + parent-link semantics, overlap policy, token counting, OCR-region handling, pipeline placement |
| enricher.md | Metadata enricher: what we tag and why, langdetect determinism, reading-level gating, section_path walker, composition contract, policy boundary |
| pii.md | PII detection: detector / enforcer split, regex scoring + Luhn check, PolicyEngine composition, event protocol (no raw PII in logs), pipeline placement; egress enforcement via PiiPolicyEngine (Step 6.5) |
| embedders.md | Embedder pipeline: 3 plugins / 4 model families, shared BatchingEmbedder (batching + retry + dim normalize), server- vs client-side dim reduction, E5 passage prefix, Cohere input_type, retry scope |
| ingest.md | Ingest pipeline orchestrator: why a dedicated package, stage order rationale, PolicyEngine consultation, per-document error isolation, gateway vs CLI split, what's deliberately deferred |
| retrieval-read-layer.md | Read-layer contract & FilterExpr push-down (Step 2.1): allowed fields, per-backend translators (pgvector SQL, Qdrant), reference evaluate() semantics, reviewer checklist |
| vector-backends.md | Vector retrieval backends (Step 2.2): IndexHint → variant mapping per backend (pgvector / Qdrant / Weaviate / Pinecone / Elasticsearch), FilterExpr translator contracts, tenant-isolation patterns, extension points |
| keyword-backends.md | Keyword (BM25) backends (Step 2.3): ElasticsearchKeywordStore + TantivyKeywordStore, per-field boosting as a backend config knob, indexed-field schema, FilterExpr push-down translators, tantivy Not quirk, when to pick which |
| graph-backends.md | Knowledge-graph backends (Step 2.4): Neo4jGraphStore + MemgraphGraphStore + NetworkXGraphStore, multi-hop expand() as the canonical traversal primitive, FilterExpr push-down on traversal edges + final neighbor, tenant-isolation pattern, relationship-type validation, when to pick which |
| hybrid-fusion.md | Hybrid retrieval fusion (Step 2.5): why RRF over score normalisation or learned fusion, parameter choices (k = 60, per_backend_top_k), graph → ChunkRef adapter rationale, partial-failure tolerance, what's deferred to Steps 2.6 – 2.10 |
| query-understanding.md | Query understanding (Step 2.6): why one package, component design (rewriter/decomposer/HyDE/glossary), parallel fan-out + partial-failure tolerance, PolicyEngine boundary, query.understand span schema, reviewer checklist |
| reranking.md | Two-stage reranking (Step 2.7, ADR-0006): why two stages, MMR diversity algorithm + embedding-source choice, composition with HybridRetriever, failure-mode rationale, rerank span schema |
| context-packing.md | Context packer (Step 2.8): dedup strategy (hash + opt-in Jaccard), lost-in-the-middle reorder rationale, heuristic conflict detection, token-budget fit semantics, pack span schema, PolicyEngine boundary |
| graphrag.md | GraphRAG (Step 2.9): community detection (Louvain default / Leiden optional), per-community LLM summarisation, query → community → entity → chunk flow, three OTel spans, PolicyEngine boundary |
| retrieval-router.md | Retrieval router (Step 2.10): query-shape classifier, in-memory health tracker + BM25-only fallback, shape-bias weight table, PolicyEngine boundary, deferred concerns, reviewer checklist |
| agent-loop-v0.md | Agent-loop validation spike (Step 2.11): why the spike exists, design constraints, stop-condition matrix, sticky-route semantics, cache integration, plan-reuse semantics, invariants, reviewer checklist |
| grpc-service.md | gRPC service (Step 3.2): codegen choice, proto package structure, server-streaming-only scope, drift gate, error mapping rationale, reviewer checklist |
| mcp-server.md | MCP server (Step 3.3): in-process backend behind a ToolBackend seam, one-Python-impl + npm launcher, lean agent-facing output models, tool↔request-model parity, composition diagram, deferred work |
| openai-compat.md | OpenAI-compatible surface (Step 3.4): retrieval pre-fetch + context injection, rag extension namespacing, header identity + dev anon fallback, OpenAI error envelope, streaming SSE, governance boundary, reviewer checklist |
| corpus-router.md | Corpus router (Step 3.5): two-question layering above the retrieval router, selection precedence + effective-strategy auto-detect, UNCONSTRAINED fallback, fan-out vs single-pass + RRF weighting, dependency-free learned classifier, PolicyEngine boundary, trace/audit, deferred concerns, reviewer checklist |
| agent-loop.md | Agent runtime (Step 3.6): why a new rag-agent package (depends only on rag-core), the explicit planning → acting → observing → finalizing state machine, four-dimension budgets + two distinct ceilings, snapshot-based resumability (last-write-wins, raw-cap budget), controller/tool seams, retrieval + egress governance at the boundary, flat AgentEvent shared by both surfaces, invariants + reviewer checklist, deferred concerns |
| sdk-generation.md | SDK generation (Step 3.7): two contracts (committed drift-gated OpenAPI spec + rag.proto), hand-written Python/TS flagships vs generated Go/Java/.NET, why generated source is a gitignored build artifact, scripts/{export_openapi,gen_sdks}.py orchestration, hand-written SDK design (identity/SSE/error mapping), testing strategy, reviewer checklist |
| admin-ui.md | Admin console (Step 3.10): client-rendered Next.js 14 App Router app, hand-rolled shadcn-style primitives + handoff tokens (the relative-content cwd invariant), the useLive live-vs-seed hybrid (Corpora + Webhooks live → gateway, rest seed, API Keys/Tenants Preview), global providers (tenant/theme/review-state/toasts), layered verification (tsc/lint/vitest/build + screenshot diff), reviewer checklist |
| webhooks.md | Outbound webhooks (Step 3.9): three-package acyclic graph (rag-core types/SPIs/Protocol → rag-webhooks engine → emitters), WebhookPublisher Protocol seam, fire-and-forget emission, replay-safe HMAC signing, at-least-once retries, dispatcher tested against noop SPIs with injected clock/sleep, tenant isolation, reviewer checklist |
| framework-adapters.md | Framework adapters (Step 3.8): adapters wrap the SDK (not the gateway), _common shared retrieval + chunk mapping, lazy framework imports + per-framework extras for dependency isolation, fake-client testing + importorskip, the integrations CI job, why CI mypy skips the integration modules, reviewer checklist |
| status-metrics-gui.md | Status & Metrics GUI (Step 3.11): the in-process read-side (MetricsCollector + LogTail) complementing OTel, WebSocket-for-snapshots / SSE-for-log-tail rationale, health = wiring + per-route error rate (no synthetic probes), the console pages + useLive seed-fallback hybrid, per-app-collector vs process-global-tail asymmetry, invariants + reviewer checklist |
| tiered-cache.md | Tiered semantic cache (Step 4.1): L0 in-memory LRU+TTL / L1 injected exact (Redis) / L2 token-set similarity behind the RetrievalCache+AnswerCache SPIs, read-through+promotion / write-through, the L2 scope token (vary only the query text), reliability invariant (L1 failures degrade never raise), cache.* events + TierStats, gateway integration + deferred corpus-version bump, reviewer checklist |
| fallback-chain.md | Fallback chain (Step 4.2): graceful degradation ladder hybrid → BM25-only → keyword → "no answer" wrapping the retrieval router; proactive budget-start-tier (ADR-0008) + reactive error/empty advance; no_answer as a graceful 200 (configurable); the ACL invariant (relax caller filters, never the policy pushdown); SupportsRoute drop-in; retrieve.fallback span + fallback.engaged event; deferred concerns + reviewer checklist |
| hallucination-guard.md | Hallucination guard (Step 4.3): per-claim NLI faithfulness of generated answers vs. retrieved context; NLIScorer SPI + dependency-free LexicalNLIScorer default; where it plugs in (after egress, on /v1/query + OpenAI chat + MCP); per-tenant threshold; annotate/redact/block; streaming = advisory-only; reliability degrade-never-raise; PII-safe guard.claim_blocked; deferred concerns + reviewer checklist |
| circuit-breaker.md | Circuit breakers (Step 4.4): per-backend three-state machine (closed/open/half-open) generalising the Step 2.10 health tracker; the SPI-wrapper integration seam (open breaker → CircuitOpenError → HybridRetriever drops the source); consecutive-failure trip + timed half-open probe; hot-path discipline (no per-call span); the breaker.opened event + status-API state + one-click force-close; health roll-up mapping; deferred concerns + reviewer checklist |
| quotas.md | Quotas & rate limiting (Step 4.5): per-tenant QPS / token / cost / queries / storage caps enforced through the PolicyEngine PDP (QuotaPolicyEngine decorator) backed by a pluggable QuotaStore (in-memory + Redis); mechanism-vs-policy split; weighted two-slot sliding-window counter (shared algorithm across stores) + storage gauge; cross-worker state rationale; reliability fail-open + atomicity + tenant isolation + PII-free quota.exceeded; operator surface (status API, admin card, ragctl); deferred Phase-6 work + reviewer checklist |
| latency.md | Latency gate, load test & profiling (Step 4.6): the gateway-overhead budget (p99 ≤ 30 ms, noop backends) vs the end-to-end target; the server-side gateway.request_duration_ms as the gate signal; in-process / sequential / single-runner / retry-tolerant rationale (incl. the GC-on finding); in-process harness vs cProfile profiler (own-time sort, warmup outside the profiler) vs Locust; invariants + deferred + reviewer checklist |
| per-query-tracing.md | Per-query tracing & provenance (Step 5.1): stable span attributes via the one span_from_trace_context choke point (rag.schema_version on every span) + the telemetry_attrs registry/contract; HMAC-signed ProvenanceRecord (privacy-by-default hashes, degrade-open recording); the TraceCollector read-side span processor grouping by rag.trace_id; tenant-isolated GET /v1/query/{id}/trace; inert-by-default wiring; invariants + reviewer checklist |
| golden-set-eval.md | Offline golden-set eval harness (Step 5.2): why the noop backends give a real signal (NoopKeywordStore token overlap + a deterministic HashingEmbedder for the zero-vector NoopEmbedder) fused via the real HybridRetriever; corpus/golden-set co-design (5 domains × 20 concepts × 3 passages, chunk_id == source_uri, committed + drift-gated); metric definitions/edge cases (nDCG, dependency-free lexical_faithfulness vs RAGAS); additive report types + self-contained HTML; relationship to the Step 5.3 gate; reviewer checklist |
| ab-shadow-mode.md | A/B testing & shadow mode (Step 5.7b): run a candidate retriever observe-only on a sampled fraction of live queries and feed the ABExperimentTracker; ShadowRunner scheduled as a FastAPI BackgroundTask (served-path latency untouched) + degrade-open + deterministic request_id-hash sampling; backend-only outcome_metric (mean retrieval score, the drift signal, since the candidate is never served); skipped on a cache hit; candidate = any SupportsRoute (config builds a RetrievalRouter differing only in RRF fusion weights; production injects one); same read_chunk PDP, event-only (experiment.shadow_*), inert + opt-in by default; key decisions + extension points |
| ab-routing.md | A/B routing (Step 5.7c): deterministically assign each query to a variant and serve the candidate config to the routing_sample_rate fraction (the first slice that can change a response); ABRouter decides/records/tags while the /v1/query + /v1/retrieve handlers own the served-path wiring; served-not-background + not-degrade-open (candidate failure → 502, no silent fallback) + variant-partitioned retrieval cache + one outcome per fresh retrieval; ExperimentAssignment wire tag on the response (REST-only for now); same read_chunk PDP, gated separately (routing_enabled) from shadow, inert + opt-in by default; governance + extension points |
| drift-monitors.md | Drift monitors (Step 5.5): the five monitors (query distribution / embedding PSI / retrieval score / citation-clickthrough / faithfulness) comparing a reference vs current window; two statistics (PSI for distributions, mean-drop for rate/score) over one scalar-window DriftMonitor; infra-scoped DriftMonitorRegistry fed via observe from the query + feedback paths (cheap O(1) appends, sparse signals report insufficient_data); detection on dashboard-poll evaluate() with transition-edge drift.detected (structured event + the Step 3.9 webhook); observe-only / inert-by-default / rebaseline; GET /v1/status/drift; invariants + extension points + reviewer checklist |
| cost-anomaly.md | Cost anomaly (Step 5.6c): per-tenant spend-spike detection — a CostTracker (in rag-observability, beside the metrics/log/trace read-side) keeping a bounded rolling window of recent per-request token costs per tenant, with a two-gate (ratio + z-score, z relaxed on a flat baseline) tri-state verdict; scale-free on tokens so it's decoupled from quota pricing; fed O(1) from record_request_usage independent of quotas; pull-based GET /v1/status/cost (no per-request span/event); inert-by-default; invariants + extension points + reviewer checklist |
| online-feedback.md | Online feedback & implicit signals (Step 5.4): the POST /v1/feedback → FeedbackRecorder (normalise score → PII-redact comment → tenant-scoped FeedbackStore.put → feedback.recorded, degrade-open) flow + the GET /v1/status/feedback → aggregate_feedback → FeedbackStats dashboard; one polymorphic signal enum for explicit + implicit; redact-don't-hash (vs provenance); [-1,1] score normalisation; body identity; event-only observability; inert-by-default wiring; tenant-isolation / no-PII / degrade-open invariants; reviewer checklist |
| ci-eval-gate.md | CI eval gate (Step 5.3): the harness → compare_to_baseline → EvalComparison → PR-comment flow; the compact byte-stable EvalBaseline (no run_id / created_at / samples) vs the full report.json; the two checks (absolute floor + regression delta), both from thresholds.yaml hard_gates, with the harness --check and the gate sharing one load_gate_thresholds; only recall/mrr/faithfulness gate (acl/pii enforced by red-team); standalone always-run eval-gate.yml (least-privilege pull-requests: write, clean required-check); sticky marker comment posted even on failure + fork-PR tolerance; invariants + extension points + reviewer checklist |
| File | Description |
|---|---|
| ragctl.md | Full ragctl command reference — public usage, internals, extension points |
| backends.md | rag-backends reference — PgVectorStore, QdrantVectorStore, WeaviateVectorStore, PineconeVectorStore, ElasticsearchDenseVectorStore, ElasticsearchKeywordStore, TantivyKeywordStore, Neo4jGraphStore, MemgraphGraphStore, NetworkXGraphStore, RedisCache, S3Storage, LocalFileStorage |
| rag-core.md | rag-core type surface — RequestContext, Budget, BlobRef, QueryPlan, ChunkRef, StageEvent, Pipeline, Batcher, cache SPIs |
| cache.md | rag-cache reference (Step 4.1) — TtlLruCache, InMemory{Retrieval,Answer,Embedding}Cache, Semantic{Retrieval,Answer}Cache, Tiered{Retrieval,Answer}Cache + TierStats; the Redis L1 tier (Redis{Retrieval,Answer}Cache); config, events, extension points |
| fallback.md | rag_retrieval.fallback reference (Step 4.2) — FallbackChain (run + drop-in route), FallbackConfig, SupportsRoute, and the FallbackTier / FallbackTrigger / FallbackResult core types; ragctl fallback; span + event schema; config + extension points |
| guard.md | rag-guard reference (Step 4.3) — HallucinationGuard (check → (GuardResult, answer)), GuardConfig, LexicalNLIScorer, the NLIScorer SPI + NoopNLIScorer, and the NLILabel / NLIScore / GuardVerdict / GuardAction / ClaimVerdict / GuardResult core types; rag.yaml guard block; guard.check span + guard.claim_blocked event; ragctl guard; extension points |
| breaker.md | rag-breaker reference (Step 4.4) — CircuitBreaker (call / allow / record_* / force_close / snapshot), CircuitBreakerConfig, BreakerRegistry, the Breaker{Vector,Keyword,Graph}RetrievalBackend SPI wrappers, and the CircuitState / BreakerSnapshot core types + CircuitOpenError; rag.yaml breakers block; GET/POST /v1/status/breakers; breaker.opened event; ragctl breaker; extension points |
| quota.md | rag-quota reference (Step 4.5) — QuotaEnforcer, QuotaPolicyEngine, QuotaPolicy / QuotaRuntimeConfig / TenantQuotaLimits, the QuotaStore SPI (InMemoryQuotaStore + RedisQuotaStore), and the QuotaVerdict / QuotaSnapshot / QuotaDimension core types + QuotaExceededError; the five dimensions; rag.yaml quotas block + extended TenantQuota; GET/POST /v1/status/quotas; quota.exceeded event; ragctl quota; extension points |
| perf.md | Performance gate, load test & profiling (Step 4.6) — rag_gateway.perf (measure_gateway_overhead, profile_gateway, LatencyReport / RouteLatency / ProfileEntry / Scenario), the tests/perf/ p99 ≤ 30 ms overhead gate (env knobs + perf marker), the published tests/contract/budgets.py registry, the eval/gateway_load_v0 harness + Locust file, ragctl perf, task perf / load-test, extension points |
| injection.md | Prompt-injection guard (rag-injection, Step 7.3) — PromptInjectionGuard.inspect (drops hijack payloads before the LLM), the pluggable InjectionDetector + dependency-free HeuristicInjectionDetector (attack-grammar regex library), InjectionConfig / InjectionResult / InjectionAction / InjectionVerdict / InjectionMatch, the INJECTION_RESISTANT_SYSTEM_PROMPT + build_user_message trust-isolation helpers, cfg.injection, the injection.blocked event; internals (noisy-OR scoring, degrade-open, PII-free telemetry) + extension points |
| pilot.md | ragctl pilot (Step 7.4) — pilot onboard (per-tenant rag.yaml block + onboarding checklist) + pilot report (weekly KPI dashboard + PASS/FAIL verdict from the platform's own feedback / drift / cost signal components; quality + latency cross-referenced to ragctl eval / perf); the KPI→signal map; seed-then-report (no infra); extension points |
| provenance.md | rag-provenance reference (Step 5.1) — ProvenanceRecorder (build / capture (degrade-open) / verify) + ProvenanceSigner (HMAC-SHA256), the ProvenanceStore SPI (NoopProvenanceStore), the ProvenanceRecord / ProvenanceCitation / ProvenanceSignature / SignedProvenanceRecord / ProvenanceVerification / SpanRecord core types + QueryTraceResponse, the TraceCollector, GET /v1/query/{id}/trace, rag.yaml provenance block, provenance.recorded event, ragctl provenance, extension points |
| drift.md | Drift monitors (Step 5.5) — rag_drift.population_stability_index / DriftMonitor / DriftMonitorRegistry; the DriftSnapshot / DriftReport / DriftMetric / DriftMethod / DriftStatus core types; the five-monitor table (metric → method → signal → drift condition); GET /v1/status/drift + POST /v1/status/drift/{metric}/rebaseline; cfg.drift; the drift.detected event + webhook; ragctl drift; PSI rule-of-thumb; extension points |
| feedback.md | Online feedback (Step 5.4) — rag_feedback.FeedbackRecorder / aggregate_feedback / score_for_signal / kind_for_signal; the FeedbackRecord / FeedbackStats / FeedbackKind / FeedbackSignal core types + FeedbackStore SPI (NoopFeedbackStore); FeedbackRequest / FeedbackAck wire types; the signal→score/kind table; POST /v1/feedback (body identity, PII-redacted comments, degrade-open ack) + GET /v1/status/feedback dashboard; cfg.feedback; feedback.recorded event; ragctl feedback; extension points |
| cost.md | Cost anomaly (Step 5.6c) — rag_observability.CostTracker (observe / snapshot) + the CostSnapshot dataclass (status ok/elevated/insufficient_data, ratio, z_score, token + micro-dollar means); the two-gate scale-free detection rule; GET /v1/status/cost (CostStatusResponse); cfg.cost; fed from record_request_usage independent of quotas; the admin Cost anomaly card; extension points |
| billing.md | Billing & metering (Step 7.9) — rag_observability.billing: UsageMeter (observe-only per-tenant, per-dimension counters) + BillingDimension + UsageSnapshot; pure generate_invoice + PricingModel / load_pricing (priced from marketplace/pricing.yaml, reconciles ±0.5% by construction); the BillingProvider Stripe/marketplace seam; GET /v1/billing/usage (BillingUsageResponse, no dist/schemas churn — mirrors 5.6c); cfg.billing; ragctl billing; extension points |
| experiments.md | A/B testing & shadow mode (Step 5.7) — the pure rag_config.eval.analyze_ab_experiment (lift + normal-approx Welch CI + significance, stdlib statistics.NormalDist) → ABAnalysisResult (rag_core.eval); rag_observability.ABExperimentTracker (bounded per-(experiment, variant) sample windows, decoupled from the analyzer); GET /v1/status/experiments (gateway composes the two); cfg.experiments (opt-in); ragctl experiments; the 5.7a–d slice plan + extension points |
| eval-harness.md | Offline eval harness + CI gate (Steps 5.2–5.3) — the five metrics (recall_at_k / mrr / ndcg_at_k / lexical_faithfulness / citation_precision) + render_html_report in rag_config.eval; EvalReport / EvalMetrics / GroupEvalMetrics (by_domain / by_difficulty); the eval/golden_set_v0 harness + generate.py (500-query / 5-domain committed golden set, drift-gated); python -m eval.golden_set_v0.harness (--domain / --queries / --k / --check); ragctl eval run --html / eval show; the gate (python -m eval.golden_set_v0.gate, committed EvalBaseline, floor + regression-delta, sticky PR comment); the regression bisector (Step 5.6d — python -m eval.golden_set_v0.bisect, binary-search good..bad in throwaway git worktrees to find the first commit that dropped a metric; pure bisect_commits → BisectResult); thresholds; extension points |
| rag-observability.md | rag-observability — structured logger, event registry, AsyncTelemetrySink (bounded, drop-on-overflow) |
| rag-policy.md | rag-policy reference — PolicyEngine, PolicyWriter, PolicyDecision, PolicyResult, FilterExpr |
| connectors.md | rag-backends connectors — FilesystemConnector, S3Connector, GCSConnector, resumable crawls via ConnectorState |
| parsers.md | rag-parsers reference — Parser SPI return shape, default_registry(), detect_mime(), format coverage, ragctl parse |
| ocr.md | rag-ocr reference — OCR SPI region-aware results, TesseractOCR, PaddleOCRBackend, ragctl ocr, extension points |
| chunker.md | rag-chunker reference — Chunker SPI, HeadingAwareChunker, TokenCounter / TiktokenCounter, ocr_result_to_parsed_document, ragctl chunk |
| enricher.md | rag-enricher reference — Enricher SPI, DefaultEnricher, LanguageDetector / LangdetectDetector, doc_type_from_mime, section_path, ragctl enrich |
| pii.md | rag-pii reference — PIIDetector SPI, RegexPIIDetector, PresidioPIIDetector, PiiProcessor, PiiPolicyEngine (Step 6.5 egress: allow/redact/mask/block over answer + context via egress_text, cfg.pii.enabled), redact_spans / mask_spans, ragctl pii |
| embedders.md | rag-embedders reference — OpenAIEmbedder, CohereEmbedder, SentenceTransformersEmbedder + bge_large_en() / e5_large_v2(), BatchingEmbedder base, RetryPolicy, normalize_dimension, ragctl embed |
| ingest.md | rag-ingest reference — IngestPipeline.ingest_document / ingest_connector, IngestResult / IngestRunSummary / IngestStatus, POST /v1/ingest/document, ragctl ingest |
| retrieval.md | rag-retrieval reference — rrf_fuse, HybridRetriever, HybridWeights, GraphAdapter / default_graph_adapter, ragctl hybrid |
| query.md | rag-query reference — UnderstoodQuery, QueryUnderstandingPipeline, LLMQueryRewriter, HeuristicQueryDecomposer / LLMQueryDecomposer, HyDEGenerator, InMemoryGlossary, ragctl understand |
| reranker.md | rag-reranker reference — RerankPipeline, MMRConfig, mmr_rerank, SentenceTransformersCrossEncoder + bge_reranker_v2_m3() / minilm_l6(), CohereReranker, JinaReranker, ragctl rerank |
| packer.md | rag-packer reference — ContextPacker, PackerConfig, hash_dedup / jaccard_dedup, lost_in_middle_order / descending_order / ascending_order, fit_to_budget, detect_conflicts, ragctl pack |
| graphrag.md | rag-graphrag reference — LouvainDetector / LeidenDetector, CommunitySummarizer, InMemoryCommunityStore, GraphRAGRetriever, graphrag_adapter, ragctl graphrag |
| router.md | rag-retrieval router (Step 2.10) — RetrievalRouter, RouterConfig, BackendHealthTracker, HealthConfig, classify_shape, RoutingDecision, QueryShape, ragctl route |
| agent-loop.md | rag-retrieval agent loop (Step 2.11 spike) — AgentLoopV0, AgentLoopConfig, AgentLoopResult, AgentLoopIterResult, QueryUnderstander / UnderstoodQueryLike Protocols, ragctl agent-loop, OTel + log schema |
| gateway.md | rag-gateway (Step 3.1, 🥈 milestone) — route catalogue (/v1/query, /v1/retrieve, /v1/corpora), request/response shapes, error envelope, OpenAPI 3.1, OTel + structured-log schema, production wiring |
| rest-api.md | Generated REST API reference (Step 7.5) — the full /v1/* endpoint catalogue (params · request/response schemas · status codes) rendered from the drift-gated dist/openapi.json by scripts/gen_api_reference.py (task docs:api) |
| grpc.md | rag-gateway gRPC contract (Step 3.2) — rag.gateway.v1.RagService RPC catalogue, streaming semantics, metadata contract, error envelope, status-code mapping, codegen + extension points |
| mcp.md | rag-gateway MCP contract (Step 3.3) — query / retrieve / ingest tool catalogue, argument tables, output models, tool↔request-model parity, ToolError envelope, ragctl mcp-query + npx @ragplatform/mcp, extension points |
| openai-compat.md | rag-gateway OpenAI-compatible API (Step 3.4) — /v1/embeddings, /v1/chat/completions (retrieval pre-fetch + streaming), /v1/models, the rag request/response extension, OpenAI error envelope, ragctl chat / embeddings, extension points |
| corpus-router.md | rag-retrieval corpus router (Step 3.5) — CorpusRouter, CorpusRouterConfig, CorpusRule, StaticRuleClassifier, LearnedCorpusClassifier, CorpusClassifier Protocol, CorpusRoutingDecision / CorpusScore / CorpusRoutingStrategy, rag.yaml routing block, ragctl corpus list/seed/route, extension points |
| agent.md | rag-agent + agent surface (Step 3.6) — AgentLoop / AgentConfig, Controller (Scripted / Heuristic / LLM), Tool / ToolRegistry / RetrieveTool / Retriever, CheckpointStore / InMemoryCheckpointStore / AgentSnapshot, the rag_core.agent_types wire models, POST /v1/agent (SSE) + gRPC Converse + ragctl agent, governance boundary, observability, extension points |
| sdks.md | Official SDKs (Step 3.7) — Python (agentcontextos) + TypeScript (@agentcontextos/sdk) hand-written clients, generated Go/Java/.NET, identity model, usage per language, the task openapi:gen / sdk:gen pipeline, extension points |
| admin-ui.md | Admin console (Step 3.10) — Next.js 14 operator GUI (apps/admin-ui); 9 pages (dashboard, corpora, connectors, glossary, webhooks, audit, API keys, tenants, config), live-vs-seed hybrid + NEXT_PUBLIC_GATEWAY_URL, header identity, running it, internals (shell/primitives/data layer), extension points |
| tenancy.md | Logical multi-tenancy (Step 6.1) — per-tenant rag.yaml config (namespace / acl_labels / pii_policy / quota); TenantResolver.resolve(id) → TenantSettings; RequestContext.namespace; GET /v1/status/tenant; ragctl tenant list / resolve; config table + scope/boundaries (6.2/6.3/6.5) + extension points; physical tenancy (6.2), ACL push-down (6.3) + egress verifier (6.4 — cfg.acl.verify_egress) sections |
| audit.md | Audit log (Step 6.6) — AuditEvent / AuditStore (append / events / verify_chain) / NoopAuditStore SHA-256 hash chain / AuditWriter (+ .store); read API GET /v1/audit (tenant-scoped, chain_verified) + GET /v1/audit/verify (whole-log); WORM signed export (6.6b) — AuditExporter (content_hash + HMAC), POST /v1/audit/export, offline verify(), ragctl audit; cfg.audit.enabled / export_secret; durable-store extension points |
| encryption.md | BYOK envelope encryption (Step 6.7) — KeyManager SPI + NoopKeyManager; EnvelopeKeyManager (AES-256-GCM DEK + tenant_id AAD) + LocalKeyManager + cloud providers AwsKmsKeyManager / GcpKmsKeyManager / AzureKeyVaultKeyManager / VaultKeyManager (behind [kms-*] extras); EncryptingStorage decorator; EncryptionError / KeyUnavailableError (sealing); provider table; cfg.kms + tenants[].kms_key_id + build_key_manager_from_config factory; RotatingKeyManager + RetiredKey + rewrap (6.7d zero-downtime rotation); ragctl kms [--rotate] |
| sso.md | SSO / SCIM (Step 6.8, rag_sso) — FederatedAuth (Auth backend) + OidcProvider/OidcSettings + SamlProvider/SamlSettings/signxml_verifier + identity_to_principal; ScimService/parse_eq_filter; low-level verify_jwt/encode_jwt_hs256 (stdlib HS256, [oidc] RS256); ScimStore SPI + NoopScimStore; FederatedIdentity/SsoProtocol/ScimUser/ScimGroup core types + ScimListResponse/ScimPatchOp/ScimErrorBody/SsoStatusResponse wire types; SsoError/ScimError/ScimNotFoundError/ScimConflictError; /scim/v2/* + GET /v1/status/sso; cfg.sso/cfg.scim/tenants[].sso; sso.*/scim.* events; ragctl sso/ragctl scim; extension points |
| airgap.md | Air-gapped install bundle (Step 6.9) — ragctl airgap build/inspect/verify/install; ragctl.airgap models (BundleManifest/BundleImage/BundleFile/BundleVerification) + pure helpers (parse_image_list/render_sha256sums/content_hash/build_manifest/verify_bundle/gateway_image_ref) + the docker/helm/cosign seam (assemble_bundle/save_images/package_chart/load_images/install_chart/cosign_*); bundle layout + SHA256SUMS/content_hash integrity (+ reason codes); infra/airgap/images.txt; standalone install.{sh,ps1} (--verify-only); --dry-run; task airgap:build/build-dry/verify; release-airgap.yml |
| compliance.md | Compliance (Step 6.10, rag_compliance) — RetentionEnforcer (purge / erase_tenant, dry_run) + compliance_posture + residency_ok; DataClass/RetentionPolicy/ErasureResult/CompliancePosture core types; ComplianceError/ResidencyViolationError; purge_before/purge_tenant SPI methods; GET /v1/status/compliance + POST /v1/compliance/erase; cfg.compliance + tenants[].data_region/retention_days; compliance.* events; ragctl compliance report/demo; extension points |
| webhooks.md | Outbound webhooks (Step 3.9) — event catalogue (ingest.completed / audit.policy_violation / drift.detected / eval.regression), event envelope, HMAC signing + verify(), at-least-once delivery, /v1/webhooks/subscriptions CRUD + test, rag.yaml block, ragctl webhooks demo, internals + extension points |
| integrations.md | Framework adapters (Step 3.8) — agentcontextos.integrations.* for LangChain / LlamaIndex / Haystack / DSPy / LangGraph / CrewAI / AutoGen / Semantic Kernel; per-framework extras, shared config + chunk metadata, usage per framework, internals + extension points |
| status-api.md | Status & Metrics API (Step 3.11) — /v1/status/health / metrics / logs (+ SSE logs/stream), WS /v1/status/ws, /v1/connectors/status; metric catalogue + request-timing middleware, the MetricsCollector / LogTail read-side, CORS + query-param identity for browser streams, extension points |
| File | Description |
|---|---|
| logging-policy.md | RAG001 policy: structured-logging requirement, allowlist, and how to extend it |
| ragctl-quickstart.md | Five-minute tour of the ragctl CLI |
| airgap-install.md | Operator runbook (Step 6.9): build + sign an offline bundle on a connected host (ragctl airgap build --sign), transfer it, then verify + install on the air-gapped target with the standalone install.sh/install.ps1 (sha256sum -c SHA256SUMS → cosign → docker load → helm upgrade --install); trimming the image set; pointing the chart at in-network backends; offline-verifying a bundle with stdlib tools |
| load-testing.md | Load + chaos testing runbook (Step 7.1): the two layers — in-process CI gates (p99 ≤ 30 ms overhead + chaos-under-load graceful degradation) vs the Locust suite on a cluster (≥ 1000 RPS sustained, e2e p99 < 500 ms); task chaos-test / load-test / perf; running Locust at scale (distributed workers, LOAD_PEAK_USERS / LOAD_HOLD_S env knobs, the ramp shape); the acceptance-target table; securing the run with an auth header |
| chaos-engineering.md | Chaos engineering runbook (Step 7.2): the two layers — the in-process kill-matrix gate (task chaos-kill: kill each hot-path backend in turn → no 5xx, breaker opens, expected degraded shape) vs LitmusChaos on a cluster (infra/chaos/: gateway pod-delete + backend network-loss/latency with httpProbe acceptance); the per-backend survival table; the two degrade-open gaps 7.2 hardened (retrieval cache + reranker); why graph is off the default read path; acceptance targets |
| red-team.md | Red-team / security runbook (Step 7.3): the four probe classes (prompt injection / PII egress / ACL bypass / tenant escape) + suites; running the gate (task redteam, the redteam-gate CI job, pip-audit for CVEs); the prompt-injection defense (filtering via PromptInjectionGuard + structural trust isolation via the hardened prompt); the corpus + acceptance (≥ 95 % block, ≤ 5 % false positives, no untrusted chunk in a system-trust position); enabling cfg.injection; the external-pentest process item |
| design-partner-pilots.md | Design-partner pilot runbook (Step 7.4): the pilot lifecycle (qualify → onboard → run → review → graduate); roles; onboarding as a configured deployment; signed success criteria (quality/latency/integration/security) each tied to a platform signal; the weekly KPI dashboard pulled from GET /v1/status/{metrics,feedback,drift,cost,health} + the eval harness (assembled by ragctl pilot report); the intake → triage → incorporate → close feedback loop; acceptance (≥ 3 consecutive green weeks · 3 referenceable tenants) + the case study; the in-repo-machine vs external-GTM split |
| documentation-site.md | The documentation site (Step 7.5): the Docusaurus app under website/ that serves docs/ in place; running/building it; the generated REST API reference; the CI gates (API-ref drift · quickstart doc-tests · spell-check · lychee link-check · site build); adding docs |
| marketplace-listings.md | Cloud marketplace listings (Step 7.6): what's in-repo (marketplace/ — one canonical pricing.yaml + shared listing copy + per-cloud AWS/Azure/GCP specs) vs the external approval/procurement process; reusing the Helm/AMI/airgap/GHCR delivery; the submission checklists + the procurement acceptance test |
| packaging-distribution.md | Packaging & distribution (Step 7.7): the one-command install per channel (PyPI rag-platform, npm @agentcontextos/sdk, GHCR images, Helm OCI, air-gap); how one vX.Y.Z tag fans out across docker.yml + release.yml + release-airgap.yml; cosign-keyless + SPDX SBOM + OIDC/provenance signing; verification commands |
| support-sla.md | Support, SLAs & on-call (Step 7.8): the four support tiers + response targets; SLA targets per tier (matching marketplace/pricing.yaml); the follow-the-sun on-call + escalation; wiring platform alerts to PagerDuty via the Step 3.9 webhook system (no new code) |
| curl-quickstart.md | 🥈 Curl-able RAG (Step 3.1): 5-minute walkthrough from curl to gateway response, including ingest, query, generate, OpenAPI |
| grpcurl-quickstart.md | gRPC quickstart (Step 3.2): 5-minute walkthrough using grpcurl against the in-process server — health check, list corpora, server-streaming query, structured errors |
| mcp-quickstart.md | MCP quickstart (Step 3.3): 5-minute walkthrough — ragctl mcp-query, running the stdio server, mounting @ragplatform/mcp in Claude Desktop, the three tools, error shape |
| openai-quickstart.md | OpenAI-compat quickstart (Step 3.4): 5-minute walkthrough — embeddings + chat with curl, the rag extension field, streaming, the official OpenAI SDK with extra_body, ragctl chat / embeddings |
| corpus-routing.md | Corpus-routing quickstart (Step 3.5): 5-minute walkthrough — declare corpora + routing in one rag.yaml, ragctl corpus list/seed/route, static-rule / explicit-pin / unconstrained paths, wiring it into the gateway via build_app_from_config |
| agent-quickstart.md | Agent quickstart (Step 3.6): 5-minute walkthrough — ragctl agent, POST /v1/agent SSE with curl, the gRPC Converse client, budget-driven early stop, resume by run_id, and what the credential-free default does under the hood |
| sdk-quickstart.md | SDK quickstart (Step 3.7): 5-minute tour — Python + TypeScript clients (query, chat, streaming agent), production bearer-token identity, typed errors, and generating the Go/Java/.NET clients with task sdk:gen |
| admin-ui-quickstart.md | Admin console quickstart (Step 3.10): 5-minute tour — run the Next.js console standalone (seed data), click through all 9 pages, then point it at a live gateway with NEXT_PUBLIC_GATEWAY_URL so Corpora + Webhooks light up |
| webhooks-quickstart.md | Webhooks quickstart (Step 3.9): 5-minute tour — ragctl webhooks demo, register a subscription via curl, fire a test delivery, verify the HMAC signature on your side + dedupe on event id, wire it via rag.yaml |
| integrations-quickstart.md | Framework integrations quickstart (Step 3.8): 5-minute tour — install one extra, plug the gateway into LangChain / LlamaIndex / Haystack / DSPy / LangGraph / CrewAI / AutoGen / Semantic Kernel as a retriever or tool |
| operator-console-live.md | Operator console quickstart (Step 3.11): 5-minute tour — curl /v1/status/*, watch the SSE log tail + WebSocket health/metrics push, open the console's Live Status / Metrics / Logs pages on seed data, then point it at a live gateway with NEXT_PUBLIC_GATEWAY_URL |
| grafana-dashboards.md | Grafana dashboards (Step 5.6e): how drift + cost become OTel observable gauges (rag_drift_* / rag_cost_*) on the OTel→Prometheus pipeline, the provisioned rag-quality-cost dashboard, viewing it via task dev-full, the dashboard-validation CI gate, and what's deferred (feedback/Loki) |
| schema-drift-gate.md | dist/schemas/ is committed; how task lint:schemas + CI catch out-of-date schema files |
| test-layout.md | Why package-local tests/ directories omit __init__.py (and the cross-package tests/ tree keeps it) |
Time-boxed investigations whose deliverable is knowledge, not a production feature. A spike memo summarises what we learned, what's broken, and what to fix before committing to the next phase.
| File | Description |
|---|---|
| agent-loop-v0-gaps.md | Step 2.11 agent-loop validation spike: 50-query harness findings, gap list with severity tiers, prioritised punch list for Phase 3.1 |
| File | Description |
|---|---|
| ADR-0001-monorepo-and-tech-stack.md | Decision: uv workspaces, Pydantic v2, Taskfile, Python 3.12+ |
| ADR-0002-eval-framework.md | Decision: pure-Python Tier 1 metrics always-on; RAGAS as optional Tier 2 |
| ADR-0003-iac-kubernetes-native.md | Decision: Kubernetes-native Terraform modules over cloud-provider-specific RDS/ElastiCache |
| ADR-0004-storage-backends.md | Decision: single rag-backends package, MinIO for S3-compatible dev storage, graceful integration test skip |
| ADR-0005-policy-engine.md | Decision: rag-policy package as central PDP for ACL/PII/quotas/redaction; the Step 6.4 ACL egress verifier ships as a complementary independent second layer (ADR-0036) |
| ADR-0006-two-stage-rerank.md | Decision: Reranker SPI ships as two-stage cascade (fast bi-encoder → cross-encoder) from day 1 |
| ADR-0007-tiered-storage.md | Decision: Chunk.text: str | BlobRef enables lazy hydration and hot/warm/cold tiering |
| ADR-0008-cost-aware-planner.md | Decision: QueryPlan carries estimated_cost; fallback triggered planner-side before dispatch, not by post-hoc timeout |
| ADR-0010-semantic-embedding-cache.md | Decision (Step 2.11 G-01): SemanticEmbeddingCache wraps any EmbeddingCache and adds Jaccard-similarity lookup; text-hash cache stays as the inner exact-match layer |
| ADR-0009-vector-index-strategy.md | Decision: IndexHint selects index by scale tier (flat / ivfflat / HNSW / IVF-PQ / DiskANN); Embedding.dtype enables int8 / binary quantization |
| ADR-0011-grpc-wire-protocol.md | Decision (Step 3.2): grpcio-tools + mypy-protobuf for codegen; server-streaming Query only (bidi deferred to Step 3.6 agent loop); proto/rag.proto stays standalone from core.proto pending a reconciliation step |
| ADR-0012-mcp-server-topology.md | Decision (Step 3.3): MCP server runs the pipeline in-process behind a ToolBackend seam (future GatewayClientBackend slots in); one Python implementation with a thin @ragplatform/mcp npm launcher rather than a second TypeScript server |
| ADR-0013-openai-compatibility.md | Decision (Step 3.4): mirror the OpenAI REST dialect for drop-in adoption; retrieval pre-fetch injects context as a system message (on by default); RAG knobs namespaced under a non-standard rag field; header identity with a dev-only NoopAuth anon fallback; OpenAI error envelope |
| ADR-0014-corpus-routing.md | Decision (Step 3.5): layered corpus selection (explicit → static rules → learned → all) above the retrieval router with effective-strategy auto-detect; UNCONSTRAINED runtime fallback preserves pre-3.5 behaviour; federated fan-out RRF-fused by default; dependency-free term-weight "learned" classifier (training deferred); corpus router makes no governed SPI calls |
| ADR-0019-admin-ui.md | Decision (Step 3.10): Next.js 14 + Tailwind operator console with hand-rolled shadcn-style primitives (deterministic offline builds, no shadcn CLI / Radix matrix) ported 1:1 from a high-fidelity design handoff; a live-vs-seed hybrid (useLive) where Corpora + Webhooks hit the gateway with seed fallback + a "Demo data" badge, the rest are seed data, and API Keys/Tenants are Preview (Phase 6 backends); verified via tsc/lint/vitest/build + a screenshot diff; console auth / SSR / real governance backends deferred |
| ADR-0018-outbound-webhooks.md | Decision (Step 3.9): HMAC-SHA256 over <timestamp>.<body> (Stripe-style, replay-safe) for signing; at-least-once delivery with bounded exponential-backoff retries + stable event id (dedupe on it); a dedicated rag-webhooks package with emitters depending only on a WebhookPublisher Protocol in rag-core (acyclic graph, fire-and-forget); signing secret returned once then masked; durable store / DLQ / drift+eval emitters / alt transports deferred |
| ADR-0017-framework-adapters.md | Decision (Step 3.8): framework adapters live in one SDK subpackage (agentcontextos.integrations.*) and wrap the public Client rather than the gateway; lazy framework imports + per-framework optional extras keep the base install light and isolate conflicting dependency trees; shared pure core (_common, format_passages) + thin wrappers tested with a fake client + importorskip; CI mypy excludes the modules (subclass Any when frameworks absent); ingestion / chat-model adapters deferred |
| ADR-0016-sdk-generation.md | Decision (Step 3.7): SDKs from two canonical contracts (OpenAPI spec + rag.proto); Python + TypeScript hand-written flagships (idiomatic, tested), Go/Java/.NET generated (openapi-generator + buf); the OpenAPI spec is committed + drift-gated while generated SDK source is a gitignored build artifact; buf skips Python to avoid colliding with the committed grpc_tools.protoc stubs; publishing pipelines + an ergonomic Python/TS gRPC facade deferred |
| ADR-0015-agent-runtime.md | Decision (Step 3.6): new rag-agent package depending only on rag-core with wire types in rag_core.agent_types; explicit state machine driven by a pluggable Controller; four-dimension budgets with distinct max_steps (config) vs budget_iter (request) ceilings; snapshot-based resumability (last-write-wins, raw-cap budget bypassing the zero-cap validator); the loop makes no governed SPI calls (retrieval governed in the wired Retriever, egress at the gateway boundary); REST SSE + gRPC Converse surfaces serialise one flat AgentEvent; durable store / measured cost / more tools / MCP+OpenAI agent-mode deferred |
| ADR-0020-status-transports.md | Decision (Step 3.11): match transport to data shape — WebSocket pushes whole-state health+metrics snapshots (client-tunable cadence/channels), SSE tails append-only log records (reusing the chat/agent data: framing, browser EventSource-native); back the read-side with two in-process rag-observability primitives (MetricsCollector complementing OTel; LogTail ring buffer polled by seq cursor to dodge cross-thread asyncio); wire models stay gateway-local; Prometheus exposition / push log streaming / deep health probes / status auth deferred |
| ADR-0021-tiered-cache.md | Decision (Step 4.1): ship the RetrievalCache/AnswerCache production impls as a tiered L0 (in-memory LRU+TTL) / L1 (injected exact, Redis) / L2 (token-set Jaccard similarity) cache in a new core-only rag-cache package so the gateway caches without pulling Redis (L1 lives in rag-backends, injected); read-through+promotion / write-through, reliability invariant (L1 failures degrade never raise), the L2 scope token holding non-text params constant, cross-tenant red-team suite; cache-hit gateway responses flagged served_from_cache (not faked); answer-cache consultation + corpus-version bump deferred to Phase 6 |
| ADR-0022-hallucination-guard.md | Decision (Step 4.3): verify generated answers against retrieved context per-claim via a swappable NLIScorer SPI (mirrors Reranker) in a new core-only rag-guard package; dependency-free LexicalNLIScorer default; per-tenant threshold; annotate/redact/block (disabled by default, behaviour-neutral); plugs in after egress_text on /v1/query + OpenAI chat + MCP; reliability degrade-never-raise; PII-safe guard.claim_blocked; rejected LLM-prompt-in-gateway + PolicyEngine-reuse + whole-answer-score |
| ADR-0023-circuit-breakers.md | Decision (Step 4.4): isolate failing backends with a per-backend three-state CircuitBreaker in a new core-only rag-breaker package; integrate by wrapping each retrieval backend SPI (open breaker raises CircuitOpenError, which HybridRetriever's gather(return_exceptions=True) already drops) rather than touching the fan-out; generalise — not replace — the Step 2.10 health tracker; consecutive-failure trip + timed single-probe half-open; on by default (closed = pass-through); no per-call OTel span (hot path), observability via breaker.opened + status snapshot; operator one-click force-close; rejected generic-proxy wrapping + replacing the health tracker + per-tenant breakers |
| ADR-0024-quotas-rate-limiting.md | Decision (Step 4.5): enforce per-tenant quotas + rate limits through the PolicyEngine PDP (a QuotaPolicyEngine decorator answering rate_limit / quota_check) per ADR-0005, in a new rag-quota package split into a QuotaStore mechanism (in-memory + injected Redis) and a QuotaEnforcer policy; weighted two-slot sliding-window counter shared across stores (QPS + monthly accruals) + storage gauge, cost in micro-dollars; disabled-by-default (can reject) + fail-open-by-default (store outage admits, never raises); per-field limit merge; rejected standalone middleware, ZSET-log windows, per-worker counters, fail-closed default, token reservation |
| ADR-0025-latency-load-testing.md | Decision (Step 4.6): gate the gateway's own overhead (noop backends) at p99 ≤ 30 ms, distinct from the backend-bound end-to-end target; the gate signal is the server-side gateway.request_duration_ms operators already watch; in-process via httpx ASGITransport, sequential, on a single-runner CI job (excluded from the cross-OS sweep), retry-tolerant; one engine (rag_gateway.perf) powers the gate + the eval/gateway_load_v0 harness + ragctl perf; profiling sorts by own time with warmup outside the profiler; Locust stays an ad-hoc install, not a locked dep; rejected end-to-end-as-gate, cross-OS gating, locked Locust, a bespoke timing path, GC-off measurement |
| ADR-0026-per-query-tracing-provenance.md | Decision (Step 5.1): make one query explainable + trustworthy via three composing parts — stamp rag.schema_version on every span at the single span_from_trace_context choke point (with a telemetry_attrs registry + contract test) rather than rewriting 21 call sites; capture an HMAC-signed ProvenanceRecord (privacy-by-default hashes, degrade-open, signing optional but record not) in a new rag-provenance package modeled on the webhook signer; a read-side TraceCollector span processor grouping by rag.trace_id behind tenant-isolated GET /v1/query/{id}/trace; inert-by-default build_app wiring (on via config); rejected per-call-site version stamping, storing raw text, request-id span grouping, fail-closed recording |
| ADR-0027-offline-eval-harness.md | Decision (Step 5.2): complete the Step 0.8 eval skeleton with a deterministic, offline harness — synthetic 5-domain corpus on the noop SPIs (a HashingEmbedder gives the zero-vector dense path real signal, fused with NoopKeywordStore token overlap via the real HybridRetriever); 500-query golden set committed under tests/eval/golden/ and generated reproducibly from the corpus (drift-gated), 3 passages/concept so nDCG + Citation Precision aren't degenerate; nDCG added + dependency-free lexical_faithfulness default (RAGAS optional) keep CI ML-free; harness in eval/golden_set_v0/ (not rag_config) so config stays light; additive report fields + self-contained HTML; --check enforces the threshold floor now (regression-delta is 5.3); rejected real backends in CI, RAGAS-as-default, hand-authored queries |
| ADR-0032-ab-testing-shadow-mode.md | Decision (Step 5.7): compare two configs on live traffic, delivered in slices (5.7a analyzer+tracker+surface, 5.7b shadow, 5.7c routing, 5.7d console); the analyzer is pure + stdlib-only in rag_config.eval (normal-approx Welch via statistics.NormalDist, no numpy/scipy — same spirit as drift PSI / cost z-score); the ABExperimentTracker is a pure sample holder in rag_observability (so it doesn't import rag_config), and the gateway composes the two for GET /v1/status/experiments; opt-in by default (A/B routing can change responses); ABAnalysisResult additive in rag_core.eval (not in dist/schemas); rejected numpy/scipy, a new package, putting the analyzer in observability, defaulting on |
| ADR-0033-logical-multi-tenancy.md | Decision (Step 6.1): make per-tenant rag.yaml config drive requests. TenantResolver (rag_config) maps a tenant id → frozen TenantSettings (rag_core), applied once at the gateway boundary; resolver in config / settings type in core keeps the config → core direction; unknown tenants → safe defaults (isolated not privileged); RequestContext.namespace defaults to tenant_id (a backend-partition primitive — Pinecone uses it — not a chunk field, so filter_pushdown is unchanged); scope stops at resolution + threading (ACL push-down 6.3, PII egress 6.5, physical tenancy 6.2); additive + inert in build_app |
| ADR-0034-physical-multi-tenancy.md | Decision (Step 6.2): a dedicated vector index/collection per tenant. TenantConfig.dedicated_index resolves to a physical_index key on TenantSettings, threaded onto RequestContext.physical_index; backends read only ctx (graph is backends→core, never rag-config) and namespace their base under it (<base>-<key>), lazily creating it; one instance + per-tenant derivation (no per-tenant instances, no SPI change); Noop is the CI conformance oracle (keyed by physical_index) for a cross-tenant probe gate that proves isolation independent of the tenant filter; Noop + Pinecone + Qdrant this step, others later |
| ADR-0035-acl-pushdown.md | Decision (Step 6.3): label-based ACL push-down at retrieval. AclPolicyEngine (a decorator like QuotaPolicyEngine) And-merges any_in("acl_labels", principal.acl_labels) into every read_chunk push-down at the canonical HybridRetriever PDP site — overlap semantics via the existing AnyIn predicate (zero backend/translator changes), fail-closed (label-less principal matches nothing; "public" = a shared label), opt-in via cfg.acl.enabled; emits acl.egress_denied on a request-level denial; graph edge ACLs + post-retrieval re-verification (6.4) deferred |
| ADR-0036-acl-egress-verifier.md | Decision (Step 6.4): a post-retrieval ACL re-check as a defense-in-depth second layer behind the 6.3 push-down. AclEgressVerifier.verify(ctx, refs) drops any returned ChunkRef whose labels don't overlap the principal's — same overlap semantics (no-op on correct results), reading ChunkRef.acl_labels (no re-hydration), independent of the PDP (consults only ctx.principal.acl_labels) so a push-down bug/bypass can't disable both; wired at the gateway as a SupportsRoute wrapper around app.state.retrieval_router (covers query/retrieve/corpus/OpenAI/agent); cfg.acl.verify_egress default on but gated by enabled; emits acl.egress_violation on a caught leak; a red-team gate proves a zero escaped-violation rate when the push-down is bypassed; backend-mislabel re-hydration + per-tenant violation metrics deferred |
| ADR-0043-load-chaos-testing.md | Decision (Step 7.1, Phase-7 hardening): split load testing into a deterministic CI gate + a cluster runbook. Chaos-under-load is a CI gate — drive the in-process gateway under concurrent load while injecting backend faults (FaultSpec + Chaos{Vector,Keyword,Graph}RetrievalBackend, SPI wrappers like the breaker wrappers, allowlisted in policy-coverage) and assert graceful degradation: no 5xx, 100% success, the relevant breaker opens. The asserted property is resilience, not throughput, so it's timing-independent + deterministic; it reuses the Phase-4 breakers + fallback (builds no new resilience — it validates them). The 1000-QPS / p99<500ms acceptance is a cluster runbook, not CI (hardware/backends-bound, same reasoning as ADR-0025) — shipped as a Locust v1 suite (weighted read/write mix + a LoadTestShape ramp + varied queries so retrieval is exercised, not the cache) + documented targets. Deferred: distributed-Locust-in-CI against an ephemeral cluster, latency-based breaker tripping, storage/LLM fault injection, soak tests; rejected asserting raw throughput in CI, killing a backend process (no separate process in-process — inject at the SPI boundary) |
| ADR-0044-chaos-engineering.md | Decision (Step 7.2, Phase-7 hardening): kill each backend, verify the fallback chain holds — a deterministic in-process kill-matrix CI gate extending 7.1 to the full hot-path set (vector/keyword/graph/embedder/retrieval_cache/reranker/llm); kill one at a time → no 5xx, on-path retrieval breaker opens, expected degraded shape; seeded keyword corpus + real hydrate so rerank/generate actually run. Chaos fixes what it finds — the matrix exposed that a down retrieval-cache or reranker 5xx-ed, so the gateway gained two degrade-open guards (gateway.cache.degraded → miss; gateway.rerank.degraded → retrieval-only, honouring RerankPipeline's "caller decides" contract); LLM + embedder already degraded. Graph is off the seed-less read path → its kill is survivable by construction (no breaker-open required). Cluster chaos is a runbook — LitmusChaos infra/chaos/ (gateway pod-delete + backend pod-network-loss/latency with httpProbe acceptance). No dist//SPI/config change (degrade kinds aren't registered events; kill wrappers are pure-raise). Deferred: latency-based breaker tripping, multi-kill-as-gate, Litmus-in-CI, soak; rejected leaving the holes documented-only, reranker fallback inside the pipeline, a synthetic graph query, pod-kill in CI |
| ADR-0045-red-team-security.md | Decision (Step 7.3, Phase-7 hardening): turn the governance stack into an adversarial probe gate (injection / PII / ACL / tenant-escape) + close the injection gap. New rag-injection package: a pluggable InjectionDetector (dependency-free HeuristicInjectionDetector, attack-grammar regexes) + PromptInjectionGuard.inspect that drops hijack chunks, paired with the INJECTION_RESISTANT_SYSTEM_PROMPT so untrusted context is fenced data in the user turn, never a system-trust position (fixes the OpenAI-chat surface that injected context as a system turn); wired on /v1/query + /v1/chat/completions + MCP; off by default (cfg.injection). Deterministic CI gate: a ≥ 500 known + ≥ 500 generated corpus (eval/redteam_v0/) at ≥ 95 % block + a false-positive bound + the no-system-position invariant end-to-end; building it hardened the detector. PII-egress probe over PiiPolicyEngine (zero leakage, second-detector verified); redteam marker + redteam-gate CI job; pip-audit is the CVE gate. Injection types are internal (no attacker signal, dist untouched). Deferred/process: the external pentest, a real ML classifier behind the seam; rejected a core SPI, surfacing the verdict, tuning to a circular 100 % |
| ADR-0046-design-partner-pilots.md | Decision (Step 7.4): ship the pilot machine in-repo (runbook + templates + per-vertical kits + a ragctl pilot KPI reader) while the partner relationships stay an external GTM deliverable (like the 7.3 pentest); each success criterion (quality / latency / integration / security) maps to a platform signal (eval / feedback / drift / cost / metrics / compliance) so KPIs are pulled, not self-reported; sliced 7.4a–d, leading with customer-support/KB, the framework vertical-extensible; no new package / core type / governed SPI call / dist change — the report reads existing status endpoints; rejected a pure-external motion, self-reported KPIs, a rag-pilot package, one generic kit |
| ADR-0047-documentation-site.md | Decision (Step 7.5): publish docs via Docusaurus sourcing the repo docs/ tree in place (single source of truth, no copy); markdown.format: 'detect' so hand-written .md renders as CommonMark; the REST API reference is generated + drift-gated from dist/openapi.json (a committed file, not a build-time plugin); doc honesty is a tests/docs/ suite (every ragctl / /v1/ reference is real) + lychee + codespell; rejected MkDocs, copying docs/, an OpenAPI plugin |
| ADR-0049-packaging-distribution.md | Decision (Step 7.7): a rag-platform PyPI meta-package pins the workspace component dists (one-command server install); keep the existing agentcontextos SDK scope (no rename); one vX.Y.Z tag fans out to PyPI/npm/GHCR/Helm-OCI/air-gap across docker.yml + a new release.yml + release-airgap.yml; cosign-keyless + SPDX SBOM + OIDC/provenance everywhere; deferred Go/Java/.NET publish + worker/eval-runner images; rejected a ragplatform rename + one mega-workflow |
| ADR-0050-support-sla-oncall.md | Decision (Step 7.8): runbooks grounded in the platform's own signals — one per alert type, each naming the exact event / /v1/status/* / SLO that raises it; paging reuses the Step 3.9 webhook system (PagerDuty as a subscriber, no new code) + Grafana SLOs; the status page reflects measured health (not manual toggles); SLA tiers match marketplace/pricing.yaml; rejected a new alerting subsystem, generic runbooks, a separate SLA datastore |
| ADR-0051-billing-metering.md | Decision (Step 7.9): mirror the 5.6c cost pattern — dataclasses in rag-observability, gateway wraps in a Pydantic response (no dist/schemas churn); metering is observe-only (no governed SPI call, fed from record_request_usage independent of quotas); invoicing is offline + pure + priced from config (marketplace/pricing.yaml, reconciles ±0.5% by construction); Stripe/marketplace via a BillingProvider Protocol seam; rejected a new rag-billing package + core types, metering-through-quotas, hard-coded prices |
| ADR-0052-ga-cutover.md | Decision (Step 7.10, GA): cut v1.0.0 with SemVer over the drift-gated public contracts (REST/OpenAPI · gRPC proto · rag.yaml · SDKs · ragctl) + a deprecation policy (≥ 1 MINOR before removal); the GA bar is a checklist of gates, not a declaration, with inherently-external items marked (external); one platform version cut as a tag that fans out to every channel; rejected staying 0.x, no platform version, declaring GA without a checklist |
| ADR-0048-marketplace-listings.md | Decision (Step 7.6): one canonical marketplace/pricing.yaml (metered dims = the Step 7.9 signals) + one shared listing copy mapped to all three clouds, so listings can't drift and prices reconcile with metering; reuse existing delivery (Helm / AMI / airgap / GHCR) — the per-cloud Marketplace wrappers are thin shells; approval + procurement are documented external process, not a faked status; rejected per-cloud pricing, custom packaging, encoding a "live" status |
| ADR-0042-compliance-posture.md | Decision (Step 6.10, Phase-6 capstone): add the three compliance pieces on top of the controls the platform already ships (audit/ACL/PII/BYOK/SSO/quotas). New rag-compliance package (config-free, like rag-feedback/rag-drift): RetentionEnforcer drives tenant-scoped purge_*; compliance_posture/residency_ok are pure. Retention is a capability on the existing stores, not a new SPI — non-abstract purge_before/purge_tenant (default no-op) on Feedback/Provenance, with dry_run in the SPI so a preview counts-without-deleting uniformly (ProvenanceStore has no list). Audit is never purged in place (the hash chain would break) — audit retention is the 6.6b WORM export; audit_days is advisory. Right-to-erasure is always-on, tenant-self-service, two-flag — POST /v1/compliance/erase erases the calling tenant's data (scope from the principal, never the body), dry-run by default, delete needs dry_run=false AND confirm=true. Residency = declared per tenant + enforced at ingest (tenants[].data_region vs cfg.compliance.region → 403), opt-in, a single-deployment assertion not multi-region routing. The SOC 2 / GDPR mapping is a doc backed by a live posture (GET /v1/status/compliance reports which controls are on, so the mapping is checkable). Deferred: subject-level (vs tenant-level) erasure, an admin retention-sweep endpoint, multi-region routing, automated audit-evidence bundles; rejected purging the audit chain, a Purgeable SPI mixin, a static doc with no live backing |
| ADR-0041-airgap-bundle.md | Decision (Step 6.9): ship the platform as one signed, self-contained offline bundle (all runtime images + Helm chart + config + installer). Integrity reuses the WORM-export pattern (6.6b): a standard SHA256SUMS whose hash is pinned as manifest.content_hash is the hard gate — verifiable with nothing but sha256sum, no network/cosign — and a cosign signature over SHA256SUMS adds authenticity; the same SHA256SUMS drives the Python verifier and the standalone shell installer so they can't diverge. The shell/pwsh install.{sh,ps1} (shipped inside the bundle) need only docker+helm (air-gap hosts lack uv/the workspace); ragctl airgap holds the typed/tested build+verify logic (pure core separated from a stubbable docker/helm/cosign subprocess seam; --dry-run = a verifiable bundle minus image blobs, so the path is testable with no Docker). Digest-pinned manifest-driven image set (infra/airgap/images.txt + the chart-derived gateway image); key-based cosign is the air-gap recommendation (keyless needs Rekor + an identity policy), keyless is the connected-release path (release-airgap.yml on tags). Deferred: ctr/podman load, registry re-tag/push, multi-arch selection, bundling backend charts, TUF-rooted offline keyless verify; rejected oras/OCI (no registry to pull from in an air-gap), a second HMAC scheme (cosign already the signer), a pure-shell build (would escape mypy/tests) |
| ADR-0040-sso-scim.md | Decision (Step 6.8): enterprise identity in two surfaces. Federation — FederatedAuth is an Auth SPI backend (the authenticate(token, tenant_id) → Principal seam already runs at the boundary, so wiring it is the whole integration — no middleware change); group claims → acl_labels so Step 6.3 push-down + 6.5 PII egress govern federated users unchanged (authorize stays a coarse allow — federation establishes who, the PDP decides what). Dependency-free defaults (stdlib HS256 JWT with full exp/nbf/iss/aud + constant-time compare; defusedxml SAML validating Issuer/Conditions/Audience) with asymmetric OIDC (PyJWT, [oidc]) + SAML XML-DSig (signxml, [saml], injected verifier → fail-closed) behind extras; algorithm-allowlist designs out alg:none/RS↔HS confusion. Per-tenant IdP on tenants[].sso (reuses Step 6.1 config; no provider → bearer rejected, header-identity still works). Provisioning — SCIM 2.0 is a separate surface with its own per-tenant bearer token (cfg.scim.tokens, not a user JWT), a tenant-scoped ScimStore SPI (NoopScimStore) + ScimService, SCIM-shaped errors, disabled→404; no new governed SPI call (linter passes). PII-free sso.*/scim.* events (hashed subject, never email/userName). Deferred: JWKS rotation, SP-initiated SAML + metadata, SCIM bulk//Me/ETag, directory-backed deprovisioning, admin-console card; rejected Authlib/python3-saml (heavy lxml/xmlsec on the default install), a dedicated SSO middleware, SCIM token on TenantConfig |
| ADR-0037-pii-egress-policies.md | Decision (Step 6.5): per-tenant PII enforcement at egress via a PiiPolicyEngine egress_text decorator (mirrors QuotaPolicyEngine / AclPolicyEngine), living in rag-pii (gains a rag-policy dep, like rag-quota). Handles both subject shapes the gateway already passes — list[Chunk] context + str answer — so it plugs into the existing egress_text call sites with no route change; maps ctx.pii_policy.action allow→delegate / redact·mask→transform / block→deny, reusing the Step 1.7 detector + rewriters and the same min_score+entities filter (no-op on clean text); opt-in cfg.pii.enabled (injects RegexPIIDetector); PII-free pii.egress_blocked (block) / pii.detected (redact·mask); post-gen answer re-check for query/OpenAI/gRPC + citation egress deferred (stored chunks are ingest-sanitised) |
| ADR-0038-immutable-audit-log.md | Decision (Step 6.6): make the 0.7c hash-chain audit log usable + provably intact, in two slices. 6.6a — the SHA-256 chain is the tamper-evidence mechanism (no second scheme); one shared AuditWriter/store on app.state (corpus router + read API write/read the same chain); GET /v1/audit tenant-scoped (a tenant sees only its own events, newest-first, chain_verified inline) + GET /v1/audit/verify whole-log {ok,event_count} (content-free, so global verification leaks nothing cross-tenant); read API on by default (cfg.audit.enabled=true — passive compliance record, unlike behaviour-changing ACL/PII). 6.6b — WORM signed export: AuditExporter builds a self-verifying AuditExport (SHA-256 content_hash over the events + HMAC signature, mirroring the ProvenanceSigner scheme), POST /v1/audit/export (tenant-scoped) + ragctl audit (whole-log), verifiable offline ({content_ok, verified, reason}); the artifact for immutable storage (S3 Object Lock) → immutability at rest; unsigned when no export_secret. Audit-coverage expansion + a durable live-store backend deferred |
| ADR-0031-cost-anomaly.md | Decision (Step 5.6c): detect per-tenant spend spikes with a rolling CostTracker (not the cumulative quota counter); detect scale-free on the token series (cost = tokens × a constant price) so detection is decoupled from quota pricing and works with quotas off; two gates (ratio + z-score, z relaxed on a flat baseline) → tri-state verdict; put it in rag-observability as a dataclass (gateway wraps it in a Pydantic CostStatusResponse) so there's no rag-core type / dist/schemas churn; feed O(1) from record_request_usage before the quota block; pull-based GET /v1/status/cost (no per-request span/event); rejected folding into the infra-scoped drift registry, a new package, a cost.anomaly_detected push event (deferred), per-model pricing, a time-series DB |
| ADR-0030-drift-monitors.md | Decision (Step 5.5): detect retrieval degradation with five drift monitors in a new rag-drift package (mirroring rag-feedback); two statistics — PSI (pure, binned, dependency-free) for the distribution monitors + mean-drop for the rate/score monitors — over one scalar-window DriftMonitor; infra-scoped registry (like breakers) fed via observe from the signals the gateway already computes (query length / retrieval score / HyDE-embedding norm / guard grounded-claim fraction / feedback citation clicks); detection on dashboard-poll evaluate() with transition-edge drift.detected (structured event + the Step 3.9 webhook, targeting alert_tenant); observe-only / inert-by-default / rebaseline; rejected per-tenant monitors, per-dimension embedding PSI, a stats library, a background scheduler, hot-path detection |
| ADR-0029-online-feedback.md | Decision (Step 5.4): capture online feedback + implicit signals in a new rag-feedback package mirroring rag-provenance (SPI + types in rag-core; recorder + pure aggregator in the package); one polymorphic POST /v1/feedback (a signal enum spanning explicit thumbs/rating/comment + implicit citation-click/copy/regenerate/dwell, kind inferred); normalise every signal to a [-1,1] score so the dashboard has one satisfaction number; redact-don't-hash free-text comments via an injected PIIDetector (default NoopPIIDetector seam, comment_redacted flag, PII-free event) — opposite of provenance's hashing; body identity like /v1/query; degrade-open + inert-by-default; GET /v1/status/feedback dashboard (event-only, no per-call span); admin-UI card deferred to 5.6; rejected separate per-signal endpoints, header-auth, hashing/raw comments, an OTel span per submission, folding into provenance |
| ADR-0028-ci-eval-gate.md | Decision (Step 5.3): gate PRs on the golden-set metrics vs a committed baseline — a compact byte-stable EvalBaseline (headline means + breakdowns, no run_id / created_at / samples) instead of the churny report.json; two checks (absolute floor + regression delta) both from thresholds.yaml hard_gates, the harness --check and the gate sharing one load_gate_thresholds; only recall/mrr/faithfulness gate (acl/pii are red-team's); standalone always-run eval-gate.yml for least-privilege pull-requests: write + a clean required check; one sticky marker comment posted even on failure, fork-PR tolerant; comparison in rag_config.eval, result types in rag_core.eval (additive, not in dist/schemas), runner in eval/; rejected folding into ci.yml/ci-pass, path-filtering the gate, report.json as baseline, floor-only, pull_request_target |
| File | Description |
|---|---|
| soc2-control-mapping.md | SOC 2 Type II — maps each AICPA Trust Service Criterion (CC / Availability / Processing Integrity / Confidentiality / Privacy) to the platform control that implements it (SSO, ACL, BYOK, audit, quotas, breakers, guard, retention, residency, …), the config that enables it, and the GET /v1/status/compliance posture key — so the mapping is checkable against the running config |
| gdpr-mapping.md | GDPR — article-by-article mapping (Art. 5 storage limitation → retention; Art. 17 erasure → /v1/compliance/erase; Art. 25 by-design → PII redaction at ingest; Art. 30 records → audit log; Art. 32 security → BYOK/ACL; Art. 44–50 transfers → data residency) + the erasure / residency flows + deferred caveats |
Design-partner pilot program (Step 7.4) — the operator runbook is guides/design-partner-pilots.md; this area holds the fill-in templates, per-vertical kits, and published case studies. Internal program material — kept on GitHub only, not published on the docs site (links below point at GitHub).
| File | Description |
|---|---|
| README.md | Pilots index — layout, the five templates, the per-vertical kit registry, the active/graduated-pilot table |
| templates/onboarding-checklist.md | Per-partner onboarding: tenant + governance + corpus + surfaces + measurement + sign-off (scripted by ragctl pilot onboard) |
| templates/success-criteria.md | Signed-before-kickoff criteria across quality / latency / integration / security, each with a threshold + the platform signal that proves it |
| templates/weekly-kpi.md | One-per-week KPI dashboard (volume / quality / satisfaction / drift / latency / cost / reliability) filled by ragctl pilot report |
| templates/feedback-log.md | The intake → triage → incorporate → close loop + the roadmap fold-back for the lessons-learned doc |
| templates/case-study.md | The referenceable customer story — challenge / solution / reference architecture / results (auditable KPI numbers) / quote |
| customer-support/README.md | Customer-support / internal-KB pilot kit (Step 7.4b) — sample corpus (PII handbook + product FAQ + a planted injection probe), domain-calibrated success criteria (deflection), the ragctl pilot seed-and-demo flow, the PII + injection security demonstration |
| customer-support/case-study.md | Worked case study (Step 7.4d) — the framework run end-to-end on the kit; real ragctl pilot report KPIs (satisfaction +0.733 · 0/5 drift · cost ok → PASS) + the PII + injection security demonstration |
Operational runbooks (Step 7.8) — one per alert the platform can raise, plus the incident-response process and the postmortem template.
| File | Description |
|---|---|
| README.md | Runbooks index — the alert → runbook table + the Step 7.8 acceptance checklist |
| alerts.md | One runbook per alert type (acl/tenant escape · breaker.opened · drift.detected · cost anomaly · quota.exceeded · injection spike · gateway degraded · latency/availability SLO · eval.regression · ingest failures), each tied to the platform event / /v1/status/* signal that raises it |
| incident-response.md | Severity ladder · IC/Comms/Ops roles · detect→declare→mitigate→communicate→resolve→postmortem flow · the status page (measured from /v1/status/health + Grafana SLOs) |
| postmortem-template.md | Blameless postmortem template (summary · impact · timeline · root cause · action items) |
| File | Description |
|---|---|
| v1.0.0.md | GA release notes (Step 7.10) — highlights across retrieval / surfaces / governance / reliability / observability / operability; one-command install per channel; the 8-phase journey; versioning + support |
| v1.0.1.md | v1.0.1 patch — fixes the container/Helm run path: the image's default CMD now serves the gateway via the config-driven rag_gateway.serve:create_app entrypoint (RAG_CONFIG_PATH), GET /readyz backs the chart's readiness probe, and the chart renders a valid rag.yaml (with a config.ragYaml passthrough) |
| File | Description |
|---|---|
| ga-readiness-checklist.md | The v1.0.0 GA readiness checklist (Step 7.10) — every Phase 7 exit gate mapped to a CI gate / in-repo artifact, with inherently-external items marked (external) |
| File | Description |
|---|---|
| enterprise-rag-problems.md | 22+ RAG problems in 7 categories, each rated 🔴 Critical / 🟠 High / 🟡 Medium |
| enterprise-rag-problems.pdf | PDF export of the problem catalog |
| rag-problems-world.html | Interactive HTML problem explorer |