Skip to content

chore(tracker): sync Step 1.2 → ✅ and log PR #55 #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
officialCodeWork merged 1 commit into from
May 24, 2026
Merged

chore(tracker): sync Step 1.2 → ✅ and log PR #55 #56

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions TRACKER.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@
| Phase | Title | Steps | ✅ Done | Remaining |
|-------|-------|------:|-------:|----------:|
| 0 | Foundation | 13 | **13** | 0 |
| 1 | Ingestion + Knowledge Store | 16 | **9** | 7 |
| 1 | Ingestion + Knowledge Store | 16 | **10** | 6 |
| 2 | Retrieval Engine | 11 | 0 | 11 |
| 3 | Gateway & Agent Runtime | 11 | 0 | 11 |
| 4 | Reliability | 6 | 0 | 6 |
| 5 | Eval & Observability | 7 | 0 | 7 |
| 6 | Governance & Tenancy | 10 | 0 | 10 |
| 7 | Pilot, Harden, GA | 10 | 0 | 10 |
| **Total** | | **84** | **22** | **62** |
| **Total** | | **84** | **23** | **61** |

---

Expand Down Expand Up @@ -74,7 +74,7 @@
| 1.1d | Pipeline + Batcher primitives | ✅ | `build/phase-1/step-1.1d-pipeline-batcher-primitives` | [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) | `Pipeline` primitive in `rag-core` (`rag_core.pipeline`): async DAG with bounded queues, per-stage worker counts, backpressure, joiner-based shutdown handling per-stage worker-count differences, `on_error="fail"`/`"skip"` policies, optional `error_handler`. `Batcher[Req, Resp]` middleware (`rag_core.batcher`, DataLoader pattern) coalescing concurrent `load(req)` calls into batched `batch_fn(reqs)` invocations; size + time triggers (`max_batch_size`, `max_wait_ms`); per-request `asyncio.Future` for isolated error propagation; `flush()` for deterministic shutdown. PEP-695-typed generics. 22 unit tests (8 Batcher + 14 Pipeline). `docs/architecture/pipeline-batcher.md` + `rag-core` reference + `performance.md` backlinks. `rag-core` 0.5.0 → 0.6.0. |
| 1.1e | Cache SPI split + perf discipline + async telemetry | ✅ | `build/phase-1/step-1.1e-cache-split-perf-telemetry` | [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | `EmbeddingCache` (`rag_core.spi.embedding_cache`, key tenant+model_id+model_version+text_hash, `invalidate_model`), `RetrievalCache` (`rag_core.spi.retrieval_cache`, key tenant+plan_hash+corpus_version, `invalidate_corpus`), `AnswerCache` (`rag_core.spi.answer_cache`, key tenant+plan_hash+corpus_version+policy_version, `invalidate_policy`). All ctx-first; signature linter extended. Noop in-memory impls + 18 conformance tests covering hit/miss/version-partition/invalidate/tenant-isolation. `AsyncTelemetrySink` in `rag-observability` — bounded buffer (default 10_000), non-blocking `submit()`, per-kind drop + exporter-error counters, idempotent start/stop, drain-timeout-cancel; 9 unit tests. Hot-path discipline doc in `performance.md` expanded with concrete hot/cold call-site table. New `docs/reference/rag-observability.md`. `caching.md` SPI shapes updated with ctx-first signatures. `rag-core` 0.6.0 → 0.7.0. |
| 1.1f | ADRs 0005–0009 + reviewer checklist | ✅ | `build/phase-1/step-1.1f-adrs-reviewer-checklist` | [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | ADR-0005 (PolicyEngine PDP), ADR-0006 (two-stage reranker default), ADR-0007 (tiered storage with BlobRef), ADR-0008 (cost-aware planner replacing reactive fallback), ADR-0009 (vector index strategy + quantization) all promoted to Accepted with implementation-path backlinks. Reviewer checklist in `docs/architecture/performance.md` extended with explicit items for plan/budget awareness, IndexHint+dtype handling, two-stage rerank shape, and BlobRef-safe consumers. ADR backlinks added to `rag_core.spi.reranker` and `rag_core.spi.storage` (the two missing module-level pointers). |
| 1.2 | Connectors framework | 🚧 | `build/phase-1/step-1.2-connectors-framework` | — | `Connector` SPI evolved to `(Document, ConnectorState)` async-iterator with resumable watermark (`ConnectorState` frozen model: connector_id, tenant_id, cursor, last_seen_at, extra). Internal `Crawler[Raw]` base class in `rag_backends.connectors._base` (iteration scaffold, error policy, monotonic watermark). Built-in connectors: `FilesystemConnector` (cross-platform pathlib, sha256 hashing, sorted resume cursor), `S3Connector` (aioboto3, ListObjectsV2 continuation token, MinIO-compatible), `GCSConnector` (gcloud-aio-storage, REST nextPageToken, `[gcs]` extra). Policy boundary documented — connectors are upstream of `PolicyEngine`; the ingest pipeline (Step 1.10) is the enforcement point. 9 contract tests (NoopConnector resume + watermark + tenant mismatch), 9 unit tests (FilesystemConnector), MinIO + GCS integration tests skip-if-no-service. `ConnectorState` schema added to `dist/schemas/`. New docs: [reference/connectors.md](docs/reference/connectors.md), [architecture/connectors.md](docs/architecture/connectors.md). |
| 1.2 | Connectors framework | ✅ | `build/phase-1/step-1.2-connectors-framework` | [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | `Connector` SPI evolved to `(Document, ConnectorState)` async-iterator with resumable watermark (`ConnectorState` frozen model: connector_id, tenant_id, cursor, last_seen_at, extra). Internal `Crawler[Raw]` base class in `rag_backends.connectors._base` (iteration scaffold, error policy, monotonic watermark). Built-in connectors: `FilesystemConnector` (cross-platform pathlib, sha256 hashing, sorted resume cursor), `S3Connector` (aioboto3, ListObjectsV2 continuation token, MinIO-compatible), `GCSConnector` (gcloud-aio-storage, REST nextPageToken, `[gcs]` extra). Policy boundary documented — connectors are upstream of `PolicyEngine`; the ingest pipeline (Step 1.10) is the enforcement point. 9 contract tests (NoopConnector resume + watermark + tenant mismatch), 9 unit tests (FilesystemConnector), MinIO + GCS integration tests skip-if-no-service. `ConnectorState` schema added to `dist/schemas/`. New docs: [reference/connectors.md](docs/reference/connectors.md), [architecture/connectors.md](docs/architecture/connectors.md). |
| 1.3 | Document parsers | ⏳ | — | — | PDF, DOCX, PPTX, XLSX, HTML, Markdown, plain text, JSON, CSV, YAML parsers; MIME detection |
| 1.4 | OCR pipeline | ⏳ | — | — | Tesseract + PaddleOCR plugins; image region extraction; confidence scoring |
| 1.5 | Structure-aware chunker | ⏳ | — | — | Heading-based chunking, parent-child `Chunk.parent_id` links, sentence-boundary, overlap, size normalization |
Expand Down Expand Up @@ -222,6 +222,7 @@
| [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | feat(core,observability): Cache SPI split + async telemetry sink (Step 1.1e) | `build/phase-1/step-1.1e-cache-split-perf-telemetry` | ✅ Merged | 2026-05-24 |
| [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | docs(adr,architecture): finalize ADRs 0005–0009 + extend reviewer checklist (Step 1.1f) | `build/phase-1/step-1.1f-adrs-reviewer-checklist` | ✅ Merged | 2026-05-24 |
| [#53](https://github.com/officialCodeWork/AgentContextOS/pull/53) | chore(tracker): sync Step 1.1f → ✅ and log PR #52 | `chore/tracker-sync-pr52` | ✅ Merged | 2026-05-24 |
| [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | feat(core,backends): connectors framework — SPI evolution + filesystem/S3/GCS (Step 1.2) | `build/phase-1/step-1.2-connectors-framework` | ✅ Merged | 2026-05-24 |

---

Expand Down
Loading