officialCodeWork · officialCodeWork · May 25, 2026 · May 25, 2026
diff --git a/TRACKER.md b/TRACKER.md
@@ -10,7 +10,7 @@
 
 **Last updated:** 2026-05-25
 **Current phase:** Phase 1 — Ingestion + Knowledge Store
-**Next action:** Phase 1 Step 1.6 — Metadata enricher: auto-tag language, doc type, created/modified dates, author, section path, reading level. Operates on `Chunk` output from Step 1.5 before PII detection (Step 1.7) and embedding (Step 1.8).
+**Next action:** Phase 1 Step 1.7 — PII detection: Presidio integration, per-tenant policy (block/redact/allow), `pii.detected` event, CI gate. Operates on enriched chunks (Step 1.6) before embedding (Step 1.8).
 
 > **Refactor window (Steps 1.1a–1.1f):** Before resuming the connectors framework (1.2), we insert a six-step refactor that locks in architecture + optimization decisions which are very expensive to retrofit later (PolicyEngine PDP, RequestContext-threaded SPIs, split Retrieval/Index backends, bulk + streaming + ID-only methods, Pipeline + Batcher primitives, three-way cache split, hot-path discipline). See [docs/adr/ADR-0005…0009] and [docs/architecture/policy-engine.md], [request-context.md], [caching.md], [performance.md].
 
@@ -32,14 +32,14 @@
 | Phase | Title | Steps | ✅ Done | Remaining |
 |-------|-------|------:|-------:|----------:|
 | 0 | Foundation | 13 | **13** | 0 |
-| 1 | Ingestion + Knowledge Store | 16 | **11** | 5 |
+| 1 | Ingestion + Knowledge Store | 16 | **12** | 4 |
 | 2 | Retrieval Engine | 11 | 0 | 11 |
 | 3 | Gateway & Agent Runtime | 11 | 0 | 11 |
 | 4 | Reliability | 6 | 0 | 6 |
 | 5 | Eval & Observability | 7 | 0 | 7 |
 | 6 | Governance & Tenancy | 10 | 0 | 10 |
 | 7 | Pilot, Harden, GA | 10 | 0 | 10 |
-| **Total** | | **84** | **24** | **60** |
+| **Total** | | **84** | **25** | **59** |
 
 ---
 
@@ -78,7 +78,7 @@
 | 1.3 | Document parsers | ✅ | `build/phase-1/step-1.3-document-parsers` | [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) | New `rag-parsers` package (`packages/parsers/`, v0.1.0) shipping `PlainTextParser`, `MarkdownParser` (markdown-it-py), `HtmlParser` (BeautifulSoup + stdlib `html.parser`), `JsonParser`, `CsvParser`, `YamlParser` (PyYAML `safe_load`), `PdfParser` (pypdf), `DocxParser` (python-docx), `PptxParser` (python-pptx), `XlsxParser` (openpyxl). `Parser` SPI evolved: `parse(...)` now returns `ParsedDocument` (`Document` + extracted text + structural `Block` list with offsets + heading levels + parser-specific `attributes`). New types in `rag_core`: `Block`, `BlockType` (8-value enum), `ParsedDocument`. `rag_parsers.mime.detect_mime()` (puremagic + OOXML ZIP central-directory disambiguation + extension shortcut + hint fallback). `ParserRegistry` + `default_registry()` (first-match-wins; specific MIMEs before generics). `ragctl parse <path>` smoke-test command (MIME detect → parser select → print summary + first-N blocks). 35 parser unit tests + 6 ragctl tests + extended contract tests (148 contract tests total pass). Schemas regenerated (`Block.json`, `ParsedDocument.json`). `rag-core` 0.7.0 → 0.8.0. Cross-platform: every parser library is pure-Python or wheel-distributed; no `libmagic` / `lxml` system deps. New docs: [reference/parsers.md](docs/reference/parsers.md), [architecture/parsers.md](docs/architecture/parsers.md). |
 | 1.4 | OCR pipeline | ✅ | `build/phase-1/step-1.4-ocr-pipeline` | [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | New `rag-ocr` package (`packages/ocr/`, v0.1.0): `TesseractOCR` (per-word regions, ISO-639-3 lang codes, `pytesseract` wrapper) + `PaddleOCRBackend` (per-line regions, quadrilateral → axis-aligned bbox, lazy engine init); both behind optional extras `[tesseract]`/`[paddle]` so default install stays pure-Python. OCR SPI evolved: `BoundingBox(x0,y0,x1,y1)` + `OCRRegion(text,confidence,bbox,page_number)` + `OCRResult(text,confidence,regions,page_number,engine)` Pydantic v2 frozen models in `rag_core.types`; aggregate confidence is text-length-weighted mean (1.0 on empty regions so "no text" doesn't read as failure); `NoopOCR` migrated; 5 contract tests updated. `ragctl ocr <path>` smoke command (`--engine tesseract\|paddleocr`, `--lang`, `--regions N`) — 5 CLI tests. 22 unit tests under `tests/ocr/` stub both engines at the `sys.modules` boundary so CI runs without Tesseract binary or PaddlePaddle wheels. Schemas regenerated (`BoundingBox.json`, `OCRRegion.json`, `OCRResult.json`). `rag-core` 0.8.0 → 0.9.0. New docs: [reference/ocr.md](docs/reference/ocr.md), [architecture/ocr.md](docs/architecture/ocr.md). |
 | 1.5 | Structure-aware chunker | ✅ | `build/phase-1/step-1.5-structure-aware-chunker` | [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | New `rag-chunker` package (`packages/chunker/`, v0.1.0): `HeadingAwareChunker` walks `ParsedDocument.blocks`, maintains a heading-level stack for `parent_id` semantics (h2 under h1 → parent=h1; new h2 sibling reparents under the same h1), accumulates body blocks under each heading up to a token budget (default 512 tokens, 64 overlap), splits oversized blocks on sentence boundaries (pure-Python regex), hard-splits sentences that still exceed the budget. Within-section overlap only; never bleeds across heading boundaries. Chunker SPI added: `rag_core.spi.chunker.Chunker` ABC (`async chunk(ctx, parsed) -> list[Chunk]`); `NoopChunker` one-chunk-per-block fallback; 7 contract tests + signature linter extension enforcing monotonic `position`, parent-refs-earlier-only, tenant/document propagation. `TokenCounter` Protocol + `TiktokenCounter` (default `cl100k_base`, swap encodings via constructor); tiktoken pulled in as default dep with wheels for all OS+arch combos. `ocr_result_to_parsed_document()` adapter turns Step 1.4 `OCRResult` into a synthetic `ParsedDocument` so OCR'd image pages flow through the same chunker (each `OCRRegion` → paragraph `Block` with `engine`/`ocr_confidence`/`bbox`/`page` in attributes). `ragctl chunk <path>` smoke command (`--max-tokens`, `--overlap-tokens`, `--mime`, `--chunks N`) — 5 CLI tests. 33 unit tests under `tests/chunker/` (sentence splitter + tokenizer + heading-aware + OCR adapter). `rag-core` 0.9.0 → 0.10.0. Cross-platform: pure-Python except `tiktoken` extension; unit tests use a `_WordCounter` stand-in so token-boundary assertions don't depend on BPE specifics. New docs: [reference/chunker.md](docs/reference/chunker.md), [architecture/chunker.md](docs/architecture/chunker.md). |
-| 1.6 | Metadata enricher | ⏳ | — | — | Auto-tag: language, doc type, created/modified dates, author, section path, reading level |
+| 1.6 | Metadata enricher | ✅ | `build/phase-1/step-1.6-metadata-enricher` | [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | New `rag-enricher` package (`packages/enricher/`, v0.1.0): `DefaultEnricher` tags each chunk's metadata with `language` (langdetect, deterministic seed; skipped < 20 chars), `doc_type` (from `Document.mime_type` via `doc_type_from_mime()` — markdown/pdf/docx/pptx/xlsx/json/csv/yaml/html/text/ocr or first-of-mime fallback), `created_at`/`modified_at` (parser-extracted dates from `Document.metadata` override `Document.created_at`/`updated_at`), `author` and `title` when present, `source_uri`, `document_id`, `section_path` (root-first heading-text list from walking `parent_id` chain via the new `section_path()` helper; O(d) via per-doc lookup table; cycle-guarded), and `reading_level` (Flesch-Kincaid grade via textstat — gated on body chunks + English + ≥80 chars so we skip rather than emit a misleading "grade 0"). Enricher SPI added: `rag_core.spi.enricher.Enricher` ABC (`async enrich(ctx, chunks, parsed) -> list[Chunk]`); `NoopEnricher` passthrough; 4 contract tests + signature linter extension enforce length/order/identifying-fields preserved (only `metadata` differs). `LanguageDetector` Protocol + `LangdetectDetector` default (seeded `DetectorFactory.seed=0` so cache keys derived from chunk metadata are stable across runs); swappable via constructor. `ragctl enrich <path>` smoke command pipes parse → chunk → enrich and prints tag-coverage summary + first-N enriched chunks with language / reading level / section path — 4 CLI tests. 30 unit tests under `tests/enricher/` (doc_type, language, section_path, default_enricher integration). `rag-core` 0.10.0 → 0.11.0. Cross-platform: langdetect + textstat + pyphen all ship pure-Python wheels — no system binaries, no native extensions. New docs: [reference/enricher.md](docs/reference/enricher.md), [architecture/enricher.md](docs/architecture/enricher.md). |
 | 1.7 | PII detection | ⏳ | — | — | Presidio integration; per-tenant policy (block/redact/allow); `pii.detected` event; CI gate |
 | 1.8 | Embedder pipeline | ⏳ | — | — | OpenAI, Cohere, bge-large, E5 plugins; batching; rate-limit back-off; dimension normalization |
 | 1.9 | CDC connectors (incremental sync) | ⏳ | — | — | Postgres CDC (pgoutput), S3 event notifications, webhook receiver; `Document.fingerprint` dedup |
@@ -230,6 +230,8 @@
 | [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | feat(core,ocr): OCR pipeline — region-aware SPI + Tesseract + PaddleOCR (Step 1.4) | `build/phase-1/step-1.4-ocr-pipeline` | ✅ Merged | 2026-05-25 |
 | [#61](https://github.com/officialCodeWork/AgentContextOS/pull/61) | chore(tracker): sync Step 1.4 → ✅ and log PRs #58, #59, #60 | `chore/tracker-sync-pr60` | ✅ Merged | 2026-05-25 |
 | [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | feat(core,chunker): structure-aware chunker (Step 1.5) | `build/phase-1/step-1.5-structure-aware-chunker` | ✅ Merged | 2026-05-25 |
+| [#63](https://github.com/officialCodeWork/AgentContextOS/pull/63) | chore(tracker): sync Step 1.5 → ✅ and log PRs #61, #62 | `chore/tracker-sync-pr62` | ✅ Merged | 2026-05-25 |
+| [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | feat(core,enricher): metadata enricher (Step 1.6) | `build/phase-1/step-1.6-metadata-enricher` | ✅ Merged | 2026-05-25 |
 
 ---