Skip to content

ocr.md

Latest commit

 

History

History
112 lines (88 loc) · 4.49 KB

ocr.md

File metadata and controls

112 lines (88 loc) · 4.49 KB
Raw

rag-ocr — reference

OCR backends for AgentContextOS. Step 1.4 ships two engine plugins — Tesseract and PaddleOCR — that implement the rag_core.spi.OCR interface and return region-aware results with axis-aligned bounding boxes, per-region confidence scores, and a length-weighted aggregate confidence.

Overview

An OCR backend consumes raw image bytes (PNG / JPEG / TIFF / BMP / GIF / WebP) and returns an OCRResult —

  • text — the concatenation of region texts in reading order (newline for line-granularity engines, space for word-granularity engines)
  • confidence — text-length-weighted mean of region confidences, normalised to 0.0–1.0; 1.0 when no regions are detected (so an image with no text reads as "fully confident in no text", not "failure")
  • regions — per-detection OCRRegion list with text, confidence, bbox: BoundingBox (axis-aligned (x0, y0, x1, y1) in pixels), and an optional page_number
  • engine — "tesseract", "paddleocr", or "noop" for trace correlation

Downstream consumers (the chunker in Step 1.5, the ingest pipeline in Step 1.10) treat region granularity as informational metadata — the OCR SPI doesn't promise word-level vs. line-level detection.

Supported engines

Plugin Engine Granularity Extras System dependency
TesseractOCR Tesseract via pytesseract per-word pip install "rag-ocr[tesseract]" tesseract binary on PATH
PaddleOCRBackend PaddleOCR per-line pip install "rag-ocr[paddle]" none (pulls paddlepaddle wheel)
NoopOCR in-process none included in rag-core none

Neither real-engine extra is installed by default — pick the engine you need. rag-ocr itself only depends on rag-core and Pillow; the heavy ML stacks live behind the extras.

Usage

from rag_core.types import (
    RequestContext, TenantId, Principal, PrincipalId, PrincipalKind,
)
from rag_ocr import TesseractOCR

ocr = TesseractOCR(lang="eng")
ctx = RequestContext(
    tenant_id=TenantId("acme"),
    principal=Principal(
        id=PrincipalId("svc-ingest"),
        kind=PrincipalKind.service,
        display_name="ingest",
        tenant_id=TenantId("acme"),
    ),
)
result = await ocr.extract(ctx, image_bytes, "image/png")
print(result.text, result.confidence)
for region in result.regions:
    box = region.bbox
    print(f"  ({box.x0},{box.y0})–({box.x1},{box.y1})  "
          f"conf={region.confidence:.2f}  {region.text}")

Smoke-test from the CLI:

ragctl ocr ./scan.png                       # Tesseract (default)
ragctl ocr ./scan.png --engine paddleocr    # PaddleOCR
ragctl ocr ./scan.png --lang eng -n 10      # show first 10 regions

ragctl ocr returns exit code 0 on success, 1 for unknown engine / missing file, 2 when the engine raises IngestionError (binary not on PATH, engine failure, malformed image).

Errors

All OCR failures surface as rag_core.errors.IngestionError:

  • Empty image bytes
  • Pillow cannot decode the image (corrupt / unsupported format)
  • Tesseract binary not found on PATH
  • Engine-internal errors (TesseractError, PaddleOCR runtime errors)
  • Missing optional extra (clear install hint in the message)

Internals

OCRResult, OCRRegion, and BoundingBox live in rag_core.types and are exported in dist/schemas/. They're Pydantic v2 frozen models — create new instances rather than mutating.

BoundingBox validates x1 >= x0 and y1 >= y0 on construction. PaddleOCR's rotated quadrilaterals are normalised to the enclosing axis-aligned box; the original quad is dropped (see docs/architecture/ocr.md for rationale and the rotation-metadata backlog item).

Extension points

To plug in a new OCR engine (AWS Textract, Google Cloud Vision, …):

  1. Subclass rag_core.spi.ocr.OCR.
  2. Implement extract(ctx, image_bytes, mime_type) -> OCRResult and health().
  3. Return OCRRegion instances with axis-aligned BoundingBox values in image pixel space, confidences normalised to 0.0–1.0, and an engine tag that identifies the backend for trace correlation.
  4. Map engine-specific exceptions to rag_core.errors.IngestionError.

The contract tests in tests/contract/test_ocr.py cover the shape; new backends should add a unit-test file under tests/ocr/ with the engine stubbed (see test_tesseract.py / test_paddle.py for the pattern).