Skip to content

packer.md

Latest commit

 

History

History
173 lines (134 loc) · 6.03 KB

packer.md

File metadata and controls

173 lines (134 loc) · 6.03 KB
Raw

Reference — rag-packer

Public API for the Step 2.8 context packer. Architectural notes live in architecture/context-packing.md.

Overview

The rag-packer package ships three pieces:

Symbol Module Purpose
ContextPacker rag_packer.pipeline Async orchestrator: dedup → reorder → budget fit → conflicts. Emits the canonical pack OTel span.
PackerConfig rag_packer.types Tunables (order, jaccard_threshold, detect_conflicts).
Pure primitives rag_packer.{dedup,reorder,budget,conflicts} hash_dedup, jaccard_dedup, lost_in_middle_order / descending_order / ascending_order, fit_to_budget, detect_conflicts.

The output type PackedContext and ConflictAnnotation live in rag-core (0.16+).

Usage

Minimal pipeline (defaults)

from rag_packer import ContextPacker

packer = ContextPacker()
packed = await packer.pack(ctx, reranked_chunks, budget=4000)
print(packed.tokens_used, "of", packed.tokens_budget)
for chunk in packed.chunks:
    ...

Default config:

  • order="lost_in_middle" (Liu et al. 2024)
  • jaccard_threshold=None (only exact SHA-256 dedup runs)
  • detect_conflicts=True (trust-mismatch + numeric-disagreement heuristics)
  • TiktokenCounter("cl100k_base") for token counting.

Aggressive dedup + chat-style ordering

from rag_packer import ContextPacker, PackerConfig

packer = ContextPacker(
    config=PackerConfig(
        order="ascending",            # most-relevant closest to the user turn
        jaccard_threshold=0.85,       # collapse near-duplicates too
        detect_conflicts=True,
    ),
)
packed = await packer.pack(ctx, reranked, budget=8000)

Custom tokenizer

from rag_chunker.tokens import TiktokenCounter

packer = ContextPacker(
    token_counter=TiktokenCounter("o200k_base"),   # GPT-4o family
)

ragctl pack — local smoke

$ ragctl pack "explain reranking" --top-k 3 --order lost_in_middle
query:        'explain reranking'
tenant:       ragctl-local
order:        lost_in_middle
budget:       4000 tokens
tokenizer:    tiktoken:cl100k_base
chunks_in:    5
chunks_out:   5
tokens_used:  47
dropped_dedup:  0
dropped_budget: 0
conflicts:    0
  #1  score=1.0000  trust=trusted        id=chunk-0  Cross-encoder rerankers score (query, document) pairs jointly.
  #2  score=1.0000  trust=trusted        id=chunk-2  MMR balances relevance against diversity in selection.
  #3  score=1.0000  trust=user_supplied  id=chunk-4  Context packing fits chunks into the LLM prompt window.

Flags:

--budget / -b         Token budget (default 4000).
--top-k / -k          How many chunks to print (default 5).
--order               lost_in_middle | descending | ascending.
--jaccard-threshold   Opt-in second-tier dedup (default 0.0 = OFF).
--conflicts / --no-conflicts
--tenant / -t

Internals

ContextPacker.pack flow

  1. Tenant safety. Every input Chunk.tenant_id must equal ctx.tenant_id; mixed-tenant input raises PackerError.
  2. Sort by descending score. Chunk.score from the reranker. None is treated as 0.0.
  3. Hash dedup (always-on). SHA-256 of normalised content (lowercase, strip punctuation, collapse whitespace). First occurrence wins.
  4. Jaccard dedup (opt-in). When jaccard_threshold is set, 5-shingle Jaccard over normalised text.
  5. Budget fit. Greedy: accumulate chunks in score order, skip any that would exceed the remaining budget but keep trying (a smaller lower-rank chunk may still fit). Uses Chunk.token_count when cached at ingest, otherwise calls the wired TokenCounter.
  6. Reorder. Apply the configured strategy (lost_in_middle_order, descending_order, or ascending_order).
  7. Conflicts. Heuristic scan; failures degrade to empty conflicts.

lost_in_middle_order

Per Liu et al. 2024 (Lost in the Middle). Highest-relevance at the start and end of the window, weakest in the middle:

[1, 2, 3, 4, 5] → [1, 3, 5, 4, 2]
[1, 2, 3, 4]    → [1, 3, 4, 2]

Even-index inputs fill the front in ascending order; odd-index inputs fill the back in descending order.

descending_order

Highest-relevance first (no-op on already-sorted input). Useful when the context window is small enough that lost-in-the-middle effects are negligible.

ascending_order

Lowest-relevance first → highest closest to the user message. Common in chat-style prompts.

Errors

PackerError (a RetrievalError subclass) raises on:

  • budget <= 0
  • Mixed-tenant input (defence-in-depth)
  • content_ref-only chunks lacking a cached token_count (ADR-0007 — packer is not in the storage business).

Conflict-detection internal failures DO NOT raise — they log packer.conflicts_failed and return conflicts=[]. Conflict detection is advisory.

Extension points

Hook What you provide Where
TokenCounter Custom tokenizer (any model family). ContextPacker(token_counter=...).
PackerConfig.order New OrderStrategy literal value. Subclass ContextPacker and extend _REORDER_FNS.
Custom dedup Pre-process the input list. Call hash_dedup / jaccard_dedup directly before packer.pack(...).
Custom conflict detectors Wrap the packer, post-process packed.conflicts. The orchestrator hard-wires the two built-in detectors; richer (LLM-based) detection is Step 5.x.

See also