Skip to content

fallback.md

Latest commit

 

History

History
136 lines (103 loc) · 5.24 KB

fallback.md

File metadata and controls

136 lines (103 loc) · 5.24 KB
Raw

rag_retrieval.fallback reference (Step 4.2)

Public API for the retrieval fallback chain. See architecture/fallback-chain.md for the design rationale.

Exports

Symbol Module Kind
FallbackChain rag_retrieval.fallback orchestrator
FallbackConfig rag_retrieval.fallback frozen dataclass
SupportsRoute rag_retrieval.router runtime-checkable Protocol
FallbackTier rag_core.types StrEnum
FallbackTrigger rag_core.types StrEnum
FallbackResult rag_core.types frozen Pydantic model

FallbackChain and FallbackConfig are re-exported from rag_retrieval.

Types

FallbackTier

hybrid · bm25_only · keyword · no_answer — the rungs of the ladder, richest first. no_answer is the terminal rung.

FallbackTrigger

none · budget · error · empty — why the chain first left the starting tier (none when it never did).

FallbackResult

Frozen decision record returned by FallbackChain.run (the fused ChunkRef list is returned alongside it, not embedded).

Field Type Meaning
served_tier FallbackTier tier that produced the returned refs (no_answer if exhausted)
start_tier FallbackTier tier the chain started at (below hybrid ⇒ budget-capped)
engaged bool True iff fallback did anything beyond a first-try hybrid serve
trigger FallbackTrigger first reason the chain left the starting tier
attempts tuple[FallbackTier, ...] tiers attempted, in order
results_n int number of refs returned
budget_capped bool the budget forced a start below the top tier
reason str advisory free-text — never parse in production

FallbackConfig

Frozen dataclass; all fields optional.

Field Default Meaning
min_results 1 usefulness bar; fewer refs ⇒ treated as empty
advance_on_empty True advance on an empty / below-bar result
relax_filters_on_keyword True drop caller metadata filters on the keyword rung
no_answer_on_exhaustion True graceful empty (200) vs. re-raise the last error (502) on error-exhaustion
tiers (hybrid, bm25_only, keyword) the ladder; canonically-ordered, duplicate-free, no no_answer
cost_estimator None CostEstimator for budget gating (None ⇒ DefaultCostEstimator)

Construction validates min_results >= 1 and the tiers ordering/uniqueness.

FallbackChain

FallbackChain(*, router: RetrievalRouter, config: FallbackConfig | None = None)

run(...) -> tuple[FallbackResult, list[ChunkRef]]

async def run(
    self,
    ctx: RequestContext,
    *,
    text: str,
    expansion_terms: dict[str, list[str]] | None = None,
    hyde_vector: list[float] | None = None,
    vector: list[float] | None = None,
    graph_seeds: list[str] | None = None,
    keyword_query: str | None = None,
    corpus_ids: Sequence[CorpusId] | None = None,
    top_k: int = 10,
    filters: FilterExpr | None = None,
    graph_hops: int = 1,
) -> tuple[FallbackResult, list[ChunkRef]]: ...

route(...) -> tuple[RoutingDecision, list[ChunkRef]]

Same signature as RetrievalRouter.route (satisfies SupportsRoute). Returns the effective backend decision for the served tier (the router's own decision for hybrid, a synthetic keyword-only decision for the degraded tiers, or a "nothing selected" decision with reason="fallback:no_answer"). Use this for drop-in wrapping; use run when you want the tier/trigger detail.

router property

The wrapped RetrievalRouter (used by tests and the smoke CLI).

ragctl fallback

ragctl fallback QUERY [--tenant T] [--corpus C] [--top-k K]
                      [--simulate-failure {vector,keyword,graph}]...
                      [--simulate-empty {vector,keyword,graph}]...
                      [--budget-tokens N] [--budget-wall-ms N]
                      [--min-results N] [--no-keyword-relax] [--strict]

Drives the chain over the noop SPIs and prints the FallbackResult (start/served tier, trigger, attempts, engaged) plus the fused chunks. No infrastructure or credentials required.

Observability

  • Span retrieve.fallback — attributes rag.fallback.{start_tier,served_tier,engaged,trigger,attempts_n,attempts,results_n,budget_capped,elapsed_ms} plus rag.tenant_id. Per-tier retrieve.route spans nest inside.
  • Event fallback.engaged — emitted only when fallback engages; fields event_kind,tenant_id,start_tier,served_tier,trigger,attempts,results_n,budget_capped,elapsed_ms. Never carries raw query text.

rag.yaml

Under retrieval.fallback (rag_config.FallbackConfig): enabled, min_results, advance_on_empty, relax_filters_on_keyword, no_answer_on_exhaustion. The gateway maps these onto the runtime rag_retrieval.FallbackConfig; enabled: false skips the wrap entirely.

Extension points

  • Inject a custom CostEstimator to change budget gating.
  • Shorten the ladder via FallbackConfig.tiers.
  • Add a new FallbackTier + an _attempt branch + a _tier_cost entry for a new rung (e.g. cached-answer or web-search).