From 00fc00d544e830f3aee5c4f2e2c7efaf75b9032b Mon Sep 17 00:00:00 2001 From: Julian Risch Date: Fri, 14 Aug 2026 20:11:15 +0200 Subject: [PATCH 1/3] docs: extend AGENTS.md with conventions mined from PR reviews Our AGENTS.md documents the mechanics -- hatch, tests, mypy, release notes -- but none of the judgement reviewers actually apply. That knowledge lived only in review comments, so agents rediscovered it one review round at a time. This adds 122 rules mined with pydantic/braindump from 2,746 review comments written by the deepset team between 2025-07-01 and today, clustered and deduplicated across PRs so that only repeatedly-enforced conventions survive. Because the corpus is mostly Haystack 2.x era, rules were filtered against the current tree before inclusion: a git diff of v2.31.0..HEAD identifies the 75 symbols removed in 3.0, and any rule whose text depends on one is dropped or rewritten to its 3.x equivalent. No rule naming a removed API survives except the handful that deliberately warn against it. Each marker traces back to its source review comments. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 305 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 305 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 81ad468be49..8910fe18f97 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,3 +57,308 @@ Every user-facing PR (not docs, not CI) must include a release note: hatch run release-note SHORT_DESCRIPTION Edit the generated file in `releasenotes/notes/`. Release notes use reStructuredText formatting; see the [release notes section in CONTRIBUTING.md](CONTRIBUTING.md#release-notes) for details. + + + +The rules below were mined from 2,746 PR review comments written by the deepset +team between 2025-07-01 and 2026-08-14, then filtered against the current source tree so that +guidance referring to APIs removed or moved in Haystack 3.0 does not survive. Each +`` marker traces back to the review comments it came from. + +They describe what reviewers actually enforce. Follow them the way you would follow a +reviewer's note: they encode reasons, not ceremony, so when a rule genuinely does not +fit the change at hand, say why rather than contorting the code to satisfy it. + +## API Design + + +- Let `Agent` own tool calls; don’t add pipeline invokers — `ToolInvoker` was removed in `3.0` + +- Use `Pipeline.run()` and `Pipeline.run_async()`; don’t add `AsyncPipeline` — merged in `3.0` + +- Use ChatGenerators like `OpenAIChatGenerator` — non-chat generators were removed in `3.0` + +- Use Haystack serialization helpers (`component_to_dict`, `default_from_dict`) — preserves wire compatibility + +- Make optional params keyword-only with `*` — preserves API compatibility + +- Build API clients and load models in `warm_up()`, not `__init__` — preserves constructibility + +- Treat `run_async` as optional in `haystack/components/` — fallback to `asyncio.to_thread(component.run, ...)` and log it + +- Implement `warm_up()` only for real setup; make it idempotent and run it before tool access — This avoids fake lifecycle APIs, duplicate guards, and premature tool access before required setup. + +- Keep `run_async`/`warm_up_async`/`close_async` paths async-only — use separate async state/hooks and only intentional non-blocking fallbacks like `asyncio.to_thread` + +- Pass explicit `required_variables` for optional prompt vars — avoids requiring optional inputs + +- Expose explicit `Agent` tool output contracts — use `raw=True` by default; document `last_message`, text-only output, and `exit_reason` when narrowing payloads + +- Subclass `Toolset` only when inherited collection APIs match — prefer composition or raise `NotImplementedError` + +- Read live Haystack `State` resources via `state.data.get(...)` or `state.data[...]` — avoids deep-copy bugs + +- Keep `run(messages, *, streaming_callback, generation_kwargs, tools)` order in `haystack/components/generators/chat/` — preserves API consistency + +- Append optional params in public signatures, including `__init__` — avoid breaking positional callers + +- Prefer existing `haystack/core/pipeline` APIs or inline logic — avoid duplicate or one-off public APIs + +- In `from_dict()`, pass only present fields to `__init__()` — keeps defaults centralized + +- Shallow-copy `from_dict` payloads; rebuild parsed nested fields — avoids costly `deepcopy()` + +- Spell out mirrored constructor params; avoid `*args`/`**kwargs` — clarifies supported config + +- Reuse splitter chunk metadata in `haystack/components/preprocessors/` — preserve traceability fields like `page_number`, `source_id`, `header`, `parent_headers`, and split positions; add new keys only for existing downstream contracts + +- Serialize tools/toolsets under `data` without flattening — preserves round-trip fidelity + +## Documentation + + +- Match markup to surface: use reStructuredText in `releasenotes/notes/`, Haystack docstring style in code — prevents broken docs rendering + +- Keep inline comments concise and non-obvious — remove restatements, but preserve durable caveats + +- Write concise, user-visible `releasenotes/notes/*.yaml` entries — clarify API, workflow, error, output, and semantic changes users rely on + +- Update docstrings with signature/type changes — remove obsolete behavior and reflect aliases like `ToolsType` + +- Document private helpers only when needed — explain non-obvious behavior, constraints, or rationale + +- Use only `releasenotes/config.yaml` sections in `releasenotes/notes/*.yaml` — keeps notes accurate and consistent + +- Document public API params and `:returns:` contracts — clarify usage, types, and behavior + +- Update `PromptBuilder`/`ChatPromptBuilder` docs with behavior changes — keep examples accurate, commented, and runnable + +- Keep `:param` docs concise and current — document meaning, defaults, and exceptions only + +- Use default constructors in docs examples; note required env vars like `OPENAI_API_KEY` nearby — Keeps docs focused on the demonstrated feature while still making required auth setup clear to readers. + +- Document raised exceptions with existing `:raises ...:` style — use `ValueError if filters have invalid syntax` for invalid filters + +- Keep doc examples minimal and local — include only used imports/code inside each snippet + +- Show example output as comments (`# ...`, `# >> ...`) — keeps snippets valid Python + +- Omit explicit defaults in docs examples — specify models only for model-specific behavior + +- Align class/API docstrings with existing style, especially in `haystack/components/` — keeps API docs readable + +- Remove `experimental` wording when promoting features — update docstrings, tutorials, `pydoc` IDs, and generated markdown filenames + +- Sync `docs-website/versioned_docs/version-*/concepts/data-classes.mdx` with dataclass/message fields — prevents stale API docs + +- Sync `haystack/components/` docs with actual behavior — prevent stale API, endpoint, integration, and example guidance + +- Document user-visible changes in `MIGRATION.md` — explain what changed, why, and required action + +## Code Style + + +- Use keyword args for multi-parameter calls — improves readability and prevents mix-ups + +- Delete lines explicitly marked to remove — resolves review feedback exactly and avoids dead code + +- Compute values once per function and reuse them — prevents drift and duplicate work + +- Keep lint suppressions current and exact — use honored codes like `# noqa: PLR0915` only while needed + +- Share sync/async logic in private helpers under `haystack/components/` — prevents drift and `# noqa: PLR0915` + +- Scope PR diffs to the stated goal — avoid unrelated refactors, formatting, serialization, or cleanup + +- Remove temporary `print()` debugging before merging — keeps tests and runtime output clean + +- Avoid filename-only headers in source files — keeps files clean and style-consistent + +- Omit args that only restate callee defaults — reduces noise and avoids stale examples + +- Revert review-rejected changes exactly — restore affected files or lines to the accepted state + +- Remove redundant branches and unused private helpers — document non-obvious compatibility or typing needs inline + +- Use `{placeholder}` logger templates with kwargs — preserves structured logs and avoids eager formatting + +- Keep private constants local and prefix module constants with `_` — clarifies API boundaries + +## Type System + + +- Use coded `# type: ignore[...]` with a nearby why-safe comment — avoids hiding unrelated typing bugs + +- Resolve callable annotations with `typing.get_type_hints()` — avoids bugs from raw or string `inspect.signature()` annotations + +- Keep union aliases non-redundant and concept-named — include base types only when subclasses inherit + +- Use `T | None`, not `Optional[T]`, in annotations and casts — keeps typing concise + +- Annotate class-or-PEP-604 inputs as `type | types.UnionType` — matches runtime values + +- Use `typing.Union[...]` instead of `|` unions until Python `3.10+` is required — Using `typing.Union[...]` keeps annotations compatible with Python versions before `3.10`. + +- Annotate helper params broadly and accurately; use `Any` for arbitrary typing objects — Accurate helper signatures improve static analysis without rejecting valid typing constructs at runtime. + +- Use `@overload` only for real API variants; fix bad types at source, not with broad `type: ignore` — Real overloads preserve API semantics, while source fixes and narrow ignores prevent hiding type bugs. + +## Imports + + +- Import via public APIs and keep only used or compatibility re-export imports — preserves API stability + +- Use top-level absolute `haystack...` imports in `haystack/components/` and `haystack/core/component/*.py` — keeps component dependencies traceable + +- Use `LazyImport` only for optional third-party deps — otherwise import directly + +- Use the narrowest clear import form, but keep module imports like `import httpx` when namespaces matter — This prevents namespace, runtime, typing, and lazy-import conflicts while keeping imports readable and minimal. + +## Config + + +- Pass `allowed_modules=` or set `HAYSTACK_DESERIALIZATION_ALLOWLIST` for YAML loads — never widen `haystack/core/serialization_security.py` allowlists in library code + +- Remove dead config in `.github/workflows/` — keeps CI behavior accurate and maintainable + +## Naming + + +- Prefix internal helpers with `_`, including `haystack/utils/` helpers — clarifies public API boundaries + +- Avoid naming locals after imported decorators/functions/utilities — prevents shadowing ambiguity + +## Testing + + +- Centralize `DocumentStore` tests in `haystack/testing/document_store.py` mixins — use `DocumentStoreBaseTests` as the minimum suite and compose capability mixins into `DocumentStoreBaseExtendedTests`; add explicit integration skips only for unsupported backend features. + +- Test `haystack/core/pipeline/` behavior changes in matching pipeline test modules — cover end-to-end edge cases like shorthands, early returns, errors, task completion, empty inputs, and outputs + +## General + + +- Define intentional package exports in `__init__.py` via `__all__`; avoid implementation-module `__all__` — Keeping package `__init__.py` exports intentional preserves stable public APIs and avoids accidental top-level imports. + +- Update `pyproject.toml` for new package APIs — declare deps and minimum versions used + +- State concrete deprecation timelines and status — helps users plan migrations + +## File-Specific Rules + +### `README.md` + +- Keep `README.md` feature lists complete and concrete — clarify broad terms with examples + +## Directory-specific conventions + +### `docs-website/` + + +- Sync `docs-website` API names and import paths with public exports — avoids stale docs + +- Omit explicit `.warm_up()` in docs unless required — lazy/idempotent warm-up handles it + +- Keep setup/usage for maintained APIs and integrations in `docs-website/docs/` — it is the authoritative, navigable source; add pages to `docs-website/sidebars.js` when needed + +- Keep `docs-website/docs/concepts/` current-facing — put history and upgrades in migration docs + +- Link data-class symbols to anchored API docs — improves discoverability and precision + +- Verify all `docs-website` MDX links — keeps external URLs current and internal routes valid + +### `docs-website/docs/` + + +- Keep `docs-website/docs/pipeline-components/generators/` examples concise and `Agent`-level — improves copyability and keeps docs focused + +- Edit `docs-website/docs/pipeline-components/agents-1/` only for outdated, incorrect, or materially useful user-facing API/behavior guidance — Keeps the agents docs focused and avoids churn while ensuring users get accurate, useful guidance. + +- Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details + +- Label `agents-1` docs sections clearly — mark examples/variants and customization paths + +- Document component outputs and link producers/API refs — keeps docs ecosystem-connected + +- Use `ChatPromptBuilder`, `ChatMessage`, and chat generators in new docs LLM pipelines — match wiring, edge names like `prompt`, and declared variables to real chat interfaces. + +- Polish `docs-website/docs/pipeline-components/**/*.mdx` prose before merge — keeps docs clear and consistent + +- Sync `docs-website/docs/pipeline-components` YAML examples with current defaults — stale model names cause config errors + +- Mark joiners/adapters optional when smart pipeline connections make them optional — Prevents docs from implying extra pipeline components are mandatory when smart connections already handle the composition. + +- Document extractor side effects and exact `doc.meta` keys — clarifies pipeline data flow + +- Prefer `result["last_message"]` for Haystack agent final responses — highlights the intended API + +- Link partial config/API summaries to authoritative references — helps users find full details + +### `haystack/hooks/compaction/` + + +- Use provider-compatible roles in `haystack/hooks/compaction/` — prefer `user` for synthetic markers + +- Name compaction retention by semantic unit (`turns`/`steps`), not `messages` — matches what is actually preserved + +- Document `haystack/hooks/compaction/` APIs by real semantics — prevents compaction misuse + +- Keep `haystack/hooks/compaction/` compactors narrative — move shared indexing, grouping, token counting, and helpers into focused utils + +### `releasenotes/notes/` + + +- Add `upgrade` notes for breaking/user-visible changes in `releasenotes/notes/` — explain affected users, old/new behavior, and migration steps + +- Add `releasenotes/notes/` entries only for in-scope user-facing PR changes — keeps release notes accurate and low-noise; leave unrelated note files untouched. + +- Name affected APIs/configs in `releasenotes/notes/` — clarifies scope and impact for users + +- Write bug/security notes around public impact, not private helpers — clarifies user risk + +- Highlight APIs in `releasenotes/notes/` only with examples or clear use cases — shows practical value + +- Keep `releasenotes/notes/` reno notes synced with shipped behavior — prevents misleading release docs + +- Use one `releasenotes/notes/` file per PR — group related change notes together + +- Proofread `releasenotes/notes/` entries — catches API typos and formatting issues + +### `test/` + + +- Name tests after verified behavior — keep names/docs current to avoid misleading coverage + +- Share `pytest` fixtures/helpers/constants only for truly common test setup — prevents brittle coupling and duplicate inputs + +- Keep tests minimal and non-redundant; prefer one smoke/contract test for brittle live paths like `test/components/generators/chat/`, `test/components/preprocessors/`, and `test/components/agents/test_agent.py` — reduces flaky, expensive, low-value coverage + +- Group tests by behavior in existing files/classes — use layouts like `test/components/test_{component}.py`, `test/components/agents/test_agent.py`, `test/core/pipeline/test_pipeline_base.py` (`TestPipelineBaseFromDict`), and `test/core/pipeline/` for breakpoints; split only when files get hard to navigate + +- Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts + +- Use real multi-chunk fixtures in `test/components/preprocessors/`; assert exact ordered content, metadata, and per-source `split_id`s. — Exact, ordered assertions catch splitter regressions in chunking, metadata propagation, and per-source `split_id` behavior that broad checks miss. + +- Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable + +- Add regression tests for pipeline socket metadata changes — protects `Variadic`/`GreedyVariadic` edge cases + +- Pair sync `run` integration tests with `run_async` tests — keep async behavior covered + +- Assert `"".join(doc.content for doc in split_docs) == text` in `DocumentSplitter` tests — catches content loss + +- Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — Keeps tests reliable in local and CI runs without requiring real OpenAI credentials unless explicitly testing the live integration. + +- Assert `RecursiveDocumentSplitter` overlap chunks and offsets — verify `split_idx_start`/`_split_overlap` as original-text character ranges, with parallel `split_unit` coverage for `word`/`token`. + +- Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs + +- Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions + +- Assert `Pipeline` fan-in order from runtime semantics — use joiners for custom order + +- Avoid `# type: ignore` in tests; narrow with `hasattr(...)` or `assert isinstance(...)` first — Explicit narrowing keeps tests type-safe and exposes real API mismatches instead of hiding bugs from `mypy`. + +- Keep test imports at module scope; remove redundant local imports — improves visibility and consistency + + From 0d8185a594b7a791bf4f150b3ccb7056da083a57 Mon Sep 17 00:00:00 2001 From: Julian Risch Date: Sat, 15 Aug 2026 09:41:16 +0200 Subject: [PATCH 2/3] docs: split AGENTS.md into nested files and slim the always-loaded set Review feedback on the single-file version: - Nested layout, as braindump generates it. Rules about tests, release notes and the docs site now live next to the code they govern, so agents load them only when work touches those directories. The root file drops 4,993 -> 2,789 tokens (-45%) with more relevant guidance loaded per session, not less. - CLAUDE.md now imports AGENTS.md via '@AGENTS.md'. Claude Code reads CLAUDE.md, not AGENTS.md, and the previous prose ('read the AGENTS.md file...') only worked if the model chose to act on it. The import inlines the content at session start. Each directory with an AGENTS.md gets the same one-line CLAUDE.md, since nested memory files are picked up lazily. - Dropped the inline markers, ~11% of every file for traceability nothing reads at runtime. Rules removed: - 'Use typing.Union[...] instead of | unions until Python 3.10+ is required'. This was wrong and contradicted the rule three lines above it: requires-python is already >=3.10, and 106 modules use PEP 604 unions against 12 using Optional[. The staleness gate could not catch it -- it diffs symbols, and this is a convention change with no symbol footprint. - 'Revert review-rejected changes exactly', 'Compute values once per function and reuse them', 'Delete lines explicitly marked to remove' -- generic developer hygiene carrying no project-specific information. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 202 ++-------------------------- CLAUDE.md | 2 +- docs-website/AGENTS.md | 12 ++ docs-website/CLAUDE.md | 3 + docs-website/docs/AGENTS.md | 18 +++ docs-website/docs/CLAUDE.md | 3 + haystack/hooks/compaction/AGENTS.md | 13 ++ haystack/hooks/compaction/CLAUDE.md | 3 + releasenotes/notes/AGENTS.md | 14 ++ releasenotes/notes/CLAUDE.md | 3 + test/AGENTS.md | 26 ++++ test/CLAUDE.md | 3 + 12 files changed, 109 insertions(+), 193 deletions(-) create mode 100644 docs-website/AGENTS.md create mode 100644 docs-website/CLAUDE.md create mode 100644 docs-website/docs/AGENTS.md create mode 100644 docs-website/docs/CLAUDE.md create mode 100644 haystack/hooks/compaction/AGENTS.md create mode 100644 haystack/hooks/compaction/CLAUDE.md create mode 100644 releasenotes/notes/AGENTS.md create mode 100644 releasenotes/notes/CLAUDE.md create mode 100644 test/AGENTS.md create mode 100644 test/CLAUDE.md diff --git a/AGENTS.md b/AGENTS.md index 8910fe18f97..3049509c408 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -62,303 +62,121 @@ Edit the generated file in `releasenotes/notes/`. Release notes use reStructured The rules below were mined from 2,746 PR review comments written by the deepset team between 2025-07-01 and 2026-08-14, then filtered against the current source tree so that -guidance referring to APIs removed or moved in Haystack 3.0 does not survive. Each -`` marker traces back to the review comments it came from. +guidance referring to APIs removed or moved in Haystack 3.0 does not survive. They describe what reviewers actually enforce. Follow them the way you would follow a reviewer's note: they encode reasons, not ceremony, so when a rule genuinely does not fit the change at hand, say why rather than contorting the code to satisfy it. +Also see directory-specific guidelines: + +- [docs-website/AGENTS.md](docs-website/AGENTS.md) +- [docs-website/docs/AGENTS.md](docs-website/docs/AGENTS.md) +- [haystack/hooks/compaction/AGENTS.md](haystack/hooks/compaction/AGENTS.md) +- [releasenotes/notes/AGENTS.md](releasenotes/notes/AGENTS.md) +- [test/AGENTS.md](test/AGENTS.md) + ## API Design - - Let `Agent` own tool calls; don’t add pipeline invokers — `ToolInvoker` was removed in `3.0` - - Use `Pipeline.run()` and `Pipeline.run_async()`; don’t add `AsyncPipeline` — merged in `3.0` - - Use ChatGenerators like `OpenAIChatGenerator` — non-chat generators were removed in `3.0` - - Use Haystack serialization helpers (`component_to_dict`, `default_from_dict`) — preserves wire compatibility - - Make optional params keyword-only with `*` — preserves API compatibility - - Build API clients and load models in `warm_up()`, not `__init__` — preserves constructibility - - Treat `run_async` as optional in `haystack/components/` — fallback to `asyncio.to_thread(component.run, ...)` and log it - - Implement `warm_up()` only for real setup; make it idempotent and run it before tool access — This avoids fake lifecycle APIs, duplicate guards, and premature tool access before required setup. - - Keep `run_async`/`warm_up_async`/`close_async` paths async-only — use separate async state/hooks and only intentional non-blocking fallbacks like `asyncio.to_thread` - - Pass explicit `required_variables` for optional prompt vars — avoids requiring optional inputs - - Expose explicit `Agent` tool output contracts — use `raw=True` by default; document `last_message`, text-only output, and `exit_reason` when narrowing payloads - - Subclass `Toolset` only when inherited collection APIs match — prefer composition or raise `NotImplementedError` - - Read live Haystack `State` resources via `state.data.get(...)` or `state.data[...]` — avoids deep-copy bugs - - Keep `run(messages, *, streaming_callback, generation_kwargs, tools)` order in `haystack/components/generators/chat/` — preserves API consistency - - Append optional params in public signatures, including `__init__` — avoid breaking positional callers - - Prefer existing `haystack/core/pipeline` APIs or inline logic — avoid duplicate or one-off public APIs - - In `from_dict()`, pass only present fields to `__init__()` — keeps defaults centralized - - Shallow-copy `from_dict` payloads; rebuild parsed nested fields — avoids costly `deepcopy()` - - Spell out mirrored constructor params; avoid `*args`/`**kwargs` — clarifies supported config - - Reuse splitter chunk metadata in `haystack/components/preprocessors/` — preserve traceability fields like `page_number`, `source_id`, `header`, `parent_headers`, and split positions; add new keys only for existing downstream contracts - - Serialize tools/toolsets under `data` without flattening — preserves round-trip fidelity ## Documentation - - Match markup to surface: use reStructuredText in `releasenotes/notes/`, Haystack docstring style in code — prevents broken docs rendering - - Keep inline comments concise and non-obvious — remove restatements, but preserve durable caveats - - Write concise, user-visible `releasenotes/notes/*.yaml` entries — clarify API, workflow, error, output, and semantic changes users rely on - - Update docstrings with signature/type changes — remove obsolete behavior and reflect aliases like `ToolsType` - - Document private helpers only when needed — explain non-obvious behavior, constraints, or rationale - - Use only `releasenotes/config.yaml` sections in `releasenotes/notes/*.yaml` — keeps notes accurate and consistent - - Document public API params and `:returns:` contracts — clarify usage, types, and behavior - - Update `PromptBuilder`/`ChatPromptBuilder` docs with behavior changes — keep examples accurate, commented, and runnable - - Keep `:param` docs concise and current — document meaning, defaults, and exceptions only - - Use default constructors in docs examples; note required env vars like `OPENAI_API_KEY` nearby — Keeps docs focused on the demonstrated feature while still making required auth setup clear to readers. - - Document raised exceptions with existing `:raises ...:` style — use `ValueError if filters have invalid syntax` for invalid filters - - Keep doc examples minimal and local — include only used imports/code inside each snippet - - Show example output as comments (`# ...`, `# >> ...`) — keeps snippets valid Python - - Omit explicit defaults in docs examples — specify models only for model-specific behavior - - Align class/API docstrings with existing style, especially in `haystack/components/` — keeps API docs readable - - Remove `experimental` wording when promoting features — update docstrings, tutorials, `pydoc` IDs, and generated markdown filenames - - Sync `docs-website/versioned_docs/version-*/concepts/data-classes.mdx` with dataclass/message fields — prevents stale API docs - - Sync `haystack/components/` docs with actual behavior — prevent stale API, endpoint, integration, and example guidance - - Document user-visible changes in `MIGRATION.md` — explain what changed, why, and required action ## Code Style - - Use keyword args for multi-parameter calls — improves readability and prevents mix-ups - -- Delete lines explicitly marked to remove — resolves review feedback exactly and avoids dead code - -- Compute values once per function and reuse them — prevents drift and duplicate work - - Keep lint suppressions current and exact — use honored codes like `# noqa: PLR0915` only while needed - - Share sync/async logic in private helpers under `haystack/components/` — prevents drift and `# noqa: PLR0915` - - Scope PR diffs to the stated goal — avoid unrelated refactors, formatting, serialization, or cleanup - - Remove temporary `print()` debugging before merging — keeps tests and runtime output clean - - Avoid filename-only headers in source files — keeps files clean and style-consistent - - Omit args that only restate callee defaults — reduces noise and avoids stale examples - -- Revert review-rejected changes exactly — restore affected files or lines to the accepted state - - Remove redundant branches and unused private helpers — document non-obvious compatibility or typing needs inline - - Use `{placeholder}` logger templates with kwargs — preserves structured logs and avoids eager formatting - - Keep private constants local and prefix module constants with `_` — clarifies API boundaries ## Type System - - Use coded `# type: ignore[...]` with a nearby why-safe comment — avoids hiding unrelated typing bugs - - Resolve callable annotations with `typing.get_type_hints()` — avoids bugs from raw or string `inspect.signature()` annotations - - Keep union aliases non-redundant and concept-named — include base types only when subclasses inherit - - Use `T | None`, not `Optional[T]`, in annotations and casts — keeps typing concise - - Annotate class-or-PEP-604 inputs as `type | types.UnionType` — matches runtime values - -- Use `typing.Union[...]` instead of `|` unions until Python `3.10+` is required — Using `typing.Union[...]` keeps annotations compatible with Python versions before `3.10`. - - Annotate helper params broadly and accurately; use `Any` for arbitrary typing objects — Accurate helper signatures improve static analysis without rejecting valid typing constructs at runtime. - - Use `@overload` only for real API variants; fix bad types at source, not with broad `type: ignore` — Real overloads preserve API semantics, while source fixes and narrow ignores prevent hiding type bugs. ## Imports - - Import via public APIs and keep only used or compatibility re-export imports — preserves API stability - - Use top-level absolute `haystack...` imports in `haystack/components/` and `haystack/core/component/*.py` — keeps component dependencies traceable - - Use `LazyImport` only for optional third-party deps — otherwise import directly - - Use the narrowest clear import form, but keep module imports like `import httpx` when namespaces matter — This prevents namespace, runtime, typing, and lazy-import conflicts while keeping imports readable and minimal. ## Config - - Pass `allowed_modules=` or set `HAYSTACK_DESERIALIZATION_ALLOWLIST` for YAML loads — never widen `haystack/core/serialization_security.py` allowlists in library code - - Remove dead config in `.github/workflows/` — keeps CI behavior accurate and maintainable ## Naming - - Prefix internal helpers with `_`, including `haystack/utils/` helpers — clarifies public API boundaries - - Avoid naming locals after imported decorators/functions/utilities — prevents shadowing ambiguity ## Testing - - Centralize `DocumentStore` tests in `haystack/testing/document_store.py` mixins — use `DocumentStoreBaseTests` as the minimum suite and compose capability mixins into `DocumentStoreBaseExtendedTests`; add explicit integration skips only for unsupported backend features. - - Test `haystack/core/pipeline/` behavior changes in matching pipeline test modules — cover end-to-end edge cases like shorthands, early returns, errors, task completion, empty inputs, and outputs ## General - - Define intentional package exports in `__init__.py` via `__all__`; avoid implementation-module `__all__` — Keeping package `__init__.py` exports intentional preserves stable public APIs and avoids accidental top-level imports. - - Update `pyproject.toml` for new package APIs — declare deps and minimum versions used - - State concrete deprecation timelines and status — helps users plan migrations ## File-Specific Rules ### `README.md` - -- Keep `README.md` feature lists complete and concrete — clarify broad terms with examples -## Directory-specific conventions - -### `docs-website/` - - -- Sync `docs-website` API names and import paths with public exports — avoids stale docs - -- Omit explicit `.warm_up()` in docs unless required — lazy/idempotent warm-up handles it - -- Keep setup/usage for maintained APIs and integrations in `docs-website/docs/` — it is the authoritative, navigable source; add pages to `docs-website/sidebars.js` when needed - -- Keep `docs-website/docs/concepts/` current-facing — put history and upgrades in migration docs - -- Link data-class symbols to anchored API docs — improves discoverability and precision - -- Verify all `docs-website` MDX links — keeps external URLs current and internal routes valid - -### `docs-website/docs/` - - -- Keep `docs-website/docs/pipeline-components/generators/` examples concise and `Agent`-level — improves copyability and keeps docs focused - -- Edit `docs-website/docs/pipeline-components/agents-1/` only for outdated, incorrect, or materially useful user-facing API/behavior guidance — Keeps the agents docs focused and avoids churn while ensuring users get accurate, useful guidance. - -- Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details - -- Label `agents-1` docs sections clearly — mark examples/variants and customization paths - -- Document component outputs and link producers/API refs — keeps docs ecosystem-connected - -- Use `ChatPromptBuilder`, `ChatMessage`, and chat generators in new docs LLM pipelines — match wiring, edge names like `prompt`, and declared variables to real chat interfaces. - -- Polish `docs-website/docs/pipeline-components/**/*.mdx` prose before merge — keeps docs clear and consistent - -- Sync `docs-website/docs/pipeline-components` YAML examples with current defaults — stale model names cause config errors - -- Mark joiners/adapters optional when smart pipeline connections make them optional — Prevents docs from implying extra pipeline components are mandatory when smart connections already handle the composition. - -- Document extractor side effects and exact `doc.meta` keys — clarifies pipeline data flow - -- Prefer `result["last_message"]` for Haystack agent final responses — highlights the intended API - -- Link partial config/API summaries to authoritative references — helps users find full details - -### `haystack/hooks/compaction/` - - -- Use provider-compatible roles in `haystack/hooks/compaction/` — prefer `user` for synthetic markers - -- Name compaction retention by semantic unit (`turns`/`steps`), not `messages` — matches what is actually preserved - -- Document `haystack/hooks/compaction/` APIs by real semantics — prevents compaction misuse - -- Keep `haystack/hooks/compaction/` compactors narrative — move shared indexing, grouping, token counting, and helpers into focused utils - -### `releasenotes/notes/` - - -- Add `upgrade` notes for breaking/user-visible changes in `releasenotes/notes/` — explain affected users, old/new behavior, and migration steps - -- Add `releasenotes/notes/` entries only for in-scope user-facing PR changes — keeps release notes accurate and low-noise; leave unrelated note files untouched. - -- Name affected APIs/configs in `releasenotes/notes/` — clarifies scope and impact for users - -- Write bug/security notes around public impact, not private helpers — clarifies user risk - -- Highlight APIs in `releasenotes/notes/` only with examples or clear use cases — shows practical value - -- Keep `releasenotes/notes/` reno notes synced with shipped behavior — prevents misleading release docs - -- Use one `releasenotes/notes/` file per PR — group related change notes together - -- Proofread `releasenotes/notes/` entries — catches API typos and formatting issues - -### `test/` - - -- Name tests after verified behavior — keep names/docs current to avoid misleading coverage - -- Share `pytest` fixtures/helpers/constants only for truly common test setup — prevents brittle coupling and duplicate inputs - -- Keep tests minimal and non-redundant; prefer one smoke/contract test for brittle live paths like `test/components/generators/chat/`, `test/components/preprocessors/`, and `test/components/agents/test_agent.py` — reduces flaky, expensive, low-value coverage - -- Group tests by behavior in existing files/classes — use layouts like `test/components/test_{component}.py`, `test/components/agents/test_agent.py`, `test/core/pipeline/test_pipeline_base.py` (`TestPipelineBaseFromDict`), and `test/core/pipeline/` for breakpoints; split only when files get hard to navigate - -- Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts - -- Use real multi-chunk fixtures in `test/components/preprocessors/`; assert exact ordered content, metadata, and per-source `split_id`s. — Exact, ordered assertions catch splitter regressions in chunking, metadata propagation, and per-source `split_id` behavior that broad checks miss. - -- Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable - -- Add regression tests for pipeline socket metadata changes — protects `Variadic`/`GreedyVariadic` edge cases - -- Pair sync `run` integration tests with `run_async` tests — keep async behavior covered - -- Assert `"".join(doc.content for doc in split_docs) == text` in `DocumentSplitter` tests — catches content loss - -- Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — Keeps tests reliable in local and CI runs without requiring real OpenAI credentials unless explicitly testing the live integration. - -- Assert `RecursiveDocumentSplitter` overlap chunks and offsets — verify `split_idx_start`/`_split_overlap` as original-text character ranges, with parallel `split_unit` coverage for `word`/`token`. - -- Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs - -- Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions - -- Assert `Pipeline` fan-in order from runtime semantics — use joiners for custom order - -- Avoid `# type: ignore` in tests; narrow with `hasattr(...)` or `assert isinstance(...)` first — Explicit narrowing keeps tests type-safe and exposes real API mismatches instead of hiding bugs from `mypy`. - -- Keep test imports at module scope; remove redundant local imports — improves visibility and consistency +- Keep `README.md` feature lists complete and concrete — clarify broad terms with examples diff --git a/CLAUDE.md b/CLAUDE.md index 8f8a6efba47..f6aa6c02628 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1,3 @@ # CLAUDE.md -Before you start working on this repository, read the AGENTS.md file and follow all the instructions. +@AGENTS.md diff --git a/docs-website/AGENTS.md b/docs-website/AGENTS.md new file mode 100644 index 00000000000..ab3575b7e7a --- /dev/null +++ b/docs-website/AGENTS.md @@ -0,0 +1,12 @@ + + +# docs-website/ Guidelines + +## Documentation + +- Sync `docs-website` API names and import paths with public exports — avoids stale docs +- Omit explicit `.warm_up()` in docs unless required — lazy/idempotent warm-up handles it +- Keep setup/usage for maintained APIs and integrations in `docs-website/docs/` — it is the authoritative, navigable source; add pages to `docs-website/sidebars.js` when needed +- Keep `docs-website/docs/concepts/` current-facing — put history and upgrades in migration docs +- Link data-class symbols to anchored API docs — improves discoverability and precision +- Verify all `docs-website` MDX links — keeps external URLs current and internal routes valid diff --git a/docs-website/CLAUDE.md b/docs-website/CLAUDE.md new file mode 100644 index 00000000000..f6aa6c02628 --- /dev/null +++ b/docs-website/CLAUDE.md @@ -0,0 +1,3 @@ +# CLAUDE.md + +@AGENTS.md diff --git a/docs-website/docs/AGENTS.md b/docs-website/docs/AGENTS.md new file mode 100644 index 00000000000..1eb1e1cbfbe --- /dev/null +++ b/docs-website/docs/AGENTS.md @@ -0,0 +1,18 @@ + + +# docs-website/docs/ Guidelines + +## Documentation + +- Keep `docs-website/docs/pipeline-components/generators/` examples concise and `Agent`-level — improves copyability and keeps docs focused +- Edit `docs-website/docs/pipeline-components/agents-1/` only for outdated, incorrect, or materially useful user-facing API/behavior guidance — Keeps the agents docs focused and avoids churn while ensuring users get accurate, useful guidance. +- Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details +- Label `agents-1` docs sections clearly — mark examples/variants and customization paths +- Document component outputs and link producers/API refs — keeps docs ecosystem-connected +- Use `ChatPromptBuilder`, `ChatMessage`, and chat generators in new docs LLM pipelines — match wiring, edge names like `prompt`, and declared variables to real chat interfaces. +- Polish `docs-website/docs/pipeline-components/**/*.mdx` prose before merge — keeps docs clear and consistent +- Sync `docs-website/docs/pipeline-components` YAML examples with current defaults — stale model names cause config errors +- Mark joiners/adapters optional when smart pipeline connections make them optional — Prevents docs from implying extra pipeline components are mandatory when smart connections already handle the composition. +- Document extractor side effects and exact `doc.meta` keys — clarifies pipeline data flow +- Prefer `result["last_message"]` for Haystack agent final responses — highlights the intended API +- Link partial config/API summaries to authoritative references — helps users find full details diff --git a/docs-website/docs/CLAUDE.md b/docs-website/docs/CLAUDE.md new file mode 100644 index 00000000000..f6aa6c02628 --- /dev/null +++ b/docs-website/docs/CLAUDE.md @@ -0,0 +1,3 @@ +# CLAUDE.md + +@AGENTS.md diff --git a/haystack/hooks/compaction/AGENTS.md b/haystack/hooks/compaction/AGENTS.md new file mode 100644 index 00000000000..cf1a8802763 --- /dev/null +++ b/haystack/hooks/compaction/AGENTS.md @@ -0,0 +1,13 @@ + + +# haystack/hooks/compaction/ Guidelines + +## API Design + +- Use provider-compatible roles in `haystack/hooks/compaction/` — prefer `user` for synthetic markers +- Name compaction retention by semantic unit (`turns`/`steps`), not `messages` — matches what is actually preserved + +## General + +- Document `haystack/hooks/compaction/` APIs by real semantics — prevents compaction misuse +- Keep `haystack/hooks/compaction/` compactors narrative — move shared indexing, grouping, token counting, and helpers into focused utils diff --git a/haystack/hooks/compaction/CLAUDE.md b/haystack/hooks/compaction/CLAUDE.md new file mode 100644 index 00000000000..f6aa6c02628 --- /dev/null +++ b/haystack/hooks/compaction/CLAUDE.md @@ -0,0 +1,3 @@ +# CLAUDE.md + +@AGENTS.md diff --git a/releasenotes/notes/AGENTS.md b/releasenotes/notes/AGENTS.md new file mode 100644 index 00000000000..c7f7c979c3b --- /dev/null +++ b/releasenotes/notes/AGENTS.md @@ -0,0 +1,14 @@ + + +# releasenotes/notes/ Guidelines + +## Documentation + +- Add `upgrade` notes for breaking/user-visible changes in `releasenotes/notes/` — explain affected users, old/new behavior, and migration steps +- Add `releasenotes/notes/` entries only for in-scope user-facing PR changes — keeps release notes accurate and low-noise; leave unrelated note files untouched. +- Name affected APIs/configs in `releasenotes/notes/` — clarifies scope and impact for users +- Write bug/security notes around public impact, not private helpers — clarifies user risk +- Highlight APIs in `releasenotes/notes/` only with examples or clear use cases — shows practical value +- Keep `releasenotes/notes/` reno notes synced with shipped behavior — prevents misleading release docs +- Use one `releasenotes/notes/` file per PR — group related change notes together +- Proofread `releasenotes/notes/` entries — catches API typos and formatting issues diff --git a/releasenotes/notes/CLAUDE.md b/releasenotes/notes/CLAUDE.md new file mode 100644 index 00000000000..f6aa6c02628 --- /dev/null +++ b/releasenotes/notes/CLAUDE.md @@ -0,0 +1,3 @@ +# CLAUDE.md + +@AGENTS.md diff --git a/test/AGENTS.md b/test/AGENTS.md new file mode 100644 index 00000000000..3b442c197e2 --- /dev/null +++ b/test/AGENTS.md @@ -0,0 +1,26 @@ + + +# test/ Guidelines + +## Testing + +- Name tests after verified behavior — keep names/docs current to avoid misleading coverage +- Share `pytest` fixtures/helpers/constants only for truly common test setup — prevents brittle coupling and duplicate inputs +- Keep tests minimal and non-redundant; prefer one smoke/contract test for brittle live paths like `test/components/generators/chat/`, `test/components/preprocessors/`, and `test/components/agents/test_agent.py` — reduces flaky, expensive, low-value coverage +- Group tests by behavior in existing files/classes — use layouts like `test/components/test_{component}.py`, `test/components/agents/test_agent.py`, `test/core/pipeline/test_pipeline_base.py` (`TestPipelineBaseFromDict`), and `test/core/pipeline/` for breakpoints; split only when files get hard to navigate +- Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts +- Use real multi-chunk fixtures in `test/components/preprocessors/`; assert exact ordered content, metadata, and per-source `split_id`s. — Exact, ordered assertions catch splitter regressions in chunking, metadata propagation, and per-source `split_id` behavior that broad checks miss. +- Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable +- Add regression tests for pipeline socket metadata changes — protects `Variadic`/`GreedyVariadic` edge cases +- Pair sync `run` integration tests with `run_async` tests — keep async behavior covered +- Assert `"".join(doc.content for doc in split_docs) == text` in `DocumentSplitter` tests — catches content loss +- Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — Keeps tests reliable in local and CI runs without requiring real OpenAI credentials unless explicitly testing the live integration. +- Assert `RecursiveDocumentSplitter` overlap chunks and offsets — verify `split_idx_start`/`_split_overlap` as original-text character ranges, with parallel `split_unit` coverage for `word`/`token`. +- Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs +- Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions +- Assert `Pipeline` fan-in order from runtime semantics — use joiners for custom order + +## General + +- Avoid `# type: ignore` in tests; narrow with `hasattr(...)` or `assert isinstance(...)` first — Explicit narrowing keeps tests type-safe and exposes real API mismatches instead of hiding bugs from `mypy`. +- Keep test imports at module scope; remove redundant local imports — improves visibility and consistency diff --git a/test/CLAUDE.md b/test/CLAUDE.md new file mode 100644 index 00000000000..f6aa6c02628 --- /dev/null +++ b/test/CLAUDE.md @@ -0,0 +1,3 @@ +# CLAUDE.md + +@AGENTS.md From 6b34d6898d1fc614b7481313c73237f05455292c Mon Sep 17 00:00:00 2001 From: Julian Risch Date: Sat, 15 Aug 2026 11:47:31 +0200 Subject: [PATCH 3/3] docs: drop the braindump marker comments and generalise instance-bound rules The comments existed so the mined block could be replaced in place on a re-run. That anchor now lives in the generator instead, so the shipped files carry no scaffolding. Clustering preserves whichever example the source review comments happened to discuss, which leaves some rules reading as if they only apply to one class or path. Generalised, with the specific case kept only where it is illustrative: - 'preserve all runtime config, including Watsonx max_retries, ...' -> every constructor argument that affects runtime behaviour must round-trip. - 'use WATSONX_API_KEY for Watsonx components' -> default each Secret from the provider's conventional env var. WATSONX_API_KEY is one of ~10 such variables in this repo (COHERE_API_KEY, NVIDIA_API_KEY, JINA_API_KEY, ...). - 'preserve Elasticsearch bulk write/delete try/except behavior' -> preserve documented bulk write/delete error behaviour. DocumentStoreError is used by 17 document stores, not just Elasticsearch. - 'update ... Google GenAI model names and RagasEvaluator ragas.metrics.collections usage' -> refresh docstrings, cookbooks and integration docs when model names or provider APIs change. - 'especially in integrations/mcp/src/haystack_integrations/tools/mcp/' -> dropped the path; the warm_up() rule is repo-wide. - 'Assert ... in DocumentSplitter tests' and the RecursiveDocumentSplitter overlap rule -> stated for splitters generally; core ships 11 splitter classes. - Test-layout and pipeline-components docs rules trimmed to the convention. Removed: 'Update integrations/amazon_bedrock/tests/ with generator changes', which is entirely about one integration, and 'Label agents-1 docs sections clearly', which is vague and pinned to a Docusaurus slug. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 4 ---- docs-website/docs/AGENTS.md | 3 +-- test/AGENTS.md | 6 +++--- 3 files changed, 4 insertions(+), 9 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3049509c408..60d5fc0929a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -58,8 +58,6 @@ hatch run release-note SHORT_DESCRIPTION Edit the generated file in `releasenotes/notes/`. Release notes use reStructuredText formatting; see the [release notes section in CONTRIBUTING.md](CONTRIBUTING.md#release-notes) for details. - - The rules below were mined from 2,746 PR review comments written by the deepset team between 2025-07-01 and 2026-08-14, then filtered against the current source tree so that guidance referring to APIs removed or moved in Haystack 3.0 does not survive. @@ -178,5 +176,3 @@ Also see directory-specific guidelines: ### `README.md` - Keep `README.md` feature lists complete and concrete — clarify broad terms with examples - - diff --git a/docs-website/docs/AGENTS.md b/docs-website/docs/AGENTS.md index 1eb1e1cbfbe..627bc05e333 100644 --- a/docs-website/docs/AGENTS.md +++ b/docs-website/docs/AGENTS.md @@ -5,9 +5,8 @@ ## Documentation - Keep `docs-website/docs/pipeline-components/generators/` examples concise and `Agent`-level — improves copyability and keeps docs focused -- Edit `docs-website/docs/pipeline-components/agents-1/` only for outdated, incorrect, or materially useful user-facing API/behavior guidance — Keeps the agents docs focused and avoids churn while ensuring users get accurate, useful guidance. +- Edit `docs-website/docs/pipeline-components/` pages only for outdated, incorrect, or materially useful user-facing guidance — avoids docs churn - Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details -- Label `agents-1` docs sections clearly — mark examples/variants and customization paths - Document component outputs and link producers/API refs — keeps docs ecosystem-connected - Use `ChatPromptBuilder`, `ChatMessage`, and chat generators in new docs LLM pipelines — match wiring, edge names like `prompt`, and declared variables to real chat interfaces. - Polish `docs-website/docs/pipeline-components/**/*.mdx` prose before merge — keeps docs clear and consistent diff --git a/test/AGENTS.md b/test/AGENTS.md index 3b442c197e2..e7af3bb51ec 100644 --- a/test/AGENTS.md +++ b/test/AGENTS.md @@ -7,15 +7,15 @@ - Name tests after verified behavior — keep names/docs current to avoid misleading coverage - Share `pytest` fixtures/helpers/constants only for truly common test setup — prevents brittle coupling and duplicate inputs - Keep tests minimal and non-redundant; prefer one smoke/contract test for brittle live paths like `test/components/generators/chat/`, `test/components/preprocessors/`, and `test/components/agents/test_agent.py` — reduces flaky, expensive, low-value coverage -- Group tests by behavior in existing files/classes — use layouts like `test/components/test_{component}.py`, `test/components/agents/test_agent.py`, `test/core/pipeline/test_pipeline_base.py` (`TestPipelineBaseFromDict`), and `test/core/pipeline/` for breakpoints; split only when files get hard to navigate +- Group tests by behavior in existing files and classes, mirroring the source layout (`test/components/test_{component}.py`) — split a file only when it becomes hard to navigate - Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts - Use real multi-chunk fixtures in `test/components/preprocessors/`; assert exact ordered content, metadata, and per-source `split_id`s. — Exact, ordered assertions catch splitter regressions in chunking, metadata propagation, and per-source `split_id` behavior that broad checks miss. - Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable - Add regression tests for pipeline socket metadata changes — protects `Variadic`/`GreedyVariadic` edge cases - Pair sync `run` integration tests with `run_async` tests — keep async behavior covered -- Assert `"".join(doc.content for doc in split_docs) == text` in `DocumentSplitter` tests — catches content loss +- In splitter tests, assert the joined split content equals the input text — catches dropped, duplicated, or reordered content - Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — Keeps tests reliable in local and CI runs without requiring real OpenAI credentials unless explicitly testing the live integration. -- Assert `RecursiveDocumentSplitter` overlap chunks and offsets — verify `split_idx_start`/`_split_overlap` as original-text character ranges, with parallel `split_unit` coverage for `word`/`token`. +- Assert splitter overlap offsets explicitly — verify `split_idx_start` and `_split_overlap` as character ranges into the original text, with coverage for each `split_unit` - Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs - Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions - Assert `Pipeline` fan-in order from runtime semantics — use joiners for custom order