docs: extend AGENTS.md with conventions mined from PR reviews

AGENTS.md

@@ -57,3 +57,122 @@ Every user-facing PR (not docs, not CI) must include a release note:
 hatch run release-note SHORT_DESCRIPTION
 
 Edit the generated file in `releasenotes/notes/`. Release notes use reStructuredText formatting; see the [release notes section in CONTRIBUTING.md](CONTRIBUTING.md#release-notes) for details.
+
+The rules below were mined from 2,746 PR review comments written by the deepset
+team between 2025-07-01 and 2026-08-14, then filtered against the current source tree so that
+guidance referring to APIs removed or moved in Haystack 3.0 does not survive.
+
+They describe what reviewers actually enforce. Follow them the way you would follow a
+reviewer's note: they encode reasons, not ceremony, so when a rule genuinely does not
+fit the change at hand, say why rather than contorting the code to satisfy it.
+
+Also see directory-specific guidelines:
+
+- [docs-website/AGENTS.md](docs-website/AGENTS.md)
+- [docs-website/docs/AGENTS.md](docs-website/docs/AGENTS.md)
+- [haystack/hooks/compaction/AGENTS.md](haystack/hooks/compaction/AGENTS.md)
+- [releasenotes/notes/AGENTS.md](releasenotes/notes/AGENTS.md)
+- [test/AGENTS.md](test/AGENTS.md)
+
+## API Design
+
+- Let `Agent` own tool calls; don’t add pipeline invokers — `ToolInvoker` was removed in `3.0`
+- Use `Pipeline.run()` and `Pipeline.run_async()`; don’t add `AsyncPipeline` — merged in `3.0`
+- Use ChatGenerators like `OpenAIChatGenerator` — non-chat generators were removed in `3.0`
+- Use Haystack serialization helpers (`component_to_dict`, `default_from_dict`) — preserves wire compatibility
+- Make optional params keyword-only with `*` — preserves API compatibility
+- Build API clients and load models in `warm_up()`, not `__init__` — preserves constructibility
+- Treat `run_async` as optional in `haystack/components/` — fallback to `asyncio.to_thread(component.run, ...)` and log it
+- Implement `warm_up()` only for real setup; make it idempotent and run it before tool access — This avoids fake lifecycle APIs, duplicate guards, and premature tool access before required setup.
+- Keep `run_async`/`warm_up_async`/`close_async` paths async-only — use separate async state/hooks and only intentional non-blocking fallbacks like `asyncio.to_thread`
+- Pass explicit `required_variables` for optional prompt vars — avoids requiring optional inputs
+- Expose explicit `Agent` tool output contracts — use `raw=True` by default; document `last_message`, text-only output, and `exit_reason` when narrowing payloads
+- Subclass `Toolset` only when inherited collection APIs match — prefer composition or raise `NotImplementedError`
+- Read live Haystack `State` resources via `state.data.get(...)` or `state.data[...]` — avoids deep-copy bugs
+- Keep `run(messages, *, streaming_callback, generation_kwargs, tools)` order in `haystack/components/generators/chat/` — preserves API consistency
+- Append optional params in public signatures, including `__init__` — avoid breaking positional callers
+- Prefer existing `haystack/core/pipeline` APIs or inline logic — avoid duplicate or one-off public APIs
+- In `from_dict()`, pass only present fields to `__init__()` — keeps defaults centralized
+- Shallow-copy `from_dict` payloads; rebuild parsed nested fields — avoids costly `deepcopy()`
+- Spell out mirrored constructor params; avoid `*args`/`**kwargs` — clarifies supported config
+- Reuse splitter chunk metadata in `haystack/components/preprocessors/` — preserve traceability fields like `page_number`, `source_id`, `header`, `parent_headers`, and split positions; add new keys only for existing downstream contracts
+- Serialize tools/toolsets under `data` without flattening — preserves round-trip fidelity
+
+## Documentation
+
+- Match markup to surface: use reStructuredText in `releasenotes/notes/`, Haystack docstring style in code — prevents broken docs rendering
+- Keep inline comments concise and non-obvious — remove restatements, but preserve durable caveats
+- Write concise, user-visible `releasenotes/notes/*.yaml` entries — clarify API, workflow, error, output, and semantic changes users rely on
+- Update docstrings with signature/type changes — remove obsolete behavior and reflect aliases like `ToolsType`
+- Document private helpers only when needed — explain non-obvious behavior, constraints, or rationale
+- Use only `releasenotes/config.yaml` sections in `releasenotes/notes/*.yaml` — keeps notes accurate and consistent
+- Document public API params and `:returns:` contracts — clarify usage, types, and behavior
+- Update `PromptBuilder`/`ChatPromptBuilder` docs with behavior changes — keep examples accurate, commented, and runnable
+- Keep `:param` docs concise and current — document meaning, defaults, and exceptions only
+- Use default constructors in docs examples; note required env vars like `OPENAI_API_KEY` nearby — Keeps docs focused on the demonstrated feature while still making required auth setup clear to readers.
+- Document raised exceptions with existing `:raises ...:` style — use `ValueError if filters have invalid syntax` for invalid filters
+- Keep doc examples minimal and local — include only used imports/code inside each snippet
+- Show example output as comments (`# ...`, `# >> ...`) — keeps snippets valid Python
+- Omit explicit defaults in docs examples — specify models only for model-specific behavior
+- Align class/API docstrings with existing style, especially in `haystack/components/` — keeps API docs readable
+- Remove `experimental` wording when promoting features — update docstrings, tutorials, `pydoc` IDs, and generated markdown filenames
+- Sync `docs-website/versioned_docs/version-*/concepts/data-classes.mdx` with dataclass/message fields — prevents stale API docs
+- Sync `haystack/components/` docs with actual behavior — prevent stale API, endpoint, integration, and example guidance
+- Document user-visible changes in `MIGRATION.md` — explain what changed, why, and required action
+
+## Code Style
+
+- Use keyword args for multi-parameter calls — improves readability and prevents mix-ups
+- Keep lint suppressions current and exact — use honored codes like `# noqa: PLR0915` only while needed
+- Share sync/async logic in private helpers under `haystack/components/` — prevents drift and `# noqa: PLR0915`
+- Scope PR diffs to the stated goal — avoid unrelated refactors, formatting, serialization, or cleanup
+- Remove temporary `print()` debugging before merging — keeps tests and runtime output clean
+- Avoid filename-only headers in source files — keeps files clean and style-consistent
+- Omit args that only restate callee defaults — reduces noise and avoids stale examples
+- Remove redundant branches and unused private helpers — document non-obvious compatibility or typing needs inline
+- Use `{placeholder}` logger templates with kwargs — preserves structured logs and avoids eager formatting
+- Keep private constants local and prefix module constants with `_` — clarifies API boundaries
+
+## Type System
+
+- Use coded `# type: ignore[...]` with a nearby why-safe comment — avoids hiding unrelated typing bugs
+- Resolve callable annotations with `typing.get_type_hints()` — avoids bugs from raw or string `inspect.signature()` annotations
+- Keep union aliases non-redundant and concept-named — include base types only when subclasses inherit
+- Use `T | None`, not `Optional[T]`, in annotations and casts — keeps typing concise
+- Annotate class-or-PEP-604 inputs as `type | types.UnionType` — matches runtime values
+- Annotate helper params broadly and accurately; use `Any` for arbitrary typing objects — Accurate helper signatures improve static analysis without rejecting valid typing constructs at runtime.
+- Use `@overload` only for real API variants; fix bad types at source, not with broad `type: ignore` — Real overloads preserve API semantics, while source fixes and narrow ignores prevent hiding type bugs.
+
+## Imports
+
+- Import via public APIs and keep only used or compatibility re-export imports — preserves API stability
+- Use top-level absolute `haystack...` imports in `haystack/components/` and `haystack/core/component/*.py` — keeps component dependencies traceable
+- Use `LazyImport` only for optional third-party deps — otherwise import directly
+- Use the narrowest clear import form, but keep module imports like `import httpx` when namespaces matter — This prevents namespace, runtime, typing, and lazy-import conflicts while keeping imports readable and minimal.
+
+## Config
+
+- Pass `allowed_modules=` or set `HAYSTACK_DESERIALIZATION_ALLOWLIST` for YAML loads — never widen `haystack/core/serialization_security.py` allowlists in library code
+- Remove dead config in `.github/workflows/` — keeps CI behavior accurate and maintainable
+
+## Naming
+
+- Prefix internal helpers with `_`, including `haystack/utils/` helpers — clarifies public API boundaries
+- Avoid naming locals after imported decorators/functions/utilities — prevents shadowing ambiguity
+
+## Testing
+
+- Centralize `DocumentStore` tests in `haystack/testing/document_store.py` mixins — use `DocumentStoreBaseTests` as the minimum suite and compose capability mixins into `DocumentStoreBaseExtendedTests`; add explicit integration skips only for unsupported backend features.
+- Test `haystack/core/pipeline/` behavior changes in matching pipeline test modules — cover end-to-end edge cases like shorthands, early returns, errors, task completion, empty inputs, and outputs
+
+## General
+
+- Define intentional package exports in `__init__.py` via `__all__`; avoid implementation-module `__all__` — Keeping package `__init__.py` exports intentional preserves stable public APIs and avoids accidental top-level imports.
+- Update `pyproject.toml` for new package APIs — declare deps and minimum versions used
+- State concrete deprecation timelines and status — helps users plan migrations
+
+## File-Specific Rules
+
+### `README.md`
+
+- Keep `README.md` feature lists complete and concrete — clarify broad terms with examples

CLAUDE.md

@@ -1,3 +1,3 @@
 # CLAUDE.md
 
-Before you start working on this repository, read the AGENTS.md file and follow all the instructions.
+@AGENTS.md

docs-website/AGENTS.md

@@ -0,0 +1,12 @@
+<!-- Mined from deepset PR reviews; see the repo-root AGENTS.md. -->
+
+# docs-website/ Guidelines
+
+## Documentation
+
+- Sync `docs-website` API names and import paths with public exports — avoids stale docs
+- Omit explicit `.warm_up()` in docs unless required — lazy/idempotent warm-up handles it
+- Keep setup/usage for maintained APIs and integrations in `docs-website/docs/` — it is the authoritative, navigable source; add pages to `docs-website/sidebars.js` when needed
+- Keep `docs-website/docs/concepts/` current-facing — put history and upgrades in migration docs
+- Link data-class symbols to anchored API docs — improves discoverability and precision
+- Verify all `docs-website` MDX links — keeps external URLs current and internal routes valid

docs-website/CLAUDE.md

@@ -0,0 +1,3 @@
+# CLAUDE.md
+
+@AGENTS.md

docs-website/docs/AGENTS.md

@@ -0,0 +1,17 @@
+<!-- Mined from deepset PR reviews; see the repo-root AGENTS.md. -->
+
+# docs-website/docs/ Guidelines
+
+## Documentation
+
+- Keep `docs-website/docs/pipeline-components/generators/` examples concise and `Agent`-level — improves copyability and keeps docs focused
+- Edit `docs-website/docs/pipeline-components/` pages only for outdated, incorrect, or materially useful user-facing guidance — avoids docs churn
+- Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details
+- Document component outputs and link producers/API refs — keeps docs ecosystem-connected
+- Use `ChatPromptBuilder`, `ChatMessage`, and chat generators in new docs LLM pipelines — match wiring, edge names like `prompt`, and declared variables to real chat interfaces.
+- Polish `docs-website/docs/pipeline-components/**/*.mdx` prose before merge — keeps docs clear and consistent
+- Sync `docs-website/docs/pipeline-components` YAML examples with current defaults — stale model names cause config errors
+- Mark joiners/adapters optional when smart pipeline connections make them optional — Prevents docs from implying extra pipeline components are mandatory when smart connections already handle the composition.
+- Document extractor side effects and exact `doc.meta` keys — clarifies pipeline data flow
+- Prefer `result["last_message"]` for Haystack agent final responses — highlights the intended API
+- Link partial config/API summaries to authoritative references — helps users find full details

docs-website/docs/CLAUDE.md

@@ -0,0 +1,3 @@
+# CLAUDE.md
+
+@AGENTS.md

haystack/hooks/compaction/AGENTS.md

@@ -0,0 +1,13 @@
+<!-- Mined from deepset PR reviews; see the repo-root AGENTS.md. -->
+
+# haystack/hooks/compaction/ Guidelines
+
+## API Design
+
+- Use provider-compatible roles in `haystack/hooks/compaction/` — prefer `user` for synthetic markers
+- Name compaction retention by semantic unit (`turns`/`steps`), not `messages` — matches what is actually preserved
+
+## General
+
+- Document `haystack/hooks/compaction/` APIs by real semantics — prevents compaction misuse
+- Keep `haystack/hooks/compaction/` compactors narrative — move shared indexing, grouping, token counting, and helpers into focused utils

haystack/hooks/compaction/CLAUDE.md

@@ -0,0 +1,3 @@
+# CLAUDE.md
+
+@AGENTS.md

releasenotes/notes/AGENTS.md

@@ -0,0 +1,14 @@
+<!-- Mined from deepset PR reviews; see the repo-root AGENTS.md. -->
+
+# releasenotes/notes/ Guidelines
+
+## Documentation
+
+- Add `upgrade` notes for breaking/user-visible changes in `releasenotes/notes/` — explain affected users, old/new behavior, and migration steps
+- Add `releasenotes/notes/` entries only for in-scope user-facing PR changes — keeps release notes accurate and low-noise; leave unrelated note files untouched.
+- Name affected APIs/configs in `releasenotes/notes/` — clarifies scope and impact for users
+- Write bug/security notes around public impact, not private helpers — clarifies user risk
+- Highlight APIs in `releasenotes/notes/` only with examples or clear use cases — shows practical value
+- Keep `releasenotes/notes/` reno notes synced with shipped behavior — prevents misleading release docs
+- Use one `releasenotes/notes/` file per PR — group related change notes together
+- Proofread `releasenotes/notes/` entries — catches API typos and formatting issues

releasenotes/notes/CLAUDE.md

@@ -0,0 +1,3 @@
+# CLAUDE.md
+
+@AGENTS.md

test/AGENTS.md

@@ -0,0 +1,26 @@
+<!-- Mined from deepset PR reviews; see the repo-root AGENTS.md. -->
+
+# test/ Guidelines
+
+## Testing
+
+- Name tests after verified behavior — keep names/docs current to avoid misleading coverage
+- Share `pytest` fixtures/helpers/constants only for truly common test setup — prevents brittle coupling and duplicate inputs
+- Keep tests minimal and non-redundant; prefer one smoke/contract test for brittle live paths like `test/components/generators/chat/`, `test/components/preprocessors/`, and `test/components/agents/test_agent.py` — reduces flaky, expensive, low-value coverage
+- Group tests by behavior in existing files and classes, mirroring the source layout (`test/components/test_{component}.py`) — split a file only when it becomes hard to navigate
+- Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts
+- Use real multi-chunk fixtures in `test/components/preprocessors/`; assert exact ordered content, metadata, and per-source `split_id`s. — Exact, ordered assertions catch splitter regressions in chunking, metadata propagation, and per-source `split_id` behavior that broad checks miss.
+- Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable
+- Add regression tests for pipeline socket metadata changes — protects `Variadic`/`GreedyVariadic` edge cases
+- Pair sync `run` integration tests with `run_async` tests — keep async behavior covered
+- In splitter tests, assert the joined split content equals the input text — catches dropped, duplicated, or reordered content
+- Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — Keeps tests reliable in local and CI runs without requiring real OpenAI credentials unless explicitly testing the live integration.
+- Assert splitter overlap offsets explicitly — verify `split_idx_start` and `_split_overlap` as character ranges into the original text, with coverage for each `split_unit`
+- Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs
+- Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions
+- Assert `Pipeline` fan-in order from runtime semantics — use joiners for custom order
+
+## General
+
+- Avoid `# type: ignore` in tests; narrow with `hasattr(...)` or `assert isinstance(...)` first — Explicit narrowing keeps tests type-safe and exposes real API mismatches instead of hiding bugs from `mypy`.
+- Keep test imports at module scope; remove redundant local imports — improves visibility and consistency

test/CLAUDE.md

@@ -0,0 +1,3 @@
+# CLAUDE.md
+
+@AGENTS.md