diff --git a/AGENTS.md b/AGENTS.md index 2b90c149bb5..b25a19ed89f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,3 +59,70 @@ Every user-facing PR (not docs, not CI) must include a release note: hatch run release-note SHORT_DESCRIPTION Edit the generated file in `releasenotes/notes/`. Release notes use reStructuredText formatting; see the [release notes section in CONTRIBUTING.md](CONTRIBUTING.md#release-notes) for details. + +In addition, there are rules inferred from previous code reviews. Follow them like a +reviewer's note: they encode reasons, not ceremony, so when a rule genuinely does not +fit the change at hand, say why rather than contorting the code to satisfy it. + +Also see directory-specific guidelines: + +- [docs-website/AGENTS.md](docs-website/AGENTS.md) +- [docs-website/docs/AGENTS.md](docs-website/docs/AGENTS.md) +- [haystack/hooks/compaction/AGENTS.md](haystack/hooks/compaction/AGENTS.md) +- [releasenotes/notes/AGENTS.md](releasenotes/notes/AGENTS.md) +- [test/AGENTS.md](test/AGENTS.md) + +## API Design + +- Target the 3.0 API: `ToolInvoker`, `AsyncPipeline`, and non-chat generators were removed — let `Agent` own tool calls, use `Pipeline.run()`/`Pipeline.run_async()`, and use ChatGenerators like `OpenAIChatGenerator` +- Serialize with the Haystack helpers (`component_to_dict`, `default_from_dict`): in `from_dict()` pass only the fields present to `__init__()` so defaults stay centralized, shallow-copy the payload instead of `deepcopy()`, and keep tools/toolsets nested under `data` — preserves wire compatibility and round-trip fidelity +- Keep public signatures explicit and compatible: optional params keyword-only after `*` and appended rather than inserted, mirrored constructor params spelled out instead of `*args`/`**kwargs`, and chat generators keeping the `run(messages, *, streaming_callback, generation_kwargs, tools)` order — avoids breaking positional callers +- Create API clients and load models in `warm_up()`, not `__init__`; implement `warm_up()` only for real setup and make it idempotent — keeps components constructible and serializable without credentials or network +- Implement `run_async` only when there is a real async execution path — `Pipeline.run_async()` already falls back to `asyncio.to_thread(component.run, ...)` when it is missing; where it exists, keep `run_async`/`warm_up_async`/`close_async` genuinely async with separate async state and hooks, and share sync/async logic through private helpers — prevents event-loop blocking and sync/async drift +- Pass explicit `required_variables` for optional prompt vars — avoids requiring optional inputs +- Subclass `Toolset` only when inherited collection APIs match — prefer composition or raise `NotImplementedError` +- Read live Haystack `State` resources via `state.data.get(...)` or `state.data[...]` — avoids deep-copy bugs +- Prefer existing `haystack/core/pipeline` APIs or inline logic — avoid duplicate or one-off public APIs +- Reuse splitter chunk metadata in `haystack/components/preprocessors/` — keep traceability fields like `page_number`, `source_id`, `header`, `parent_headers`, and split positions; add new keys only for existing downstream contracts + +## Documentation + +- Keep inline comments and private-helper docs to what is non-obvious — remove restatements, keep durable caveats and rationale +- Keep docstrings current with signatures and behavior, in the existing Haystack style: each public `:param` by meaning, default, and constraints; `:returns:` contracts; exceptions in the existing `:raises ValueError:` style; aliases like `ToolsType` reflected — stale docs mislead users and assistants +- Keep doc examples minimal, runnable, and local: default constructors with required env vars like `OPENAI_API_KEY` noted nearby, only the imports the snippet uses, no restated defaults (name a model only for model-specific behavior), expected output as comments, generally `# >> ...` +- When behavior, fields, or names change, update every surface in the same PR: `haystack/components/` docstrings and examples, `docs-website/docs/` plus the current `versioned_docs/version-*/` page (e.g. `concepts/data-classes.mdx`), and `experimental` wording, `pydoc` IDs, and generated markdown filenames when promoting features + +## Code Style + +- Use keyword args for multi-parameter calls — improves readability and prevents mix-ups +- Keep lint suppressions exact and current — a `# noqa: PLR0915` only while the code needs it +- Scope diffs to the stated goal and leave them clean: no unrelated refactors or formatting, no stray `print()`, no filename-only header comments, no unused private helpers or redundant branches, no args that only restate callee defaults +- Use `{placeholder}` logger templates with kwargs — preserves structured logs and avoids eager formatting +- Prefix internal helpers and module-level constants with `_` and keep private constants local; never name locals after imported functions, decorators, or utilities — clarifies API boundaries and avoids shadowing + +## Type System + +- Fix types at the source; where a suppression is unavoidable use a coded `# type: ignore[...]` with a nearby why-safe comment, and add `@overload` only for real API variants +- Resolve callable annotations with `typing.get_type_hints()` — avoids bugs from raw or string `inspect.signature()` annotations +- Keep union aliases non-redundant and concept-named — include base types only when subclasses inherit +- Use `T | None`, not `Optional[T]`, in annotations and casts — keeps typing concise +- Annotate helper params broadly and accurately — `type | types.UnionType` for class-or-PEP-604 inputs, `Any` for arbitrary typing objects + +## Imports + +- Import via public APIs in the narrowest clear form (keep module imports like `import httpx` when namespaces matter); keep only used imports plus deliberate compatibility re-exports +- Use top-level absolute `haystack...` imports in `haystack/components/` and `haystack/core/component/*.py` — keeps component dependencies traceable +- Use `LazyImport` only for optional third-party deps — otherwise import directly + +## Config + +- Pass `allowed_modules=` or set `HAYSTACK_DESERIALIZATION_ALLOWLIST` for YAML loads — never widen `haystack/core/serialization_security.py` allowlists in library code + +## Testing + +- Centralize `DocumentStore` tests in `haystack/testing/document_store.py` mixins — use `DocumentStoreBaseTests` as the minimum suite and compose capability mixins into `DocumentStoreBaseExtendedTests`; add explicit integration skips only for unsupported backend features. + +## General + +- Define package exports in `__init__.py` via `__all__`; avoid `__all__` in implementation modules — keeps public APIs intentional +- Update `pyproject.toml` for new package APIs — declare deps and minimum versions used diff --git a/CLAUDE.md b/CLAUDE.md index 8f8a6efba47..43c994c2d36 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1 @@ -# CLAUDE.md - -Before you start working on this repository, read the AGENTS.md file and follow all the instructions. +@AGENTS.md diff --git a/docs-website/AGENTS.md b/docs-website/AGENTS.md new file mode 100644 index 00000000000..1159e0fdf8b --- /dev/null +++ b/docs-website/AGENTS.md @@ -0,0 +1,7 @@ +# docs-website/ Guidelines + +## Documentation + +- Keep `docs-website` API names, import paths, and links current with public exports — link data-class symbols to anchored API docs and verify every MDX link (internal routes and external URLs) resolves +- Omit explicit `.warm_up()` in docs unless required — lazy/idempotent warm-up handles it +- Keep setup/usage for maintained APIs and integrations in `docs-website/docs/` — it is the authoritative, navigable source; add pages to `docs-website/sidebars.js` when needed diff --git a/docs-website/CLAUDE.md b/docs-website/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/docs-website/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/docs-website/docs/AGENTS.md b/docs-website/docs/AGENTS.md new file mode 100644 index 00000000000..0f7e01ae2a1 --- /dev/null +++ b/docs-website/docs/AGENTS.md @@ -0,0 +1,9 @@ +# docs-website/docs/ Guidelines + +## Documentation + +- Edit `docs-website/docs/pipeline-components/` pages only for outdated, incorrect, or materially useful guidance; keep examples concise and `Agent`-level and polish prose before merge — avoids churn while keeping docs copyable +- Add `## Overview` near the top of `docs-website/docs/pipeline-components/**` pages — explains what the component does and why to use it before details +- Document component outputs, extractor side effects, and exact `doc.meta` keys; link producers, API references, and the authoritative reference for any partial config summary +- Use current chat APIs in new docs pipelines — `ChatPromptBuilder`, `ChatMessage`, chat generators, and `result["last_message"]` for agent output — matching wiring, edge names like `prompt`, and declared variables; keep YAML examples on current default model names +- Mark joiners/adapters optional where smart pipeline connections already handle the composition diff --git a/docs-website/docs/CLAUDE.md b/docs-website/docs/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/docs-website/docs/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/haystack/hooks/compaction/AGENTS.md b/haystack/hooks/compaction/AGENTS.md new file mode 100644 index 00000000000..29da53fef58 --- /dev/null +++ b/haystack/hooks/compaction/AGENTS.md @@ -0,0 +1,10 @@ +# haystack/hooks/compaction/ Guidelines + +## API Design + +- Use provider-compatible roles in `haystack/hooks/compaction/` — prefer `user` for synthetic markers +- Name and document compaction settings by their real semantics — retention in `turns`/`steps`, not `messages`; state when compaction runs, what is retained, and which tool-call context is preserved + +## General + +- Keep `haystack/hooks/compaction/` compactors narrative — move shared indexing, grouping, token counting, and helpers into focused utils diff --git a/haystack/hooks/compaction/CLAUDE.md b/haystack/hooks/compaction/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/haystack/hooks/compaction/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/releasenotes/notes/AGENTS.md b/releasenotes/notes/AGENTS.md new file mode 100644 index 00000000000..90ed0761c47 --- /dev/null +++ b/releasenotes/notes/AGENTS.md @@ -0,0 +1,8 @@ +# releasenotes/notes/ Guidelines + +## Documentation + +- Add `upgrade` notes for breaking/user-visible changes in `releasenotes/notes/` — explain affected users, old/new behavior, and migration steps +- Write one concise, user-facing note file per PR, only for in-scope changes and only under sections from `releasenotes/config.yaml`; name the affected APIs/configs and the old and new behavior, describe impact in user terms rather than private helpers, and leave unrelated note files untouched +- Highlight APIs in `releasenotes/notes/` only with examples or clear use cases — shows practical value +- Check the PR's own release note against the shipped behavior — API names, and reStructuredText formatting with double backticks for inline code; leave existing notes alone diff --git a/releasenotes/notes/CLAUDE.md b/releasenotes/notes/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/releasenotes/notes/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/test/AGENTS.md b/test/AGENTS.md new file mode 100644 index 00000000000..592cf95e0e9 --- /dev/null +++ b/test/AGENTS.md @@ -0,0 +1,18 @@ +# test/ Guidelines + +## Testing + +- Name tests after the behavior they verify and group them by behavior in the existing file mirroring the source layout (`test/components//test_{component}.py`); keep the suite minimal — one smoke/contract test for brittle live paths such as chat generators, preprocessors, and `test_agent.py` — and split a file only when it becomes hard to navigate +- Keep test imports at module scope and share `pytest` fixtures/helpers/constants only for truly common setup — prevents brittle coupling +- Test serialization via public round trips — use `component_to_dict`/`component_from_dict`, `to_dict()`/`from_dict()`, or constructors; avoid hardcoded deep dicts +- In splitter tests use real multi-chunk fixtures and assert exact ordered content, metadata, and per-source `split_id`s; assert the joined split content equals the input text; check `split_idx_start`/`_split_overlap` as character ranges into the original text for each `split_unit` +- Assert exception messages with `pytest.raises(..., match=...)`; fully match one related invalid case — catches user-visible error regressions while keeping tests readable +- Cover `haystack/core/pipeline/` changes in the matching pipeline test module with end-to-end edge cases (shorthands, early returns, errors, empty inputs, outputs); add regression tests for socket metadata around `Variadic`/`GreedyVariadic`; assert fan-in order from runtime semantics and use joiners when a custom order matters +- Pair sync `run` integration tests with `run_async` tests — keep async behavior covered +- Skip OpenAI integration tests without `OPENAI_API_KEY`; use dummy keys for non-live tests — keeps local and CI runs credential-free unless testing live +- Test PEP 604 unions (`X | Y`, `X | None`) with `typing.Union`/`Optional` — catches annotation-compat bugs +- Assert full dict shapes in `test/components/generators/chat/` — catches schema regressions + +## General + +- Avoid `# type: ignore` in tests; narrow with `hasattr(...)` or `assert isinstance(...)` first — exposes real API mismatches diff --git a/test/CLAUDE.md b/test/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/test/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md