Merge pull request Pipelex#112 from Pipelex/release/v0.4.5

### Changed
- **Test structure overhaul**: Reorganized test directory structure for better organization:
  - Tests now separated into `unit/`, `integration/`, and `e2e/` directories
  - Created `tests/cases/` package for pure test data and constants
  - Created `tests/helpers/` package for test utilities
  - Cleaned up test imports and removed empty `__init__.py` files
- **Class registry refactoring**: Updated kajson from 0.1.6 to 0.2.0, adapted to changes in [Kajson](https://github.com/Pipelex/kajson)'s class registry with new `ClassRegistryUtils` (better separation of concerns)
- **Dependency updates**:
  - Added pytest-mock to dev dependencies for improved unit testing

### Added
- **Coverage commands**: New Makefile targets for test coverage analysis:
  - `make cov`: Run tests with coverage report
  - `make cov-missing` (or `make cm`): Show coverage with missing lines
- **Test configuration**: Set `xfail_strict = true` in pytest config for stricter test failure handling
- **Pydantic validation errors**: Enhanced error formatting to properly handle model_type errors

### Fixed
- **External links**: Removed broken Markdown target="_blank" syntax from MANIFESTO.md links
- **Variable naming consistency**: Fixed redundant naming in OpenAI config (openai_openai_config → openai_config)
- **Makefile optimization**: Removed parallel test execution (`-n auto`) from codex-tests, works better now

### Tests
- **Unit tests added**: New comprehensive unit tests for:
  - `ClassRegistryUtils`
  - `FuncRegistry` 
  - `ModuleInspector`
  - File finding utilities

Author: lchoquel

.cursor/rules/pytest.mdc

@@ -10,11 +10,11 @@ These rules apply when writing unit tests.
 
 - Name test files with `test_` prefix
 - Use descriptive names that match the functionality being tested
-- Place test files in the appropriate subdirectory of `tests/`:
-    - `tests/tools/` for tests related to sub-package `pipelex.tools`
-    - `tests/pipelex/` for tests related to `pipelex`and its sub-packages
-    - `tests/pipelex/cogt/` for tests related to sub-package `pipelex.cogt`
-- More precisely, for `pipelex` and `pipelex.cogt` the async tests are placed inside subdirectories named `cogt_asynch` and `pipelex_asynch`
+- Place test files in the appropriate test category directory:
+    - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+    - `tests/integration/` - for integration tests that test component interactions
+    - `tests/e2e/` - for end-to-end tests that test complete workflows
+    - `tests/test_pipelines/` - for test pipeline definitions (TOML files and their structuring python files)
 - Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
 - Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `tests.pipelex.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
 - Always put test inside Test classes.
@@ -55,7 +55,7 @@ class TestFooBar:
         # Test implementation
 ```
 
-Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get th test name using `cast(str, request.node.originalname),  # type: ignore`. 
+Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname),  # type: ignore`. 
 
 # Pipe tests
 

.cursor/rules/standards.mdc

@@ -14,6 +14,7 @@ This document outlines the coding standards and quality control procedures that
 Before finalizing a task, you must run the following command to check for linting issues, type errors, and code quality problems:
 
 ```bash
+make fix-unused-imports
 make check
 ```
 
@@ -34,20 +35,16 @@ We have several make commands for running tests:
    ```
    Use this for quick test runs that don't require LLM or image generation.
 
-2. `make ti`: Runs all tests with these markers:
-   ```
-   inference and not imgg
-   ```
-   Use this for testing LLM functionality without image generation.
-
-3. To run specific tests:
+2. To run specific tests:
    ```bash
    make tp TEST=TestClassName
    # or
    make tp TEST=test_function_name
    ```
    It matches names, so `TEST=test_function_name` is going to run all test with the function name that STARTS with `test_function_name`.
 
+Note: never run `make ti`, `make test-inference`, `make to`, `make test-ocr`, `make tg`, or `make test-imgg`: these all use inference which is costly.
+
 ## Important Project Directories
 
 ### Pipelines Directory

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [v0.4.5] - 2025-06-23
+
+### Changed
+- **Test structure overhaul**: Reorganized test directory structure for better organization:
+  - Tests now separated into `unit/`, `integration/`, and `e2e/` directories
+  - Created `tests/cases/` package for pure test data and constants
+  - Created `tests/helpers/` package for test utilities
+  - Cleaned up test imports and removed empty `__init__.py` files
+- **Class registry refactoring**: Updated kajson from 0.1.6 to 0.2.0, adapted to changes in [Kajson](https://github.com/Pipelex/kajson)'s class registry with new `ClassRegistryUtils` (better separation of concerns)
+- **Dependency updates**:
+  - Added pytest-mock to dev dependencies for improved unit testing
+
+### Added
+- **Coverage commands**: New Makefile targets for test coverage analysis:
+  - `make cov`: Run tests with coverage report
+  - `make cov-missing` (or `make cm`): Show coverage with missing lines
+- **Test configuration**: Set `xfail_strict = true` in pytest config for stricter test failure handling
+- **Pydantic validation errors**: Enhanced error formatting to properly handle model_type errors
+
+### Fixed
+- **External links**: Removed broken Markdown target="_blank" syntax from MANIFESTO.md links
+- **Variable naming consistency**: Fixed redundant naming in OpenAI config (openai_openai_config → openai_config)
+- **Makefile optimization**: Removed parallel test execution (`-n auto`) from codex-tests, works better now
+
+### Tests
+- **Unit tests added**: New comprehensive unit tests for:
+  - `ClassRegistryUtils`
+  - `FuncRegistry` 
+  - `ModuleInspector`
+  - File finding utilities
+
 ## [v0.4.4] - 2025-06-20
 
 ### Fixed

MANIFESTO.md

@@ -1,6 +1,6 @@
-### This is our manifesto, this is why we build [Pipelex](https://pipelex.com){:target="_blank"}.
+### This is our manifesto, this is why we built [Pipelex](https://pipelex.com).
 
-First published on our [blog](https://www.pipelex.com/post/repeatable-ai-workflows-knowledge-pipelines){:target="_blank"} on June 3, 2025.
+First published on our [blog](https://www.pipelex.com/post/repeatable-ai-workflows-knowledge-pipelines) on June 3, 2025.
 
 ---
 # The Knowledge Pipeline Manifesto
@@ -54,7 +54,7 @@ We've seen this movie before. Docker transformed deployment by giving us a simpl
 
 Today's AI workflows are ready for the same breakthrough.
 
-This is why we built [**Pipelex**](https://github.com/Pipelex/pipelex){:target="_blank"} as an open-source standard for repeatable AI workflows. Not just another tool, but a foundation for the community to build on.
+This is why we built [**Pipelex**](https://github.com/Pipelex/pipelex) as an open-source standard for repeatable AI workflows. Not just another tool, but a foundation for the community to build on.
 
 The manifesto is simple: **When you discover a method that works, capture it as a knowledge pipeline. When you build a pipeline that roars, share it with the world!**
 

Makefile

@@ -207,7 +207,7 @@ cleanall: cleanderived cleanenv cleanlibraries
 codex-tests: env
 	$(call PRINT_TITLE,"Unit testing for Codex")
 	@echo "• Running unit tests for Codex (excluding inference and codex_disabled)"
-	$(VENV_PYTEST) -n auto --exitfirst --quiet -m "(dry_runnable or not inference) and not (needs_output or pipelex_api or codex_disabled)" || [ $$? = 5 ]
+	$(VENV_PYTEST) --exitfirst -m "(dry_runnable or not inference) and not (needs_output or pipelex_api or codex_disabled)" || [ $$? = 5 ]
 
 gha-tests: env
 	$(call PRINT_TITLE,"Unit testing for github actions")
@@ -318,6 +318,27 @@ test-pipelex-api: env
 ta: test-pipelex-api
 	@echo "> done: ta = test-pipelex-api"
 
+cov: env
+	$(call PRINT_TITLE,"Unit testing with coverage")
+	@echo "• Running unit tests with coverage"
+	@if [ -n "$(TEST)" ]; then \
+		$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+	else \
+		$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+	fi
+
+cov-missing: env
+	$(call PRINT_TITLE,"Unit testing with coverage and missing lines")
+	@echo "• Running unit tests with coverage and missing lines"
+	@if [ -n "$(TEST)" ]; then \
+		$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+	else \
+		$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+	fi
+
+cm: cov-missing
+	@echo "> done: cm = cov-missing"
+
 ############################################################################################
 ############################               Linting              ############################
 ############################################################################################

docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md

@@ -133,7 +133,6 @@ Analyze the document page shown in the image and explain how it relates to the p
 2. **Fixed Multiple Outputs**: Use `nb_output = N` (where N is a positive integer) when you need exactly N outputs. For example, `nb_output = 3` will try to generate 3 results. The parameter `_nb_output` will be available in the prompt template, e.g. "Give me the names of $_nb_output flowers".
 
 3. **Variable Multiple Outputs**: Use `multiple_output = true` when you need a variable-length list where the LLM determines how many outputs to generate based on the content and context.
-| `output_multiplicity`       | string or integer   | Defines the number of outputs. Use `"list"` for a variable-length list, or an integer (e.g., `3`) for a fixed-size list.                                                       | No       |
 
 ## Examples
 

pipelex/core/stuff_factory.py

@@ -1,7 +1,7 @@
 from typing import Any, Dict, List, Optional, Tuple
 
 import shortuuid
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, ValidationError
 
 from pipelex.config import get_config
 from pipelex.core.concept import Concept
@@ -11,6 +11,7 @@
 from pipelex.core.stuff_content import StuffContent, StuffContentInitableFromStr
 from pipelex.exceptions import ConceptError, PipelexError
 from pipelex.hub import get_class_registry, get_required_concept
+from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error
 
 
 class StuffFactoryError(PipelexError):
@@ -148,14 +149,22 @@ def make_multiple_stuff_from_str(cls, str_stuff_and_concepts_dict: Dict[str, Tup
         return result
 
     @classmethod
-    def combine_stuffs(cls, concept_code: str, stuff_contents: Dict[str, StuffContent], name: Optional[str] = None) -> Stuff:
+    def combine_stuffs(
+        cls,
+        concept_code: str,
+        stuff_contents: Dict[str, StuffContent],
+        name: Optional[str] = None,
+    ) -> Stuff:
         """
         Combine a dictionary of stuffs into a single stuff.
         """
         the_concept = get_required_concept(concept_code=concept_code)
         the_subclass_name = the_concept.structure_class_name
         the_subclass = get_class_registry().get_required_subclass(name=the_subclass_name, base_class=StuffContent)
-        the_stuff_content = the_subclass.model_validate(obj=stuff_contents)
+        try:
+            the_stuff_content = the_subclass.model_validate(obj=stuff_contents)
+        except ValidationError as exc:
+            raise StuffFactoryError(f"Error combining stuffs: {format_pydantic_validation_error(exc=exc)}") from exc
         return cls.make_stuff(
             concept_str=concept_code,
             content=the_stuff_content,

pipelex/libraries/library_manager.py

@@ -24,6 +24,7 @@
     StaticValidationError,
 )
 from pipelex.libraries.library_config import LibraryConfig
+from pipelex.tools.class_registry_utils import ClassRegistryUtils
 from pipelex.tools.misc.file_utils import find_files_in_dir
 from pipelex.tools.misc.json_utils import deep_update
 from pipelex.tools.misc.toml_utils import load_toml_from_path
@@ -74,13 +75,13 @@ def teardown(self) -> None:
     def load_libraries(self):
         log.debug("LibraryManager loading separate libraries")
 
-        KajsonManager.get_class_registry().register_classes_in_folder(
+        ClassRegistryUtils.register_classes_in_folder(
             folder_path=LibraryConfig.loaded_pipelines_path,
         )
         library_paths = [LibraryConfig.loaded_pipelines_path]
         if runtime_manager.is_unit_testing:
             log.debug("Registering test pipeline structures for unit testing")
-            KajsonManager.get_class_registry().register_classes_in_folder(
+            ClassRegistryUtils.register_classes_in_folder(
                 folder_path=LibraryConfig.test_pipelines_path,
             )
             library_paths += [LibraryConfig.test_pipelines_path]
@@ -117,6 +118,7 @@ def _load_combo_libraries(self, library_paths: List[str]):
                 pattern="*.toml",
                 is_recursive=True,
             )
+            log.debug(f"Searching for TOML files in {libraries_path}, found '{found_file_paths}'")
             if not found_file_paths:
                 log.warning(f"No TOML files found in library path: {libraries_path}")
             toml_file_paths.extend(found_file_paths)

pipelex/pipe_operators/pipe_llm_factory.py

@@ -131,7 +131,7 @@ def make_pipe_from_blueprint(
             user_images=user_images or None,
         )
 
-        llm_settings = LLMSettingChoices(
+        llm_choices = LLMSettingChoices(
             for_text=pipe_blueprint.llm,
             for_object=pipe_blueprint.llm_to_structure,
             for_object_direct=pipe_blueprint.llm_to_structure_direct,
@@ -152,7 +152,7 @@ def make_pipe_from_blueprint(
             inputs=PipeInputSpec(root=pipe_blueprint.inputs or {}),
             output_concept_code=pipe_blueprint.output,
             pipe_llm_prompt=pipe_llm_prompt,
-            llm_choices=llm_settings,
+            llm_choices=llm_choices,
             structuring_method=pipe_blueprint.structuring_method,
             prompt_template_to_structure=pipe_blueprint.prompt_template_to_structure,
             system_prompt_to_structure=pipe_blueprint.system_prompt_to_structure,

pipelex/plugins/openai/openai_factory.py

@@ -50,8 +50,8 @@ def make_openai_client(cls, llm_platform: LLMPlatform) -> openai.AsyncClient:
                     base_url=endpoint,
                 )
             case LLMPlatform.OPENAI:
-                openai_openai_config = get_config().plugins.openai_config
-                api_key = openai_openai_config.get_api_key(secrets_provider=get_secrets_provider())
+                openai_config = get_config().plugins.openai_config
+                api_key = openai_config.get_api_key(secrets_provider=get_secrets_provider())
                 the_client = openai.AsyncOpenAI(api_key=api_key)
             case LLMPlatform.VERTEXAI:
                 vertexai_config = get_config().plugins.vertexai_config