Merge pull request Pipelex#107 from Pipelex/dev

dev to release/v0.4.4

Author: thomashebrard

.cursor/rules/best_practices.mdc

@@ -0,0 +1,77 @@
+---
+description: 
+globs: 
+alwaysApply: false
+---
+# Best Practices Guide
+
+This document outlines the core best practices and patterns used in our codebase.
+
+## Type Hints
+
+1. **Always Use Type Hints**
+   - Every function parameter must be typed
+   - Every function return must be typed
+   - Use type hints for all variables where type is not obvious
+
+2. **StrEnum**
+   - Import StrEnum from pipelex.types
+   ```python
+   from pipelex.types import StrEnum
+   
+   class ModelType(StrEnum):
+       GPT4 = "gpt-4"
+       GPT35 = "gpt-3.5-turbo"
+   ```
+
+## Factory Pattern
+
+1. **Use Factory Pattern for Object Creation**
+   - Create factories when dealing with multiple implementations
+
+## Documentation
+
+1. **Docstring Format**
+   - Quick description of the function/class
+   - List args and their types
+   - Document return values
+   - Example:
+   ```python
+   def process_image(image_path: str, size: Tuple[int, int]) -> bytes:
+       """Process and resize an image.
+       
+       Args:
+           image_path: Path to the source image
+           size: Tuple of (width, height) for resizing
+           
+       Returns:
+           Processed image as bytes
+       """
+       pass
+   ```
+
+2. **Class Documentation**
+   - Document class purpose and behavior
+   - Include examples if complex
+   ```python
+   class ImageProcessor:
+       """Handles image processing operations.
+       
+       Provides methods for resizing, converting, and optimizing images.
+       """
+   ```
+
+## Custom Exceptions
+
+1. **Graceful Error Handling**
+   - Use try/except blocks with specific exceptions
+   - Convert third-party exceptions to custom ones
+   ```python
+   try:
+       from fal_client import AsyncClient as FalAsyncClient
+   except ImportError as exc:
+       raise MissingDependencyError(
+           "fal-client", "fal", 
+           "The fal-client SDK is required to use FAL models."
+       ) from exc
+   ```

.cursor/rules/pipes.mdc

@@ -5,20 +5,9 @@ alwaysApply: false
 ---
 This rule explains how to build pipes.
 
-# File Naming & Structure
-
-## The pipelines/ directory
-
-- Pipelines and structures are defined in the `pipelex/libraries/pipelines` directory
-
-pipelex/libraries
-└── pipelines
-
 ## Pipeline file naming
 
-- Pipeline TOML files should be placed in the `pipelex/libraries/pipelines/` directory or a subdirectory of it
 - The file name should be descriptive, in snake_case and end with `.toml`
-- The file should contain both concepts and pipes definitions
 
 ## Pipeline file structure
 
@@ -167,7 +156,6 @@ Your summary should not be longer than 2 sentences.
 PipeLLM = "Convert table screenshot to HTML"
 inputs = { table_screenshot = "TableScreenshot" }
 output = "HtmlTable"
-images = ["table_screenshot"]
 system_prompt = """
 You are a vision-based table extractor.
 """

.cursor/rules/pytest.mdc

@@ -17,6 +17,8 @@ These rules apply when writing unit tests.
 - More precisely, for `pipelex` and `pipelex.cogt` the async tests are placed inside subdirectories named `cogt_asynch` and `pipelex_asynch`
 - Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
 - Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `tests.pipelex.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes.
+- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
 
 ## Markers
 

.cursor/rules/standards.mdc

@@ -0,0 +1,60 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+# Coding Standards
+
+This document outlines the coding standards and quality control procedures that must be followed when contributing to this project.
+
+## Code Quality Checks
+
+### Linting and Type Checking
+
+Before finalizing a task, you must run the following command to check for linting issues, type errors, and code quality problems:
+
+```bash
+make check
+```
+
+This command runs multiple code quality tools:
+- Pyright: Static type checking
+- Ruff: Fast Python linter
+- Mypy: Static type checker
+
+Always fix any issues reported by these tools before proceeding.
+
+### Running Tests
+
+We have several make commands for running tests:
+
+1. `make tp`: Runs all tests with these markers:
+   ```
+   (dry_runnable or not (inference or llm or imgg or ocr)) and not (needs_output or pipelex_api)
+   ```
+   Use this for quick test runs that don't require LLM or image generation.
+
+2. `make ti`: Runs all tests with these markers:
+   ```
+   inference and not imgg
+   ```
+   Use this for testing LLM functionality without image generation.
+
+3. To run specific tests:
+   ```bash
+   make tp TEST=TestClassName
+   # or
+   make tp TEST=test_function_name
+   ```
+   It matches names, so `TEST=test_function_name` is going to run all test with the function name that STARTS with `test_function_name`.
+
+## Important Project Directories
+
+### Pipelines Directory
+- All pipeline definitions go in `pipelex/libraries/pipelines/`
+
+### Tests Directory
+- All tests are located in the `tests/` directory
+
+### Documentation Directory
+- All documentation is located in the `docs/` directory

.cursor/rules/structures.mdc

@@ -59,3 +59,4 @@ Structure classes defined within `pipelex_libraries/pipelines/` are automaticall
 - Use Pydantic validators for data cleaning/validation
 - Remove timezone info from datetime fields
 - Respect rules of @base_models.
+

.cursor/rules/tdd.mdc

@@ -0,0 +1,28 @@
+---
+description: 
+globs: 
+alwaysApply: false
+---
+# Test-Driven Development Guide
+
+This document outlines our test-driven development (TDD) process and the tools available for testing.
+
+## TDD Cycle
+
+1. **Write a Test First**
+[pytest.mdc](mdc:.cursor/rules/pytest.mdc)
+
+2. **Write the Code**
+   - Implement the minimum amount of code needed to pass the test
+   - Follow the project's coding standards
+   - Keep it simple - don't write more than needed
+
+3. **Run Linting and Type Checking**
+[standards.mdc](mdc:.cursor/rules/standards.mdc)
+
+4. **Refactor if needed**
+If the code needs refactoring, with the best practices [best_practices.mdc](mdc:.cursor/rules/best_practices.mdc)
+
+5. **Validate tests**
+
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.

.github/workflows/guard-branches.yml

@@ -36,7 +36,7 @@ jobs:
           if [[ "$HEAD" == "dev" ]]; then
             exit 0
           fi
-          if [[ ! "$HEAD" =~ ^(fix|feature|refactor|chore|doc|ci-cd|changelog|codex)\/[A-Za-z0-9._\/\<\>\=\-]+$ ]]; then
+          if [[ ! "$HEAD" =~ ^(fix|feature|refactor|chore|docs|ci-cd|changelog|codex)\/[A-Za-z0-9._\/\<\>\=\-]+$ ]]; then
             echo "::error::Branch must start with fix/, feature/, refactor/, chore/, docs/, or ci-cd/."
             exit 1
           fi

CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## [v0.4.4] - 2025-06-20
+
+### Fixed
+- Changed the allowed base branch names in the GHA `guard-branches.yml`: `doc` -> `docs`
+- Fixed `kajson` dependency (see [kajson v0.1.6 changelog](https://github.com/Pipelex/kajson/blob/main/CHANGELOG.md))
+
+### Cursor rules
+- Added Cursor rules for coding best practices and standards (including linting methods). Added TDD (Test Driven Development) rule on demand.
+- Various changes
+
+### Documentation
+- Added documentation for referencing images in PipeLLM.
+- Fixed typos
+
+### Refactor
+- Removed the `images` field from PipeLLM - images can now be referenced directly in the `inputs`
+- Moved the list-pipes CLI function to the `PipeLibrary` class.
+
 ## [v0.4.3] - 2025-06-19
 
 ### Fixed
@@ -35,9 +53,9 @@
     - Added GHA workflows for documentation deployment and validation
 - **Added to docs:**
     - [**Manifesto**](https://docs.pipelex.com/manifesto/) explaining the Pipelex viewpoint
-    - [**The Pipelex Paradigm**](https://docs.pipelex.com/pages/pipelex-paradigm-for-repeatable-ai-workflows/) explaining the fundamentals of Pipelex’s solution
+    - [**The Pipelex Paradigm**](https://docs.pipelex.com/pages/pipelex-paradigm-for-repeatable-ai-workflows/) explaining the fundamentals of Pipelex's solution
     - [**Cookbook examples](https://docs.pipelex.com/pages/cookbook-examples/)** presented and explained, commented code, some event with [mermaid](https://docs.pipelex.com/pages/cookbook-examples/invoice-extractor/) [flow](https://docs.pipelex.com/pages/cookbook-examples/extract-gantt/) [charts](https://docs.pipelex.com/pages/cookbook-examples/write-tweet/)
-    - And plenty of details about **using Pipelex** and **developing for Pipelex,** from **structured generation** to PipeOperators (**LLM**, **Image generation**, **OCR**…) to PipeControllers (**Sequence**, **Parallel**, **Batch**, **Condition**…), workflow **optimization**, workflow static **validation** and dry run… there’s still work to do, but we move fast!
+    - And plenty of details about **using Pipelex** and **developing for Pipelex,** from **structured generation** to PipeOperators (**LLM**, **Image generation**, **OCR**…) to PipeControllers (**Sequence**, **Parallel**, **Batch**, **Condition**…), workflow **optimization**, workflow static **validation** and dry run… there's still work to do, but we move fast!
 - **Also a major update of Cursor rules**
 
 ### Tooling Improvements

CONTRIBUTING.md

@@ -12,7 +12,7 @@ We are open to contributions in all areas of our core Pipelex library:
 - **Feature**: New API, CLI flag, module, test coverage
 - **Refactor**: Rethink architecture
 - **Chore**: Dependency updates, config tweaks, file renames
-- **Doc**: Main docs, SWE Agent rules, tutorials, examples, READMEs
+- **Docs**: Main docs, SWE Agent rules, tutorials, examples, READMEs
 - **CI/CD**: GitHub Actions, packaging, release tooling
 
 ## Contribution process
@@ -22,7 +22,7 @@ We are open to contributions in all areas of our core Pipelex library:
 - Install dependencies: `make install` (creates .venv and installs dependencies)
 - Copy `.env.example` to `.env` and fill in required API keys (at least OpenAI)
 - Run checks to make sure all is good: `make check` & `make test`
-- Create a branch with the format user_name/category/short_slug where category is one of: `feature`, `fix`, `refactor`, `doc`, `cicd` or `chore`
+- Create a branch with the format user_name/category/short_slug where category is one of: `feature`, `fix`, `refactor`, `docs`, `cicd` or `chore`
 - Make and commit changes
 - Push your local branch to your fork
 - Open a PR that [links to an existing Issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue) which does not include the `needs triage` label