Skip to content

Commit f06826a

Browse files
actions-useractions-user
committed
docs: update codeboarding documentation in advanced-customization
## 📚 Documentation Update This commit contains updated documentation files fetched from the CodeBoarding service and copied to the advanced-customization section. ### 📊 Summary - Documentation files created/updated: 8 - JSON files created/updated: 9 - Documentation directory: .codeboarding/ - JSON directory: .codeboarding/ - Output format: .md - Repository analyzed: https://github.com/CodeBoarding/pipelex - Destination: docs/pages/advanced-customization/ The generated .md files have been automatically copied to the advanced-customization documentation section. 🤖 This commit was automatically generated by the CodeBoarding documentation update workflow.
1 parent b5ff219 commit f06826a

25 files changed

Lines changed: 2498 additions & 0 deletions

‎.codeboarding/AI_Integration_Layer.json‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
{
2+
"description": "The AI Integration Layer in pipelex provides a unified interface to various AI service providers, abstracting away their specific APIs, and is utilized by the Pipe Operators for AI-related tasks. This layer is primarily encapsulated within the pipelex.cogt package.",
3+
"components": [
4+
{
5+
"name": "InferenceManager",
6+
"description": "The central orchestrator for all AI inference tasks (LLM, Image Generation, OCR). It receives inference requests and dispatches them to the appropriate specialized AI workers.",
7+
"referenced_source_code": [
8+
{
9+
"qualified_name": "pipelex.cogt.inference.inference_manager",
10+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/inference/inference_manager.py",
11+
"reference_start_line": null,
12+
"reference_end_line": null
13+
}
14+
],
15+
"can_expand": true
16+
},
17+
{
18+
"name": "LLMWorker",
19+
"description": "Defines the abstract interface for interacting with various Large Language Model (LLM) providers. Concrete implementations (e.g., Anthropic, OpenAI) handle provider-specific API calls.",
20+
"referenced_source_code": [
21+
{
22+
"qualified_name": "pipelex.cogt.llm.llm_worker_abstract",
23+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/llm/llm_worker_abstract.py",
24+
"reference_start_line": null,
25+
"reference_end_line": null
26+
}
27+
],
28+
"can_expand": true
29+
},
30+
{
31+
"name": "ImggWorker",
32+
"description": "Defines the abstract interface for interacting with different Image Generation (Imgg) providers. Concrete implementations handle provider-specific API calls.",
33+
"referenced_source_code": [
34+
{
35+
"qualified_name": "pipelex.cogt.imgg.imgg_worker_abstract",
36+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/imgg/imgg_worker_abstract.py",
37+
"reference_start_line": null,
38+
"reference_end_line": null
39+
}
40+
],
41+
"can_expand": true
42+
},
43+
{
44+
"name": "OcrWorker",
45+
"description": "Defines the abstract interface for interacting with various Optical Character Recognition (OCR) providers. Concrete implementations handle provider-specific API calls.",
46+
"referenced_source_code": [
47+
{
48+
"qualified_name": "pipelex.cogt.ocr.ocr_worker_abstract",
49+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/ocr/ocr_worker_abstract.py",
50+
"reference_start_line": null,
51+
"reference_end_line": null
52+
}
53+
],
54+
"can_expand": true
55+
},
56+
{
57+
"name": "LLMDeck",
58+
"description": "Manages and validates configurations for various LLMs, ensuring correct setup and parameters for AI interactions.",
59+
"referenced_source_code": [
60+
{
61+
"qualified_name": "pipelex.cogt.llm.llm_models.llm_deck",
62+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/llm/llm_models/llm_deck.py",
63+
"reference_start_line": null,
64+
"reference_end_line": null
65+
}
66+
],
67+
"can_expand": true
68+
},
69+
{
70+
"name": "LLMModelLibrary",
71+
"description": "Provides access to loaded LLM model definitions and acts as a central repository for LLM models, enabling workers to retrieve necessary model information.",
72+
"referenced_source_code": [
73+
{
74+
"qualified_name": "pipelex.cogt.llm.llm_models.llm_model_library",
75+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/llm/llm_models/llm_model_library.py",
76+
"reference_start_line": null,
77+
"reference_end_line": null
78+
}
79+
],
80+
"can_expand": true
81+
},
82+
{
83+
"name": "AI Configuration",
84+
"description": "Encapsulates specific configuration models (`LLMConfig`, `ImggConfig`, `OcrConfig`, `InferenceManagerConfig`) used to set up and run various AI tasks across the subsystem.",
85+
"referenced_source_code": [
86+
{
87+
"qualified_name": "pipelex.cogt.config_cogt",
88+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cogt/config_cogt.py",
89+
"reference_start_line": null,
90+
"reference_end_line": null
91+
}
92+
],
93+
"can_expand": false
94+
}
95+
],
96+
"components_relations": [
97+
{
98+
"relation": "orchestrates",
99+
"src_name": "InferenceManager",
100+
"dst_name": "LLMWorker"
101+
},
102+
{
103+
"relation": "orchestrates",
104+
"src_name": "InferenceManager",
105+
"dst_name": "ImggWorker"
106+
},
107+
{
108+
"relation": "orchestrates",
109+
"src_name": "InferenceManager",
110+
"dst_name": "OcrWorker"
111+
},
112+
{
113+
"relation": "is configured by",
114+
"src_name": "InferenceManager",
115+
"dst_name": "AI Configuration"
116+
},
117+
{
118+
"relation": "receives tasks from",
119+
"src_name": "LLMWorker",
120+
"dst_name": "InferenceManager"
121+
},
122+
{
123+
"relation": "uses configurations from",
124+
"src_name": "LLMWorker",
125+
"dst_name": "LLMDeck"
126+
},
127+
{
128+
"relation": "uses configurations from",
129+
"src_name": "LLMWorker",
130+
"dst_name": "LLMModelLibrary"
131+
},
132+
{
133+
"relation": "receives tasks from",
134+
"src_name": "ImggWorker",
135+
"dst_name": "InferenceManager"
136+
},
137+
{
138+
"relation": "is configured by",
139+
"src_name": "ImggWorker",
140+
"dst_name": "AI Configuration"
141+
},
142+
{
143+
"relation": "receives tasks from",
144+
"src_name": "OcrWorker",
145+
"dst_name": "InferenceManager"
146+
},
147+
{
148+
"relation": "is configured by",
149+
"src_name": "OcrWorker",
150+
"dst_name": "AI Configuration"
151+
},
152+
{
153+
"relation": "provides configurations to",
154+
"src_name": "LLMDeck",
155+
"dst_name": "LLMWorker"
156+
},
157+
{
158+
"relation": "provides models to",
159+
"src_name": "LLMModelLibrary",
160+
"dst_name": "LLMWorker"
161+
},
162+
{
163+
"relation": "configures",
164+
"src_name": "AI Configuration",
165+
"dst_name": "InferenceManager"
166+
},
167+
{
168+
"relation": "configures",
169+
"src_name": "AI Configuration",
170+
"dst_name": "LLMWorker"
171+
},
172+
{
173+
"relation": "configures",
174+
"src_name": "AI Configuration",
175+
"dst_name": "ImggWorker"
176+
},
177+
{
178+
"relation": "configures",
179+
"src_name": "AI Configuration",
180+
"dst_name": "OcrWorker"
181+
}
182+
]
183+
}

‎.codeboarding/AI_Integration_Layer.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
```mermaid
2+
graph LR
3+
InferenceManager["InferenceManager"]
4+
LLMWorker["LLMWorker"]
5+
ImggWorker["ImggWorker"]
6+
OcrWorker["OcrWorker"]
7+
LLMDeck["LLMDeck"]
8+
LLMModelLibrary["LLMModelLibrary"]
9+
AI_Configuration["AI Configuration"]
10+
InferenceManager -- "orchestrates" --> LLMWorker
11+
InferenceManager -- "orchestrates" --> ImggWorker
12+
InferenceManager -- "orchestrates" --> OcrWorker
13+
InferenceManager -- "is configured by" --> AI_Configuration
14+
LLMWorker -- "receives tasks from" --> InferenceManager
15+
LLMWorker -- "uses configurations from" --> LLMDeck
16+
LLMWorker -- "uses configurations from" --> LLMModelLibrary
17+
ImggWorker -- "receives tasks from" --> InferenceManager
18+
ImggWorker -- "is configured by" --> AI_Configuration
19+
OcrWorker -- "receives tasks from" --> InferenceManager
20+
OcrWorker -- "is configured by" --> AI_Configuration
21+
LLMDeck -- "provides configurations to" --> LLMWorker
22+
LLMModelLibrary -- "provides models to" --> LLMWorker
23+
AI_Configuration -- "configures" --> InferenceManager
24+
AI_Configuration -- "configures" --> LLMWorker
25+
AI_Configuration -- "configures" --> ImggWorker
26+
AI_Configuration -- "configures" --> OcrWorker
27+
```
28+
29+
[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org)
30+
31+
## Details
32+
33+
The AI Integration Layer in pipelex provides a unified interface to various AI service providers, abstracting away their specific APIs, and is utilized by the Pipe Operators for AI-related tasks. This layer is primarily encapsulated within the pipelex.cogt package.
34+
35+
### InferenceManager
36+
The central orchestrator for all AI inference tasks (LLM, Image Generation, OCR). It receives inference requests and dispatches them to the appropriate specialized AI workers.
37+
38+
39+
**Related Classes/Methods**:
40+
41+
- `pipelex.cogt.inference.inference_manager`
42+
43+
44+
### LLMWorker
45+
Defines the abstract interface for interacting with various Large Language Model (LLM) providers. Concrete implementations (e.g., Anthropic, OpenAI) handle provider-specific API calls.
46+
47+
48+
**Related Classes/Methods**:
49+
50+
- `pipelex.cogt.llm.llm_worker_abstract`
51+
52+
53+
### ImggWorker
54+
Defines the abstract interface for interacting with different Image Generation (Imgg) providers. Concrete implementations handle provider-specific API calls.
55+
56+
57+
**Related Classes/Methods**:
58+
59+
- `pipelex.cogt.imgg.imgg_worker_abstract`
60+
61+
62+
### OcrWorker
63+
Defines the abstract interface for interacting with various Optical Character Recognition (OCR) providers. Concrete implementations handle provider-specific API calls.
64+
65+
66+
**Related Classes/Methods**:
67+
68+
- `pipelex.cogt.ocr.ocr_worker_abstract`
69+
70+
71+
### LLMDeck
72+
Manages and validates configurations for various LLMs, ensuring correct setup and parameters for AI interactions.
73+
74+
75+
**Related Classes/Methods**:
76+
77+
- `pipelex.cogt.llm.llm_models.llm_deck`
78+
79+
80+
### LLMModelLibrary
81+
Provides access to loaded LLM model definitions and acts as a central repository for LLM models, enabling workers to retrieve necessary model information.
82+
83+
84+
**Related Classes/Methods**:
85+
86+
- `pipelex.cogt.llm.llm_models.llm_model_library`
87+
88+
89+
### AI Configuration
90+
Encapsulates specific configuration models (`LLMConfig`, `ImggConfig`, `OcrConfig`, `InferenceManagerConfig`) used to set up and run various AI tasks across the subsystem.
91+
92+
93+
**Related Classes/Methods**:
94+
95+
- `pipelex.cogt.config_cogt`
96+
97+
98+
99+
100+
### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq)

‎.codeboarding/Client_API_Layer.json‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
{
2+
"description": "The `Client & API Layer` is fundamentally important as it defines how users and other systems access the workflow orchestration capabilities. Separating it into `CLI` and `Pipelex (Core Engine API)` components aligns with the \"Workflow Orchestration Engine / AI Workflow Framework\" pattern by distinguishing between direct human interaction (CLI) and programmatic integration (API). This separation promotes modularity, allowing different types of clients to interact with the core engine through appropriate interfaces, which is crucial for extensibility and maintainability in an AI workflow framework.",
3+
"components": [
4+
{
5+
"name": "Client & API Layer",
6+
"description": "This layer serves as the primary interface for external users and systems to interact with the Pipelex engine. It encompasses both the command-line interface for direct user interaction and the programmatic API exposed by the core `Pipelex` engine for integration with other systems or custom scripts. It acts as the gateway for initiating, managing, and monitoring AI workflows.",
7+
"referenced_source_code": [
8+
{
9+
"qualified_name": "pipelex.cli._cli",
10+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cli/_cli.py",
11+
"reference_start_line": null,
12+
"reference_end_line": null
13+
},
14+
{
15+
"qualified_name": "pipelex.pipelex.Pipelex",
16+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/pipelex.py",
17+
"reference_start_line": 60,
18+
"reference_end_line": 304
19+
}
20+
],
21+
"can_expand": true
22+
},
23+
{
24+
"name": "CLI (Command-Line Interface)",
25+
"description": "The `CLI` component provides a user-friendly command-line interface, enabling users to directly execute Pipelex commands, validate pipeline configurations, and manage workflow executions. It is the primary entry point for manual user interaction with the Pipelex system.",
26+
"referenced_source_code": [
27+
{
28+
"qualified_name": "pipelex.cli._cli",
29+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/cli/_cli.py",
30+
"reference_start_line": null,
31+
"reference_end_line": null
32+
}
33+
],
34+
"can_expand": true
35+
},
36+
{
37+
"name": "Pipelex (Core Engine API)",
38+
"description": "While `Pipelex` represents the core orchestration engine, its public methods and attributes form the primary programmatic API. This API allows external clients, including the `CLI` and other integrated systems, to programmatically control and manage workflows, define pipelines, and interact with the engine's functionalities.",
39+
"referenced_source_code": [
40+
{
41+
"qualified_name": "pipelex.pipelex.Pipelex",
42+
"reference_file": "/mnt/e/StartUp/pipelex/pipelex/pipelex.py",
43+
"reference_start_line": 60,
44+
"reference_end_line": 304
45+
}
46+
],
47+
"can_expand": true
48+
}
49+
],
50+
"components_relations": [
51+
{
52+
"relation": "interacts with",
53+
"src_name": "CLI (Command-Line Interface)",
54+
"dst_name": "Pipelex (Core Engine API)"
55+
},
56+
{
57+
"relation": "exposes API to",
58+
"src_name": "Pipelex (Core Engine API)",
59+
"dst_name": "Client & API Layer"
60+
}
61+
]
62+
}

‎.codeboarding/Client_API_Layer.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
```mermaid
2+
graph LR
3+
Client_API_Layer["Client & API Layer"]
4+
CLI_Command_Line_Interface_["CLI (Command-Line Interface)"]
5+
Pipelex_Core_Engine_API_["Pipelex (Core Engine API)"]
6+
CLI_Command_Line_Interface_ -- "interacts with" --> Pipelex_Core_Engine_API_
7+
Pipelex_Core_Engine_API_ -- "exposes API to" --> Client_API_Layer
8+
click Client_API_Layer href "https://github.com/CodeBoarding/pipelex/blob/main/.codeboarding/Client_API_Layer.md" "Details"
9+
```
10+
11+
[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org)
12+
13+
## Details
14+
15+
The `Client & API Layer` is fundamentally important as it defines how users and other systems access the workflow orchestration capabilities. Separating it into `CLI` and `Pipelex (Core Engine API)` components aligns with the "Workflow Orchestration Engine / AI Workflow Framework" pattern by distinguishing between direct human interaction (CLI) and programmatic integration (API). This separation promotes modularity, allowing different types of clients to interact with the core engine through appropriate interfaces, which is crucial for extensibility and maintainability in an AI workflow framework.
16+
17+
### Client & API Layer [[Expand]](./Client_API_Layer.md)
18+
This layer serves as the primary interface for external users and systems to interact with the Pipelex engine. It encompasses both the command-line interface for direct user interaction and the programmatic API exposed by the core `Pipelex` engine for integration with other systems or custom scripts. It acts as the gateway for initiating, managing, and monitoring AI workflows.
19+
20+
21+
**Related Classes/Methods**:
22+
23+
- `pipelex.cli._cli`
24+
- <a href="https://github.com/CodeBoarding/pipelex/blob/main/pipelex/pipelex.py#L60-L304" target="_blank" rel="noopener noreferrer">`pipelex.pipelex.Pipelex` (60:304)</a>
25+
26+
27+
### CLI (Command-Line Interface)
28+
The `CLI` component provides a user-friendly command-line interface, enabling users to directly execute Pipelex commands, validate pipeline configurations, and manage workflow executions. It is the primary entry point for manual user interaction with the Pipelex system.
29+
30+
31+
**Related Classes/Methods**:
32+
33+
- `pipelex.cli._cli`
34+
35+
36+
### Pipelex (Core Engine API)
37+
While `Pipelex` represents the core orchestration engine, its public methods and attributes form the primary programmatic API. This API allows external clients, including the `CLI` and other integrated systems, to programmatically control and manage workflows, define pipelines, and interact with the engine's functionalities.
38+
39+
40+
**Related Classes/Methods**:
41+
42+
- <a href="https://github.com/CodeBoarding/pipelex/blob/main/pipelex/pipelex.py#L60-L304" target="_blank" rel="noopener noreferrer">`pipelex.pipelex.Pipelex` (60:304)</a>
43+
44+
45+
46+
47+
### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq)

0 commit comments

Comments
 (0)