Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/source/en/_redirects.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# old_local: new_local
stable_diffusion: optimization/overview
api/pipelines/p_h3_image: api/pipelines/p_image_oss
api/models/autoencoderkl_p_h3_image: api/models/autoencoderkl_p_image_oss
8 changes: 4 additions & 4 deletions docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -515,8 +515,8 @@
title: AutoencoderKLMiniMaxH3Image
- local: api/models/autoencoderkl_mochi
title: AutoencoderKLMochi
- local: api/models/autoencoderkl_p_h3_image
title: AutoencoderKLPH3Image
- local: api/models/autoencoderkl_p_image_oss
title: AutoencoderKLPImageOSS
- local: api/models/autoencoderkl_qwenimage
title: AutoencoderKLQwenImage
- local: api/models/autoencoderkl_qwenimage21
Expand Down Expand Up @@ -665,8 +665,8 @@
title: OmniGen
- local: api/pipelines/ovis_image
title: Ovis-Image
- local: api/pipelines/p_h3_image
title: P-H3-Image
- local: api/pipelines/p_image_oss
title: P-Image-OSS
- local: api/pipelines/pag
title: PAG
- local: api/pipelines/pixart
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ Unless required by applicable law or agreed to in writing, software distributed
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License. -->

# AutoencoderKLPH3Image
# AutoencoderKLPImageOSS

The image VAE of [P-H3-Image](https://huggingface.co/PrunaAI/P-H3-Image) by Pruna AI. It uses the encoder and latent space of MiniMax-H3's [video VAE](./autoencoderkl_minimax_h3), with a ViT decoder that decodes a single latent frame into an image.
The image VAE of [P-Image-OSS](https://huggingface.co/PrunaAI/P-Image-OSS) by Pruna AI. It uses the encoder and latent space of MiniMax-H3's [video VAE](./autoencoderkl_minimax_h3), with a ViT decoder that decodes a single latent frame into an image.

It follows the conventions of [`AutoencoderKLMiniMaxH3`]:

Expand All @@ -22,18 +22,18 @@ It follows the conventions of [`AutoencoderKLMiniMaxH3`]:

```python
import torch
from diffusers import AutoencoderKLPH3Image
from diffusers import AutoencoderKLPImageOSS

vae = AutoencoderKLPH3Image.from_pretrained("PrunaAI/P-H3-Image", subfolder="vae", dtype=torch.bfloat16).to(
vae = AutoencoderKLPImageOSS.from_pretrained("PrunaAI/P-Image-OSS", subfolder="vae", dtype=torch.bfloat16).to(
"cuda"
) # or "mps", "xpu", "cpu"
```

The encoder and the decoder's output projection stay in float32 whatever `dtype` the model is loaded in.

## AutoencoderKLPH3Image
## AutoencoderKLPImageOSS

[[autodoc]] AutoencoderKLPH3Image
[[autodoc]] AutoencoderKLPImageOSS
- encode
- decode
- all
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,12 @@ Unless required by applicable law or agreed to in writing, software distributed
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License. -->

# P-H3-Image
# P-Image-OSS

P-H3-Image is an image model by Pruna AI based on [MiniMax-H3](./minimax_h3), for text-to-image generation and instruction-based image editing. You can find the checkpoint at [PrunaAI/P-H3-Image](https://huggingface.co/PrunaAI/P-H3-Image).
P-Image-OSS is an image model by Pruna AI based on [MiniMax-H3](./minimax_h3), for text-to-image generation and instruction-based image editing. You can find the checkpoint at [PrunaAI/P-Image-OSS](https://huggingface.co/PrunaAI/P-Image-OSS).

> [!TIP]
> P-Image-OSS was previously released as P-H3-Image. The former class names (`PH3ImageModularPipeline`, `PH3ImageBlocks`, `PH3ImageTextConnector`, and `AutoencoderKLPH3Image`) are kept as aliases of the new ones.

```py
import torch
Expand All @@ -20,7 +23,7 @@ from diffusers import ComponentsManager, ModularPipeline
manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda") # or "mps", "xpu"

pipe = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image", components_manager=manager)
pipe = ModularPipeline.from_pretrained("PrunaAI/P-Image-OSS", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)

prompt = (
Expand All @@ -39,8 +42,8 @@ Every request goes through the [prompt upsampler](#prompt-upsampling) first. Wit
|---|---|
| `transformer` | [`MiniMaxH3Transformer3DModel`] |
| `text_encoder` | `Qwen3VLForConditionalGeneration` (Qwen3-VL-4B) |
| `text_connector` | [`PH3ImageTextConnector`] |
| `vae` | [`AutoencoderKLPH3Image`] |
| `text_connector` | [`PImageOSSTextConnector`] |
| `vae` | [`AutoencoderKLPImageOSS`] |
| `tokenizer`, `processor` | The text encoder's tokenizer, and a `Qwen3VLProcessor` with MiniMax-H3's image processor |
| `scheduler`, `audio_scheduler` | [`MiniMaxH3Scheduler`] |
| `prompt_upsampler`, `prompt_upsampler_processor` | `Qwen3_5ForConditionalGeneration` (based on Qwen3.5-9B) and its processor |
Expand All @@ -54,7 +57,7 @@ The workflow is selected on the `image` input:

## Prompt upsampling

P-H3-Image is meant to render full prompts rather than short requests, so the pipeline runs the prompt upsampler first on every request. It thinks about the request and then writes the prompt: a detailed text-to-image prompt for the aspect ratio of `height` and `width`, or, when editing, a brief of what to keep and what to change in the images. The prompt it wrote comes back as the `prompt` output.
P-Image-OSS is meant to render full prompts rather than short requests, so the pipeline runs the prompt upsampler first on every request. It thinks about the request and then writes the prompt: a detailed text-to-image prompt for the aspect ratio of `height` and `width`, or, when editing, a brief of what to keep and what to change in the images. The prompt it wrote comes back as the `prompt` output.

```py
out = pipe(prompt=prompt, height=3360, width=2496, output=["images", "prompt"])
Expand All @@ -69,14 +72,14 @@ To render `prompt` as given, pop the step from the blocks. The popped block also
manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

blocks = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image").blocks
blocks = ModularPipeline.from_pretrained("PrunaAI/P-Image-OSS").blocks
upsampler_block = blocks.sub_blocks.pop("prompt_upsampler")

upsampler = upsampler_block.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
upsampler = upsampler_block.init_pipeline("PrunaAI/P-Image-OSS", components_manager=manager)
upsampler.load_components(dtype=torch.bfloat16)
upsampled = upsampler(prompt=prompt, height=3360, width=2496, output="prompt")

pipe = blocks.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
pipe = blocks.init_pipeline("PrunaAI/P-Image-OSS", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
image = pipe(prompt=upsampled, height=3360, width=2496, output="images")[0]
```
Expand All @@ -88,14 +91,14 @@ Without the step, an edit prompt in either form the upsampler writes, the six-se
- The 8 steps run over the checkpoint's `sample_sigmas`. Pass `sigmas` to use another grid.
- The model is guidance-distilled, so there is no guider, `negative_prompt` or `guidance_scale`.

## PH3ImageModularPipeline
## PImageOSSModularPipeline

[[autodoc]] PH3ImageModularPipeline
[[autodoc]] PImageOSSModularPipeline

## PH3ImageBlocks
## PImageOSSBlocks

[[autodoc]] PH3ImageBlocks
[[autodoc]] PImageOSSBlocks

## PH3ImageTextConnector
## PImageOSSTextConnector

[[autodoc]] PH3ImageTextConnector
[[autodoc]] PImageOSSTextConnector
8 changes: 8 additions & 0 deletions src/diffusers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,7 @@
"AutoencoderKLMiniMaxH3Image",
"AutoencoderKLMochi",
"AutoencoderKLPH3Image",
"AutoencoderKLPImageOSS",
"AutoencoderKLQwenImage",
"AutoencoderKLQwenImage21",
"AutoencoderKLTemporalDecoder",
Expand Down Expand Up @@ -358,6 +359,7 @@
"PAN2Transformer3DModel",
"ParallelConfig",
"PH3ImageTextConnector",
"PImageOSSTextConnector",
"PixArtTransformer2DModel",
"PriorTransformer",
"PRXTransformer2DModel",
Expand Down Expand Up @@ -611,6 +613,8 @@
"PAN2SafetyChecker",
"PH3ImageBlocks",
"PH3ImageModularPipeline",
"PImageOSSBlocks",
"PImageOSSModularPipeline",
"PikaH3FlexBlocks",
"PikaH3FlexModularPipeline",
"QwenImage21AutoBlocks",
Expand Down Expand Up @@ -1180,6 +1184,7 @@
AutoencoderKLMiniMaxH3Image,
AutoencoderKLMochi,
AutoencoderKLPH3Image,
AutoencoderKLPImageOSS,
AutoencoderKLQwenImage,
AutoencoderKLQwenImage21,
AutoencoderKLTemporalDecoder,
Expand Down Expand Up @@ -1276,6 +1281,7 @@
PAN2Transformer3DModel,
ParallelConfig,
PH3ImageTextConnector,
PImageOSSTextConnector,
PixArtTransformer2DModel,
PriorTransformer,
PRXTransformer2DModel,
Expand Down Expand Up @@ -1509,6 +1515,8 @@
PH3ImageModularPipeline,
PikaH3FlexBlocks,
PikaH3FlexModularPipeline,
PImageOSSBlocks,
PImageOSSModularPipeline,
QwenImage21AutoBlocks,
QwenImage21ModularPipeline,
QwenImage21TurboAutoBlocks,
Expand Down
9 changes: 7 additions & 2 deletions src/diffusers/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
_import_structure["autoencoders.autoencoder_kl_minimax_h3_audio"] = ["AutoencoderKLMiniMaxH3Audio"]
_import_structure["autoencoders.autoencoder_kl_minimax_h3_image"] = ["AutoencoderKLMiniMaxH3Image"]
_import_structure["autoencoders.autoencoder_kl_mochi"] = ["AutoencoderKLMochi"]
_import_structure["autoencoders.autoencoder_kl_p_h3_image"] = ["AutoencoderKLPH3Image"]
_import_structure["autoencoders.autoencoder_kl_p_image_oss"] = ["AutoencoderKLPH3Image", "AutoencoderKLPImageOSS"]
_import_structure["autoencoders.autoencoder_kl_qwenimage"] = ["AutoencoderKLQwenImage"]
_import_structure["autoencoders.autoencoder_kl_qwenimage21"] = ["AutoencoderKLQwenImage21"]
_import_structure["autoencoders.autoencoder_kl_temporal_decoder"] = ["AutoencoderKLTemporalDecoder"]
Expand All @@ -83,7 +83,10 @@
_import_structure["condition_embedders.condition_embedder_gligen"] = ["CLIPImageProjection"]
_import_structure["condition_embedders.condition_embedder_ltx2"] = ["LTX2TextConnectors"]
_import_structure["condition_embedders.condition_embedder_minimax_music3"] = ["MiniMaxMusic3ConditionEncoder"]
_import_structure["condition_embedders.condition_embedder_p_h3_image"] = ["PH3ImageTextConnector"]
_import_structure["condition_embedders.condition_embedder_p_image_oss"] = [
"PH3ImageTextConnector",
"PImageOSSTextConnector",
]
_import_structure["condition_embedders.condition_embedder_stable_audio"] = ["StableAudioProjectionModel"]
_import_structure["condition_embedders.condition_embedder_stable_audio_3"] = ["StableAudio3DurationEmbedder"]
_import_structure["condition_embedders.condition_embedder_stable_unclip"] = ["StableUnCLIPImageNormalizer"]
Expand Down Expand Up @@ -236,6 +239,7 @@
AutoencoderKLMiniMaxH3Image,
AutoencoderKLMochi,
AutoencoderKLPH3Image,
AutoencoderKLPImageOSS,
AutoencoderKLQwenImage,
AutoencoderKLQwenImage21,
AutoencoderKLTemporalDecoder,
Expand Down Expand Up @@ -267,6 +271,7 @@
LTX2TextConnectors,
MiniMaxMusic3ConditionEncoder,
PH3ImageTextConnector,
PImageOSSTextConnector,
ReduxImageEncoder,
StableAudio3DurationEmbedder,
StableAudioProjectionModel,
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/models/autoencoders/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
from .autoencoder_kl_minimax_h3_audio import AutoencoderKLMiniMaxH3Audio
from .autoencoder_kl_minimax_h3_image import AutoencoderKLMiniMaxH3Image
from .autoencoder_kl_mochi import AutoencoderKLMochi
from .autoencoder_kl_p_h3_image import AutoencoderKLPH3Image
from .autoencoder_kl_p_image_oss import AutoencoderKLPH3Image, AutoencoderKLPImageOSS
from .autoencoder_kl_qwenimage import AutoencoderKLQwenImage
from .autoencoder_kl_qwenimage21 import AutoencoderKLQwenImage21
from .autoencoder_kl_temporal_decoder import AutoencoderKLTemporalDecoder
Expand Down
Loading