Skip to content

Add Comfy quantization support for INT8 ConvRot checkpoints - #15209

Open
asaiyume wants to merge 2 commits into
huggingface:mainfrom
asaiyume:feature/convrot-int8
Open

asaiyume wants to merge 2 commits into
huggingface:mainfrom
asaiyume:feature/convrot-int8

Conversation

@asaiyume

@asaiyume asaiyume commented Nov 8, 2026 •

Copy link
Copy Markdown

What does this PR do?

Adds prequantized Comfy checkpoint loading through from_single_file and ComfyQuantizationConfig, with per-layer format detection and a shared Comfy-Kitchen linear runtime. Quantization metadata stays aligned when checkpoint conversion splits or concatenates weights.

Fixes #14403 & #14705.

Usage

pip install "comfy-kitchen>=0.2.41" accelerate
import torch
from diffusers import ComfyQuantizationConfig, ZImagePipeline, ZImageTransformer2DModel

transformer = ZImageTransformer2DModel.from_single_file(
    "https://huggingface.co/Comfy-Org/Z-Image-Turbo/blob/main/split_files/diffusion_models/z_image_turbo_int8_convrot.safetensors",
    config="Tongyi-MAI/Z-Image-Turbo",
    subfolder="transformer",
    quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16),
)
pipe = ZImagePipeline.from_pretrained(
    "Tongyi-MAI/Z-Image-Turbo", transformer=transformer, dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()

image = pipe(
    "A small orange cat sitting on a mossy stone wall in a mountain village.",
    height=1024,
    width=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
    generator=torch.Generator("cpu").manual_seed(0),
).images[0]
image.save("convrot.png")

Example

image

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@github-actions github-actions Bot added documentation Improvements or additions to documentation quantization models tests utils single-file size/L PR with diff > 200 LOC labels Nov 8, 2026
@asaiyume
asaiyume marked this pull request as ready for review November 8, 2026 16:19
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Hi @asaiyume, thanks for the PR! It does not appear to link an issue it fixes.

  • If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked.
  • If you are the author of the model/pipeline this PR relates to, or part of the team we work with on it, tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box in the PR description (if you removed the template, paste that line back, checked). This reminder then no longer applies.
  • Otherwise there is nothing you need to do: a maintainer will look at the PR and add the no-issue-needed label to silence this reminder if it should stay open without an issue.

See the contribution guide for more details.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an existing issue, the author box is ticked, or the no-issue-needed label is added, you can ignore this message — it stays here as a comment, but it no longer applies.

@sayakpaul sayakpaul added performance Anything related to performance improvements, profiling and benchmarking no-issue-needed for PRs that do not require link to an issue labels Nov 9, 2026

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR. It mostly looks quite clean.

Comment thread docs/source/en/quantization/convrot.md Resolved

def to_diff_dict(self):
default = ConvRotQuantizationConfig().to_dict()
return {key: value for key, value in self.to_dict().items() if key == "quant_method" or value != default[key]}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can value != default[key] arise?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asaiyume this seems to be unaddressed, still.

@sayakpaul
sayakpaul requested a review from DN6 November 9, 2026 09:08
@sayakpaul

Copy link
Copy Markdown
Member

Wanted to post some results after running everything on one H100 80GB (torch 2.13+cu130, comfy-kitchen 0.2.41 on its native CUDA kernels).

Z-Image:

image

QwenImage 2.1:

image

Artifact: https://claude.ai/artifact/RkvvHy5LRNTpdUrWQvC37t

@asaiyume

asaiyume commented Nov 9, 2026 •

Copy link
Copy Markdown
Author

Wanted to post some results after running everything on one H100 80GB (torch 2.13+cu130, comfy-kitchen 0.2.41 on its native CUDA kernels).

Z-Image:

image QwenImage 2.1: image Artifact: https://claude.ai/artifact/RkvvHy5LRNTpdUrWQvC37t

Thank you for the results! I did suspect that offloading might have affected the results, so I ran it again on a L40S without any offloading, and was consistent with the results speedups I got locally.

Model Generation BF16 → INT8 Denoiser BF16 → INT8 Denoiser speedup
Z-Image Turbo, 8 steps 3.211 → 2.495 s 3.001 → 2.284 s 1.31×
Qwen-Image 2.1, 40 steps 16.686 → 12.166 s 16.281 → 11.761 s 1.38×
image image

@asaiyume

Copy link
Copy Markdown
Author

Also added support for loading int8 VAE, which decreased VRAM usage by 50% and improves video decoding speed.

@asaiyume

Copy link
Copy Markdown
Author

Just noticed #14705 , should the PR support ComfyQuantizationConfig() instead, since int8 w8a8 + cr is just a subset of it?

@sayakpaul @DN6 Appreciate any directions or guidance on this one.

@sayakpaul

Copy link
Copy Markdown
Member

Just noticed #14705 , should the PR support ComfyQuantizationConfig() instead, since int8 w8a8 + cr is just a subset of it?

I think this sounds quite reasonable. We could also do something like the following:

  • Introduce ComfyQuantizationConfig
  • Only keep ConvRot inside it
  • Follow up with the rest of the scemes supported there

Obviously, it depends on the amount of work and reviewing needed but my hunch says doing it the above way will keep things more manageable.

@asaiyume

Copy link
Copy Markdown
Author

Just noticed #14705 , should the PR support ComfyQuantizationConfig() instead, since int8 w8a8 + cr is just a subset of it?

I think this sounds quite reasonable. We could also do something like the following:

  • Introduce ComfyQuantizationConfig
  • Only keep ConvRot inside it
  • Follow up with the rest of the scemes supported there

Obviously, it depends on the amount of work and reviewing needed but my hunch says doing it the above way will keep things more manageable.

Definitely, I will try to keep it small, then extend model supports & other quant variants.

@asaiyume asaiyume changed the title Add native INT8 ConvRot checkpoint loading Add Comfy quantization support for INT8 ConvRot checkpoints Nov 12, 2026
@asaiyume

Copy link
Copy Markdown
Author

@sayakpaul Appreciate your help reviewing this when you have a moment so we can get this base merged. Currently supports QI2.1, Z-Image & MMH3 (pruned + VAE) loading using ComfyInt8Linear. Also updated usage script for quick reproductions.

Remaining works (not in this PR scope):

  • model adapters for converting different architectures
  • lora loading & fusing
  • other comfy quantization variants

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Pages changed in this PR:

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! I left some comments. I also ran the tests on an H100 and they pass.

Comment thread docs/source/en/quantization/comfy.md Resolved
Comment thread docs/source/en/quantization/comfy.md Resolved
Comment thread src/diffusers/quantizers/comfy/utils.py Resolved
@sayakpaul sayakpaul added this to the Release 0.44.0 milestone Nov 13, 2026
@asaiyume

Copy link
Copy Markdown
Author

@sayakpaul gentle ping when you have a moment

@DN6

DN6 commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

@asaiyume I think this can be expanded to support the full set of Comfy Quants via Comfy Kitchen.
Here is what I recommend

  1. Reduce the config to ComfyQuantizationConfig(compute_dtype) and drop quant_type, since a Comfy checkpoint can mix quant types per layer.
  2. Hard code self.pre_quantized = True, similar to GGUF, to only support pre-quantized checkpoints.
  3. Introduce a ComfyQuantizedTensor(QuantizedTensor) that holds a layer's weight together with the sidecar tensors from the Comfy checkpoint (weight_scale, input_scale, etc.). It keeps them aligned through chunk, split and cat when the checkpoint is converted to diffusers format.
  4. Implement maybe_update_state_dict on ComfyQuantizer; from_single_file already calls it before conversion. It should replace each quantized layer's weight in the loaded state dict with a ComfyQuantizedTensor, which is then passed to the conversion functions.
  5. Create a single ComfyLinear for all formats and swap it in for nn.Linear in _process_model_before_weight_loading, similar to the BFL quantizer.

We also want to avoid quantizer-specific logic in the model loading utilities and in the model files. It should live in the quantizer. For example, _get_projection_dtype in transformer_minimax_h3.py should not be needed once the weight is a QuantizedTensor, since it reports the compute dtype.

Can we also move the MiniMax-H3 changes to a separate PR that covers pruned checkpoint support (adaln_curve_grid, adaln_t_table, curve_proj) and the H3 VAE single-file loader.

@asaiyume

Copy link
Copy Markdown
Author

Thanks, that was very insightful! I will work on that direction.

@asaiyume
asaiyume force-pushed the feature/convrot-int8 branch from 55a13ed to 70a1993 Compare November 17, 2026 11:31
@asaiyume

Copy link
Copy Markdown
Author

@DN6 The PR is reworked based on your guidance. The reproduction script is also updated for testing. The current quantizer only receives per-layer markers .comfy_quant, so pre-quant models using _quantization_metadata might not work directly and requires a adapter.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation fixes-issue no-issue-needed for PRs that do not require link to an issue performance Anything related to performance improvements, profiling and benchmarking quantization size/L PR with diff > 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

convrot for int8 and fp4

4 participants