Skip to content

[Quantization] Add support for Comfy-Kitchen / Comfy-Quants - #14747

Open
PrakshaaleJain wants to merge 10 commits into
huggingface:mainfrom
PrakshaaleJain:feat/comfy-quant-support
Open

[Quantization] Add support for Comfy-Kitchen / Comfy-Quants#14747
PrakshaaleJain wants to merge 10 commits into
huggingface:mainfrom
PrakshaaleJain:feat/comfy-quant-support

Conversation

@PrakshaaleJain

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR adds support for the Comfy Quant quantization toolkit by integrating its inference engine, Comfy-Kitchen, directly into Diffusers library.

Key additions:

  • Added ComfyQuantConfig and ComfyQuantizer to handle wrapping of tensors.
  • Safely intercepts weights via create_quantized_param and wraps them via QuantizedTensor.from_float to dispatch to Comfy-Kitchen's optimized GPU kernels.
  • Implemented robust format validation and auto-resolution of standard layout classes (fp8, int8, mxfp8, nvfp4, int4_svd, int4_awq).
  • Added full unit test coverage under tests/quantization/comfy_quant/.

Fixes #14705

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?
  • Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

AI Agent Self-Review Notes

Agent: Antigravity (Google DeepMind)
Review Focus: Diffusers Quantization Standards & Guidelines

  1. Backwards Compatibility:
    • Status: Passed. The ComfyQuantizer relies on is_comfy_kitchen_available(). If the backend is not installed, Diffusers will safely fall back to standard behavior or fail gracefully if ComfyQuantConfig is passed explicitly without the toolkit.
  2. Coding Style & Formatting:
    • Status: Passed. The code was fully run through make style && make fix-copies. utils/check_dummies.py successfully generated dummy_comfy_kitchen_objects.py and correctly registered the imports in `diffusers/__init
  3. Testing:
    • Status: Passed. Added tests/quantization/comfy_quant/test_comfy_quant.py. Tests cover both valid format wrapping (FP8 mapping to TensorCoreFP8Layout) and error handling for unsupported formats. Tests ran locally and passed with 100% success.

@github-actions github-actions Bot added documentation Improvements or additions to documentation quantization fixes-issue tests utils pipelines size/L PR with diff > 200 LOC labels Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation fixes-issue pipelines quantization size/L PR with diff > 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Quantization] Add support for Comfy Quants backend

1 participant