Repository navigation
Conversation
|
Hi @asaiyume, thanks for the PR! It does not appear to link an issue it fixes.
See the contribution guide for more details. Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice. Once the PR links an existing issue, the author box is ticked, or the |
sayakpaul
left a comment
There was a problem hiding this comment.
Thanks a lot for the PR. It mostly looks quite clean.
|
|
||
| def to_diff_dict(self): | ||
| default = ConvRotQuantizationConfig().to_dict() | ||
| return {key: value for key, value in self.to_dict().items() if key == "quant_method" or value != default[key]} |
There was a problem hiding this comment.
When can value != default[key] arise?
|
Also added support for loading int8 VAE, which decreased VRAM usage by 50% and improves video decoding speed. |
|
Just noticed #14705 , should the PR support ComfyQuantizationConfig() instead, since int8 w8a8 + cr is just a subset of it? @sayakpaul @DN6 Appreciate any directions or guidance on this one. |
I think this sounds quite reasonable. We could also do something like the following:
Obviously, it depends on the amount of work and reviewing needed but my hunch says doing it the above way will keep things more manageable. |
Definitely, I will try to keep it small, then extend model supports & other quant variants. |
|
@sayakpaul Appreciate your help reviewing this when you have a moment so we can get this base merged. Currently supports QI2.1, Z-Image & MMH3 (pruned + VAE) loading using ComfyInt8Linear. Also updated usage script for quick reproductions. Remaining works (not in this PR scope):
|
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. Pages changed in this PR: |
sayakpaul
left a comment
There was a problem hiding this comment.
Thanks for working on this! I left some comments. I also ran the tests on an H100 and they pass.
|
@sayakpaul gentle ping when you have a moment |
|
@asaiyume I think this can be expanded to support the full set of Comfy Quants via Comfy Kitchen.
We also want to avoid quantizer-specific logic in the model loading utilities and in the model files. It should live in the quantizer. For example, Can we also move the MiniMax-H3 changes to a separate PR that covers pruned checkpoint support ( |
|
Thanks, that was very insightful! I will work on that direction. |
55a13ed to
70a1993
Compare
|
@DN6 The PR is reworked based on your guidance. The reproduction script is also updated for testing. The current quantizer only receives per-layer markers |






What does this PR do?
Adds prequantized Comfy checkpoint loading through
from_single_fileandComfyQuantizationConfig, with per-layer format detection and a shared Comfy-Kitchen linear runtime. Quantization metadata stays aligned when checkpoint conversion splits or concatenates weights.Fixes #14403 & #14705.
Usage
pip install "comfy-kitchen>=0.2.41" accelerateExample
Before submitting
self-reviewskill on the diff?