Skip to content

Support mixed-type quantization within a single quantization backend #15320

Open

Description

@moon-bot-app

Motivation

Other tools like ComfyUI and Unsloth already support mixed-type quants: different layers or modules of a model are quantized with different schemes or precisions. Some layers stay in high precision, and others drop to lower bit-widths. This usually gets a better quality/memory trade-off than quantizing every layer the same way.

Most of the quantization backends Diffusers integrates with already support this natively. Diffusers should let users use that capability.

Proposal

  • Allow mixed quantization types within a single backend (for example, different torchao or bnb configs per module/layer pattern), when the underlying backend framework supports it.
  • Out of scope for now: mixing backends in one checkpoint (for example, some layers bnb and others torchao). Not all backends are feature-complete, so this would be messy. We also haven't seen such checkpoints in practice.

Open questions

  • API design: how users specify per-module/per-layer quant configs (pattern-based mapping, modules_to_not_convert-style extension, etc.).
  • Serialization and loading of mixed-quant checkpoints, plus compatibility with existing pre-quantized checkpoints (for example, from the ComfyUI / Unsloth ecosystems).
  • Which backends support this today, and what's missing in each.

Benchmarks / guidance

As part of this work, produce:

  • Benchmarks (memory, latency) comparing uniform vs. mixed quantization on representative pipelines.
  • Qualitative comparisons of outputs.
  • Better-grounded heuristics for choosing which layers to keep in higher precision. Existing community guides give heuristics, but they're hit and miss.

Requested by Sayak Paul - Slack thread - Agent trace

Activity

  1. self-assigned this
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions