Skip to content

KolorsPAGPipeline: load_lora_weights raises AttributeError ('text_encoder_2') #15348

Closed

Description

@chsu-amd

Describe the bug

KolorsPAGPipeline (pipeline_pag_kolors.py) inherits StableDiffusionXLLoraLoaderMixin, which also loads into text_encoder_2. Kolors has a single text encoder, so load_lora_weights raises AttributeError on self.text_encoder_2, even for a UNet-only LoRA.

KolorsPipeline had the same problem and was switched to StableDiffusionLoraLoaderMixin in #11198; the PAG pipeline was not updated. The fix is the same mixin switch. I have it with a LoRA test class for Kolors PAG and am happy to open a PR.

KolorsImg2ImgPipeline has the same problem; it is already listed in #13612 (Issue 6), so it is left out here.

Reproduction

Tiny random components and a random UNet LoRA; runs on CPU:

import torch
from peft import LoraConfig
from peft.utils import get_peft_model_state_dict
from transformers import CLIPImageProcessor

from diffusers import AutoencoderKL, EulerDiscreteScheduler, KolorsPAGPipeline, UNet2DConditionModel
from diffusers.pipelines.kolors import ChatGLMModel, ChatGLMTokenizer


def tiny_unet():
    # Tiny random Kolors UNet (same layout as tests/pipelines/pag/test_pag_kolors.py)
    torch.manual_seed(0)
    return UNet2DConditionModel(
        block_out_channels=(2, 4),
        layers_per_block=2,
        sample_size=32,
        in_channels=4,
        out_channels=4,
        down_block_types=("DownBlock2D", "CrossAttnDownBlock2D"),
        up_block_types=("CrossAttnUpBlock2D", "UpBlock2D"),
        attention_head_dim=(2, 4),
        use_linear_projection=True,
        addition_embed_type="text_time",
        addition_time_embed_dim=8,
        transformer_layers_per_block=(1, 2),
        projection_class_embeddings_input_dim=56,
        cross_attention_dim=8,
        norm_num_groups=1,
    )


pipe = KolorsPAGPipeline(
    vae=AutoencoderKL(
        block_out_channels=[32, 64],
        down_block_types=["DownEncoderBlock2D"] * 2,
        up_block_types=["UpDecoderBlock2D"] * 2,
        latent_channels=4,
    ),
    text_encoder=ChatGLMModel.from_pretrained("hf-internal-testing/tiny-random-chatglm3-6b", dtype=torch.float32),
    tokenizer=ChatGLMTokenizer.from_pretrained("hf-internal-testing/tiny-random-chatglm3-6b"),
    unet=tiny_unet(),
    scheduler=EulerDiscreteScheduler(),
    feature_extractor=CLIPImageProcessor(size=32, crop_size=32),
    pag_applied_layers=["mid"],
)

# A random UNet LoRA in the diffusers format
unet = tiny_unet()
unet.add_adapter(LoraConfig(r=4, target_modules=["to_q", "to_k", "to_v", "to_out.0"], init_lora_weights=False))
lora = {f"unet.{k}": v for k, v in get_peft_model_state_dict(unet).items()}

pipe.load_lora_weights(lora, adapter_name="test")
print(f"load_lora_weights: OK, adapters {pipe.get_list_adapters()}")

Context

I was running documented features (PAG, IP-Adapter, LoRA) across the Kolors pipelines on current main, looking for bugs in the library with an AI agent (Claude Code). The script above was run as shown, on main (2bb9468) and with the fix, on a machine with an RTX 5090 (the script itself runs on CPU), with PYTHONNOUSERSITE=1 and HF_HOME set to a separate cache. The logs below are its output on main; with the fix it prints load_lora_weights: OK, adapters {'unet': ['test']}.

The same failure happens with the real weights on the RTX 5090 (Kwai-Kolors/Kolors-diffusers fp16, 1024x1024, pag_applied_layers=["mid"], pag_scale=1.5): load_lora_weights("khadim-hussain/spacevision-kolors-lora") fails with the same AttributeError.

Logs

Traceback (most recent call last):
  File "C:\Temp\dfbug\final5090\repro_kolors_pag_lora.py", line 52, in <module>
    pipe.load_lora_weights(lora, adapter_name="test")
  File "C:\temp\dfbug\wt_final_main\src\diffusers\utils\peft_utils.py", line 376, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\temp\dfbug\wt_final_main\src\diffusers\loaders\lora_pipeline.py", line 654, in load_lora_weights
    text_encoder=self.text_encoder_2,
                 ^^^^^^^^^^^^^^^^^^^
  File "C:\temp\dfbug\wt_final_main\src\diffusers\configuration_utils.py", line 175, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'KolorsPAGPipeline' object has no attribute 'text_encoder_2'. Did you mean: 'text_encoder'?

System Info

  • 🤗 Diffusers version: 0.44.0.dev0
  • Platform: Windows-11-10.0.26200-SP0
  • Running on Google Colab?: No
  • Python version: 3.12.16
  • PyTorch version (GPU?): 2.14.1+cu130 (True)
  • Huggingface_hub version: 1.32.0
  • Transformers version: 5.16.1
  • Accelerate version: 1.9.0
  • PEFT version: 0.21.3
  • Safetensors version: 0.8.0
  • xFormers version: not installed
  • Accelerator: NVIDIA GeForce RTX 5090, 32607 MiB
  • Using GPU in script?: No (the script runs on CPU)
  • Using distributed or parallel set-up in script?: No

Who can help?

@sayakpaul

Activity

  1. sayakpaul commented on Nov 19, 2026

    @sayakpaul
    Member

    Same as the other one. A PR is appreciated :)

  2. chsu-amd commented on Nov 19, 2026

    @chsu-amd
    ContributorAuthor

    Thanks! The fix is ready and verified: with it, the snippet above prints load_lora_weights: OK, and the Kolors PAG LoRA tests pass. PR: #15350

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions