Skip to content

HunyuanVideo: lora_state_dict / load_lora_weights crash on musubi-tuner (kohya-style) LoRAs #15344

Closed

Description

@chsu-amd

Describe the bug

The LoRA used in the HunyuanVideo docs example (hunyuan_video.md, lucataco/hunyuan-steamboat-willie-10) cannot be loaded. It was trained with musubi-tuner, so its keys look like lora_unet_double_blocks_0_img_attn_qkv.lora_down.weight / .lora_up.weight / .alpha.

HunyuanVideoLoraLoaderMixin.lora_state_dict detects the original format with any("img_attn_qkv" in k ...), which also matches these keys, and sends them to _convert_hunyuan_video_lora_to_diffusers. That converter only knows the lora_A / lora_B layout: it calls chunk(3) on the 0-dim .alpha tensor and fails. The underscore-joined module names and lora_down / lora_up are not handled either.

Wan already handles the same musubi-tuner format with _convert_musubi_wan_lora_to_diffusers (added in #11243). A similar step for HunyuanVideo (rename the musubi keys to the original layout, fold alpha / rank into the weights, then reuse the existing converter) fixes it: the snippet below then prints 1040 keys, and each converted key matches the shape of its module in the full-size HunyuanVideoTransformer3DModel (checked on the meta device). Happy to open a PR with a test.

Open PR #15307 also edits _convert_hunyuan_video_lora_to_diffusers (final-layer scale/shift swap) and adds a test to TestHunyuanVideoTransformerLoRA. That is a different bug; the snippet below fails the same way on that PR's branch.

Related: #14088 lists HunyuanVideo kohya-format LoRAs as unsupported in its converter table; this is a concrete public LoRA (the one in the HunyuanVideo docs) that fails to load.

Reproduction

from diffusers import HunyuanVideoPipeline

# LoRA from the HunyuanVideo docs example; only the LoRA file is downloaded
state_dict = HunyuanVideoPipeline.lora_state_dict("lucataco/hunyuan-steamboat-willie-10")
print(len(state_dict), "keys")

pipeline.load_lora_weights("lucataco/hunyuan-steamboat-willie-10") fails with the same error. The docs pass the full https://huggingface.co/... URL, which fails earlier with HFValidationError; with the repo id it hits this error.

Context

I was running the documented examples of popular pipelines on current main, looking for bugs in the library with an AI agent (Claude Code); this is the LoRA example in the HunyuanVideo docs. The snippet above was run as shown, on main (2bb9468) and with the fix, on a machine with an RTX 5090 (the snippet itself runs on CPU), with PYTHONNOUSERSITE=1 and HF_HOME set to a separate cache. The logs below are its output on main.

Logs

Traceback (most recent call last):
  File "C:\Temp\dfbug\final5090\repro_hunyuan_musubi_lora.py", line 4, in <module>
    state_dict = HunyuanVideoPipeline.lora_state_dict("lucataco/hunyuan-steamboat-willie-10")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\miniforge3\envs\dfop-nv\Lib\site-packages\huggingface_hub\utils\_validators.py", line 91, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\temp\dfbug\wt_final_main\src\diffusers\loaders\lora_pipeline.py", line 3575, in lora_state_dict
    state_dict = _convert_hunyuan_video_lora_to_diffusers(state_dict)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\temp\dfbug\wt_final_main\src\diffusers\loaders\lora_conversion_utils.py", line 1760, in _convert_hunyuan_video_lora_to_diffusers
    handler_fn_inplace(key, converted_state_dict)
  File "C:\temp\dfbug\wt_final_main\src\diffusers\loaders\lora_conversion_utils.py", line 1630, in remap_img_attn_qkv_
    to_q, to_k, to_v = weight.chunk(3, dim=0)
                       ^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: chunk expects at least a 1-dimensional tensor

System Info

  • 🤗 Diffusers version: 0.44.0.dev0
  • Platform: Windows-11-10.0.26200-SP0
  • Running on Google Colab?: No
  • Python version: 3.12.16
  • PyTorch version (GPU?): 2.14.1+cu130 (True)
  • Huggingface_hub version: 1.32.0
  • Transformers version: 5.16.1
  • Accelerate version: 1.9.0
  • PEFT version: 0.21.3
  • Safetensors version: 0.8.0
  • xFormers version: not installed
  • Accelerator: NVIDIA GeForce RTX 5090, 32607 MiB
  • Using GPU in script?: No (the snippet runs on CPU)
  • Using distributed or parallel set-up in script?: No

Who can help?

@sayakpaul

Activity

  1. sayakpaul commented on Nov 19, 2026

    @sayakpaul
    Member

    Thanks for reporting. Do you want to contribute a fix?

  2. chsu-amd commented on Nov 19, 2026

    @chsu-amd
    ContributorAuthor

    Yes, thanks! The fix is ready and verified: with it, the snippet above prints 1040 keys (520 LoRA modules x lora_A/lora_B), every converted key matches its module shape in the full-size transformer, and the HunyuanVideo LoRA tests pass. I'm opening the PR now and will link it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions