You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[docs] Context parallelism guide only parallelizes transformer; Wan 2.2 A14B's transformer_2 stays unsplit #15317
Is your feature request related to a problem? Please describe.
Wan 2.2 A14B pipelines (WanPipeline, WanImageToVideoPipeline) have two transformers. transformer runs the high-noise steps and transformer_2 runs the low-noise steps. Every example in the context parallelism guide calls pipeline.transformer.enable_parallelism(...), and the guide doesn't say that enable_parallelism only applies to the model it's called on. If you follow it, the transformer_2 steps are not split, and every GPU runs them in full. With the default 40 steps and boundary_ratio=0.9, transformer_2 runs 29 of the 40 steps. The output is still correct, so nothing tells the user that most of the speedup is missing.
Full pipeline call, Wan-AI/Wan2.2-I2V-A14B-Diffusers, bf16, 480x832, 33 frames, 40 steps, Ulysses degree 2, 2 GCDs of one AMD MI250 (ROCm, perf level auto). The two configs were interleaved (A B A B).
CP enabled on
run 1
run 2
transformer only (current docs)
365.1 s
363.8 s
transformer and transformer_2
229.4 s
237.0 s
Describe the solution you'd like.
Add a short note to the context parallelism guide, docs/source/en/optimization/context_parallelism.md, right after the Ring Attention example, which every other variant section points back to. It explains that enable_parallelism only applies to the model it's called on, and shows how to enable it on both Wan 2.2 A14B transformers:
This is a docs-only change. I have it ready and can open a PR if the maintainers agree with the approach.
Describe alternatives you've considered.
A pipeline-level enable_parallelism() that applies the config to every model with a _cp_plan, similar to enable_group_offload. That adds a new public method to DiffusionPipeline, so I'd leave that decision to the maintainers and start with the docs.
Additional context.
I was testing context parallelism on popular multi-GPU workloads on AMD MI250 when I found this. LoRA (#12074, load_into_transformer_2) and MagCache (#14025) also had to handle Wan 2.2's second transformer.
I think context parallelism is an advanced topic anyway and it's not unreasonable to expect the users to have some background of it + how it's integrated in the library. So, I don't think this would add much value but I will defer this @stevhliu.
I think it's worth a short note, since the issue is more about the API scope than CP background, and its relatively cheap to do so, but I'd make it a bit more general rather than about Wan.
Thanks @stevhliu and @sayakpaul! Based on @stevhliu's suggestion, I opened #15334. It adds a short, general note after the Ring Attention example: enable_parallelism only applies to the model it's called on, so call it on each transformer when a pipeline has more than one. Could you please take a look when you get a chance? Happy to change the wording or placement if needed. Thank you!
Activity
sayakpaul commentedon Nov 17, 2026
I think context parallelism is an advanced topic anyway and it's not unreasonable to expect the users to have some background of it + how it's integrated in the library. So, I don't think this would add much value but I will defer this @stevhliu.
stevhliu commentedon Nov 17, 2026
I think it's worth a short note, since the issue is more about the API scope than CP background, and its relatively cheap to do so, but I'd make it a bit more general rather than about Wan.
menglcai commentedon Nov 18, 2026
Thanks @stevhliu and @sayakpaul! Based on @stevhliu's suggestion, I opened #15334. It adds a short, general note after the Ring Attention example:
enable_parallelismonly applies to the model it's called on, so call it on each transformer when a pipeline has more than one. Could you please take a look when you get a chance? Happy to change the wording or placement if needed. Thank you!