Repository navigation
Conversation
…tead of overwriting
| from ..testing_utils import require_torch_accelerator, torch_device | ||
|
|
||
|
|
||
| class TestAssignComponentsToDevices: |
There was a problem hiding this comment.
I think we should find a more general way to test it instead of using the private methods.
There was a problem hiding this comment.
Ahh, yeah, makes sense, thanks.
I replaced them with one test that goes through from_pretrained: it loads the tiny SD pipeline with device_map="balanced" and a max_memory budget that only fits the text encoder, then checks that unet and vae are both in hf_device_map under "cpu" and that inference runs. On main it fails with KeyError: 'unet'.
It needs an accelerator, so it's skipped on the CPU jobs. I ran it on CUDA and MPS.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Fixes #15204
With
device_map="balanced", each component that didn't fit on an accelerator overwrote the"cpu"entry in_assign_components_to_devices. Only the last one stayed in the map. The dropped components then loaded as plain CPU modules without accelerate's offload hooks, so inference failed with a device mismatch (repro in the linked issue).This PR appends to the
"cpu"entry instead, the same way the accelerator entries are handled a few lines below.Behavior change: if every component overflows,
hf_device_mapnow has more than one entry._is_pipeline_device_mapped()then returnsTrue, so.to()/enable_model_cpu_offload()ask forreset_device_map()first. That's the same as any other balanced map. Before, the single-entry map slipped past that check.Tests
Added
TestAssignComponentsToDevicestotests/pipelines/test_pipeline_utils.py(CPU-only, no downloads).End-to-end repro from the issue (SD 1.5, fp16,
max_memory={0: "700MB"}, NVIDIA B200):Same result on Apple Silicon (MPS).
ruff checkandruff format --checkpass on the changed files.Self-review notes
Ran the
self-reviewskill against.ai/references/review-rules.mdandtesting.md. No blocking issues and no dead code.Left for the reviewer:
device_id_component_mapping.setdefault("cpu", []).append(component)would be one line shorter. I kept theif/elseso it matches the accelerator branch right below it.Before submitting
self-reviewskill on the diff?@sayakpaul