You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Group offloading to disk leaves buffers read without a forward call uninitialized (FLUX.2 Klein all-black image) #15332
With group offloading to disk (offload_to_disk_path), buffers that a pipeline reads directly, without calling the owning module's forward, hold uninitialized values. Flux2KleinPipeline reads vae.bn.running_mean / vae.bn.running_var to scale the latents, so with the VAE group-offloaded to disk the output image is all black. With CPU-memory offloading the same repro gives an identical image.
_offload_to_disk replaces every offloaded tensor with torch.empty_like(...). That is fine for tensors the hooks onload before use, but vae.bn's forward is never called, so its buffers are never onloaded and the pipeline reads the uninitialized CPU tensors. Memory offloading keeps the values (it moves the tensors to the CPU, or restores pinned CPU copies with streams). The same direct reads are in pipeline_flux2.py, pipeline_flux2_klein*.py, pipeline_ernie_image.py, pipeline_bria_fibo2.py, pipeline_ideogram4.py and the flux2 / flux3 / ernie_image / ideogram4 modular blocks.
I have a fix (keep buffer values on the offload device in _offload_to_disk, as memory offloading does) with a regression test and can open a PR.
Reproduction
importtempfileimportnumpyasnpimporttorchfromdiffusersimportFlux2KleinPipelinepipe=Flux2KleinPipeline.from_pretrained("black-forest-labs/FLUX.2-klein-4B", torch_dtype=torch.bfloat16)
pipe.to("cuda")
kwargs=dict(
prompt="A cat holding a sign that says hello world",
height=1024,
width=1024,
guidance_scale=1.0,
num_inference_steps=4,
)
ref=pipe(**kwargs, generator=torch.Generator("cpu").manual_seed(0)).images[0]
running_mean=pipe.vae.bn.running_mean.clone()
pipe.vae.to("cpu")
pipe.vae.enable_group_offload(
onload_device=torch.device("cuda"),
offload_device=torch.device("cpu"),
offload_type="leaf_level",
offload_to_disk_path=tempfile.mkdtemp(),
)
out=pipe(**kwargs, generator=torch.Generator("cpu").manual_seed(0)).images[0]
print("bn.running_mean unchanged:", torch.equal(pipe.vae.bn.running_mean.to("cuda"), running_mean))
diff=np.abs(np.asarray(ref, dtype=np.int16) -np.asarray(out, dtype=np.int16))
print(f"image vs. no offload: max abs diff {diff.max()}, mean abs diff {diff.mean():.2f}")
ref.save("ref.png")
out.save("vae_disk_offload.png")
Logs
bn.running_mean unchanged: False
image vs. no offload: max abs diff 255, mean abs diff 149.37
vae_disk_offload.png is all black. With the fix the output is identical to ref.png (max abs diff 0). Checking only the bn buffers after enable_group_offload on main: leaf_level and block_level, with and without use_stream, are all wrong with offload_to_disk_path and all correct with CPU-memory offloading.
Activity