Skip to content

PRX pipelines ignore the dict returned by callback_on_step_end #15355

Open

Description

@Toraastrta

Describe the bug

PRXPipeline and PRXPixelPipeline call callback_on_step_end but throw away the dict it returns:

callback_on_step_end(self, i, t, callback_kwargs)

(pipeline_prx.py around line 806, pipeline_prx_pixel.py around line 642). So a callback that changes latents has no effect. Every other pipeline pops the returned keys back, e.g. latents = callback_outputs.pop("latents", latents).

The test suite already knows about this: tests/pipelines/prx/test_pipeline_prx.py and test_pipeline_prx_pixel.py mark test_callback_inputs with a strict xfail (CALLBACK_OUTPUTS_IGNORED), with the note that "whoever adds the pop-back will see it XPASS and can drop this marker".

I'd like to open a PR for this: assign the callback output and pop latents back in both pipelines (+4 lines), and remove the two xfail markers. I left prompt_embeds out on purpose: it's in _callback_tensor_inputs, but it isn't read inside the denoising loop after the cross-attention embeddings are built, so popping it would change nothing.

Reproduction

The existing test, with the xfail marker ignored:

pytest tests/pipelines/prx -k callback --runxfail

On main (4f78832):

FAILED tests/pipelines/prx/test_pipeline_prx.py::TestPRXPipeline::test_callback_inputs
FAILED tests/pipelines/prx/test_pipeline_prx_pixel.py::TestPRXPixelPipeline::test_callback_inputs
2 failed, 2 passed, 73 deselected

The failing assertion is assert output.abs().sum() == 0: the test's callback sets latents to zeros on the last step, but the output isn't zero because the edit is dropped.

With the fix, pytest tests/pipelines/prx gives 61 passed, 16 skipped.

I verified this with the tiny components from the test suite only; I didn't run the Photoroom/prx-512-t2i-sft checkpoint.

Logs

No response

System Info

  • 🤗 Diffusers version: 0.44.0.dev0 (main @ 4f78832)
  • Platform: macOS-26.6.2-arm64-arm-64bit
  • Python version: 3.12.16
  • PyTorch version (GPU?): 2.15.0 (False)
  • Huggingface_hub version: 2.4.0
  • Transformers version: 5.23.0
  • Accelerate version: 1.16.0

Who can help?

@DN6 @yiyixuxu

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions