Skip to content

[MiniMax-H3, Pika H3 Flex] Resample reference soundtracks without torchaudio - #15199

Open
christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_audio_resample_without_torchaudio
Open

christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_audio_resample_without_torchaudio

Conversation

@christopher5106

@christopher5106 christopher5106 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #15245

The ref2va setup steps of MiniMax-H3 (MiniMaxH3Ref2VASetupStep) and Pika H3 Flex (PikaH3FlexRef2VASetupStep) resample a reference soundtrack onto the audio VAE's sample rate with torchaudio.transforms.Resample. torchaudio is not a diffusers dependency, so a reference soundtrack at any other rate than the VAE's 32 kHz (the 44.1 / 48 kHz audio track of most videos) fails:

ImportError: Resampling a MiniMax-H3 reference soundtrack from 48000 Hz to 32000 Hz needs `torchaudio`. Pass a waveform already at the audio VAE's sample rate to do without it.

Installing torchaudio doesn't help everywhere: it has no release for recent torch versions (the last on PyPI is 2.11.1).

This PR adds diffusers.utils.torch_utils.resample_audio, a plain-PyTorch port of torchaudio's default band-limited resampler (Hann-windowed sinc, lowpass_filter_width=6, rolloff=0.99), and uses it in both steps. Its output matches torchaudio.transforms.Resample to float32 rounding: identical at 48 → 32 kHz, within 4e-5 at 44.1 kHz. torchaudio's own transforms.Resample and functional.resample differ from each other by about as much.

echo still imports torchaudio, because it also needs MelSpectrogram, so this PR doesn't change it.

Tests

  • tests/others/test_utils.py::TestResampleAudio: parity with torchaudio.transforms.Resample for 48→32, 44.1→32, 16→48 and 22.05→44.1 kHz (skipped without torchaudio), a 1 kHz tone keeping its frequency through 48 → 32 kHz, and checks on shape, dtype, the same-rate identity and the integer-input error.
  • tests/modular_pipelines/minimax_h3: _normalize_audio_condition resamples a 48 kHz mono reference with torchaudio blocked in sys.modules.

The full minimax_h3 and pika_h3_flex suites pass on CPU, except 4 Pika H3 Flex ref2va tests (test_inference_is_not_nan, test_save_from_pretrained, and the two auto_cpu_offload ones), which fail the same way on main without this change.

Before submitting

…chaudio

The ref2va setup steps of MiniMax-H3 and Pika H3 Flex resample a reference
soundtrack onto the audio VAE's sample rate with `torchaudio.transforms.Resample`,
an optional dependency that is not installed with diffusers. Any soundtrack at
another rate than the VAE's (e.g. the 44.1 / 48 kHz track of most videos) fails
with an ImportError unless torchaudio is installed, and torchaudio has no release
for recent torch versions.

Add `diffusers.utils.torch_utils.resample_audio`, a plain-PyTorch port of
torchaudio's default windowed-sinc resampler (`sinc_interp_hann`,
`lowpass_filter_width=6`, `rolloff=0.99`), and use it in both steps. It matches
`torchaudio.transforms.Resample` to float32 rounding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Hi @christopher5106, thanks for the PR! It does not appear to link an issue it fixes.

  • If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked.
  • If you are the author of the model/pipeline this PR relates to, or part of the team we work with on it, tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box in the PR description (if you removed the template, paste that line back, checked). This reminder then no longer applies.
  • Otherwise there is nothing you need to do: a maintainer will look at the PR and add the no-issue-needed label to silence this reminder if it should stay open without an issue.

See the contribution guide for more details.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an existing issue, the author box is ticked, or the no-issue-needed label is added, you can ignore this message — it stays here as a comment, but it no longer applies.

@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically closed because it does not link an issue and the reminder above was not addressed within 10 days. If this PR is still relevant, please link the issue it fixes (e.g. Fixes #1234), tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box if it applies to you, or ask a maintainer to add the no-issue-needed label, and it can be reopened.

We are experimenting with this process to keep the review queue manageable, and it will sometimes get it wrong. If you think this PR should stay open, please just say so here and we will reopen it — no need to justify it at length.

Thanks again for contributing, and sorry for the noise if we closed this by mistake!

@github-actions github-actions Bot closed this Nov 18, 2026
@christopher5106

christopher5106 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

@sayakpaul this PR should not have been closed, because it links an issue

@sayakpaul sayakpaul reopened this Nov 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically closed because it does not link an issue and the reminder above was not addressed within 10 days. If this PR is still relevant, please link the issue it fixes (e.g. Fixes #1234), tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box if it applies to you, or ask a maintainer to add the no-issue-needed label, and it can be reopened.

We are experimenting with this process to keep the review queue manageable, and it will sometimes get it wrong. If you think this PR should stay open, please just say so here and we will reopen it — no need to justify it at length.

Thanks again for contributing, and sorry for the noise if we closed this by mistake!

@github-actions github-actions Bot closed this Nov 19, 2026
@christopher5106

Copy link
Copy Markdown
Contributor Author

@sayakpaul second closing 😂

@yiyixuxu yiyixuxu reopened this Nov 20, 2026
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yiyixuxu

Copy link
Copy Markdown
Collaborator

hey @christopher5106
thanks for the PR! however I don't think we want to host a copy of torchaudio's resampler in diffusers

for this, I think it should work on recent torch version now

Installing torchaudio doesn't help everywhere: it has no release for recent torch versions (the last on PyPI is 2.11.1).

let me know if it's not the case, otherwise we will close the PR and issue if it works when user install torchaudio

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MiniMax-H3, Pika H3 Flex] ref2va fails on a reference soundtrack that is not at 32 kHz without torchaudio

4 participants