Skip to content

Standardize how audio pipelines expose their output sample rate #15341

Open

Description

@yiyixuxu

An audio pipeline returns a bare waveform, so anything that saves it has to find the sample rate somewhere else. Today each pipeline keeps it in a different place:

Where Pipelines
pipe.vocoder.config.output_sampling_rate LTX-2
pipe.vae.config.sampling_rate Stable Audio
pipe.audio_sampling_rate LTX-2, MiniMax-H3, Pika H3 Flex, Echo (modular)
pipe.sound_sampling_rate Cosmos (modular)
pipe.sampling_rate MiniMax Music 3 (modular)

The same name also doesn't always mean the same thing: in LTX-2, audio_sampling_rate is the audio VAE's rate (16000), while the returned waveform is at vocoder.config.output_sampling_rate (24000).

This makes generic tooling guess:

  • diffusers-cli run falls back to 16000 unless the user passes --sampling-rate.
  • diffusers serve (Add diffusers serve to CLI #15017) tries a list of six attribute paths in order.

Proposal

Pick one way for a pipeline to say what rate its output audio is at. Two options:

  1. A single property on every audio pipeline (e.g. audio_sampling_rate) that always means the rate of the returned waveform.
  2. Return the rate with the output: a sampling_rate field on the output class, or an output param next to audio for modular pipelines.

(1) is a small change and modular pipelines are most of the way there. (2) is more robust since the rate travels with the data.

Posted by Claude on behalf of @yiyixuxu. The text was drafted together and approved by yiyixuxu.

Activity

  1. yiyixuxu commented on Nov 19, 2026

    @yiyixuxu
    CollaboratorAuthor

    cc @DN6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions