From 1af8589c53acabb9d23bce4bf95b9927cd5cfd07 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?apolin=C3=A1rio?= Date: Tue, 17 Nov 2026 09:34:41 -0300 Subject: [PATCH 1/2] Add Generative Media Spec docs page and link it from serve/CLI docs --- docs/source/en/_toctree.yml | 2 + docs/source/en/using-diffusers/cli.md | 4 +- docs/source/en/using-diffusers/gms.md | 72 +++++++++++++++++++++++++ docs/source/en/using-diffusers/serve.md | 4 +- 4 files changed, 78 insertions(+), 4 deletions(-) create mode 100644 docs/source/en/using-diffusers/gms.md diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index 628f2f6ff636..0e6a86393f09 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -9,6 +9,8 @@ title: Command line interface - local: using-diffusers/serve title: Serving + - local: using-diffusers/gms + title: Generative Media Spec title: Get started - isExpanded: true sections: diff --git a/docs/source/en/using-diffusers/cli.md b/docs/source/en/using-diffusers/cli.md index 7dd6de84ca68..b962fe57bedd 100644 --- a/docs/source/en/using-diffusers/cli.md +++ b/docs/source/en/using-diffusers/cli.md @@ -22,7 +22,7 @@ installed as `diffusers`, so `diffusers serve` and `diffusers-cli serve` are the | [`env`](#env) | Print environment info for bug reports. | | [`schema`](#schema) | Inspect a pipeline's `__call__` signature without downloading weights. | | [`run`](#run) | Run a pipeline locally or in a Hugging Face Sandbox. | -| [`serve`](#serve) | Serve a pipeline over HTTP with the Generative Media Spec API. | +| [`serve`](#serve) | Serve a pipeline over HTTP with the [Generative Media Spec](./gms) API. | | [`custom_blocks`](#customblocks) | Package a local `ModularPipelineBlocks` subclass for the Hub. | | [`fp16_safetensors`](#fp16safetensors) | Convert a checkpoint to fp16 `.safetensors`. | | [`skills`](#skills) | Install pre-authored skill bundles into your AI coding agent. | @@ -291,7 +291,7 @@ hf sandbox kill ## `serve` -Serve a pipeline over HTTP with the Generative Media Spec API. See [Serving](./serve) for the full guide. +Serve a pipeline over HTTP with the [Generative Media Spec](./gms) API. See [Serving](./serve) for the full guide. ```bash pip install "diffusers[serve]" diff --git a/docs/source/en/using-diffusers/gms.md b/docs/source/en/using-diffusers/gms.md new file mode 100644 index 000000000000..f6c626df38d0 --- /dev/null +++ b/docs/source/en/using-diffusers/gms.md @@ -0,0 +1,72 @@ + + +# Generative Media Spec + +The [Generative Media Spec](https://github.com/huggingface/gms-alpha) (GMS) is an open HTTP API for serving image, +video, and audio generation models. Any GMS client works with any GMS server, and a model exposes the same interface +everywhere it is hosted. [`diffusers serve`](./serve) is a GMS server. + +> [!TIP] +> GMS is a draft (`gms/0.1`). The full specification, the [OpenAPI definition](https://github.com/huggingface/gms-alpha/blob/main/openapi.yaml) +> and the [manifest JSON Schema](https://github.com/huggingface/gms-alpha/blob/main/schemas/manifest.schema.json) +> live in the [GMS repository](https://github.com/huggingface/gms-alpha). Feedback is welcome there. + +Every request follows one loop: + +1. **Discover.** `GET /v1/models/{id}` returns the model's manifest: its tasks, inputs, outputs, parameters and + presets, plus what the server supports. +2. **Generate.** `POST /v1/generations` creates a generation from a model, its `inputs` and its `parameters`. +3. **Follow along.** A generation goes from `queued` to `generating` to `complete`. Poll it, or subscribe to + `GET /v1/generations/{id}/events` for progress and previews. +4. **Read the results.** The completed generation lists its named outputs, the seed that was used, and the model + revision that ran. + +## Manifests + +A manifest is the machine-readable description of one model. Clients render interfaces and validate requests from it, +so a new model works in an existing client without client changes. + +Each task declares its parameters in one flat, typed map. A small core vocabulary has the same meaning in every +manifest, and everything else is model-specific. `serve` maps the core names to pipeline arguments: + +| GMS name | Pipeline argument | +|---|---| +| `steps` | `num_inference_steps` | +| `seed` | `generator` | +| `num_outputs` | `num_images_per_prompt`, `num_videos_per_prompt`, `num_waveforms_per_prompt` | +| `fps` | `fps`, `frame_rate` | +| `duration` | `num_frames`, converted from seconds with the manifest's frame lattice | +| `mask` (input) | `mask_image` | + +The manifest travels with the model: canonically, a `gms.json` file in the model repository. A server SHOULD serve a +published manifest as is, so the same model reads the same on every host. `serve` loads it automatically, and falls +back to a manifest derived from the pipeline's `__call__` signature when the repository has none. + +## Publishing a manifest + +A derived manifest has types and defaults, but no ranges, presets, or durations in seconds. Write a `gms.json` to +add them, and test it locally before uploading: + +```bash +diffusers serve black-forest-labs/FLUX.1-dev --manifest gms.json +``` + +`serve` checks the manifest against the pipeline at startup and refuses to start if it declares something the +pipeline can't do. Once it loads, upload it to the root of the model repository: + +```bash +hf upload black-forest-labs/FLUX.1-dev gms.json gms.json +``` + +See the [example manifests](https://github.com/huggingface/gms-alpha/tree/main/manifests) for FLUX.2, LTX 2.5 and +ACE-Step 1.5. diff --git a/docs/source/en/using-diffusers/serve.md b/docs/source/en/using-diffusers/serve.md index 8a24b66637f6..0e9eb04ffac3 100644 --- a/docs/source/en/using-diffusers/serve.md +++ b/docs/source/en/using-diffusers/serve.md @@ -12,7 +12,7 @@ specific language governing permissions and limitations under the License. # Serving -`diffusers serve` serves a pipeline over HTTP with the Generative Media Spec (GMS) API: clients discover what the +`diffusers serve` serves a pipeline over HTTP with the [Generative Media Spec](./gms) (GMS) API: clients discover what the model accepts, create a generation, follow it while it runs, and download the results. ```bash @@ -141,7 +141,7 @@ curl localhost:8000/v1/models/black-forest-labs/FLUX.1-dev ``` `serve` uses the file passed with `--manifest`, else a `gms.json` in the model repo, else a manifest derived from the -pipeline's `__call__` signature. Publish a `gms.json` to add value ranges, presets, or a duration in seconds. The +pipeline's `__call__` signature. [Publish a `gms.json`](./gms#publishing-a-manifest) to add value ranges, presets, or a duration in seconds. The server checks the manifest at startup and refuses to start if it declares something the pipeline can't do. `x-diffusers` maps a manifest's names to pipeline arguments, and `capabilities` lists what the server supports: From 49bf89bd8d8cfe25646519979c3c5b43383973e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?apolin=C3=A1rio?= Date: Tue, 17 Nov 2026 09:44:58 -0300 Subject: [PATCH 2/2] Say GMS is in alpha --- docs/source/en/using-diffusers/gms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/using-diffusers/gms.md b/docs/source/en/using-diffusers/gms.md index f6c626df38d0..487575f6dbcb 100644 --- a/docs/source/en/using-diffusers/gms.md +++ b/docs/source/en/using-diffusers/gms.md @@ -17,7 +17,7 @@ video, and audio generation models. Any GMS client works with any GMS server, an everywhere it is hosted. [`diffusers serve`](./serve) is a GMS server. > [!TIP] -> GMS is a draft (`gms/0.1`). The full specification, the [OpenAPI definition](https://github.com/huggingface/gms-alpha/blob/main/openapi.yaml) +> GMS is in alpha. The full specification, the [OpenAPI definition](https://github.com/huggingface/gms-alpha/blob/main/openapi.yaml) > and the [manifest JSON Schema](https://github.com/huggingface/gms-alpha/blob/main/schemas/manifest.schema.json) > live in the [GMS repository](https://github.com/huggingface/gms-alpha). Feedback is welcome there.