@@ -11,18 +12,18 @@ keywords:
-Manifest ships as a verified n8n community node, [`n8n-nodes-manifest`](https://www.npmjs.com/package/n8n-nodes-manifest). It installs from n8n's own **Community Nodes** screen on n8n Cloud and on self-hosted n8n alike, and gives you two nodes:
+The gateway ships as a verified n8n community node, [`n8n-nodes-manifest`](https://www.npmjs.com/package/n8n-nodes-manifest). It installs from n8n's own **Community Nodes** screen on n8n Cloud and on self-hosted n8n alike, and gives you two nodes:
@@ -12,48 +13,48 @@ keywords:
-Manifest is an open-source LLM router for agents and AI apps. Connect the providers you already use — subscriptions like ChatGPT or Claude, pay-per-token API keys, local models, and custom endpoints — and route every request through one OpenAI- and Anthropic-compatible URL.
+The Manifest LLM Gateway connects the models and subscriptions you already use behind one OpenAI- and Anthropic-compatible endpoint.
-Send `auto` as the model and Manifest routes each request by the rules you set, with fallbacks when one fails. Pin a real model ID instead, and the request goes straight to that model, with no routing and no fallback. Either way, you cap what each harness spends and see the cost of every request.
+Send `auto` to apply your routing rules and fallbacks, or name a model to send the request there directly. Either way, you can cap what each harness spends and see the cost of every request.
## Key features
+
```bash
@@ -72,12 +73,12 @@ curl -X POST https://app.manifest.build/v1/chat/completions \
-d '{"model": "auto", "messages": [{"role": "user", "content": "Hello"}]}'
```
-`auto` asks Manifest to route the request. The request appears on the Requests page with the model that served it and its cost.
+`auto` asks the gateway to route the request. The request appears on the **Requests** page with the model that served it and its cost.
-Install Manifest first: [Self-hosted](/self-hosted) covers Docker and the managed platforms. Your instance starts empty. [First request](/self-hosted#first-request) takes you from there to a routed request, against `http://localhost:2099/v1`.
+Install the gateway first. [Self-hosted](/llm-gateway/docs/self-hosted) covers Docker and the managed platforms. Your instance starts empty. [First request](/llm-gateway/docs/self-hosted#first-request) takes you from there to a routed request against `http://localhost:2099/v1`.
diff --git a/llm-gateway.mdx b/llm-gateway.mdx
index db2ec17..0d2b215 100644
--- a/llm-gateway.mdx
+++ b/llm-gateway.mdx
@@ -1,30 +1,31 @@
---
title: "LLM Gateway"
-description: "Every request passes through the gateway: limits check it, routing picks the model, and fallback catches it when that model fails."
+description: "Route requests across models, enforce limits, and use fallbacks when a provider fails."
icon: "split"
keywords:
["LLM gateway", "model routing", "LLM router", "fallback chain", "hard limit", "custom tier", "default model", "fallback exhausted", "429 too many requests"]
+canonical: "https://manifest.build/llm-gateway/docs/llm-gateway"
---
-Point your client at Manifest instead of a provider, and every request goes through the gateway. Three things happen there: limits decide whether the request runs at all, routing picks the model, and fallback steps in when that model fails.
+Point your client at the gateway instead of a provider, and every request goes through the gateway. Three things happen there: limits decide whether the request runs at all, routing picks the model, and fallback steps in when that model fails.
## How a request flows
+
+
+
+
+Some requests fail because of the request, not the model. A parameter the provider doesn't accept, a tool schema in the wrong shape, a model name that moved. Switching to a fallback model doesn't help, because the next model rejects the same body.
+
+Autofix handles that case. When a request fails with a repairable error, the gateway sends the failed request and the provider's error to a healing service, gets a corrected request back, and sends it once more. It runs **before** the fallback chain, so fallback is still the safety net if the repair doesn't land.
+
+## When it runs
+
+Autofix only touches request-side 4xx errors, where the provider is telling you the request was wrong:
+
+| Status | Meaning |
+| ------ | ---------------------------------------------- |
+| `400` | Bad request, usually a malformed parameter |
+| `404` | Not found, usually a model name that moved |
+| `422` | Unprocessable, usually a schema the API rejects |
+
+It deliberately skips `401`, `403`, `429`, and every `5xx`. A bad key, a rate limit, or a provider outage isn't something a rewritten body can fix, so those go straight to [fallback](/llm-gateway/docs/llm-gateway#fallback).
+
+Renamed models are covered through the provider itself. When a request names a model the gateway hasn't catalogued but the provider and credentials are unambiguous, the gateway forwards it anyway. The provider answers with its real error, typically a `404` for a model that no longer exists, and that error goes through the normal Autofix path. A patch that corrects the model name serves the request.
+
+## How it works
+
+
+
+Nothing to deploy or configure: your install talks to the hosted Manifest Autofix service directly.
+
+
+
+### When you create a harness
+
+The Connect Harness modal shows an Autofix toggle under the name field, pre-set to on. You decide there, and the choice is stored on the harness. This works the same on Cloud and self-hosted.
+
+
+ 
+
+
+### Harnesses you already have
+
+A harness that existed before Autofix keeps its state: the gateway never flips the toggle for you.
+
+
+
+On LLM Gateway Cloud, those harnesses are already on. You can switch any of them off on its **Settings** page.
+
+
+
+
+
+Enable each one from its **Settings** page or from the sidebar card.
+
+
+
+The dashboard shows a sidebar card while any harness has Autofix off. It opens a modal listing those harnesses with individual toggles, and you can dismiss it for the browser session.
+
+
+ 
+
+
+
+ 
+
+
+## Your data
+
+
+
+Here is exactly what Autofix sends to the healing service, and when nothing is sent.
+
+
+
+
+
+Self-hosting is often a privacy choice, so here is exactly what leaves your machine, and when nothing does.
+
+The first time you enable Autofix, your install records a one-time consent: enabling it means you agree that failing requests are sent to the hosted healing service, as set out in the [terms](https://manifest.build/terms/). Any enable path records it. Your install identifies itself with an anonymous install id, the same id [telemetry](/llm-gateway/docs/reference/telemetry) uses.
+
+
+
+When Autofix repairs a request, the gateway sends the failed request, including its message content, plus the provider's error response to the healing service.
+
+Provider API keys and OAuth tokens are never sent: credentials travel in request headers, which are not part of the payload sent to the healing service. The provider's error response is scrubbed for secrets before it is sent.
+
+Nothing is sent for requests that succeed. No request data is ever sent while the toggle is off; the only call that can still happen is the boot health check, which carries no request content.
+
+
+
+Self-hosted deployments can also switch Autofix off globally, which stops every call to the healing service, including the health check at boot. See [Environment variables](#environment-variables).
+
+
+
+See the [privacy policy](https://manifest.build/privacy/) and the [terms](https://manifest.build/terms/).
+
+## Guardrails
+
+Autofix is built to stay out of the way when it isn't working:
+
+- **One attempt.** A patched request is sent once. It never loops.
+- **Failures degrade quietly.** If the healing service errors or times out, your agent gets the original provider error. Autofix never turns a provider's 400 into a gateway 500.
+- **A slow healer stops being asked.** After three consecutive transport failures, the gateway stops calling it for 30 seconds so a down service doesn't add latency to every failing request. One successful call clears the streak.
+- **Startup never waits.** In production, the gateway pings the healing service once at boot, fire-and-forget, and logs a warning if it's unreachable. It never blocks startup, and it's skipped entirely when Autofix is disabled globally.
+- **Fallback still runs.** Autofix sits in front of the chain, it doesn't replace it.
+
+## Seeing what it did
+
+Open any request in the dashboard's request log. Requests that went through Autofix carry a panel showing what changed and whether the retry succeeded. Only a successful patched retry counts as recovered by Autofix.
+
+
+ 
+
+
+The failed original and its patched retry are separate [provider attempts](/llm-gateway/docs/observability#requests-and-provider-attempts), so with [request logs](/llm-gateway/docs/request-logs) on you can read both bodies side by side and see exactly which field was rewritten.
+
+The **Overview** page aggregates the same data across a time range:
+
+| Card | What it measures |
+|---|---|
+| **Success rate** | Share of requests that ended successfully, however many attempts it took |
+| **Recovered requests** | Share of requests that failed at least one attempt and still succeeded |
+| **Recovered by Autofix** | Count of requests where a patched retry is what saved it |
+
+Both **Recovered** cards link through to the matching filter on the request log.
+
+
+
+## Environment variables
+
+
+ Self-hosted only. On [LLM Gateway Cloud](https://app.manifest.build) there is no
+ server to configure: everything Autofix related lives in the dashboard.
+
+
+Autofix needs no setup. Every variable below is optional and adjusts behavior:
+
+| Variable | Default | Description |
+| ----------------------------- | ------------- | ------------------------------------------------------ |
+| `AUTOFIX_GLOBAL_ENABLED` | `true` | Set `false` to turn Autofix off for the whole deployment. No call reaches the healing service, including the boot health check |
+| `AUTOFIX_TIMEOUT_MS` | `10000` | Timeout per healing call, in milliseconds |
+| `AUTOFIX_REPAIRABLE_STATUSES` | `400,404,422` | Which provider statuses are eligible for a repair |
+| `AUTOFIX_HEALING_API_KEY` | unset | Sent as `x-api-key` to the healing service. Only relevant when provided by the hosted healing service |
+| `AUTOFIX_REPORT_ALL_4XX` | `false` | Opt-in. Also report request-side 4xx errors (except `401`/`402`/`403`/`429`) from harnesses with Autofix on, as diagnostic evidence with the secret-scrubbed request body. Bodies over 256 KB are dropped, not truncated. No fix comes back from it |
+
+You don't need `AUTOFIX_HEALING_API_KEY`. With no key set, the install announces its anonymous install id instead. The full list of variables lives in [Environment variables](/llm-gateway/docs/reference/environment-variables).
+
+
+
+## Related
+
+- [LLM Gateway](/llm-gateway/docs/llm-gateway): where Autofix sits in the request flow
+- [Observability](/llm-gateway/docs/observability): finding failing requests in the log
+- [Request logs](/llm-gateway/docs/request-logs): reading the original and patched bodies
+- [Error codes](/llm-gateway/docs/errors)
diff --git a/llm-gateway/docs/deploy.mdx b/llm-gateway/docs/deploy.mdx
new file mode 100644
index 0000000..ec96add
--- /dev/null
+++ b/llm-gateway/docs/deploy.mdx
@@ -0,0 +1,98 @@
+---
+title: "Deploy the LLM Gateway"
+sidebarTitle: "Overview"
+description: "Run the Manifest LLM Gateway with Docker, on a managed platform, or in your cloud account."
+icon: "rocket"
+keywords:
+ [
+ "self-hosted gateway",
+ "Manifest deployment",
+ "Docker",
+ "Railway",
+ "Render",
+ "AWS",
+ "GCP",
+ "Fly.io",
+ "Easypanel",
+ "Coolify",
+ ]
+canonical: "https://manifest.build/llm-gateway/docs/deploy"
+---
+
+Everything on this page runs the same open-source gateway image on infrastructure you control, with your own PostgreSQL database. Pick the path that matches where you want it to live. If you'd rather not run anything yourself, [LLM Gateway Cloud](https://app.manifest.build) is the hosted alternative.
+
+## Your own machine
+
+
+
+
+
+
+The gateway ships as a verified n8n community node, [`n8n-nodes-manifest`](https://www.npmjs.com/package/n8n-nodes-manifest). It installs from n8n's own **Community Nodes** screen on n8n Cloud and on self-hosted n8n alike, and gives you two nodes:
+
+
+
+| Field | Value |
+| ------------ | --------------------------------------------------------------------- |
+| **Base URL** | `https://app.manifest.build` |
+| **API Key** | The harness key, starting with `mnfst_`, from the harness **Settings** page |
+
+
+
+
+
+| Field | Value |
+| ------------ | --------------------------------------------------------------------- |
+| **Base URL** | The origin of your instance, for example `http://192.168.1.20:2099` |
+| **API Key** | The harness key, starting with `mnfst_`, from the harness **Settings** page |
+
+
+
+The Base URL is the origin only, without `/v1`. The nodes append `/v1/models`, `/v1/chat/completions`, and `/v1/responses` themselves. A Base URL ending in `/v1` sends requests to `/v1/v1/...` and every call fails.
+
+Select **Test** on the credential. It calls `GET /v1/models`, so a passing test confirms that n8n can reach your instance and that the key is valid.
+
+
+
+### Reaching a self-hosted instance from n8n
+
+The credential test fails with a connection error when n8n cannot reach the gateway. Where the two run decides the address:
+
+- **n8n and the gateway in Docker on the same host.** `localhost` inside the n8n container is n8n itself. Put both containers on a shared Docker network and use the gateway service name as the host, or use `http://host.docker.internal:2099` on Docker Desktop.
+- **n8n on another machine.** The bundled compose file binds the gateway to `127.0.0.1` only. Set `HOST_BIND_ADDRESS=0.0.0.0` in the gateway's `.env`, as described in [Exposing on the LAN](/llm-gateway/docs/self-hosted#exposing-on-the-lan), so the port answers on the network.
+- **Plain `http://` is fine.** The calls are server to server. The dashboard's browser security rules do not apply to them.
+
+The keyless loopback shortcut that a development build of the gateway accepts does not exist in the production image. n8n always needs a real `mnfst_` key.
+
+
+
+## Use the gateway behind an AI Agent
+
+
+
+
+
+
+The Manifest LLM Gateway connects the models and subscriptions you already use behind one OpenAI- and Anthropic-compatible endpoint.
+
+Send `auto` to apply your routing rules and fallbacks, or name a model to send the request there directly. Either way, you can cap what each harness spends and see the cost of every request.
+
+## Key features
+
+
+
+Sign up at [app.manifest.build](https://app.manifest.build) and a guided setup walks you through it. Step one creates a harness. Then connect a provider, and give your default tier a model to route to. The last step shows the gateway address and the harness key, which starts with `mnfst_`. A harness you create later reuses the providers your workspace already has.
+
+
+
+
+
+```bash
+curl -X POST https://app.manifest.build/v1/chat/completions \
+ -H "Authorization: Bearer mnfst_YOUR_KEY" \
+ -H "Content-Type: application/json" \
+ -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello"}]}'
+```
+
+`auto` asks the gateway to route the request. The request appears on the **Requests** page with the model that served it and its cost.
+
+
+
+
+
+
+```bash
+curl -X POST https://app.manifest.build/v1/chat/completions \
+ -H "Authorization: Bearer mnfst_YOUR_KEY" \
+ -H "Content-Type: application/json" \
+ -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello"}]}'
+```
+
+`auto` asks the gateway to route the request. The request appears on the **Requests** page with the model that served it and its cost.
+
+
+
+Install the gateway first. [Self-hosted](/llm-gateway/docs/self-hosted) covers Docker and the managed platforms. Your instance starts empty. [First request](/llm-gateway/docs/self-hosted#first-request) takes you from there to a routed request against `http://localhost:2099/v1`.
+
+
diff --git a/llm-gateway/docs/llm-gateway.mdx b/llm-gateway/docs/llm-gateway.mdx
new file mode 100644
index 0000000..0d2b215
--- /dev/null
+++ b/llm-gateway/docs/llm-gateway.mdx
@@ -0,0 +1,167 @@
+---
+title: "LLM Gateway"
+description: "Route requests across models, enforce limits, and use fallbacks when a provider fails."
+icon: "split"
+keywords:
+ ["LLM gateway", "model routing", "LLM router", "fallback chain", "hard limit", "custom tier", "default model", "fallback exhausted", "429 too many requests"]
+canonical: "https://manifest.build/llm-gateway/docs/llm-gateway"
+---
+
+Point your client at the gateway instead of a provider, and every request goes through the gateway. Three things happen there: limits decide whether the request runs at all, routing picks the model, and fallback steps in when that model fails.
+
+## How a request flows
+
+
+
+
+
+
+Every request through the gateway is recorded: which model served it, what it cost, how long it took, and what failed. The dashboard is built on that record, and you can query the same numbers from your own code.
+
+## Requests and provider attempts
+
+The gateway tracks two things, and the difference matters once fallback is involved:
+
+| | What it is | When you get more than one |
+|---|---|---|
+| **Request** | One call from your agent to the gateway | Never. One call in, one Request. |
+| **Provider attempt** | One call from the gateway to a provider | Every fallback step and every [Autofix](/llm-gateway/docs/autofix) retry adds another |
+
+A request that succeeds on the first try is one Request with one attempt. A request that fails twice before succeeding is still one Request, with three attempts. A request blocked by the gateway, like a [hard limit](/llm-gateway/docs/llm-gateway#hard-limits) or a malformed body, is one Request with one attempt that never reached a provider. That attempt is listed under the model you asked for, or **No provider** when the request named none, and carries the error that stopped it.
+
+This is why the request count and the provider call count don't match, and why cost is attributed to attempts: a failed attempt can still burn tokens.
+
+## The dashboard
+
+
+
+On the Free plan, the Overview covers up to 7 days of history; the 30-, 90-, and 365-day ranges need Pro.
+
+
+
+### What's on each request
+
+Opening a request in the log shows the whole story: the tier that routed it, the model and provider that served it, input and output tokens, computed cost, latency, the auth type used ([API key, subscription, or local](/llm-gateway/docs/reference/glossary#auth-type)), and every fallback hop with its own status. Failures carry an error code, so a gateway rejection like [M100](/llm-gateway/docs/errors/M100) reads differently from a provider's own 500.
+
+That's all metadata. To read the messages themselves, turn on [request logs](/llm-gateway/docs/request-logs) for the harness. Open a request, pick an attempt, and its **Messages** tab shows the body sent to the provider and the body that came back.
+
+
+
+Alerts go to your account email. Check it's current in your profile settings.
+
+
+
+
+
+On the **Limits** page, under **Configure email provider**, pick Resend, Mailgun, or SendGrid. Enter the API key. Mailgun also asks for a sending domain. **Notification email** defaults to your account email; change it to send alerts elsewhere. **Test & Connect** sends a real test email and saves nothing if it fails. The saved provider appears as a card on the same page, where you can edit or remove it.
+
+Your workspace holds one provider, shared by every harness. Removing it stops alerts for all of them.
+
+Alerts use the saved provider first, and fall back to the [`EMAIL_*` environment variables](/llm-gateway/docs/reference/environment-variables) when no provider is saved. Those variables also send verification and password-reset email, which the saved provider never handles.
+
+Without an email provider from either path, alerts are skipped. Blocking rules still work, since they don't need to send anything.
+
+
+
+## Reading usage from your own code
+
+A harness can query its own numbers with its `mnfst_` key, which is useful for showing a budget in your own UI or stopping work before a limit blocks you.
+
+```bash
+curl https://app.manifest.build/api/v1/agent/usage \
+ -H "Authorization: Bearer mnfst_YOUR_KEY"
+```
+
+| Endpoint | Returns |
+|---|---|
+| `GET /api/v1/agent/usage` | Token usage for the calling harness |
+| `GET /api/v1/agent/costs` | Cost data for the calling harness |
+
+
+
+Self-hosted installs swap the base URL for their own host.
+
+## Anonymous telemetry
+
+Separately from your dashboard, each self-hosted install sends a small daily report so the project can see version adoption and which providers deserve attention. It's aggregates only, never prompts or content, and one environment variable turns it off. Full field list and opt-out: [Data and telemetry](/llm-gateway/docs/reference/telemetry).
+
+
+
+## Related
+
+- [LLM Gateway](/llm-gateway/docs/llm-gateway) — routing, fallback, and hard limits
+- [Request logs](/llm-gateway/docs/request-logs) — the request and response bodies
+- [Autofix](/llm-gateway/docs/autofix)
+- [Error codes](/llm-gateway/docs/errors)
+- [Glossary](/llm-gateway/docs/reference/glossary)
diff --git a/llm-gateway/docs/providers/api-key-providers.mdx b/llm-gateway/docs/providers/api-key-providers.mdx
new file mode 100644
index 0000000..ec942ee
--- /dev/null
+++ b/llm-gateway/docs/providers/api-key-providers.mdx
@@ -0,0 +1,108 @@
+---
+title: "API key providers"
+sidebarTitle: "API keys"
+description: "Connect provider API keys for OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Groq, Cerebras, AWS Bedrock, NVIDIA NIM, OpenRouter, Hugging Face, and more."
+icon: "key"
+keywords:
+ ["OpenAI", "Anthropic", "Google Gemini", "xAI", "DeepSeek", "Mistral", "Qwen", "Moonshot", "MiniMax", "Z.ai", "OpenRouter", "Groq", "Cerebras", "Fireworks", "AWS Bedrock", "NVIDIA NIM", "Xiaomi MiMo", "Hugging Face", "Gemini Free", "API key"]
+canonical: "https://manifest.build/llm-gateway/docs/providers/api-key-providers"
+---
+
+Most providers work the standard way: sign up, generate an API key, paste it into the gateway. Routed requests go out with that key as the credential.
+
+## Supported providers
+
+| Provider | Where to get a key |
+| ------------------------------------------------- | ---------------------------------------------------------------- |
+| [OpenAI](https://platform.openai.com) | [platform.openai.com](https://platform.openai.com) |
+| [Anthropic](https://www.anthropic.com) | [console.anthropic.com](https://console.anthropic.com) |
+| [Google](https://ai.google.dev) | [ai.google.dev](https://ai.google.dev) |
+| [xAI](https://x.ai) | [x.ai/api](https://x.ai/api) |
+| [Meta](https://dev.meta.ai) | [dev.meta.ai](https://dev.meta.ai) |
+| [DeepSeek](https://www.deepseek.com) | [platform.deepseek.com](https://platform.deepseek.com) |
+| [Mistral](https://mistral.ai) | [console.mistral.ai](https://console.mistral.ai) |
+| [Alibaba Cloud (Qwen)](https://qwen.ai) | [bailian.console.aliyun.com](https://bailian.console.aliyun.com) |
+| [Moonshot (Kimi)](https://platform.moonshot.cn) | [platform.moonshot.cn](https://platform.moonshot.cn) |
+| [MiniMax](https://www.minimaxi.com) | [platform.minimaxi.com](https://platform.minimaxi.com) |
+| [Z.ai (Zhipu)](https://z.ai) | [bigmodel.cn](https://bigmodel.cn) |
+| [OpenRouter](https://openrouter.ai) | [openrouter.ai/keys](https://openrouter.ai/keys) |
+| [Groq](https://groq.com) | [console.groq.com](https://console.groq.com) |
+| [Cerebras](https://cerebras.ai) | [cloud.cerebras.ai](https://cloud.cerebras.ai) |
+| [Fireworks AI](https://fireworks.ai) | [fireworks.ai](https://fireworks.ai) |
+| [NVIDIA NIM](https://build.nvidia.com) | [build.nvidia.com](https://build.nvidia.com) |
+| [AWS Bedrock](https://aws.amazon.com/bedrock) | [aws.amazon.com/bedrock](https://aws.amazon.com/bedrock) |
+| [OpenCode Zen](https://opencode.ai) | [opencode.ai/auth](https://opencode.ai/auth) |
+| [Xiaomi MiMo](https://platform.xiaomimimo.com) | [platform.xiaomimimo.com](https://platform.xiaomimimo.com/console/api-keys) |
+| [Kilo](https://kilo.ai) | [app.kilo.ai](https://app.kilo.ai) |
+| [Pioneer](https://pioneer.ai) | [pioneer.ai](https://pioneer.ai) |
+| [Hugging Face](https://huggingface.co) | [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) |
+
+New providers are added often, so **Providers → Usage-based** in the dashboard always shows the current list.
+
+### Gemini Free
+
+Gemini Free is a managed tile. The gateway provisions the credential automatically, so there's no key to fetch from Google first: add the tile and you have a free model to route to. It behaves like any other provider once connected.
+
+
+
+
+## Refreshing the model list
+
+A connection keeps the model list it discovered when you saved it. Nothing re-checks it on a schedule. When a provider adds, renames, or drops a model, click **Refresh models**: on your harness's **Routing** page, or on the connection itself under **Providers**.
+
+The refresh replaces the list, it does not merge it. A tier or a fallback pinned to a model id that no longer exists stops matching, so pin it to one of the new ids.
+
+## Key prefixes
+
+Each provider issues keys with a recognizable prefix. Useful for catching paste mistakes:
+
+| Provider | Prefix | Example |
+| ---------- | --------- | ------------ |
+| OpenAI | `sk-` | `sk-...` |
+| Anthropic | `sk-ant-` | `sk-ant-...` |
+| xAI | `xai-` | `xai-...` |
+| Meta | `LLM_` | `LLM_...` |
+| DeepSeek | `sk-` | `sk-...` |
+| Moonshot | `sk-` | `sk-...` |
+| MiniMax | `sk-` | `sk-...` |
+| Qwen | `sk-` | `sk-...` |
+| OpenRouter | `sk-or-` | `sk-or-...` |
+| Google | none | API key |
+| Mistral | none | API key |
+| Z.ai | none | API key |
+| Fireworks | `fw_` | `fw_...` |
+| Groq | `gsk_` | `gsk_...` |
+| Pioneer | `pio_sk_` | `pio_sk_...` |
+| Xiaomi MiMo | `sk-` | `sk-...` |
+| Hugging Face | `hf_` | `hf_...` |
+
+Providers not listed here (Cerebras, NVIDIA NIM, AWS Bedrock, Kilo, OpenCode Zen) issue keys with no fixed prefix.
+
+## Regional providers
+
+Some providers serve more than one region. Alibaba Cloud, AWS Bedrock, and MiniMax let you pick a region when you connect them, and the gateway routes to the matching endpoint.
diff --git a/llm-gateway/docs/providers/custom-providers.mdx b/llm-gateway/docs/providers/custom-providers.mdx
new file mode 100644
index 0000000..a514a68
--- /dev/null
+++ b/llm-gateway/docs/providers/custom-providers.mdx
@@ -0,0 +1,66 @@
+---
+title: "Custom providers"
+sidebarTitle: "Custom"
+description: "Connect any OpenAI- or Anthropic-compatible endpoint, including vLLM, TGI, LocalAI, or an internal service."
+icon: "plug"
+keywords:
+ ["vLLM", "TGI", "text-generation-inference", "LocalAI", "Xinference", "OpenLLM", "OpenAI-compatible", "Anthropic-compatible", "custom provider"]
+canonical: "https://manifest.build/llm-gateway/docs/providers/custom-providers"
+---
+
+If your endpoint speaks OpenAI or Anthropic, the gateway can route to it. Useful for self-hosted inference servers (vLLM, TGI, LocalAI), internal endpoints behind your VPN, or providers that aren't on the built-in list yet.
+
+## Compatible servers
+
+Any server exposing one of these endpoints works out of the box:
+
+| Format | Endpoint |
+| -------------------- | --------------------------- |
+| OpenAI-compatible | `POST /v1/chat/completions` |
+| Anthropic-compatible | `POST /v1/messages` |
+
+Common options that ship with one of these formats: [vLLM](https://github.com/vllm-project/vllm), [TGI](https://github.com/huggingface/text-generation-inference), [LocalAI](https://localai.io), [Xinference](https://github.com/xorbitsai/inference), [OpenLLM](https://github.com/bentoml/OpenLLM).
+
+## Add a custom provider
+
+
+
+
+
+
+The [Requests page](/llm-gateway/docs/observability) tells you what a request cost and which model served it. Request logs tell you what was actually said: the body the gateway sent to the provider, and the body that came back. The dashboard calls this feature **Logs**.
+
+It's one toggle per harness.
+
+## What gets recorded
+
+Recording happens per **provider attempt**, not per request. A request that falls back twice before succeeding produces three recordings, each with its own body pair, so you can see how the conversation was reshaped for each provider. An [Autofix](/llm-gateway/docs/autofix) retry is its own attempt too, which is how you compare the request that failed against the patched one that worked.
+
+| | What's stored |
+|---|---|
+| **Request** | The body the gateway forwarded to the provider, after routing rewrote the model and translated the protocol. Not the body your client sent |
+| **Response** | The parsed JSON body, or for a streamed attempt, the raw SSE stream as it arrived |
+| **Wire format** | Which protocol the exchange used, so the drawer can render it correctly |
+
+A request blocked by the gateway, on a [hard limit](/llm-gateway/docs/llm-gateway#hard-limits) or a malformed body, still lists one attempt, labelled with the model you asked for or **No provider**. It carries the failure status and no **Messages**, **Tools**, or **Raw** tab: the gateway sent nothing to a provider, so there is nothing to read back.
+
+## Reading it back
+
+Open a request in the dashboard's **Requests** page and pick an attempt in the **Attempts** list. The initial call, each fallback, and an [Autofix](/llm-gateway/docs/autofix) retry are separate attempts, and each one shows its own conversation. The **Messages**, **Tools**, and **Raw** tabs appear when that attempt has a recording.
+
+
+ 
+
+
+The **Messages** tab renders the exchange as a conversation, with a rail down the side listing every turn. Search the rail or filter it by role, and click any row to jump to that turn in the main pane. Assistant turns show the tool calls the model actually made, not the tool definitions your client offered it.
+
+
+ 
+
+
+## Turning it on
+
+The switch is **Enable logs**, in the **Logs** section of each harness's **Settings** page.
+
+New harnesses have it on. To make sure a harness records its logs, check **Settings → Logs**.
+
+
+ 
+
+
+
+
+
+ Logs were reintroduced in **v6.18.0**. On an older self-hosted install,
+ [upgrade the gateway](/llm-gateway/docs/self-hosted#upgrading) first.
+
+
+
+
+Recordings don't live in Postgres. They're gzipped and written to object storage, which you pick per instance.
+
+
@@ -15,14 +16,14 @@ Every request through the gateway is recorded: which model served it, what it co
## Requests and provider attempts
-Manifest tracks two things, and the difference matters once fallback is involved:
+The gateway tracks two things, and the difference matters once fallback is involved:
| | What it is | When you get more than one |
|---|---|---|
-| **Request** | One call from your agent to Manifest | Never. One call in, one Request. |
-| **Provider attempt** | One call from Manifest to a provider | Every fallback step and every [Autofix](/autofix) retry adds another |
+| **Request** | One call from your agent to the gateway | Never. One call in, one Request. |
+| **Provider attempt** | One call from the gateway to a provider | Every fallback step and every [Autofix](/llm-gateway/docs/autofix) retry adds another |
-A request that succeeds on the first try is one Request with one attempt. A request that fails twice before succeeding is still one Request, with three attempts. A request Manifest blocked itself, like a [hard limit](/llm-gateway#hard-limits) or a malformed body, is one Request with one attempt that never reached a provider. That attempt is listed under the model you asked for, or **No provider** when the request named none, and carries the error that stopped it.
+A request that succeeds on the first try is one Request with one attempt. A request that fails twice before succeeding is still one Request, with three attempts. A request blocked by the gateway, like a [hard limit](/llm-gateway/docs/llm-gateway#hard-limits) or a malformed body, is one Request with one attempt that never reached a provider. That attempt is listed under the model you asked for, or **No provider** when the request named none, and carries the error that stopped it.
This is why the request count and the provider call count don't match, and why cost is attributed to attempts: a failed attempt can still burn tokens.
@@ -51,13 +52,13 @@ On the Free plan, the Overview covers up to 7 days of history; the 30-, 90-, and
### What's on each request
-Opening a request in the log shows the whole story: the tier that routed it, the model and provider that served it, input and output tokens, computed cost, latency, the auth type used ([API key, subscription, or local](/reference/glossary#auth-type)), and every fallback hop with its own status. Failures carry an error code, so a Manifest rejection like [M100](/errors/M100) reads differently from a provider's own 500.
+Opening a request in the log shows the whole story: the tier that routed it, the model and provider that served it, input and output tokens, computed cost, latency, the auth type used ([API key, subscription, or local](/llm-gateway/docs/reference/glossary#auth-type)), and every fallback hop with its own status. Failures carry an error code, so a gateway rejection like [M100](/llm-gateway/docs/errors/M100) reads differently from a provider's own 500.
-That's all metadata. To read the messages themselves, turn on [request logs](/request-logs) for the harness. Open a request, pick an attempt, and its **Messages** tab shows the body sent to the provider and the body that came back.
+That's all metadata. To read the messages themselves, turn on [request logs](/llm-gateway/docs/request-logs) for the harness. Open a request, pick an attempt, and its **Messages** tab shows the body sent to the provider and the body that came back.
Local models record `cost = 0` with real token counts and latency, since
- nothing was billed. Custom providers do the same, because Manifest can't
+ nothing was billed. Custom providers do the same, because the gateway can't
infer pricing for a model it doesn't know.
@@ -72,7 +73,7 @@ A limit rule watches one metric for one harness over a rolling period, and acts
| **Period** | Hour, day, week, or month |
| **Action** | Every rule emails you. Turn on blocking to also stop requests over the threshold |
-Blocking is the gateway half of this and is covered in [hard limits](/llm-gateway#hard-limits). The rest of this section is the alerting half.
+Blocking is the gateway half of this and is covered in [hard limits](/llm-gateway/docs/llm-gateway#hard-limits). The rest of this section is the alerting half.
@@ -105,7 +106,7 @@ On the **Limits** page, under **Configure email provider**, pick Resend, Mailgun
Your workspace holds one provider, shared by every harness. Removing it stops alerts for all of them.
-Alerts use the saved provider first, and fall back to the [`EMAIL_*` environment variables](/reference/environment-variables) when no provider is saved. Those variables also send verification and password-reset email, which the saved provider never handles.
+Alerts use the saved provider first, and fall back to the [`EMAIL_*` environment variables](/llm-gateway/docs/reference/environment-variables) when no provider is saved. Those variables also send verification and password-reset email, which the saved provider never handles.
Without an email provider from either path, alerts are skipped. Blocking rules still work, since they don't need to send anything.
@@ -131,14 +132,14 @@ Self-hosted installs swap the base URL for their own host.
## Anonymous telemetry
-Separately from your dashboard, each self-hosted install sends a small daily report so the project can see version adoption and which providers deserve attention. It's aggregates only, never prompts or content, and one environment variable turns it off. Full field list and opt-out: [Data and telemetry](/reference/telemetry).
+Separately from your dashboard, each self-hosted install sends a small daily report so the project can see version adoption and which providers deserve attention. It's aggregates only, never prompts or content, and one environment variable turns it off. Full field list and opt-out: [Data and telemetry](/llm-gateway/docs/reference/telemetry).
## Related
-- [LLM Gateway](/llm-gateway) — routing, fallback, and hard limits
-- [Request logs](/request-logs) — the request and response bodies
-- [Autofix](/autofix)
-- [Error codes](/errors)
-- [Glossary](/reference/glossary)
+- [LLM Gateway](/llm-gateway/docs/llm-gateway) — routing, fallback, and hard limits
+- [Request logs](/llm-gateway/docs/request-logs) — the request and response bodies
+- [Autofix](/llm-gateway/docs/autofix)
+- [Error codes](/llm-gateway/docs/errors)
+- [Glossary](/llm-gateway/docs/reference/glossary)
diff --git a/providers/api-key-providers.mdx b/providers/api-key-providers.mdx
index df5c61e..ec942ee 100644
--- a/providers/api-key-providers.mdx
+++ b/providers/api-key-providers.mdx
@@ -1,13 +1,14 @@
---
title: "API key providers"
sidebarTitle: "API keys"
-description: "Bring your own API key for OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Groq, Cerebras, AWS Bedrock, NVIDIA NIM, OpenRouter, Hugging Face, and more."
+description: "Connect provider API keys for OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Groq, Cerebras, AWS Bedrock, NVIDIA NIM, OpenRouter, Hugging Face, and more."
icon: "key"
keywords:
["OpenAI", "Anthropic", "Google Gemini", "xAI", "DeepSeek", "Mistral", "Qwen", "Moonshot", "MiniMax", "Z.ai", "OpenRouter", "Groq", "Cerebras", "Fireworks", "AWS Bedrock", "NVIDIA NIM", "Xiaomi MiMo", "Hugging Face", "Gemini Free", "API key"]
+canonical: "https://manifest.build/llm-gateway/docs/providers/api-key-providers"
---
-Most providers work the standard way: sign up, generate an API key, paste it into Manifest. Routed requests go out with that key as the credential.
+Most providers work the standard way: sign up, generate an API key, paste it into the gateway. Routed requests go out with that key as the credential.
## Supported providers
@@ -40,13 +41,13 @@ New providers are added often, so **Providers → Usage-based** in the dashboard
### Gemini Free
-Gemini Free is a managed tile. Manifest provisions the credential through its own gateway, so there's no key to fetch from Google first: add the tile and you have a free model to route to. It behaves like any other provider once connected.
+Gemini Free is a managed tile. The gateway provisions the credential automatically, so there's no key to fetch from Google first: add the tile and you have a free model to route to. It behaves like any other provider once connected.
@@ -11,25 +12,25 @@ keywords:
-The [Requests page](/observability) tells you what a request cost and which model served it. Request logs tell you what was actually said: the body Manifest sent to the provider, and the body that came back. The dashboard calls this feature **Logs**.
+The [Requests page](/llm-gateway/docs/observability) tells you what a request cost and which model served it. Request logs tell you what was actually said: the body the gateway sent to the provider, and the body that came back. The dashboard calls this feature **Logs**.
It's one toggle per harness.
## What gets recorded
-Recording happens per **provider attempt**, not per request. A request that falls back twice before succeeding produces three recordings, each with its own body pair, so you can see how the conversation was reshaped for each provider. An [Autofix](/autofix) retry is its own attempt too, which is how you compare the request that failed against the patched one that worked.
+Recording happens per **provider attempt**, not per request. A request that falls back twice before succeeding produces three recordings, each with its own body pair, so you can see how the conversation was reshaped for each provider. An [Autofix](/llm-gateway/docs/autofix) retry is its own attempt too, which is how you compare the request that failed against the patched one that worked.
| | What's stored |
|---|---|
-| **Request** | The body Manifest forwarded to the provider, after routing rewrote the model and translated the protocol. Not the body your client sent |
+| **Request** | The body the gateway forwarded to the provider, after routing rewrote the model and translated the protocol. Not the body your client sent |
| **Response** | The parsed JSON body, or for a streamed attempt, the raw SSE stream as it arrived |
| **Wire format** | Which protocol the exchange used, so the drawer can render it correctly |
-A request Manifest blocked itself, on a [hard limit](/llm-gateway#hard-limits) or a malformed body, still lists one attempt, labelled with the model you asked for or **No provider**. It carries the failure status and no **Messages**, **Tools**, or **Raw** tab: Manifest sent nothing to a provider, so there is nothing to read back.
+A request blocked by the gateway, on a [hard limit](/llm-gateway/docs/llm-gateway#hard-limits) or a malformed body, still lists one attempt, labelled with the model you asked for or **No provider**. It carries the failure status and no **Messages**, **Tools**, or **Raw** tab: the gateway sent nothing to a provider, so there is nothing to read back.
## Reading it back
-Open a request in the dashboard's **Requests** page and pick an attempt in the **Attempts** list. The initial call, each fallback, and an [Autofix](/autofix) retry are separate attempts, and each one shows its own conversation. The **Messages**, **Tools**, and **Raw** tabs appear when that attempt has a recording.
+Open a request in the dashboard's **Requests** page and pick an attempt in the **Attempts** list. The initial call, each fallback, and an [Autofix](/llm-gateway/docs/autofix) retry are separate attempts, and each one shows its own conversation. The **Messages**, **Tools**, and **Raw** tabs appear when that attempt has a recording.
@@ -52,9 +53,9 @@ New harnesses have it on. To make sure a harness records its logs, check **Setti