[M3 Bug] Token Plan + Cloudflare AI Gateway (BYOK) does not forward SSE — chunks are buffered and returned as a single payload

[FireTable]

Which inference path did you use?

MiniMax API (Token Plan, via Cloudflare AI Gateway / BYOK)

Inference parameters

• model: MiniMax-M3
• stream: true
• max_tokens: default
• temperature: default
• request path: POST https://<gateway>.gateway.ai.cloudflare.com/v1/chat/completions (Cloudflare AI Gateway)
• upstream: api.minimaxi.com (大陆版 Token Plan) — configured via BYOK (Bring Your Own Key)
• client: raw curl -N (also reproduced from @langchain/openai TS / openai-python SDK)

Prompt / input

# A. Via Cloudflare AI Gateway (BYOK)  ← 问题路径
curl -N -X POST "https://<gateway>.gateway.ai.cloudflare.com/v1/chat/completions" \
  -H "Authorization: Bearer $CF_AIGW_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMax-M3",
    "stream": true,
    "messages": [{"role": "user", "content": "Count from 1 to 10 slowly"}]
  }'

# B. Direct to api.minimaxi.com (Token Plan)  ← 对照，正常
curl -N -X POST "https://api.minimaxi.com/v1/chat/completions" \
  -H "Authorization: Bearer $MINIMAX_TOKEN_PLAN_KEY" \
  -H "Content-Type: application/json" \
  -d '{ ... 同上 ... }'

Expected behavior

Cloudflare AI Gateway 在 BYOK + stream: true 时应该透明转发上游 SSE：

• 响应 Content-Type: text/event-stream + Transfer-Encoding: chunked
• 上游每个 data: {...}\n\n 增量到客户端
• TTFB ≈ TTFT（几百 ms），不是"等生成完再一次性返回"

A / B 两条路径的流式体验应完全一致。

Actual behavior

SSE 没有被 CF 透传。观察：

• B（直连 api.minimaxi.com）：正常流式，chunks 一个个到，前端可以增量渲染 ✅
• A（CF AI Gateway BYOK）：chunks 全部存在且顺序正确，但挤在一起一次性返回 ⛔
  • 客户端在 curl / OpenAI SDK / @langchain/openai 上都看不到 chunk-by-chunk 到达
  • 整段 data: {...}\n\n 行连成一片在生成结束后才一次性 flush
  • 没有"逐字打出"的效果，TTFB 接近"整个 response 生成完成"的耗时
  • 任何依赖 incremental render 的客户端（OpenAI SDK stream iterator、LCEL streaming、assistant-ui、自定义 SSE consumer）都不可用

Additional context

• 根因方向：CF AI Gateway edge 在 BYOK 模式下对上游 text/event-stream 响应做了 buffering（典型表现：buffer 到 size threshold 或 end-of-stream 才 flush），没有透传给下游。属于 CF 侧行为，但提在这里因为：
  1. 用 MiniMax Token Plan 作为 upstream 可稳定复现
  2. MiniMax 端加 X-Accel-Buffering: no 等 header 可能绕过
  3. 走 CF 转发 Token Plan 的其他用户也会撞到同样的问题
• 与模型无关：MiniMax-M2.7 和 MiniMax-M3 在 Token Plan + CF BYOK 路径下表现一致
• 还没测过 Anthropic-compatible 端点经 CF BYOK 是否同样问题
• 直连 Token Plan 一切正常，问题专属于 CF AI Gateway BYOK 这一跳

Suggested mitigations

MiniMax（上游）侧可考虑：

• streaming 响应加 X-Accel-Buffering: no header
• 确保 Content-Type: text/event-stream 在第一个 byte 就 set，而不是 buffering 完成后才设
• 文档里给出推荐的 CF AI Gateway / 反代层流式配置说明

Cloudflare（gateway）侧可考虑：

• 上游是 text/event-stream 时跳过 response buffering
• 允许把 BYOK upstream 标记为 "streaming-required"，让 edge 不要 buffer

[FireTable]

另外使用 cloudflare AI Gateway 链接其它厂商没出现此问题，所以初步判断为 MiniMax 处理 SSE 返回的问题

[FireTable]

Update 2026-06-19 (UTC+8) — verified by the reporter

1. Anthropic-compatible endpoint is also affected / not affected

Tested POST https://api.minimaxi.com/anthropic/v1/messages directly with stream: true on MiniMax-M3:

• HTTP 200, Content-Type: text/event-stream, Transfer-Encoding: chunked, no Content-Length
• 57 raw chunks over ~8.6s, TTFT 1.2s, avg chunk interval 133.5ms, 3.6% chunks < 5ms
• Anthropic-compatible direct-to-MiniMax: streaming works ✅

We have not yet tested the Anthropic-compatible endpoint through Cloudflare AI Gateway BYOK. Likely it hits the same wrap path and is also buffered.

2. Direct-to-MiniMax (OpenAI-compatible) is fine

For comparison, POST https://api.minimaxi.com/v1/chat/completions with stream: true directly:

• 137 raw chunks, TTFT 1.36s, avg chunk interval 116ms, 8.1% chunks < 5ms
• Direct-to-MiniMax OpenAI-compat: streaming works ✅

3. The bug is isolated to the Cloudflare AI Gateway (BYOK / custom upstream) path

Reproduced matrix:

Path	Streaming?
Direct to api.minimaxi.com (OpenAI-compat)	✅
Direct to api.minimaxi.com (Anthropic-compat)	✅
Through a self-hosted streaming proxy (pipe upstream.body directly)	✅
Through Cloudflare AI Gateway (BYOK / custom upstream)	⛔ buffered

4. Likely root cause (refined)

Cloudflare AI Gateway appears to use a "known provider fast-path" for OpenAI / Anthropic / Workers AI / etc. (where upstream.body is returned as a ReadableStream without being read), and a "generic OpenAI-compatible wrap" for BYOK / custom upstreams. The generic path almost certainly reads the entire response body before returning (await resp.text() or equivalent) in order to log it, compute usage, and apply cache rules — which is what kills streaming.

If anyone on the MiniMax team is in contact with the Cloudflare team, this is a CF AI Gateway bug, not a MiniMax one. A fix would be: when relaying text/event-stream from a BYOK upstream, pass upstream.body through without buffering, and either capture the log/usage asynchronously or skip it for streaming responses.

5. Workaround (Cloudflare Worker, pass-through, deployable in 5 minutes)

Verified locally: a Cloudflare Worker that pipes upstream.body directly into a Response (without await resp.text()) preserves streaming end-to-end with the same chunk timing as direct-to-MiniMax. Drop-in replacement for the Cloudflare AI Gateway baseURL for streaming requests.

// worker.js
export default {
  async fetch(request, env) {
    const url = new URL(request.url);

    // Auth gate
    const incomingAuth = request.headers.get("authorization") || "";
    if (incomingAuth !== `Bearer ${env.PROXY_AUTH_TOKEN}`) {
      return new Response("unauthorized", { status: 401 });
    }

    // Health check
    if (url.pathname === "/health") {
      return new Response(JSON.stringify({ ok: true, upstream: env.UPSTREAM_URL }), {
        headers: { "content-type": "application/json" },
      });
    }

    if (!url.pathname.startsWith("/v1/") && !url.pathname.startsWith("/anthropic/")) {
      return new Response("not found", { status: 404 });
    }

    const upstreamBase = new URL(env.UPSTREAM_URL);
    const upstreamUrl = upstreamBase.origin + url.pathname + url.search;

    const headers = new Headers(request.headers);
    headers.set("Authorization", `Bearer ${env.MINIMAX_TOKEN_PLAN_KEY}`);
    headers.set("Host", upstreamBase.host);
    headers.delete("cf-connecting-ip");
    headers.delete("cf-ray");

    const upstream = await fetch(upstreamUrl, {
      method: request.method,
      headers,
      body: request.method === "GET" || request.method === "HEAD" ? undefined : request.body,
    });

    // KEY: do not read the body. Pass the ReadableStream straight through.
    return new Response(upstream.body, {
      status: upstream.status,
      headers: upstream.headers,
    });
  },
};

wrangler.toml:

name = "minimax-stream-proxy"
main = "worker.js"
compatibility_date = "2025-01-01"

[vars]
UPSTREAM_URL = "https://api.minimaxi.com"
# MINIMAX_TOKEN_PLAN_KEY + PROXY_AUTH_TOKEN set via `wrangler secret put`

Deploy:

npx wrangler deploy worker.js --name minimax-stream-proxy
npx wrangler secret put MINIMAX_TOKEN_PLAN_KEY
npx wrangler secret put PROXY_AUTH_TOKEN   # any random string

Then use the Worker URL as the OpenAI baseURL:

const client = new OpenAI({
  apiKey: process.env.PROXY_AUTH_TOKEN,        // the proxy's auth token, not MiniMax's
  baseURL: "https://minimax-stream-proxy.<your-subdomain>.workers.dev/v1",
});

6. Test setup that produced the numbers above

// test.js — measures per-chunk arrival time to detect buffering
const https = require("https");
const t0 = Date.now();
const req = https.request({
  hostname: "api.minimaxi.com",
  port: 443,
  path: "/v1/chat/completions",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.MINIMAX_API_KEY}`,
    "Content-Type": "application/json",
  },
}, (res) => {
  const dts = [];
  let lastT = 0, firstByteAt = 0;
  res.on("data", (chunk) => {
    const now = Date.now() - t0;
    if (firstByteAt === 0) firstByteAt = now;
    if (lastT > 0) dts.push(now - lastT);
    lastT = now;
  });
  res.on("end", () => {
    const avg = dts.reduce((a, b) => a + b, 0) / dts.length;
    const max = Math.max(...dts), min = Math.min(...dts);
    const p50 = [...dts].sort((a, b) => a - b)[Math.floor(dts.length / 2)];
    console.log({ TTFT: firstByteAt, count: dts.length + 1, dt: { min, avg, p50, max } });
  });
});
req.write(JSON.stringify({
  model: "MiniMax-M3",
  stream: true,
  max_tokens: 1500,
  messages: [{ role: "user", content: "用中文写一个关于火星上发现古代遗迹的科幻短篇，500 字左右，要有人物、悬念和反转。" }],
}));
req.end();

If the avg chunk interval is in the 80–200ms range → streaming is healthy. If avg < 5ms → it's buffered.

[FireTable]

8## 进一步排查 — 自己写透传 + 部署海外 Worker 也无法排除问题

今晚（2026-06-19 UTC+8）做了更深的排查，跑了 4 条路径的对比：

路径	streaming 行为
直连 api.minimaxi.com (OpenAI-compat)	✅ 正常, p50 97ms
直连 api.minimaxi.com (Anthropic-compat)	✅ 正常, p50 133ms
本地 Node 透传 proxy (Mac 出口, 大陆 IP)	✅ 正常, p50 82ms
CF Worker 自部署透传 (CF 海外 edge)	✅ 正常, p50 97ms
CF AI Gateway BYOK	⛔ buffered

我能排除的

• ❌ 不是 MiniMax 端 SSE 处理问题（直连 streaming 正常）
• ❌ 不是"海外 IP 触发 MiniMax 端 buffer"（CF Worker 海外 edge 同样 streaming 正常）
• ❌ 不是反代通用 await resp.text() 这种明显的代码 bug（透传模式全部正常）

我无法排除的（请 MiniMax 团队协助）

我自己写 CF Worker 透传 workaround 跑得通，但这只证明"绕过 CF AI Gateway 能解决"，无法 100% 反向证明根因就在 CF AI Gateway 的 wrap 层——可能还有我没考虑到的变量：

1. Token Plan vs Pay-as-you-go：我只测了 Token Plan。这两套是走同一套 infra 还是不同代码路径？建议团队在 Pay-as-you-go + CF AI Gateway BYOK 路径下也复现一下
2. OpenAI-compat vs Anthropic-compat 端点：CF AI Gateway BYOK 下两个端点是否都受影响？
3. BYOK 模式 vs Stored Key 模式：CF AI Gateway 也有让用户自己提供 key 的 Stored Key 模式（不需要 BYOK），这个模式是否一样有 buffering？
4. edge region：CF AI Gateway 部署在哪个 region？我测试时 Worker 走的是默认 region

希望 MiniMax 团队结合 Cloudflare BYOK 模式确认

• 能否在你们的测试环境复现"CF AI Gateway BYOK + stream: true"路径下的 buffering？
• 你们和 Cloudflare AI Gateway 团队有沟通渠道吗？能否把"BYOK + streaming"路径的 wrap 层（特别是日志 / 计量 / 缓存这几块）让他们 review 一下？
• 如果根因确实在 CF 端，文档里能否加一条"经 CF AI Gateway 透传时不要用 stream: true"的工作说明？
• 同时建议在 MiniMax 端试一下加 X-Accel-Buffering: no header（nginx 风格），可能能绕过某些反代的默认 buffer 行为

---

Update 2026-06-19 04:41 UTC+8 — by the reporter