Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b1bd017
feat(images): add Grok image bridge for non-OpenAI models
Jul 24, 2026
74ee934
fix(images): address Wibias review blockers for #424
Jul 27, 2026
144cc98
fix(images): address Wibias R2 review blockers for #424
Jul 27, 2026
b76d549
fix(images): address Codex P2 follow-ups for image bridge
Wibias Jul 27, 2026
c6a7d1b
fix(images): address Codex and CodeRabbit feedback on #528
Wibias Jul 27, 2026
34a950f
fix(images): abort stalled runTurn and merge incomplete usage
Wibias Jul 27, 2026
b4e8814
fix(images): pin validated DNS for artifact downloads against rebinding
Wibias Jul 27, 2026
8660f1a
fix(images): stream pinned downloads with byte cap and idle timeout
Wibias Jul 27, 2026
24036e0
fix(images): destroy pinned HTTPS response on non-2xx status
Wibias Jul 27, 2026
a714723
fix(docs): point image-bridge Providers link at guides/providers
Wibias Jul 27, 2026
f535247
fix(images): address Codex review on #528
Wibias Jul 27, 2026
e51fe53
fix(images): unblock runTurn idle timeout without waiting on the adapter
Wibias Jul 27, 2026
8034e33
fix(test): raise Windows CI timeout for injected cleanup rollback cases
Wibias Jul 27, 2026
07b53e9
fix(images): address CodeRabbit review on image bridge P2
Wibias Jul 27, 2026
ee7f783
fix(images): maintainer takeover hardening for image bridge (#424)
Wibias Jul 27, 2026
88e9613
test(images): cover URL-canonicalized IPv4 shorthand SSRF rejects
Wibias Jul 27, 2026
d96ef8b
fix(test): raise Windows timeout for archived cleanup read-failure case
Wibias Jul 27, 2026
d261411
fix(images): address Codex review on image-bridge takeover
Wibias Jul 27, 2026
e623e90
fix(images): harden DNS family classify and image budget charging
Wibias Jul 27, 2026
0aebcb0
Merge upstream/dev into image-bridge takeover
Wibias Jul 27, 2026
ce571e6
fix(test): raise Windows timeouts for storage trash and Kiro login fl…
Wibias Jul 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs-site/src/content/docs/guides/codex-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,11 @@ The proxy listens on port `10100` by default and serves `POST /v1/responses`,
Codex's built-in `image_gen` tool does not go through `/v1/responses` — the codex-rs extension
POSTs `{base_url}/images/generations` (or `/images/edits` when reference images are attached)
directly, with the same ChatGPT bearer auth it uses for chat. Because the injected `base_url`
points at opencodex, the proxy relays those calls to the OpenAI upstream:
points at opencodex, the proxy relays those calls to the OpenAI upstream.

This is separate from the [Image Bridge](/guides/image-bridge/), which only activates when a
**Responses** turn lists the hosted `image_generation` tool while a non-OpenAI model is selected.
Standalone `/images/generations` calls never enter that bridge.

- **One mode-aware forward candidate:** Pool selects an eligible main/added account; Direct uses the
caller OAuth bearer. The configured mode applies consistently to the image request.
Expand Down
95 changes: 95 additions & 0 deletions docs-site/src/content/docs/guides/image-bridge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: Image Bridge
description: Route image_generation hosted-tool calls to xAI Grok Imagine when using a non-OpenAI provider.
---

## Overview

When you route Codex through a non-OpenAI model (Claude, Gemini, Grok, etc.), the
`image_generation` **hosted tool** normally doesn't work — it requires OpenAI's server-side
execution environment. The Image Bridge detects these calls and transparently reroutes them to
xAI Grok Imagine, so the model you're actually chatting with can still generate images.

## Prerequisites

- **Enable the bridge** by setting `images.bridgeEnabled: true` in your config (it is off by
default to avoid unexpected xAI charges — see [Configuration](#configuration) below).
- An `xai` provider entry with an **API key**. The bridge pins fulfillment to the registry xAI
Images endpoint (`https://api.x.ai/v1`); any configured `baseUrl` override is ignored for image
calls. OAuth / `ocx login xai` alone does **not** arm the bridge (the Grok CLI OAuth transport is
chat-oriented and is not used for `/images/*`).

```json
{
"providers": {
"xai": { "adapter": "openai-chat", "apiKey": "xai-…", "authMode": "key" }
}
}
```

- A non-OpenAI model selected as your active provider. (When the active provider is OpenAI,
the native hosted tool is used directly and the bridge is bypassed.)

## Configuration

Image Bridge options live under `images` in `~/.opencodex/config.json`. Bridging is
**opt-in** — you must set `bridgeEnabled: true` to enable paid xAI Grok Imagine generation:

```json
{
"images": {
"bridgeEnabled": true,
"bridgeModel": "grok-imagine-image-quality",
"maxRounds": 3,
"timeoutMs": 60000
}
}
```

| Option | Default | Description |
| --- | --- | --- |
| `bridgeEnabled` | `false` | Master switch. Set `true` to enable bridging. Off by default to avoid unexpected xAI charges. |
| `bridgeModel` | `grok-imagine-image-quality` | The xAI image model id to send prompts to. |
| `maxRounds` | `3` | Maximum image-generation loop iterations per turn. Floored to an integer and clamped to `[0, 10]`; non-finite values fall back to `3`. |
| `timeoutMs` | `60000` | Per-call xAI deadline in milliseconds. Finite positive values are floored and passed to the xAI request. |
| `artifactsKeepCount` | `200` | Maximum number of files retained under `artifacts/`. When exceeded, the oldest files are deleted after each fulfilled call. Set to `0` or a negative value to disable pruning. |

## Artifact Retention

Generated images are written to `~/.opencodex/artifacts/`. To prevent unbounded disk
growth in long-running sessions, the directory is pruned automatically after each fulfilled
image call (once the full batch for that call is on disk) — the oldest files (by modification
time) are deleted when the count exceeds the configured maximum (default 200, configurable via
`images.artifactsKeepCount`). Only paths that survive pruning are returned to the model.

## How It Works

The Image Bridge activates only on **Responses** turns that include the hosted
`image_generation` tool in the `/v1/responses` tools array while a **non-OpenAI**
model is selected. It does **not** intercept Codex's built-in `image_gen` tool,
which POSTs directly to `/v1/images/generations` (or `/images/edits`) — that path
is covered separately in [Codex Integration](/guides/codex-integration/#built-in-image-generation-image_gen).

1. When a Responses request lists `image_generation` in `tools`, OpenCodex detects it
during request preprocessing.
2. The hosted tool is replaced with a **synthetic function tool** that the routed model can call
normally — the model sees a callable tool rather than an opaque hosted tool it can't execute.
3. When the model invokes that tool, OpenCodex intercepts the call and sends the prompt to xAI's
image generation API.
4. Generated images are saved to `~/.opencodex/artifacts/` and the **local file path** is returned
to the model as the tool result.
5. The model continues the conversation with knowledge of the generated image and its location.

From the model's perspective nothing changed — it called a tool and got a result. From the user's
perspective, image generation works with any routed provider instead of silently failing.

## Limitations

- **Only xAI Grok Imagine is supported.** DALL-E and other image providers may be added later.
- **Web search takes priority** on adapters that support the web-search sidecar loop. If both web
search and image generation are requested in the same turn, web-search runs and image
generation is skipped. Cursor/`runTurn` adapters cannot use that sidecar today, so the image
bridge may still run for those dual-tool turns.
- **xAI costs apply.** Image generation via xAI requires an active xAI subscription or API credits.
- **Streaming only.** The bridge works by intercepting the SSE response stream; requests with
`stream: false` are rejected with a 400 error.
Comment on lines +86 to +95

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# What does the Responses handler actually do for stream:false + hosted image_generation?
ast-grep outline src/server/responses/core.ts --items all --match 'Image|image'
rg -nP -C6 'runWithImageBridge|imageBridge|planImageBridge' src/server/responses/core.ts
rg -nP -C4 'stream' src/images/plan.ts src/images/loop.ts | rg -nP -C4 '400|stream'

Repository: lidge-jun/opencodex

Length of output: 15374


Clarify the streaming-path wording. In docs-site/src/content/docs/guides/image-bridge.md:94-95, keep the stream: false400 claim, but rephrase the lead-in to “Responses tool-event streaming path” instead of “intercepting the SSE response stream”; src/server/responses/core.ts:1545-1549 rejects non-streaming image-bridge requests with 400 image bridge requires stream=true, and src/images/loop.ts:287-290 shows runTurn adapters replay queued events into the bridge rather than exposing raw SSE.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs-site/src/content/docs/guides/image-bridge.md` around lines 86 - 95,
Update the “Streaming only” limitation in the image-bridge documentation to
describe the “Responses tool-event streaming path” instead of intercepting the
SSE response stream. Preserve the existing claim that stream: false requests are
rejected with a 400 error.

Source: Path instructions

Loading
Loading