-
Notifications
You must be signed in to change notification settings - Fork 471
feat(images): Grok image bridge (maintainer takeover of #424) #577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Wibias
merged 21 commits into
lidge-jun:dev
from
Wibias:maintainer-takeover/424-feat-image-bridge
Jul 27, 2026
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
b1bd017
feat(images): add Grok image bridge for non-OpenAI models
74ee934
fix(images): address Wibias review blockers for #424
144cc98
fix(images): address Wibias R2 review blockers for #424
b76d549
fix(images): address Codex P2 follow-ups for image bridge
Wibias c6a7d1b
fix(images): address Codex and CodeRabbit feedback on #528
Wibias 34a950f
fix(images): abort stalled runTurn and merge incomplete usage
Wibias b4e8814
fix(images): pin validated DNS for artifact downloads against rebinding
Wibias 8660f1a
fix(images): stream pinned downloads with byte cap and idle timeout
Wibias 24036e0
fix(images): destroy pinned HTTPS response on non-2xx status
Wibias a714723
fix(docs): point image-bridge Providers link at guides/providers
Wibias f535247
fix(images): address Codex review on #528
Wibias e51fe53
fix(images): unblock runTurn idle timeout without waiting on the adapter
Wibias 8034e33
fix(test): raise Windows CI timeout for injected cleanup rollback cases
Wibias 07b53e9
fix(images): address CodeRabbit review on image bridge P2
Wibias ee7f783
fix(images): maintainer takeover hardening for image bridge (#424)
Wibias 88e9613
test(images): cover URL-canonicalized IPv4 shorthand SSRF rejects
Wibias d96ef8b
fix(test): raise Windows timeout for archived cleanup read-failure case
Wibias d261411
fix(images): address Codex review on image-bridge takeover
Wibias e623e90
fix(images): harden DNS family classify and image budget charging
Wibias 0aebcb0
Merge upstream/dev into image-bridge takeover
Wibias ce571e6
fix(test): raise Windows timeouts for storage trash and Kiro login fl…
Wibias File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| --- | ||
| title: Image Bridge | ||
| description: Route image_generation hosted-tool calls to xAI Grok Imagine when using a non-OpenAI provider. | ||
| --- | ||
|
|
||
| ## Overview | ||
|
|
||
| When you route Codex through a non-OpenAI model (Claude, Gemini, Grok, etc.), the | ||
| `image_generation` **hosted tool** normally doesn't work — it requires OpenAI's server-side | ||
| execution environment. The Image Bridge detects these calls and transparently reroutes them to | ||
| xAI Grok Imagine, so the model you're actually chatting with can still generate images. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - **Enable the bridge** by setting `images.bridgeEnabled: true` in your config (it is off by | ||
| default to avoid unexpected xAI charges — see [Configuration](#configuration) below). | ||
| - An `xai` provider entry with an **API key**. The bridge pins fulfillment to the registry xAI | ||
| Images endpoint (`https://api.x.ai/v1`); any configured `baseUrl` override is ignored for image | ||
| calls. OAuth / `ocx login xai` alone does **not** arm the bridge (the Grok CLI OAuth transport is | ||
| chat-oriented and is not used for `/images/*`). | ||
|
|
||
| ```json | ||
| { | ||
| "providers": { | ||
| "xai": { "adapter": "openai-chat", "apiKey": "xai-…", "authMode": "key" } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| - A non-OpenAI model selected as your active provider. (When the active provider is OpenAI, | ||
| the native hosted tool is used directly and the bridge is bypassed.) | ||
|
|
||
| ## Configuration | ||
|
|
||
| Image Bridge options live under `images` in `~/.opencodex/config.json`. Bridging is | ||
| **opt-in** — you must set `bridgeEnabled: true` to enable paid xAI Grok Imagine generation: | ||
|
|
||
| ```json | ||
| { | ||
| "images": { | ||
| "bridgeEnabled": true, | ||
| "bridgeModel": "grok-imagine-image-quality", | ||
| "maxRounds": 3, | ||
| "timeoutMs": 60000 | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| | Option | Default | Description | | ||
| | --- | --- | --- | | ||
| | `bridgeEnabled` | `false` | Master switch. Set `true` to enable bridging. Off by default to avoid unexpected xAI charges. | | ||
| | `bridgeModel` | `grok-imagine-image-quality` | The xAI image model id to send prompts to. | | ||
| | `maxRounds` | `3` | Maximum image-generation loop iterations per turn. Floored to an integer and clamped to `[0, 10]`; non-finite values fall back to `3`. | | ||
| | `timeoutMs` | `60000` | Per-call xAI deadline in milliseconds. Finite positive values are floored and passed to the xAI request. | | ||
| | `artifactsKeepCount` | `200` | Maximum number of files retained under `artifacts/`. When exceeded, the oldest files are deleted after each fulfilled call. Set to `0` or a negative value to disable pruning. | | ||
|
|
||
| ## Artifact Retention | ||
|
|
||
| Generated images are written to `~/.opencodex/artifacts/`. To prevent unbounded disk | ||
| growth in long-running sessions, the directory is pruned automatically after each fulfilled | ||
| image call (once the full batch for that call is on disk) — the oldest files (by modification | ||
| time) are deleted when the count exceeds the configured maximum (default 200, configurable via | ||
| `images.artifactsKeepCount`). Only paths that survive pruning are returned to the model. | ||
|
|
||
| ## How It Works | ||
|
|
||
| The Image Bridge activates only on **Responses** turns that include the hosted | ||
| `image_generation` tool in the `/v1/responses` tools array while a **non-OpenAI** | ||
| model is selected. It does **not** intercept Codex's built-in `image_gen` tool, | ||
| which POSTs directly to `/v1/images/generations` (or `/images/edits`) — that path | ||
| is covered separately in [Codex Integration](/guides/codex-integration/#built-in-image-generation-image_gen). | ||
|
|
||
| 1. When a Responses request lists `image_generation` in `tools`, OpenCodex detects it | ||
| during request preprocessing. | ||
| 2. The hosted tool is replaced with a **synthetic function tool** that the routed model can call | ||
| normally — the model sees a callable tool rather than an opaque hosted tool it can't execute. | ||
| 3. When the model invokes that tool, OpenCodex intercepts the call and sends the prompt to xAI's | ||
| image generation API. | ||
| 4. Generated images are saved to `~/.opencodex/artifacts/` and the **local file path** is returned | ||
| to the model as the tool result. | ||
| 5. The model continues the conversation with knowledge of the generated image and its location. | ||
|
|
||
| From the model's perspective nothing changed — it called a tool and got a result. From the user's | ||
| perspective, image generation works with any routed provider instead of silently failing. | ||
|
|
||
| ## Limitations | ||
|
|
||
| - **Only xAI Grok Imagine is supported.** DALL-E and other image providers may be added later. | ||
| - **Web search takes priority** on adapters that support the web-search sidecar loop. If both web | ||
| search and image generation are requested in the same turn, web-search runs and image | ||
| generation is skipped. Cursor/`runTurn` adapters cannot use that sidecar today, so the image | ||
| bridge may still run for those dual-tool turns. | ||
| - **xAI costs apply.** Image generation via xAI requires an active xAI subscription or API credits. | ||
| - **Streaming only.** The bridge works by intercepting the SSE response stream; requests with | ||
| `stream: false` are rejected with a 400 error. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: lidge-jun/opencodex
Length of output: 15374
Clarify the streaming-path wording. In
docs-site/src/content/docs/guides/image-bridge.md:94-95, keep thestream: false→400claim, but rephrase the lead-in to “Responses tool-event streaming path” instead of “intercepting the SSE response stream”;src/server/responses/core.ts:1545-1549rejects non-streaming image-bridge requests with400 image bridge requires stream=true, andsrc/images/loop.ts:287-290showsrunTurnadapters replay queued events into the bridge rather than exposing raw SSE.🤖 Prompt for AI Agents
Source: Path instructions