diff --git a/CLAUDE.md b/CLAUDE.md index 4a6c4601..3d837676 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -118,7 +118,11 @@ ones). which the Discovery document lacks. `gemini-video.ts` builds one endpoint per video model at sync time: Google's Interactions OpenAPI request with `model` pinned and `response_format.duration` an enum from the model - page's `Output video 3s-15s` limit. The endpoint id is the model id and + page's `Output video 3s-15s` limit. A page that states resolution, fps + and a token rate instead (#697, Genie 4) is still video on the poll, and + the spec sync names it as a real-time stream with no fixed-duration + schema. A page that fails to fetch or parse rejects the video fetch, so + the sync keeps the stored video endpoints. The endpoint id is the model id and the path is shared, so `GET /v1/openapi/gemini` merges them into one operation whose body is an `anyOf` of components tagged `x-modelschemas-model` (`mergeModelBodies`). The Models API lists these diff --git a/src/server/providers/fixtures/gemini-genie-4.md.txt b/src/server/providers/fixtures/gemini-genie-4.md.txt new file mode 100644 index 00000000..3b6ffe10 --- /dev/null +++ b/src/server/providers/fixtures/gemini-genie-4.md.txt @@ -0,0 +1,367 @@ +> ## AI Agent Context +> - Full documentation index: https://aistudio.google.com/llms.txt +> - Gemini Docs MCP server (consistent live info): https://gemini-api-docs-mcp.dev +> - Coding agent skills & setup guide: https://aistudio.google.com/docs/coding-agents.md + +# Genie 4 + +Genie 4 (`models/genie-4`) is a real-time interactive world generation model on +the [Interactions API](/docs/interactions-overview). Seed a world from a text +prompt, a 16:9 reference image, or both, then explore and steer the live 24 fps +video and 48 kHz stereo audio stream in real time using keyboard movement +controls (`WASD` and arrow keys) and mid-session text events. + +For offline, fixed-duration clip generation, multi-turn video editing, and +upsampling up to 4K, use +[Gemini Omni 1.2 Flash](/docs/models/gemini-omni-flash) instead. + +{% button href="https://aistudio.google.com/live?model=genie-4" variant="ais" %}Try in Google AI Studio{% /button %} + +## Documentation {% id="documentation" %} + +* [Interactive world generation with Genie 4](/docs/genie): End-to-end setup + with `@google/genai` (WebRTC and WebSocket) and `google-genai` (async + WebSocket), ephemeral session tokens, event protocol, and media stream + decoding. +* [Genie 4 prompt and controls guide](/docs/genie-prompting): Tested prompt + patterns for causal object rules, custom key-bound action verbs, 16:9 + landscape painting seeds, broadband Foley audio, and Google AI Studio Live + playground controls. + +## Quickstart {% id="quickstart" %} + +Genie 4 sessions require a [paid-tier Gemini API key](/docs/pricing#genie-4) +(`~$0.10/sec` of streamed output) and run over a live bidirectional connection +on `v1beta`. Always call `complete()` when ending a session so server-side +generation and token billing stop immediately. + +{% tabs %} + {% tab label=JavaScript (WebRTC) %} + ```javascript + import { GoogleGenAI } from '@google/genai'; + + // 1. Server-side: create the interaction and mint a single-session auth token + const serverAi = new GoogleGenAI({ + apiKey: process.env.GEMINI_API_KEY, + apiVersion: 'v1beta', + }); + const interaction = await serverAi.interactions.create({ + model: 'models/genie-4', + api_version: 'v1beta', + }); + const { auth_token } = await serverAi.interactions.authTokens.create( + interaction.id, + { api_version: 'v1beta' } + ); + + // 2. Browser: connect over WebRTC and attach a