feat: move sampling options (temperature/topP/maxTokens) into modelOptions

.changeset/sampling-options-to-model-options.md

@@ -0,0 +1,31 @@
+---
+'@tanstack/ai': minor
+'@tanstack/openai-base': minor
+'@tanstack/ai-openai': minor
+'@tanstack/ai-anthropic': minor
+'@tanstack/ai-gemini': minor
+'@tanstack/ai-grok': minor
+'@tanstack/ai-groq': minor
+'@tanstack/ai-ollama': minor
+'@tanstack/ai-openrouter': minor
+---
+
+**BREAKING:** Sampling options (`temperature`, `topP`, `maxTokens`) have moved off the root of `chat()` / `ai()` / `generate()` and into provider-native `modelOptions`. There is no longer a generic root-level sampling surface — each provider accepts its own native keys, fully typed per model:
+
+- OpenAI (Responses): `modelOptions: { temperature, top_p, max_output_tokens }`
+- Anthropic: `modelOptions: { temperature, top_p, max_tokens }`
+- Gemini: `modelOptions: { temperature, topP, maxOutputTokens }`
+- Grok: `modelOptions: { temperature, top_p, max_tokens }`
+- Groq: `modelOptions: { temperature, top_p, max_completion_tokens }`
+- Ollama: `modelOptions: { options: { temperature, top_p, num_predict } }` (nested)
+- OpenRouter (chat): `modelOptions: { temperature, topP, maxCompletionTokens }`
+
+Middleware no longer sees `temperature`/`topP`/`maxTokens` as first-class fields on `ChatMiddlewareConfig`; mutate `config.modelOptions` (with the provider-native keys above) instead. `metadata` is unaffected and stays at the root.
+
+Migrate automatically with the codemod, which resolves the provider from the adapter and rewrites the keys for you:
+
+```bash
+pnpm codemod:move-sampling-to-model-options "src/**/*.{ts,tsx}"
+```
+
+See the [Sampling Options migration guide](https://tanstack.com/ai/latest/docs/migration/sampling-options-to-model-options) for details.

codemods/README.md

@@ -6,9 +6,10 @@ Each codemod lives in its own subdirectory and is named after the migration it c
 
 ## Available codemods
 
-| Codemod                                  | Migrates                                                                                                                                                                                                                                                      |
-| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| [`ag-ui-compliance`](./ag-ui-compliance) | Client-side renames introduced by the AG-UI client/server compliance release: `body` → `forwardedProps` on `useChat` / `ChatClient` / `updateOptions`, Svelte's `updateBody` → `updateForwardedProps`, and `chat({ conversationId })` → `chat({ threadId })`. |
+| Codemod                                                              | Migrates                                                                                                                                                                                                                                                      |
+| -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [`ag-ui-compliance`](./ag-ui-compliance)                             | Client-side renames introduced by the AG-UI client/server compliance release: `body` → `forwardedProps` on `useChat` / `ChatClient` / `updateOptions`, Svelte's `updateBody` → `updateForwardedProps`, and `chat({ conversationId })` → `chat({ threadId })`. |
+| [`move-sampling-to-model-options`](./move-sampling-to-model-options) | Moves root `temperature` / `topP` / `maxTokens` off `chat()` / `ai()` / `generate()` / `createChatOptions()` into provider-native `modelOptions`, renamed per provider (with ollama nesting under `options`).                                                 |
 
 ## Running a codemod
 

codemods/move-sampling-to-model-options/README.md

@@ -0,0 +1,83 @@
+# `move-sampling-to-model-options`
+
+Moves the root-level convenience sampling props — `temperature`, `topP`, and
+`maxTokens` — off `chat()` / `ai()` / `generate()` / `createChatOptions()`
+calls (imported from `@tanstack/ai`) and into the provider-native
+`modelOptions` object, renaming each one to its provider's canonical option
+name.
+
+This is a **breaking change**: the root-level props have been removed, so run
+this codemod to migrate existing call sites onto the new `modelOptions` shape.
+
+## What it changes
+
+The provider is resolved from the `adapter:` property's factory call (e.g.
+`openaiText('gpt-4o')` → `openai`). Each present root prop is moved into
+`modelOptions` under its provider-specific name:
+
+| Root prop     | openai              | anthropic     | gemini            | grok          | groq                    | openrouter            | ollama (nested)       |
+| ------------- | ------------------- | ------------- | ----------------- | ------------- | ----------------------- | --------------------- | --------------------- |
+| `temperature` | `temperature`       | `temperature` | `temperature`     | `temperature` | `temperature`           | `temperature`         | `options.temperature` |
+| `topP`        | `top_p`             | `top_p`       | `topP`            | `top_p`       | `top_p`                 | `topP`                | `options.top_p`       |
+| `maxTokens`   | `max_output_tokens` | `max_tokens`  | `maxOutputTokens` | `max_tokens`  | `max_completion_tokens` | `maxCompletionTokens` | `options.num_predict` |
+
+For **ollama**, the renamed keys are nested inside a `options` object **within**
+`modelOptions` (e.g. `modelOptions: { options: { temperature, num_predict } }`).
+
+### Example (openai)
+
+```ts
+// before
+chat({
+  adapter: openaiText('gpt-4o'),
+  messages,
+  temperature: 0.3,
+  maxTokens: 100,
+})
+
+// after
+chat({
+  adapter: openaiText('gpt-4o'),
+  messages,
+  modelOptions: {
+    temperature: 0.3,
+    max_output_tokens: 100,
+  },
+})
+```
+
+If `modelOptions` already exists (as an object literal), the renamed keys are
+merged into it. Original value expressions are preserved, and shorthand props
+(`{ temperature }`) are expanded to `key: temperature`.
+
+## Running it
+
+From this repo:
+
+```bash
+pnpm codemod:move-sampling-to-model-options "src/**/*.{ts,tsx}"
+```
+
+Or directly against the published transform — no clone needed:
+
+```bash
+npx jscodeshift \
+  --parser=tsx \
+  -t https://raw.githubusercontent.com/TanStack/ai/main/codemods/move-sampling-to-model-options/transform.ts \
+  src/**/*.{ts,tsx}
+```
+
+Add `--dry --print` to preview the rewrite without modifying files.
+
+## Report / skip behavior
+
+The codemod never partially transforms a single call. It leaves the call
+untouched and emits an `api.report(...)` message in these cases:
+
+- **Unresolvable adapter** — no `adapter` prop, the adapter value isn't a
+  recognized provider-factory call (e.g. `makeAdapter()`), or it's
+  dynamic/spread.
+- **`modelOptions` is not a plain object literal** — e.g. a spread or an
+  identifier reference.
+- **Key conflict** — a target renamed key already exists in `modelOptions`
+  (or in `modelOptions.options` for ollama). Resolve these by hand.

codemods/move-sampling-to-model-options/__testfixtures__/anthropic-merge.input.ts

@@ -0,0 +1,12 @@
+import { chat } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function run(messages: Array<unknown>) {
+  const temperature = 0.5
+  return chat({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+    modelOptions: { top_k: 40 },
+    temperature,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/anthropic-merge.output.ts

@@ -0,0 +1,15 @@
+import { chat } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function run(messages: Array<unknown>) {
+  const temperature = 0.5
+  return chat({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+
+    modelOptions: {
+      top_k: 40,
+      temperature: temperature,
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/conflict.input.ts

@@ -0,0 +1,14 @@
+// Conflict case: root `temperature` AND `modelOptions.temperature` are
+// both present. The codemod must leave the whole call alone and report.
+
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+    modelOptions: { temperature: 0.9 },
+    temperature: 0.3,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/conflict.output.ts

@@ -0,0 +1,14 @@
+// Conflict case: root `temperature` AND `modelOptions.temperature` are
+// both present. The codemod must leave the whole call alone and report.
+
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+    modelOptions: { temperature: 0.9 },
+    temperature: 0.3,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/create-chat-options.input.ts

@@ -0,0 +1,8 @@
+import { createChatOptions } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export const options = createChatOptions({
+  adapter: openaiText('gpt-4o'),
+  temperature: 0.2,
+  topP: 0.8,
+})

codemods/move-sampling-to-model-options/__testfixtures__/create-chat-options.output.ts

@@ -0,0 +1,11 @@
+import { createChatOptions } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export const options = createChatOptions({
+  adapter: openaiText('gpt-4o'),
+
+  modelOptions: {
+    temperature: 0.2,
+    top_p: 0.8,
+  },
+})

codemods/move-sampling-to-model-options/__testfixtures__/gemini-rename.input.ts

@@ -0,0 +1,11 @@
+import { chat } from '@tanstack/ai'
+import { geminiText } from '@tanstack/ai-gemini'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: geminiText('gemini-1.5-pro'),
+    messages,
+    topP: 0.9,
+    maxTokens: 512,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/gemini-rename.output.ts

@@ -0,0 +1,14 @@
+import { chat } from '@tanstack/ai'
+import { geminiText } from '@tanstack/ai-gemini'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: geminiText('gemini-1.5-pro'),
+    messages,
+
+    modelOptions: {
+      topP: 0.9,
+      maxOutputTokens: 512,
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/generate-and-ai.input.ts

@@ -0,0 +1,18 @@
+import { ai, generate } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function viaAi(messages: Array<unknown>) {
+  return ai({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+    maxTokens: 64,
+  })
+}
+
+export function viaGenerate(messages: Array<unknown>) {
+  return generate({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+    topP: 0.95,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/generate-and-ai.output.ts

@@ -0,0 +1,24 @@
+import { ai, generate } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function viaAi(messages: Array<unknown>) {
+  return ai({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+
+    modelOptions: {
+      max_tokens: 64,
+    },
+  })
+}
+
+export function viaGenerate(messages: Array<unknown>) {
+  return generate({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+
+    modelOptions: {
+      top_p: 0.95,
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/groq-maxtokens.input.ts

@@ -0,0 +1,10 @@
+import { chat } from '@tanstack/ai'
+import { groqText } from '@tanstack/ai-groq'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: groqText('llama-3.1-70b'),
+    messages,
+    maxTokens: 256,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/groq-maxtokens.output.ts

@@ -0,0 +1,13 @@
+import { chat } from '@tanstack/ai'
+import { groqText } from '@tanstack/ai-groq'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: groqText('llama-3.1-70b'),
+    messages,
+
+    modelOptions: {
+      max_completion_tokens: 256,
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/no-import.input.ts

@@ -0,0 +1,13 @@
+// Negative case: no `@tanstack/ai` import. A local `chat` helper that
+// happens to share the name and use `temperature`/`maxTokens` must be
+// left completely untouched.
+
+function chat(opts: { temperature?: number; maxTokens?: number }) {
+  return opts
+}
+
+export const result = chat({
+  adapter: 'whatever',
+  temperature: 0.3,
+  maxTokens: 100,
+})

codemods/move-sampling-to-model-options/__testfixtures__/no-import.output.ts

@@ -0,0 +1,13 @@
+// Negative case: no `@tanstack/ai` import. A local `chat` helper that
+// happens to share the name and use `temperature`/`maxTokens` must be
+// left completely untouched.
+
+function chat(opts: { temperature?: number; maxTokens?: number }) {
+  return opts
+}
+
+export const result = chat({
+  adapter: 'whatever',
+  temperature: 0.3,
+  maxTokens: 100,
+})

codemods/move-sampling-to-model-options/__testfixtures__/ollama-nested.input.ts

@@ -0,0 +1,11 @@
+import { chat } from '@tanstack/ai'
+import { ollamaText } from '@tanstack/ai-ollama'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: ollamaText('llama3'),
+    messages,
+    temperature: 0.7,
+    maxTokens: 200,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/ollama-nested.output.ts

@@ -0,0 +1,16 @@
+import { chat } from '@tanstack/ai'
+import { ollamaText } from '@tanstack/ai-ollama'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: ollamaText('llama3'),
+    messages,
+
+    modelOptions: {
+      options: {
+        temperature: 0.7,
+        num_predict: 200,
+      },
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/openai-basic.input.ts

@@ -0,0 +1,11 @@
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+    temperature: 0.3,
+    maxTokens: 100,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/openai-basic.output.ts

@@ -0,0 +1,14 @@
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+
+    modelOptions: {
+      temperature: 0.3,
+      max_output_tokens: 100,
+    },
+  })
+}