feat(providers): prompt caching capability and usage-based cache pricing

Replace the arbitrary cached-rate heuristic with a single cache-aware pricing
function, and add prompt caching as an opt-in capability for Anthropic.

Pricing: priceModelUsage in cost-policy.ts is now the only place cache
arithmetic happens. Provider adapters normalize their wire shape into
ModelUsage (input always excludes cache buckets); the pricing function never
branches on provider. This removes five divergent behaviors, including the
!!request.context heuristic that gave Router and Evaluator an unearned 10x
input discount, and the overwrite that silently billed Anthropic cache reads
and writes at zero. Also parses OpenAI cache_write_tokens, previously ignored.

Caching: Anthropic gets a capability-gated advanced switch that places
cache_control on the last tool and last system block; system is now always a
TextBlockParam array. OpenAI gets a stable per-block prompt_cache_key with no
UI, since its caching is automatic.

Author: icecrasher321

apps/docs/content/docs/en/workflows/blocks/agent.mdx

@@ -80,8 +80,11 @@ Some settings live under advanced, or appear only for models that support them:
 - **Temperature.** How varied the output is. Stay low (0–0.3) when you need accuracy and repeatability, middle (around 0.5) for everyday work, higher (0.7+) when you want creative variety.
 - **Max output tokens.** Caps the response length. Defaults to the model's full limit.
 - **Reasoning effort / Thinking level.** For models with extended reasoning, how much the model thinks before answering. Higher is more thorough but slower and costs more tokens.
+- **Prompt caching.** For Anthropic Claude models, reuses the system prompt and tool definitions between runs instead of re-reading them every time. Cached input costs a tenth of the normal rate, but writing the cache costs 1.25x, so leave it off for one-off runs and turn it on when the same agent runs repeatedly. The cache covers a prefix only if it reaches 1,024 tokens (2,048 on Haiku) — below that Anthropic ignores it and nothing changes. Entries expire after five minutes of no use.
 - **API key.** Your key for the chosen provider. Hidden on hosted Sim, which supplies one.
 
+OpenAI and Gemini cache automatically at no extra cost and need no setting; their discount is already reflected in what you are charged.
+
 ## Outputs
 
 After the agent runs, later blocks read its result by name:
@@ -141,6 +144,7 @@ The Agent reads the message from Start with `<start.input>` and returns a result
   { question: "What is the difference between the tool usage controls (Auto, Force, None)?", answer: "In Auto, the model decides when to call a tool based on context. In Force, the model must call the tool on every run. In None, the tool is hidden from the model and never sent, which disables it without removing it from the block." },
   { question: "How does the Response Format work?", answer: "It enforces structured output by providing a JSON Schema. When set, the model's response is constrained to match the schema exactly, and each field is read directly by downstream blocks using <agent.fieldName>. Without a response format, the agent returns its standard outputs: content, model, tokens, and toolCalls." },
   { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI o-series and GPT-5 models) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." },
+  { question: "When should I turn on Prompt Caching?", answer: "Turn it on when the same agent runs repeatedly with a large, stable system prompt or tool set — cached input bills at a tenth of the normal input rate. Leave it off for one-off runs, because writing the cache costs 1.25x and nothing reads it back. The setting appears only for Anthropic Claude models; OpenAI and Gemini cache automatically with no setting and no write fee. Anthropic only caches a prefix of at least 1,024 tokens (2,048 on Haiku), and entries expire after five minutes of no use." },
   { question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." },
   { question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." },
 ]} />

apps/docs/content/docs/en/workflows/deployment/agent-events.mdx

@@ -12,9 +12,11 @@ Agent blocks can emit more than answer text while they run: **provider-exposed t
 Sim does **not** invent thinking for providers that do not stream it. Bedrock Converse, many OpenAI-compat models, and non-reasoning chat models stay text-only (plus tools when a live tool loop is wired).
 </Callout>
 
-## Independent event gates (public chat / simple SSE)
+## Protocol opt-in vs. event gates (public chat / simple SSE)
 
-Thinking and tool frames each require the stream protocol header plus their own deployment policy:
+Sending the header is a statement about the **client**: it understands v1 framing, so answer text can stream live and be retracted with `chunk_reset`. That alone does not expose anything — it only changes cadence, and it applies even when both event policies are off.
+
+Thinking and tool frames need the header **plus** their own deployment policy:
 
 1. Thinking frames require chat `includeThinking` (default **off**).
 2. Tool lifecycle frames require chat `includeToolCalls` (default **off**).
@@ -24,15 +26,15 @@ Thinking and tool frames each require the stream protocol header plus their own
 X-Sim-Stream-Protocol: agent-events-v1
 ```
 
-Legacy clients that omit the header keep today’s text-only SSE even if either deployment policy is enabled. The hosted chat UI always sends the header for its own deployments.
+Legacy clients that omit the header keep today’s text-only SSE even if either deployment policy is enabled. The hosted chat UI always sends the header for its own deployments, so hosted chats stream token by token regardless of the toggles.
 
 ## Simple SSE frame shapes
 
 Answer text stays on `chunk`. Thinking and tools never reuse `chunk` (so older clients that append every `chunk` into the answer cannot leak thinking).
 
 | Frame | Meaning |
 |-------|---------|
-| `{ "blockId", "chunk": "…" }` | Answer text. Legacy clients receive settled final-turn text; opted-in clients receive it live as the model generates (see below) |
+| `{ "blockId", "chunk": "…" }` | Answer text. Legacy clients receive settled final-turn text in one piece; opted-in clients receive it live as the model generates (see below) |
 | `{ "blockId", "event": "chunk_reset" }` | Opted-in only: discard the block’s streamed answer text — it belonged to a turn that resolved to tool calls |
 | `{ "blockId", "event": "thinking", "data": "…" }` | Thinking / reasoning summary delta |
 | `{ "blockId", "event": "tool", "phase": "start"\|"end", "id", "name", "status?" }` | Tool lifecycle (no args / results) |
@@ -45,8 +47,8 @@ Answer text stays on `chunk`. Thinking and tools never reuse `chunk` (so older c
 
 During a live tool loop, the model can’t be classified mid-turn: text it emits may turn out to be the final answer or preamble before a tool call (the stop reason arrives only at turn end).
 
-- **Opted-in clients** (protocol header + either event policy) receive answer text as `chunk` frames **live**, token by token. If the turn then resolves to tool calls, a `chunk_reset` frame tells the client to discard that block’s streamed text — the final turn re-streams live after tools settle. Append `chunk`, honor `chunk_reset`, and the displayed answer always converges to the block’s final content.
-- **Legacy clients** (no header) never see provisional text: only settled final-turn text is emitted as `chunk`, delivered when the turn completes.
+- **Opted-in clients** (protocol header; no event policy required) receive answer text as `chunk` frames **live**, token by token. If the turn then resolves to tool calls, a `chunk_reset` frame tells the client to discard that block’s streamed text — the final turn re-streams live after tools settle. Append `chunk`, honor `chunk_reset`, and the displayed answer always converges to the block’s final content.
+- **Legacy clients** (no header) never see provisional text: only settled final-turn text is emitted as `chunk`, delivered in one piece when the turn completes. Honoring `chunk_reset` is what buys live cadence, so send the header if you want it.
 
 Logs, memory, and the block’s `content` output always contain final-turn text only — intermediate preamble is never persisted.
 
@@ -71,7 +73,9 @@ The terminal output panel shows Thinking / Tools chrome above the block output w
 
 ## Chat deployment toggles
 
-In **Deploy → Chat**, enable **Include thinking** for provider-exposed thinking and **Include tool calls** for tool names and lifecycle status. The switches are independent, and both event types still require the protocol header. Tool arguments and results are never included in lifecycle frames. Redeploy or update the chat after changing Agent models or tools.
+In **Deploy → Chat**, enable **Include thinking** for provider-exposed thinking and **Include tool calls** for tool names and lifecycle status. The switches are independent, and both event types still require the protocol header. Neither switch affects answer-text cadence.
+
+Tool arguments and results are never exposed to a public chat — not in lifecycle frames, and not through the terminal `final` envelope, where the block's own tool calls are reduced to the same name-and-lifecycle shape. The authenticated workflow API still returns full tool results. Redeploy or update the chat after changing Agent models or tools.
 
 ## Capability honesty (high level)
 
@@ -81,7 +85,7 @@ Per-model support is generated from the model registry on the [Agent block page]
 |--------|----------|------------|
 | Anthropic / Azure Anthropic | Yes (incl. redacted blocks in traces). The newest Claude generations omit full thinking; Sim requests summarized thinking for them on streaming runs | Yes |
 | Gemini / Vertex | Yes when a thinking level is set (thought summaries requested on agent-events runs) | Yes |
-| OpenAI Responses | Reasoning **summaries** when streamed (requires OpenAI organization verification; unverified orgs fall back to no summaries) | Silent tool loop; tool chips arrive settled (start + end together) once tools finish, ahead of the streamed answer — no live in-progress chips yet |
+| OpenAI Responses | Reasoning **summaries** when streamed (requires OpenAI organization verification; unverified orgs fall back to no summaries) | Yes |
 | OpenAI-compat (Groq, DeepSeek, …) | Only if vendor streams `reasoning` / `reasoning_content` | Live loop where wired (e.g. Groq, DeepSeek) |
 | Bedrock | Not invented | Yes when streaming tool loop is used |
 

apps/sim/app/api/chat/[identifier]/route.ts

@@ -331,11 +331,7 @@ export const POST = withRouteHandler(
           status: 200,
           headers: {
             ...SSE_HEADERS,
-            ...agentStreamProtocolResponseHeaders({
-              includeThinking,
-              includeToolCalls,
-              requestHeaders: request.headers,
-            }),
+            ...agentStreamProtocolResponseHeaders({ requestHeaders: request.headers }),
           },
         })
         return streamResponse

apps/sim/app/api/chat/manage/[id]/route.test.ts

@@ -236,6 +236,38 @@ describe('Chat Edit API Route', () => {
       expect(data.message).toBe('Chat deployment updated successfully')
     })
 
+    it('turns grandfathered tool calls off when the same update disables thinking', async () => {
+      // Before includeToolCalls existed, includeThinking gated tool frames too.
+      // Resolving the fallback against the stored value would materialize the
+      // stale `true` and silently leave tool frames on.
+      authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-id' } })
+
+      mockCheckChatAccess.mockResolvedValue({
+        hasAccess: true,
+        chat: {
+          id: 'chat-123',
+          identifier: 'test-chat',
+          title: 'Test Chat',
+          authType: 'public',
+          workflowId: 'workflow-123',
+          includeThinking: true,
+          includeToolCalls: null,
+        },
+        workspaceId: 'workspace-123',
+      })
+
+      const req = new NextRequest('http://localhost:3000/api/chat/manage/chat-123', {
+        method: 'PATCH',
+        body: JSON.stringify({ includeThinking: false }),
+      })
+      const response = await PATCH(req, { params: Promise.resolve({ id: 'chat-123' }) })
+
+      expect(response.status).toBe(200)
+      expect(dbChainMockFns.set).toHaveBeenCalledWith(
+        expect.objectContaining({ includeThinking: false, includeToolCalls: false })
+      )
+    })
+
     it('returns 403 when the updated auth type changes to a blocked mode', async () => {
       authMockFns.mockGetSession.mockResolvedValue({
         user: { id: 'user-id' },

apps/sim/app/api/chat/manage/[id]/route.ts

@@ -257,11 +257,16 @@ export const PATCH = withRouteHandler(
         updateData.includeThinking = includeThinking
       }
 
+      /**
+       * Grandfathering must resolve against the thinking value this request is
+       * setting, not the stored one. Before `includeToolCalls` existed,
+       * `includeThinking` gated tool frames too, so a partial update turning
+       * thinking off has to turn them off as well rather than materializing the
+       * stale `true` and silently leaving tool frames enabled.
+       */
+      const effectiveIncludeThinking = includeThinking ?? existingChatRecord.includeThinking
       updateData.includeToolCalls =
-        includeToolCalls ??
-        existingChatRecord.includeToolCalls ??
-        existingChatRecord.includeThinking ??
-        false
+        includeToolCalls ?? existingChatRecord.includeToolCalls ?? effectiveIncludeThinking ?? false
 
       const emailCount = Array.isArray(updateData.allowedEmails)
         ? updateData.allowedEmails.length

apps/sim/app/api/resume/[workflowId]/[executionId]/[contextId]/route.ts

@@ -285,11 +285,7 @@ export const POST = withRouteHandler(
           headers: {
             ...SSE_HEADERS,
             // Echo the negotiated stream protocol (same as the public chat route).
-            ...agentStreamProtocolResponseHeaders({
-              includeThinking,
-              includeToolCalls,
-              requestHeaders: request.headers,
-            }),
+            ...agentStreamProtocolResponseHeaders({ requestHeaders: request.headers }),
             'X-Execution-Id': enqueueResult.resumeExecutionId,
           },
         })

apps/sim/app/api/tools/firecrawl/parse/route.ts

@@ -88,9 +88,15 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
 
     logger.info(`[${requestId}] Firecrawl parse successful`)
 
+    const document = firecrawlData.data ?? firecrawlData
     return NextResponse.json({
       success: true,
-      output: firecrawlData.data ?? firecrawlData,
+      output:
+        // Credits reported on the envelope would otherwise be dropped with it,
+        // leaving a paid parse with nothing to meter.
+        firecrawlData.creditsUsed != null
+          ? { ...document, creditsUsed: firecrawlData.creditsUsed }
+          : document,
     })
   } catch (error) {
     const notReady = docNotReadyResponse(error)

apps/sim/blocks/blocks/agent.ts

@@ -13,6 +13,7 @@ import {
   getMaxTemperature,
   getModelsWithDeepResearch,
   getModelsWithoutMemory,
+  getModelsWithPromptCaching,
   getModelsWithReasoningEffort,
   getModelsWithThinking,
   getModelsWithVerbosity,
@@ -29,6 +30,7 @@ const logger = createLogger('AgentBlock')
 const MODELS_WITH_REASONING_EFFORT = getModelsWithReasoningEffort()
 const MODELS_WITH_VERBOSITY = getModelsWithVerbosity()
 const MODELS_WITH_THINKING = getModelsWithThinking()
+const MODELS_WITH_PROMPT_CACHING = getModelsWithPromptCaching()
 const MODELS_WITH_DEEP_RESEARCH = getModelsWithDeepResearch()
 const MODELS_WITHOUT_MEMORY = getModelsWithoutMemory()
 
@@ -308,6 +310,19 @@ Return ONLY the JSON array.`,
         value: MODELS_WITH_THINKING,
       },
     },
+    {
+      id: 'promptCaching',
+      title: 'Prompt Caching',
+      type: 'switch',
+      description:
+        'Cache the system prompt and tool definitions so repeat runs reuse them at a reduced rate. Writing the cache costs more than a normal request, so this pays off when the same prompt runs repeatedly.',
+      defaultValue: false,
+      mode: 'advanced',
+      condition: {
+        field: 'model',
+        value: MODELS_WITH_PROMPT_CACHING,
+      },
+    },
 
     ...getProviderCredentialSubBlocks(),
     {
@@ -646,6 +661,10 @@ Return ONLY the JSON array.`,
       type: 'string',
       description: 'Thinking level for models with extended thinking (Anthropic Claude, Gemini 3)',
     },
+    promptCaching: {
+      type: 'boolean',
+      description: 'Cache the system prompt and tool definitions on models that support it',
+    },
     tools: { type: 'json', description: 'Available tools configuration' },
     skills: { type: 'json', description: 'Selected skills configuration' },
   },

apps/sim/executor/handlers/agent/agent-handler.ts

@@ -954,6 +954,7 @@ export class AgentBlockHandler implements BlockHandler {
       reasoningEffort: inputs.reasoningEffort,
       verbosity: inputs.verbosity,
       thinkingLevel: inputs.thinkingLevel,
+      promptCaching: inputs.promptCaching === true,
       previousInteractionId: inputs.previousInteractionId,
       /** Agent-events remains the opt-in for exposing thinking and tool lifecycle events. */
       agentEvents: streaming && ctx.metadata?.agentEvents === true,
@@ -1030,6 +1031,9 @@ export class AgentBlockHandler implements BlockHandler {
         reasoningEffort: providerRequest.reasoningEffort,
         verbosity: providerRequest.verbosity,
         thinkingLevel: providerRequest.thinkingLevel,
+        promptCaching: providerRequest.promptCaching,
+        // Stable per-block identity; providers use it to route cache lookups.
+        blockId: block.id,
         previousInteractionId: providerRequest.previousInteractionId,
         agentEvents: providerRequest.agentEvents,
         abortSignal: ctx.abortSignal,

apps/sim/executor/handlers/agent/types.ts

@@ -39,6 +39,7 @@ export interface AgentInputs {
   reasoningEffort?: string
   verbosity?: string
   thinkingLevel?: string
+  promptCaching?: boolean
   files?: unknown
 }