diff --git a/.agents/skills/add-block/SKILL.md b/.agents/skills/add-block/SKILL.md index 60778644da4..2361233e08a 100644 --- a/.agents/skills/add-block/SKILL.md +++ b/.agents/skills/add-block/SKILL.md @@ -927,6 +927,11 @@ bun run apps/sim/scripts/check-canvas-sentences.ts --block={service} ## Generated artifacts +When adding or changing `sunset.replacedBy`, run `bun run generate:block-successors` and commit +`apps/sim/lib/permission-groups/block-successors.generated.ts`. Authorization uses this generated +map to resolve legacy and current block IDs consistently without importing the executable registry. +Verify it with `bun run check:block-successors`. + Adding a block on its own needs no **tool metadata** regeneration — a block references existing tool IDs through `tools.access` and does not change any tool's shape. @@ -969,6 +974,7 @@ changes. - [ ] Tools.config.tool returns correct tool ID (snake_case) - [ ] Outputs match tool outputs - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`) +- [ ] If `sunset.replacedBy` changed: regenerated and committed the block successor map; `bun run check:block-successors` passes - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts - [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes - [ ] `bun run integration-catalog:check` passes diff --git a/.agents/skills/add-tools/SKILL.md b/.agents/skills/add-tools/SKILL.md index e0f68e70c34..85d34b491cd 100644 --- a/.agents/skills/add-tools/SKILL.md +++ b/.agents/skills/add-tools/SKILL.md @@ -264,6 +264,38 @@ stale/missing sidecars, and scope isolation. ## Critical Rules for Outputs +### File Downloads and Generated Files + +Internal operations return `createInternalToolFileResult` / `createInternalToolFilesResult` from +`lib/internal/tool-operations/file-result.ts` with bounded Buffers and a callback that places the +stored descriptors in the response. Their handlers preserve this result through dispatch, using +`InternalToolOperationHandler`. Do not serialize file bytes as base64 +JSON: the executor's 10 MiB response cap runs before ordinary file postprocessing or large-value +externalization. The shared executor stores files using trusted run or Copilot ownership. + +External endpoints that return raw binary files explicitly declare `request.responseType: 'binary'` +and return `output.file` with `{ name, mimeType, data: buffer, size }` from `transformResponse`. +The executor applies the bounded file-transfer budget and persists the descriptor. This opt-in is +for raw binary responses, not provider JSON containing base64 or tools that fetch attachments later. +Keep provider-specific limits and bounded reads; a file declaration is not permission to enlarge +arbitrary JSON responses. + +Attachment readers that download files inside `transformResponse` need their own bounded reads: +the first response cap does not cover subsequent fetches. Accept `ToolResponseContext` as the third +transform argument, forward its `signal`, and share one `AttachmentDownloadBudget` across sequential +downloads. Prefer raw provider endpoints over base64 metadata. Return the same file object in the +declared `file` / `file[]` output and nested message associations; `FileToolProcessor` stores it once +and replaces every alias with the same `UserFile` in both workflow and Copilot execution. + +Preserve stored `UserFile` fields (`id`, `key`, `url`, `context`, `type`, `name`, `size`) in transforms; +rebuilding the old `{ name, mimeType, data, size }` shape discards the reference. File outputs do not +need duplicate inline text/base64 aliases; the file system handles content materialization. When +an existing tool explicitly exposes content aliases in its contract, preserve its legacy version and +use the existing block/tool version pattern for a file-only output. Test a file over 10 MiB through +executor admission, single persistence, trusted ownership, and the unchanged JSON cap. Avoid adding +top-level filename, size, MIME type, URL, or success fields that merely repeat the canonical file or +tool result; keep additional provider fields only when they convey distinct information. + ### Output Types - `'string'`, `'number'`, `'boolean'` - Primitives - `'json'` - Complex objects (use this, NOT 'object') diff --git a/.agents/skills/validate-integration/SKILL.md b/.agents/skills/validate-integration/SKILL.md index 308df1d6691..a32df2b8bb3 100644 --- a/.agents/skills/validate-integration/SKILL.md +++ b/.agents/skills/validate-integration/SKILL.md @@ -345,6 +345,21 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba - [ ] List/search tools expose API limits and do not auto-fetch every page into memory - [ ] Transform logic does not build unbounded arrays, maps, sets, or `Promise.all` fan-outs - [ ] File and HTTP body reads use explicit byte caps or existing stream-limit helpers +- [ ] Internal file results reach `createInternalToolFileResult` / `createInternalToolFilesResult` + before JSON serialization; external raw downloads explicitly use `request.responseType: 'binary'` + and return a buffered `output.file`. Provider base64 JSON needs separate handling +- [ ] Transforms retain stored `UserFile` identity/access fields, and tests cover a >10 MiB file + crossing executor admission without another upload. New file outputs contain references only, + without inline content aliases; preserve legacy versions when removing existing inline fields +- [ ] Scan every file-producing path, including attachment fetches inside `transformResponse`, URL + descriptors, export operations, and old/new block versions; checking download-named tools alone + misses late reads that occur after the first response admission +- [ ] Late attachment reads share a per-call byte budget, bound actual streamed bytes independently + of provider size metadata, and forward `ToolResponseContext.signal` through every fetch/read +- [ ] Both workflow and Copilot tests produce compact `UserFile` outputs; nested message attachment + aliases reference the same stored files, with no duplicate upload or raw bytes left behind +- [ ] New output contracts omit redundant copies of file name, MIME type, size, URL, and success; + retained provider metadata has a distinct purpose, and types match the stored-file runtime shape - [ ] Large result payloads are summarized, paginated, referenced, or capped rather than raw-dumped - [ ] Pagination and download tests cover caps, early stop behavior, or partial-result preservation when relevant diff --git a/.github/workflows/test-build.yml b/.github/workflows/test-build.yml index dacc08540b9..083fc8a3edf 100644 --- a/.github/workflows/test-build.yml +++ b/.github/workflows/test-build.yml @@ -190,6 +190,7 @@ jobs: lib/knowledge/__integration__/search-reference-batching.integration.ts lib/core/outbox/service.integration.ts lib/knowledge/__integration__/connector-upload.integration.ts + lib/uploads/contexts/organization-logo/application.integration.ts test-build: name: Lint and Test diff --git a/apps/docs/components/ui/icon-mapping.ts b/apps/docs/components/ui/icon-mapping.ts index 51431ca589b..6178ca24e12 100644 --- a/apps/docs/components/ui/icon-mapping.ts +++ b/apps/docs/components/ui/icon-mapping.ts @@ -339,6 +339,7 @@ export const blockTypeToIconMap: Record = { azure_devops: AzureIcon, bitbucket: BitbucketIcon, box: BoxCompanyIcon, + box_v2: BoxCompanyIcon, brandfetch: BrandfetchIcon, brex: BrexIcon, brightdata: BrightDataIcon, @@ -380,9 +381,11 @@ export const blockTypeToIconMap: Record = { docusign: DocuSignIcon, downdetector: DowndetectorIcon, dropbox: DropboxIcon, + dropbox_v2: DropboxIcon, dropcontact: DropcontactIcon, dspy: DsPyIcon, dub: DubIcon, + dub_v2: DubIcon, duckduckgo: DuckDuckGoIcon, dynamodb: DynamoDBIcon, dynatrace: DynatraceIcon, @@ -475,6 +478,7 @@ export const blockTypeToIconMap: Record = { jotform: JotformIcon, jsm: JiraServiceManagementIcon, jupyter: JupyterIcon, + jupyter_v2: JupyterIcon, kalshi: KalshiIcon, kalshi_v2: KalshiIcon, ketch: KetchIcon, @@ -506,6 +510,7 @@ export const blockTypeToIconMap: Record = { memory: BrainIcon, microsoft_ad: AzureIcon, microsoft_dataverse: MicrosoftDataverseIcon, + microsoft_dataverse_v2: MicrosoftDataverseIcon, microsoft_dynamics_365: MicrosoftDataverseIcon, microsoft_excel: MicrosoftExcelIcon, microsoft_excel_v2: MicrosoftExcelIcon, @@ -557,6 +562,7 @@ export const blockTypeToIconMap: Record = { quartr: QuartrIcon, quickbooks: QuickBooksIcon, quiver: QuiverIcon, + quiver_v2: QuiverIcon, rabbitmq: RabbitmqIcon, railway: RailwayIcon, rb2b: RB2BIcon, @@ -588,8 +594,10 @@ export const blockTypeToIconMap: Record = { sentry: SentryIcon, serper: SerperIcon, servicenow: ServiceNowIcon, + servicenow_v2: ServiceNowIcon, ses: SESIcon, sftp: SftpIcon, + sftp_v2: SftpIcon, sharepoint: MicrosoftSharepointIcon, sharepoint_v2: MicrosoftSharepointIcon, shopify: ShopifyIcon, @@ -607,6 +615,7 @@ export const blockTypeToIconMap: Record = { sqs: SQSIcon, square: SquareIcon, ssh: SshIcon, + ssh_v2: SshIcon, ssm: SSMIcon, stagehand: StagehandIcon, start_trigger: StartIcon, diff --git a/apps/docs/content/docs/cli/usage-data.mdx b/apps/docs/content/docs/cli/usage-data.mdx index 61528c8554c..229bfa374e5 100644 --- a/apps/docs/content/docs/cli/usage-data.mdx +++ b/apps/docs/content/docs/cli/usage-data.mdx @@ -31,6 +31,13 @@ workspace ids, error messages, environment variable values, credentials, or the address of the deployment you talk to. The report leaves your machine from a separate short-lived process that is not given your API key. +## Location + +The report carries no location of its own. The analytics service derives an +approximate location (country and city) from the address the report arrived +from, the way any HTTP request exposes one, and Sim uses that only to see +which regions use the CLI. + ## Identity The first run mints a random device id and stores it in `telemetry.json` under diff --git a/apps/docs/content/docs/integrations/box.mdx b/apps/docs/content/docs/integrations/box.mdx index db628a6699d..f165d7f9058 100644 --- a/apps/docs/content/docs/integrations/box.mdx +++ b/apps/docs/content/docs/integrations/box.mdx @@ -6,7 +6,7 @@ description: Manage files, folders, and e-signatures with Box import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -63,7 +63,6 @@ Download a file from Box | Parameter | Type | Description | | --------- | ---- | ----------- | | `file` | file | Downloaded file stored in execution files | -| `content` | string | Base64 encoded file content | ### Box Get File Info diff --git a/apps/docs/content/docs/integrations/dropbox.mdx b/apps/docs/content/docs/integrations/dropbox.mdx index 27b95131829..ebd17a220a4 100644 --- a/apps/docs/content/docs/integrations/dropbox.mdx +++ b/apps/docs/content/docs/integrations/dropbox.mdx @@ -6,7 +6,7 @@ description: Upload, download, share, and manage files in Dropbox import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -66,7 +66,7 @@ Upload a file to Dropbox ### Dropbox Download File -Download a file from Dropbox with metadata and content +Download a file from Dropbox with metadata #### Input @@ -81,7 +81,6 @@ Download a file from Dropbox with metadata and content | `file` | file | Downloaded file stored in execution files | | `metadata` | json | The file metadata | | `temporaryLink` | string | Temporary link to download the file \(valid for ~4 hours\) | -| `content` | string | Base64 encoded file content \(if fetched\) | ### Dropbox List Folder diff --git a/apps/docs/content/docs/integrations/dub.mdx b/apps/docs/content/docs/integrations/dub.mdx index beb890bbf74..5ad7e6c23f1 100644 --- a/apps/docs/content/docs/integrations/dub.mdx +++ b/apps/docs/content/docs/integrations/dub.mdx @@ -6,7 +6,7 @@ description: Link management with Dub import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -474,7 +474,6 @@ Generate a customizable QR code (PNG) for a short link, with control over size, | Parameter | Type | Description | | --------- | ---- | ----------- | | `file` | file | Generated QR code image stored in execution files | -| `content` | string | Base64-encoded PNG image data | ### Dub List Domains diff --git a/apps/docs/content/docs/integrations/jupyter.mdx b/apps/docs/content/docs/integrations/jupyter.mdx index ceb5266a318..2025e5fc79f 100644 --- a/apps/docs/content/docs/integrations/jupyter.mdx +++ b/apps/docs/content/docs/integrations/jupyter.mdx @@ -6,7 +6,7 @@ description: Manage files, notebooks, kernels, and sessions on a Jupyter server import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -61,7 +61,7 @@ List files, notebooks, and subdirectories at a path on a Jupyter server ### Jupyter Get Content -Read a file or notebook from a Jupyter server +Download a file as a stored file, or read structured notebook and directory content #### Input @@ -75,11 +75,11 @@ Read a file or notebook from a Jupyter server | Parameter | Type | Description | | --------- | ---- | ----------- | -| `name` | string | File or notebook name | -| `path` | string | Path relative to the server root | -| `mimetype` | string | MIME type of the content | -| `text` | string | Text content, for text files and notebooks \(JSON-stringified\) | -| `file` | file | Binary content stored as a file, for base64-format content | +| `file` | file | Downloaded file | +| `text` | string | JSON-stringified notebook or directory content | +| `name` | string | Notebook or directory name | +| `path` | string | Notebook or directory path | +| `mimetype` | string | Notebook or directory MIME type | ### Jupyter Create File diff --git a/apps/docs/content/docs/integrations/microsoft_dataverse.mdx b/apps/docs/content/docs/integrations/microsoft_dataverse.mdx index 596dd1db9a7..91662cfea31 100644 --- a/apps/docs/content/docs/integrations/microsoft_dataverse.mdx +++ b/apps/docs/content/docs/integrations/microsoft_dataverse.mdx @@ -6,7 +6,7 @@ description: Manage records in Microsoft Dataverse tables import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -136,7 +136,7 @@ Remove an association between two records in Microsoft Dataverse. For collection ### Download File from Microsoft Dataverse -Download a file from a file or image column on a Dataverse record. Stores the file in execution storage and returns a file reference, plus the base64 content and metadata directly. +Download a file from a Dataverse file or image column and return its stored file reference and metadata #### Input @@ -152,12 +152,7 @@ Download a file from a file or image column on a Dataverse record. Stores the fi | Parameter | Type | Description | | --------- | ---- | ----------- | | `file` | file | Downloaded file stored in execution files | -| `fileContent` | string | Base64-encoded file content | -| `fileName` | string | Name of the downloaded file | -| `fileSize` | number | File size in bytes | -| `mimeType` | string | MIME type of the file | | `fileColumn` | string | File column the file was downloaded from | -| `success` | boolean | Whether the file was downloaded successfully | ### Execute Microsoft Dataverse Action diff --git a/apps/docs/content/docs/integrations/quiver.mdx b/apps/docs/content/docs/integrations/quiver.mdx index 282ca9aca7b..43a658acd42 100644 --- a/apps/docs/content/docs/integrations/quiver.mdx +++ b/apps/docs/content/docs/integrations/quiver.mdx @@ -6,7 +6,7 @@ description: Generate and vectorize SVGs import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -57,16 +57,12 @@ Generate SVG images from text prompts using QuiverAI | Parameter | Type | Description | | --------- | ---- | ----------- | -| `success` | boolean | Whether the SVG generation succeeded | -| `output` | object | Generated SVG output | -| ↳ `file` | file | First generated SVG file | -| ↳ `files` | json | All generated SVG files \(when n > 1\) | -| ↳ `svgContent` | string | Raw SVG markup content of the first result | -| ↳ `id` | string | Generation request ID | -| ↳ `usage` | json | Token usage statistics | -| ↳ `totalTokens` | number | Total tokens used | -| ↳ `inputTokens` | number | Input tokens used | -| ↳ `outputTokens` | number | Output tokens used | +| `files` | file[] | All generated SVG files | +| `id` | string | Request ID | +| `usage` | json | Token usage statistics | +| ↳ `totalTokens` | number | Total tokens used | +| ↳ `inputTokens` | number | Input tokens used | +| ↳ `outputTokens` | number | Output tokens used | ### Quiver Image to SVG @@ -90,15 +86,12 @@ Convert raster images into vector SVG format using QuiverAI | Parameter | Type | Description | | --------- | ---- | ----------- | -| `success` | boolean | Whether the vectorization succeeded | -| `output` | object | Vectorized SVG output | -| ↳ `file` | file | Generated SVG file | -| ↳ `svgContent` | string | Raw SVG markup content | -| ↳ `id` | string | Vectorization request ID | -| ↳ `usage` | json | Token usage statistics | -| ↳ `totalTokens` | number | Total tokens used | -| ↳ `inputTokens` | number | Input tokens used | -| ↳ `outputTokens` | number | Output tokens used | +| `files` | file[] | All generated SVG files | +| `id` | string | Request ID | +| `usage` | json | Token usage statistics | +| ↳ `totalTokens` | number | Total tokens used | +| ↳ `inputTokens` | number | Input tokens used | +| ↳ `outputTokens` | number | Output tokens used | ### Quiver List Models diff --git a/apps/docs/content/docs/integrations/servicenow.mdx b/apps/docs/content/docs/integrations/servicenow.mdx index 84482fba5fc..9fb36bf30f2 100644 --- a/apps/docs/content/docs/integrations/servicenow.mdx +++ b/apps/docs/content/docs/integrations/servicenow.mdx @@ -6,7 +6,7 @@ description: Create, read, update, and delete ServiceNow records import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -189,7 +189,6 @@ Download an attachment file from ServiceNow by its sys_id | Parameter | Type | Description | | --------- | ---- | ----------- | | `file` | file | Downloaded attachment stored in execution files | -| `content` | string | Base64 encoded file content | ### Upload ServiceNow Attachment @@ -258,7 +257,7 @@ Create an incident in ServiceNow. Reference fields (caller, assignment group, as | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -306,7 +305,7 @@ Retrieve a single ServiceNow incident by number (e.g., INC0010001) or sys_id. Re | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -362,7 +361,7 @@ Search ServiceNow incidents by state, priority, assignment, caller, or text. All | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -424,7 +423,7 @@ Update fields on an existing ServiceNow incident. Only the fields you supply are | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -476,7 +475,7 @@ Move a ServiceNow incident to Resolved (state 6) with a resolution code and reso | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -528,7 +527,7 @@ Move a ServiceNow incident to Closed (state 7) with a resolution code and resolu | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -578,7 +577,7 @@ Append an internal work note or a customer-visible additional comment to a Servi | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -643,7 +642,7 @@ Create a change request in ServiceNow. Reference fields (assignment group, assig | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -691,7 +690,7 @@ Retrieve a single ServiceNow change request by number (e.g., CHG0030001) or sys_ | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -747,7 +746,7 @@ Search ServiceNow change requests by state, type, risk, assignment, or text. All | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -809,7 +808,7 @@ Update fields on an existing ServiceNow change request. Only the fields you supp | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -862,7 +861,7 @@ Move a ServiceNow change request to another state. Base-system change model stat | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1040,7 +1039,7 @@ List requested items (RITMs) from the ServiceNow Requested Item [sc_req_item] ta | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1088,7 +1087,7 @@ Retrieve a single ServiceNow requested item (RITM) by number (e.g., RITM0010001) | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1140,7 +1139,7 @@ List approval records from the ServiceNow Approval [sysapproval_approver] table. | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1190,7 +1189,7 @@ Approve or reject a ServiceNow approval record by setting its state on the Appro | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1242,7 +1241,7 @@ Search the ServiceNow CMDB for configuration items. Defaults to the base cmdb_ci | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1324,7 +1323,7 @@ List rows from the CI Relationship [cmdb_rel_ci] table for a configuration item. | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1444,7 +1443,7 @@ Look up ServiceNow users by email, user name, or display name. Use this to resol | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | @@ -1495,7 +1494,7 @@ List the members of a ServiceNow group from the Group Member [sys_user_grmember] | `count` | number | Aggregate matching record count | | `attachments` | json | Attachment metadata list — record attachments \(sys_id, file_name, content_type, download_link\) or knowledge article attachments \(sys_id, file_name, size_bytes, state\) | | `file` | file | Downloaded attachment file | -| `content` | string | Base64-encoded downloaded file content, or the HTML body of a knowledge article | +| `content` | string | HTML body of a knowledge article | | `attachment` | json | Uploaded attachment metadata | | `tasks` | json | Change tasks belonging to a change request. Unlike the Table API operations, the Change Management API always returns every field as \{value, display_value\} regardless of the display value setting | | `items` | json | Service catalog items | diff --git a/apps/docs/content/docs/integrations/sftp.mdx b/apps/docs/content/docs/integrations/sftp.mdx index 60b2397b18a..a62e82ca437 100644 --- a/apps/docs/content/docs/integrations/sftp.mdx +++ b/apps/docs/content/docs/integrations/sftp.mdx @@ -6,7 +6,7 @@ description: Transfer files via SFTP (SSH File Transfer Protocol) import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -77,19 +77,12 @@ Download a file from a remote SFTP server | `privateKey` | string | No | Private key for authentication \(OpenSSH format\) | | `passphrase` | string | No | Passphrase for encrypted private key | | `remotePath` | string | Yes | Path to the file on the remote server | -| `encoding` | string | No | Output encoding: utf-8 for text, base64 for binary \(default: utf-8\) | #### Output | Parameter | Type | Description | | --------- | ---- | ----------- | -| `success` | boolean | Whether the download was successful | | `file` | file | Downloaded file stored in execution files | -| `fileName` | string | Name of the downloaded file | -| `content` | string | File content \(text or base64 encoded\) | -| `size` | number | File size in bytes | -| `encoding` | string | Content encoding \(utf-8 or base64\) | -| `message` | string | Operation status message | ### SFTP List Directory diff --git a/apps/docs/content/docs/integrations/ssh.mdx b/apps/docs/content/docs/integrations/ssh.mdx index 3b6e7631df7..23e16880e0e 100644 --- a/apps/docs/content/docs/integrations/ssh.mdx +++ b/apps/docs/content/docs/integrations/ssh.mdx @@ -6,7 +6,7 @@ description: Connect to remote servers via SSH import { BlockInfoCard } from "@/components/ui/block-info-card" @@ -161,13 +161,8 @@ Download a file from a remote SSH server | Parameter | Type | Description | | --------- | ---- | ----------- | -| `downloaded` | boolean | Whether the file was downloaded successfully | | `file` | file | Downloaded file stored in execution files | -| `fileContent` | string | File content \(base64 encoded for binary files\) | -| `fileName` | string | Name of the downloaded file | | `remotePath` | string | Source path on the remote server | -| `size` | number | File size in bytes | -| `message` | string | Operation status message | ### SSH List Directory diff --git a/apps/sim/.env.example b/apps/sim/.env.example index 5b8106a17b7..c8b8583d103 100644 --- a/apps/sim/.env.example +++ b/apps/sim/.env.example @@ -254,6 +254,9 @@ CRON_SECRET=your_cron_secret # Use `openssl rand -hex 32` to generate. Authentic # MISTRAL_OCR_QUOTA_GROUPS={"<64-character lowercase key fingerprint>":"organization-id"} # Official Sim Search Slack app (optional; requires existing Search access) -# Register the company app with bun scripts/register-platform-slack-app.ts --search. +# Supply these through the deployment environment (for example, ECS Secrets Manager injection). # SLACK_SEARCH_APP_ID= +# SLACK_SEARCH_CLIENT_ID= +# SLACK_SEARCH_CLIENT_SECRET= +# SLACK_SEARCH_SIGNING_SECRET= # SLACK_SEARCH_SHARED_APP=false # Off-production fallback for the global slack-search-shared-app flag diff --git a/apps/sim/app/_shell/consent/consent-banner.test.tsx b/apps/sim/app/_shell/consent/consent-banner.test.tsx new file mode 100644 index 00000000000..76af8161697 --- /dev/null +++ b/apps/sim/app/_shell/consent/consent-banner.test.tsx @@ -0,0 +1,87 @@ +/** + * @vitest-environment jsdom + */ +import { act } from 'react' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockUseConsentManager, mockUseHeadlessConsentUI } = vi.hoisted(() => ({ + mockUseConsentManager: vi.fn(), + mockUseHeadlessConsentUI: vi.fn(), +})) + +vi.mock('@c15t/nextjs/headless', () => ({ + useConsentManager: mockUseConsentManager, + useHeadlessConsentUI: mockUseHeadlessConsentUI, +})) + +vi.mock('@/app/_shell/consent/consent-preferences', () => ({ + CONSENT_LINK_CLASS: 'link', + ConsentPreferences: () => , +})) + +import { ConsentBanner } from '@/app/_shell/consent/consent-banner' + +let root: Root | null = null + +function render(): HTMLDivElement { + ;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true + const container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) + act(() => root?.render()) + return container +} + +function isBannerShown(container: HTMLDivElement): boolean { + return container.querySelector('section[aria-label="Cookie preferences"]') !== null +} + +beforeEach(() => { + mockUseHeadlessConsentUI.mockReturnValue({ + banner: { isVisible: true, allowedActions: ['accept', 'reject', 'customize'] }, + dialog: { isVisible: false, allowedActions: [] }, + openDialog: vi.fn(), + performAction: vi.fn(), + saveCustomPreferences: vi.fn(), + }) +}) + +afterEach(() => { + act(() => root?.unmount()) + root = null + vi.clearAllMocks() +}) + +describe('ConsentBanner', () => { + it.each(['backend', 'backend-cache-hit', 'ssr'])( + 'asks for consent when the policy came from %s', + (initDataSource) => { + mockUseConsentManager.mockReturnValue({ initDataSource }) + + expect(isBannerShown(render())).toBe(true) + } + ) + + it('still renders a dialog the visitor opened, so the published control works', () => { + mockUseHeadlessConsentUI.mockReturnValue({ + banner: { isVisible: false, allowedActions: [] }, + dialog: { isVisible: true, allowedActions: ['accept', 'reject', 'customize'] }, + openDialog: vi.fn(), + performAction: vi.fn(), + saveCustomPreferences: vi.fn(), + }) + mockUseConsentManager.mockReturnValue({ initDataSource: 'offline-fallback' }) + + expect(isBannerShown(render())).toBe(true) + }) + + it('asks nothing when the policy lookup fell back', () => { + // A bot challenge on the third-party `/init` makes the runtime substitute a + // generic opt-in policy, which would otherwise re-prompt visitors who had + // already consented under the real one. + mockUseConsentManager.mockReturnValue({ initDataSource: 'offline-fallback' }) + + expect(isBannerShown(render())).toBe(false) + }) +}) diff --git a/apps/sim/app/_shell/consent/consent-banner.tsx b/apps/sim/app/_shell/consent/consent-banner.tsx index a3878c0d02b..bd3143edb1c 100644 --- a/apps/sim/app/_shell/consent/consent-banner.tsx +++ b/apps/sim/app/_shell/consent/consent-banner.tsx @@ -1,6 +1,6 @@ 'use client' -import { useHeadlessConsentUI } from '@c15t/nextjs/headless' +import { useConsentManager, useHeadlessConsentUI } from '@c15t/nextjs/headless' import { Chip } from '@sim/emcn' import { AnimatePresence, motion, useReducedMotion } from 'framer-motion' import Link from 'next/link' @@ -29,12 +29,29 @@ const CATEGORIES_OPEN = { height: 'auto', opacity: 1 } as const * the light layer on `` through `ThemeProvider`'s forced theme, or is a * themed app page where inheriting is what should happen — the card no longer * decides for itself. + * + * Nothing is asked *unprompted* when the policy lookup failed. `/init` answers + * from a third-party origin, and when that origin refuses the request — a bot + * challenge returns `403` with an HTML body — the runtime substitutes a generic + * opt-in policy rather than surfacing the failure. Volunteering a banner from + * it asks a question the visitor's jurisdiction may not require, and asks it of + * people who already answered, because the substituted policy's fingerprint + * never matches the one their stored consent was recorded under. The next load + * that reaches the real policy asks properly if it still needs to. + * + * A dialog the visitor opened themselves still renders, fallback or not: the + * Cookie Policy promises the choice can be changed at any time, and a control + * that silently does nothing breaks that promise. A choice saved during a + * fallback is recorded against the substituted policy and will be asked for + * again once the real one resolves, which is the lesser of the two failures. */ export function ConsentBanner() { const { banner, dialog, openDialog, performAction, saveCustomPreferences } = useHeadlessConsentUI() + const { initDataSource } = useConsentManager() const prefersReducedMotion = useReducedMotion() + const isPolicyResolved = initDataSource !== 'offline-fallback' const isExpanded = dialog.isVisible const surfaceName = isExpanded ? 'dialog' : 'banner' const { allowedActions } = isExpanded ? dialog : banner @@ -42,7 +59,7 @@ export function ConsentBanner() { return ( - {(banner.isVisible || dialog.isVisible) && ( + {((isPolicyResolved && banner.isVisible) || dialog.isVisible) && ( ({ })) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: { - publishStatusChanged: mockPublishStatusChanged, - }, + publishChatStatusChanged: mockPublishStatusChanged, })) import { POST } from '@/app/api/copilot/chat/stop/route' @@ -59,7 +57,7 @@ describe('copilot chat stop route', () => { user: { id: 'user-1' }, session: { id: 'session-1' }, }) - mockGetAccessibleChat.mockResolvedValue({ id: 'chat-1' }) + mockGetAccessibleChat.mockResolvedValue({ id: 'chat-1', workspaceId: 'ws-1', userId: 'user-1' }) }) it('does not persist stopped content after organization access is removed', async () => { @@ -120,12 +118,14 @@ describe('copilot chat stop route', () => { contentBlocks: [{ type: 'complete', status: 'cancelled' }], }) - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'ws-1', - chatId: 'chat-1', - type: 'completed', - streamId: 'stream-1', - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'ws-1' }), + { + chatId: 'chat-1', + type: 'completed', + streamId: 'stream-1', + } + ) }) it('appends a stopped assistant message if the stream marker was already cleared', async () => { @@ -145,12 +145,14 @@ describe('copilot chat stop route', () => { const [, appended] = mockAppendCopilotChatMessages.mock.calls[0] expect(appended[0]).toMatchObject({ role: 'assistant', content: 'partial' }) - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'ws-1', - chatId: 'chat-1', - type: 'completed', - streamId: 'stream-1', - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'ws-1' }), + { + chatId: 'chat-1', + type: 'completed', + streamId: 'stream-1', + } + ) }) it('republishes completed status when the assistant was already persisted', async () => { @@ -167,11 +169,13 @@ describe('copilot chat stop route', () => { expect(await response.json()).toEqual({ success: true }) expect(mockAppendCopilotChatMessages).not.toHaveBeenCalled() expect(dbChainMockFns.set).not.toHaveBeenCalled() - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'ws-1', - chatId: 'chat-1', - type: 'completed', - streamId: 'stream-1', - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'ws-1' }), + { + chatId: 'chat-1', + type: 'completed', + streamId: 'stream-1', + } + ) }) }) diff --git a/apps/sim/app/api/copilot/chat/stop/route.ts b/apps/sim/app/api/copilot/chat/stop/route.ts index ef02d470844..29cf8900229 100644 --- a/apps/sim/app/api/copilot/chat/stop/route.ts +++ b/apps/sim/app/api/copilot/chat/stop/route.ts @@ -11,7 +11,7 @@ import { withStoppedContentBlock, } from '@/lib/copilot/chat/persisted-message' import { finalizeAssistantTurn } from '@/lib/copilot/chat/terminal-state' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { CopilotChatFinalizeOutcome, CopilotStopOutcome, @@ -87,9 +87,8 @@ export const POST = withRouteHandler((req: NextRequest) => const shouldPublishCompleted = result.updated || result.outcome === CopilotChatFinalizeOutcome.AssistantAlreadyPersisted - if (shouldPublishCompleted && result.workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId: result.workspaceId, + if (shouldPublishCompleted) { + publishChatStatusChanged(chat, { chatId, type: 'completed', streamId, diff --git a/apps/sim/app/api/files/authorization.test.ts b/apps/sim/app/api/files/authorization.test.ts index 22525582e75..b449eab962f 100644 --- a/apps/sim/app/api/files/authorization.test.ts +++ b/apps/sim/app/api/files/authorization.test.ts @@ -63,6 +63,14 @@ function grantAccess(cloudKey: string) { } describe('verifyKBFileAccess (binding-only)', () => { + it.each(['mothership', 'profile-pictures', 'general'] as const)( + 'refuses organization image keys through legacy %s authorization', + async (context) => { + await expect( + verifyFileAccess('assistant/org-1/user-1/upload-1/image.png', USER_ID, undefined, context) + ).resolves.toBe(false) + } + ) beforeEach(() => { vi.clearAllMocks() // Default liveness query result: one active document references the exact storage key. @@ -174,6 +182,22 @@ describe('public-context access (profile-pictures / og-images / workspace-logos) return verifyFileAccess(cloudKey, USER_ID, undefined, context, false, { requireWrite: true }) } + it('allows organization logo reads and denies generic deletes even for the uploader', async () => { + const key = 'organization-logos/org-1/logo.png' + mockGetFileMetadata.mockResolvedValue({ userId: USER_ID }) + await expect(verifyFileAccess(key, USER_ID, undefined, 'organization-logos')).resolves.toBe( + true + ) + await expect( + verifyFileAccess(key, USER_ID, undefined, 'organization-logos', false, { requireWrite: true }) + ).resolves.toBe(false) + await expect( + verifyFileAccess(key, USER_ID, undefined, 'general', false, { requireWrite: true }) + ).resolves.toBe(false) + expect(mockGetFileMetadata).not.toHaveBeenCalled() + expect(mockGetUserEntityPermissions).not.toHaveBeenCalled() + }) + it('grants public reads without any ownership check', async () => { await expect(read('og-images/banner.png', 'og-images')).resolves.toBe(true) await expect(read('profile-pictures/123-avatar.png', 'profile-pictures')).resolves.toBe(true) diff --git a/apps/sim/app/api/files/authorization.ts b/apps/sim/app/api/files/authorization.ts index 618b4e9711e..647e5b5792b 100644 --- a/apps/sim/app/api/files/authorization.ts +++ b/apps/sim/app/api/files/authorization.ts @@ -150,9 +150,13 @@ export async function verifyFileAccess( isLocal?: boolean, options?: { requireWrite?: boolean; knowledgeAccess?: KnowledgeFileAccess } ): Promise { + /** Organization images require the Principal-aware Assistant application resolver. */ + if (cloudKey.startsWith('assistant/')) return false const requireWrite = options?.requireWrite ?? false try { const keyContext = inferContextFromKey(cloudKey) + /** Organization logos are changed only through the organization-authorized upload lifecycle. */ + if (keyContext === 'organization-logos') return !requireWrite if (keyContext === 'knowledge-base') { return requireWrite ? verifyKBFileWriteAccess(cloudKey, userId) diff --git a/apps/sim/app/api/files/serve/[...path]/route.test.ts b/apps/sim/app/api/files/serve/[...path]/route.test.ts index 755861df05f..e3937b9e78d 100644 --- a/apps/sim/app/api/files/serve/[...path]/route.test.ts +++ b/apps/sim/app/api/files/serve/[...path]/route.test.ts @@ -3,7 +3,12 @@ * * @vitest-environment node */ -import { hybridAuthMockFns, storageServiceMock, storageServiceMockFns } from '@sim/testing' +import { + authMockFns, + hybridAuthMockFns, + storageServiceMock, + storageServiceMockFns, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' @@ -34,6 +39,7 @@ const { mockCreateErrorResponse, FileNotFoundError, serveLogger, + mockReadOrganizationAssistantImage, } = vi.hoisted(() => { class FileNotFoundErrorClass extends Error { constructor(message: string) { @@ -43,6 +49,7 @@ const { } return { serveLogger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() }, + mockReadOrganizationAssistantImage: vi.fn(), mockVerifyFileAccess: vi.fn(), mockReadFile: vi.fn(), mockIsUsingCloudStorage: vi.fn(), @@ -62,6 +69,10 @@ const { } }) +vi.mock('@/lib/uploads/contexts/organization-assistant/application', () => ({ + readOrganizationAssistantImage: mockReadOrganizationAssistantImage, +})) + vi.mock('fs/promises', () => ({ readFile: mockReadFile, access: vi.fn().mockResolvedValue(undefined), @@ -204,6 +215,43 @@ describe('File Serve API Route', () => { }) }) + it('serves private Assistant images through session authorization and disables caching', async () => { + authMockFns.mockGetSession.mockResolvedValue({ + user: { id: 'user-1' }, + session: { id: 'session-1' }, + }) + const key = 'assistant/org-1/user-1/upload-1/image.png' + mockReadOrganizationAssistantImage.mockResolvedValue({ + name: 'image.png', + contentType: 'image/webp', + buffer: Buffer.from('decoded-image'), + }) + const response = await GET(new NextRequest(`http://localhost/api/files/serve/${key}`), { + params: Promise.resolve({ path: key.split('/') }), + }) + expect(response.status).toBe(200) + expect(mockReadOrganizationAssistantImage).toHaveBeenCalledWith({ + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + key, + signal: expect.any(AbortSignal), + }) + expect(mockCreateFileResponse).toHaveBeenCalledWith( + expect.objectContaining({ cacheControl: 'private, no-store', contentType: 'image/webp' }) + ) + expect(mockVerifyFileAccess).not.toHaveBeenCalled() + expect(hybridAuthMockFns.mockCheckSessionOrInternalAuth).not.toHaveBeenCalled() + }) + + it('requires a real session for private Assistant images even when legacy auth succeeds', async () => { + authMockFns.mockGetSession.mockResolvedValue(null) + const key = 'assistant/org-1/user-1/upload-1/image.png' + const response = await GET(new NextRequest(`http://localhost/api/files/serve/${key}`), { + params: Promise.resolve({ path: key.split('/') }), + }) + expect(response.status).toBe(401) + expect(mockReadOrganizationAssistantImage).not.toHaveBeenCalled() + }) + it('bounds the local read rather than trusting the stored size', async () => { await GET(new NextRequest('http://localhost:3000/api/files/serve/workspace/ws/test-file.txt'), { params: Promise.resolve({ path: ['workspace', 'ws', 'test-file.txt'] }), @@ -408,6 +456,28 @@ describe('File Serve API Route', () => { }) }) + it('serves organization logos through the existing public asset path', async () => { + mockIsUsingCloudStorage.mockReturnValue(true) + mockInferContextFromKey.mockReturnValue('organization-logos') + const key = 'organization-logos/org-1/upload-1-logo.png' + const response = await GET(new NextRequest(`http://localhost/api/files/serve/s3/${key}`), { + params: Promise.resolve({ path: ['s3', 'organization-logos', 'org-1', 'upload-1-logo.png'] }), + }) + expect(response.status).toBe(200) + expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({ + key, + context: 'organization-logos', + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + }) + expect(mockCreateFileResponse).toHaveBeenCalledWith( + expect.objectContaining({ + cacheControl: 'public, max-age=31536000', + }) + ) + expect(mockVerifyFileAccess).not.toHaveBeenCalled() + expect(mockAuthenticateWorkspaceFile).not.toHaveBeenCalled() + }) + it('should return 404 when file not found', async () => { mockVerifyFileAccess.mockResolvedValue(false) mockFindLocalFile.mockReturnValue(null) diff --git a/apps/sim/app/api/files/serve/[...path]/route.ts b/apps/sim/app/api/files/serve/[...path]/route.ts index ffb8845aeff..1bd88eeffd9 100644 --- a/apps/sim/app/api/files/serve/[...path]/route.ts +++ b/apps/sim/app/api/files/serve/[...path]/route.ts @@ -7,6 +7,7 @@ import { fileServeParamsSchema, fileServeQuerySchema } from '@/lib/api/contracts import { concealCrossTenantResourceError, InternalUnauthenticatedError, + internalSessionAuth, } from '@/lib/api/server/routes' import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid' import { resolveServableDocBytes } from '@/lib/copilot/tools/server/files/doc-compile' @@ -16,6 +17,7 @@ import { assertKnownSizeWithinLimit, isPayloadSizeLimitError } from '@/lib/core/ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { CopilotFiles, isUsingCloudStorage } from '@/lib/uploads' import type { StorageContext } from '@/lib/uploads/config' +import { readOrganizationAssistantImage } from '@/lib/uploads/contexts/organization-assistant/application' import { parseWorkspaceFileKey } from '@/lib/uploads/contexts/workspace/workspace-file-manager' import { downloadFile } from '@/lib/uploads/core/storage-service' import { resolveServableImageBytes } from '@/lib/uploads/server/image-derivative' @@ -218,10 +220,26 @@ export const GET = withRouteHandler( const isCloudPath = isS3Path || isBlobPath || isGcsPath const cloudKey = isCloudPath ? path.slice(1).join('/') : fullPath + if (cloudKey.startsWith('assistant/')) { + const principal = await internalSessionAuth.authenticate() + const image = await readOrganizationAssistantImage({ + principal, + key: cloudKey, + signal: request.signal, + }) + return createFileResponse({ + buffer: image.buffer, + filename: image.name, + contentType: image.contentType, + cacheControl: 'private, no-store', + }) + } + const isPublicByKeyPrefix = cloudKey.startsWith('profile-pictures/') || cloudKey.startsWith('og-images/') || - cloudKey.startsWith('workspace-logos/') + cloudKey.startsWith('workspace-logos/') || + cloudKey.startsWith('organization-logos/') if (isPublicByKeyPrefix) { const context = inferContextFromKey(cloudKey) diff --git a/apps/sim/app/api/files/uploads/finalizers.ts b/apps/sim/app/api/files/uploads/finalizers.ts index bc51d3a034a..963b421720c 100644 --- a/apps/sim/app/api/files/uploads/finalizers.ts +++ b/apps/sim/app/api/files/uploads/finalizers.ts @@ -1,6 +1,7 @@ import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { type Principal, resolvePrincipalAuditAttribution } from '@sim/auth/principal' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { type WorkspaceFileRow, workspaceFileColumns, workspaceFiles } from '@sim/db/schema' import { generateId } from '@sim/utils/id' import { eq, sql } from 'drizzle-orm' @@ -9,6 +10,11 @@ import type { OrchestrationRequestContext } from '@/lib/core/orchestration/types import { captureServerEvent } from '@/lib/posthog/server' import { notifyWorkspaceFilesChanged } from '@/lib/realtime/notify' import { getServeStoragePrefix } from '@/lib/uploads/config' +import { finalizeOrganizationAssistantAttachment } from '@/lib/uploads/contexts/organization-assistant/application' +import { + finalizeOrganizationLogoUpload, + organizationLogoUploadResult, +} from '@/lib/uploads/contexts/organization-logo/application' import { getWorkspaceFile, registerUploadedWorkspaceFile, @@ -104,9 +110,14 @@ export async function finalizeUploadPurpose({ ) case 'profile_picture': return { value: storedAssetResult(session, 'profile-pictures') } + case 'organization_logo': + return finalizeOrganizationLogoUpload(principal, session, request) case 'workspace_logo': return finalizeWorkspaceLogo(session, actor, request) case 'mothership_attachment': + if (session.workspaceId === null) { + return { value: await finalizeOrganizationAssistantAttachment(principal, session) } + } return finalizeMothershipAttachment(session) case 'execution_attachment': return finalizeExecutionAttachment(session) @@ -132,6 +143,8 @@ export async function loadCompletedUploadPurpose( switch (session.purpose) { case 'workspace_file': return toV2File(await loadCompletedWorkspaceFileUpload(session)) + case 'organization_logo': + return organizationLogoUploadResult(session) case 'profile_picture': case 'workspace_logo': case 'mothership_attachment': @@ -354,7 +367,7 @@ async function insertOrLoadFileMetadata( const now = new Date() const [inserted] = await db - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ id: generateId(), key: input.key, diff --git a/apps/sim/app/api/files/uploads/purposes.ts b/apps/sim/app/api/files/uploads/purposes.ts index dddddf82663..204de756eb8 100644 --- a/apps/sim/app/api/files/uploads/purposes.ts +++ b/apps/sim/app/api/files/uploads/purposes.ts @@ -24,6 +24,7 @@ const INTERNAL_UPLOAD_PURPOSES = new Set([ 'workspace_file', 'profile_picture', 'workspace_logo', + 'organization_logo', 'mothership_attachment', 'execution_attachment', ]) @@ -57,6 +58,11 @@ export async function createPurposeUploadSession( localOrigin, }) } + case 'organization_logo': + throw new UploadSessionError( + 'validation', + 'Organization logos require organization authorization' + ) case 'profile_picture': return createUploadSession({ purpose: body.purpose, @@ -78,6 +84,7 @@ export async function createPurposeUploadSession( localOrigin, }) case 'mothership_attachment': + if (!body.workspaceId) throw new UploadSessionError('validation', 'workspaceId is required') await requireWorkspacePermission(userId, body.workspaceId, 'write') return createUploadSession({ purpose: body.purpose, @@ -120,6 +127,11 @@ export async function reauthorizeUploadPurpose( case 'mothership_attachment': await requireWorkspacePermission(userId, requireSessionScope(session.workspaceId), 'write') return + case 'organization_logo': + throw new UploadSessionError( + 'forbidden', + 'Organization logos require organization authorization' + ) case 'profile_picture': return case 'workspace_logo': diff --git a/apps/sim/app/api/files/uploads/route.test.ts b/apps/sim/app/api/files/uploads/route.test.ts index f36e29862d1..4dc5aab2175 100644 --- a/apps/sim/app/api/files/uploads/route.test.ts +++ b/apps/sim/app/api/files/uploads/route.test.ts @@ -234,6 +234,49 @@ describe('/api/files/uploads', () => { ) }) + it.each([ + { organizationId: 'org-1', contentType: 'application/pdf', size: 100 }, + { organizationId: 'org-1', contentType: 'image/svg+xml', size: 100 }, + { organizationId: 'org-1', contentType: 'image/png', size: 5 * 1024 * 1024 + 1 }, + { organizationId: 'org-1', workspaceId: 'ws-1', contentType: 'image/png', size: 100 }, + ])('rejects unsupported organization attachments before application loading', async (body) => { + const response = await createUpload( + new NextRequest('http://localhost/api/files/uploads', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ purpose: 'mothership_attachment', name: 'image.png', ...body }), + }) + ) + expect(response.status).toBe(400) + expect(mockCreateInternalPurposeUploadSession).not.toHaveBeenCalled() + }) + + it('creates organization image attachments through the same upload lifecycle', async () => { + mockCreateInternalPurposeUploadSession.mockResolvedValue({ + ...session({ purpose: 'mothership_attachment', storageContext: 'mothership' }), + transfer: { method: 'put', url: 'https://storage.example/upload', headers: {} }, + }) + const response = await createUpload( + new NextRequest('http://localhost/api/files/uploads', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + purpose: 'mothership_attachment', + organizationId: 'org-1', + name: 'image.png', + contentType: 'image/png', + size: 100, + }), + }) + ) + expect(response.status).toBe(201) + expect(mockCreateInternalPurposeUploadSession).toHaveBeenCalledWith( + expect.objectContaining({ kind: 'session', userId: 'user-1' }), + expect.objectContaining({ purpose: 'mothership_attachment', organizationId: 'org-1' }), + expect.anything() + ) + }) + it('rejects mothership attachments above the 5 GiB direct-to-storage limit', async () => { const request = new NextRequest('http://localhost/api/files/uploads', { method: 'POST', diff --git a/apps/sim/app/api/mothership/chats/[chatId]/fork/route.test.ts b/apps/sim/app/api/mothership/chats/[chatId]/fork/route.test.ts index 4e806387e8c..a3e687bdf27 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/fork/route.test.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/fork/route.test.ts @@ -65,7 +65,7 @@ vi.mock('@/lib/copilot/chat/messages-store', () => ({ })) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: { publishStatusChanged: mockPublishStatusChanged }, + publishChatStatusChanged: mockPublishStatusChanged, })) vi.mock('@/lib/copilot/request/go/fetch', () => ({ @@ -291,11 +291,13 @@ describe('POST /api/mothership/chats/[chatId]/fork', () => { userId: 'user-1', }) - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'ws-1', - chatId: body.id, - type: 'created', - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'ws-1' }), + { + chatId: body.id, + type: 'created', + } + ) expect(mockCaptureServerEvent).toHaveBeenCalledWith( 'user-1', 'task_forked', diff --git a/apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts b/apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts index 740456f750b..0bd87841838 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts @@ -19,7 +19,7 @@ import { rewriteMessageFileRefs, rewriteResourceFileRefs, } from '@/lib/copilot/chat/rewrite-file-references' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { fetchGo } from '@/lib/copilot/request/go/fetch' import { authenticateCopilotRequestSessionOnly, @@ -267,13 +267,7 @@ export const POST = withRouteHandler( logger.warn('Failed to fork copilot-service conversation, skipping', { err }) } - if (newChat.workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId: newChat.workspaceId, - chatId: newId, - type: 'created', - }) - } + publishChatStatusChanged({ ...parent, userId }, { chatId: newId, type: 'created' }) captureServerEvent( userId, diff --git a/apps/sim/app/api/mothership/chats/[chatId]/restore/route.test.ts b/apps/sim/app/api/mothership/chats/[chatId]/restore/route.test.ts index ee9438038b8..adfe941a4e0 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/restore/route.test.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/restore/route.test.ts @@ -26,7 +26,7 @@ vi.mock('@/lib/workspaces/permissions/utils', () => ({ })) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: { publishStatusChanged: mockPublishStatusChanged }, + publishChatStatusChanged: mockPublishStatusChanged, })) vi.mock('@/lib/posthog/server', () => ({ @@ -101,11 +101,13 @@ describe('POST /api/mothership/chats/[chatId]/restore', () => { updatedAt: expect.any(Date), lastSeenAt: expect.any(Date), }) - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'workspace-1', - chatId: 'chat-1', - type: 'created', - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'workspace-1' }), + { + chatId: 'chat-1', + type: 'created', + } + ) }) it('returns 404 when the chat is restored concurrently before the update lands', async () => { diff --git a/apps/sim/app/api/mothership/chats/[chatId]/restore/route.ts b/apps/sim/app/api/mothership/chats/[chatId]/restore/route.ts index b98f6824e1f..ed1b9d84c24 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/restore/route.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/restore/route.ts @@ -6,7 +6,7 @@ import { type NextRequest, NextResponse } from 'next/server' import { restoreMothershipChatContract } from '@/lib/api/contracts/mothership-chats' import { parseRequest } from '@/lib/api/server' import { authorizeOrganizationChat } from '@/lib/copilot/chat/organization-chats' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { authenticateCopilotRequestSessionOnly, createForbiddenResponse, @@ -95,12 +95,8 @@ export const POST = withRouteHandler( return NextResponse.json({ success: false, error: 'Chat not found' }, { status: 404 }) } + publishChatStatusChanged({ ...restoredChat, userId }, { chatId, type: 'created' }) if (restoredChat.workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId: restoredChat.workspaceId, - chatId, - type: 'created', - }) captureServerEvent( userId, 'task_restored', diff --git a/apps/sim/app/api/mothership/chats/[chatId]/route.test.ts b/apps/sim/app/api/mothership/chats/[chatId]/route.test.ts index d0f2a64c00e..ae07dee5be7 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/route.test.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/route.test.ts @@ -59,7 +59,7 @@ vi.mock('@/lib/copilot/chat/persisted-message', () => ({ })) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: { publishStatusChanged: vi.fn() }, + publishChatStatusChanged: vi.fn(), })) vi.mock('@/lib/billing/storage', () => ({ @@ -71,7 +71,8 @@ vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn(), })) -import { DELETE, GET } from '@/app/api/mothership/chats/[chatId]/route' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' +import { DELETE, GET, PATCH } from '@/app/api/mothership/chats/[chatId]/route' function makeContext(chatId: string) { return { params: Promise.resolve({ chatId }) } @@ -307,3 +308,67 @@ describe('DELETE /api/mothership/chats/[chatId]', () => { expect(mockDecrementStorageUsageForBillingContextInTx).not.toHaveBeenCalled() }) }) + +describe('organization chat mutations publish private owner updates', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValue({ + userId: 'user-1', + isAuthenticated: true, + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + }) + mockGetAccessibleCopilotChat.mockResolvedValue({ + id: 'chat-1', + type: 'mothership', + organizationId: 'org-1', + userId: 'user-1', + }) + dbChainMockFns.returning.mockResolvedValue([ + { id: 'chat-1', workspaceId: null, organizationId: 'org-1' }, + ]) + }) + + it.each([{ title: 'New title' }, { pinned: true }, { isUnread: true }, { isUnread: false }])( + 'publishes after updating %j', + async (body) => { + const response = await PATCH( + new NextRequest('http://localhost/api/mothership/chats/chat-1', { + method: 'PATCH', + body: JSON.stringify(body), + }), + makeContext('chat-1') + ) + expect(response.status).toBe(200) + expect(publishChatStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ organizationId: 'org-1', userId: 'user-1' }), + { chatId: 'chat-1', type: 'title' in body ? 'renamed' : 'updated' } + ) + } + ) + + it('publishes deletion under the same owner', async () => { + const response = await DELETE( + new NextRequest('http://localhost/api/mothership/chats/chat-1', { method: 'DELETE' }), + makeContext('chat-1') + ) + expect(response.status).toBe(200) + expect(publishChatStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ organizationId: 'org-1', userId: 'user-1' }), + { chatId: 'chat-1', type: 'deleted' } + ) + }) + + it('does not publish if a concurrent deletion leaves no updated row', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([]) + const response = await PATCH( + new NextRequest('http://localhost/api/mothership/chats/chat-1', { + method: 'PATCH', + body: JSON.stringify({ pinned: true }), + }), + makeContext('chat-1') + ) + expect(response.status).toBe(404) + expect(publishChatStatusChanged).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/mothership/chats/[chatId]/route.ts b/apps/sim/app/api/mothership/chats/[chatId]/route.ts index 6d321e169f3..e8e6eccbf8c 100644 --- a/apps/sim/app/api/mothership/chats/[chatId]/route.ts +++ b/apps/sim/app/api/mothership/chats/[chatId]/route.ts @@ -18,7 +18,7 @@ import { } from '@/lib/copilot/chat/lifecycle' import { normalizeMessage } from '@/lib/copilot/chat/persisted-message' import { reconcileChatStreamMarkers } from '@/lib/copilot/chat/stream-liveness' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { authenticateCopilotRequestSessionOnly, createInternalServerErrorResponse, @@ -199,19 +199,22 @@ export const PATCH = withRouteHandler( .returning({ id: copilotChats.id, workspaceId: copilotChats.workspaceId, + organizationId: copilotChats.organizationId, }) if (!updatedChat) { return NextResponse.json({ success: false, error: 'Chat not found' }, { status: 404 }) } + publishChatStatusChanged( + { ...updatedChat, userId }, + { + chatId, + type: title !== undefined ? 'renamed' : 'updated', + } + ) if (updatedChat.workspaceId) { if (title !== undefined) { - chatPubSub?.publishStatusChanged({ - workspaceId: updatedChat.workspaceId, - chatId, - type: 'renamed', - }) captureServerEvent( userId, 'task_renamed', @@ -281,18 +284,15 @@ export const DELETE = withRouteHandler( ) .returning({ workspaceId: copilotChats.workspaceId, + organizationId: copilotChats.organizationId, }) if (!deletedChat) { return NextResponse.json({ success: false, error: 'Chat not found' }, { status: 404 }) } + publishChatStatusChanged({ ...deletedChat, userId }, { chatId, type: 'deleted' }) if (deletedChat.workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId: deletedChat.workspaceId, - chatId, - type: 'deleted', - }) captureServerEvent( userId, 'task_deleted', diff --git a/apps/sim/app/api/mothership/chats/read/route.test.ts b/apps/sim/app/api/mothership/chats/read/route.test.ts index 5a67be8f8a6..b69926d7cf8 100644 --- a/apps/sim/app/api/mothership/chats/read/route.test.ts +++ b/apps/sim/app/api/mothership/chats/read/route.test.ts @@ -17,6 +17,9 @@ vi.mock('@/lib/copilot/chat/lifecycle', () => ({ getAccessibleCopilotChatAuth: mockGetAccessibleChat, })) +vi.mock('@/lib/copilot/chat-status', () => ({ publishChatStatusChanged: vi.fn() })) + +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { POST } from '@/app/api/mothership/chats/read/route' function createRequest() { @@ -67,6 +70,22 @@ describe('POST /api/mothership/chats/read', () => { ) }) + it('broadcasts only a changed read marker, avoiding read/refetch loops', async () => { + mockGetAccessibleChat.mockResolvedValue({ + id: 'chat-1', + type: 'mothership', + organizationId: 'org-1', + userId: 'user-1', + }) + dbChainMockFns.returning.mockResolvedValueOnce([{ id: 'chat-1' }]).mockResolvedValueOnce([]) + await POST(createRequest()) + await POST(createRequest()) + expect(publishChatStatusChanged).toHaveBeenCalledExactlyOnceWith( + expect.objectContaining({ organizationId: 'org-1', userId: 'user-1' }), + { chatId: 'chat-1', type: 'updated' } + ) + }) + it('does not update a chat the caller can no longer access', async () => { mockGetAccessibleChat.mockResolvedValueOnce(null) const res = await POST(createRequest()) diff --git a/apps/sim/app/api/mothership/chats/read/route.ts b/apps/sim/app/api/mothership/chats/read/route.ts index 1c2cc149f72..8eaf98e955c 100644 --- a/apps/sim/app/api/mothership/chats/read/route.ts +++ b/apps/sim/app/api/mothership/chats/read/route.ts @@ -6,6 +6,7 @@ import { type NextRequest, NextResponse } from 'next/server' import { markMothershipChatReadContract } from '@/lib/api/contracts/mothership-chats' import { parseRequest } from '@/lib/api/server' import { getAccessibleCopilotChatAuth } from '@/lib/copilot/chat/lifecycle' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { authenticateCopilotRequestSessionOnly, createInternalServerErrorResponse, @@ -28,7 +29,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => { const chat = await getAccessibleCopilotChatAuth(chatId, userId, { principal }) if (!chat) return NextResponse.json({ success: true }) - await db + const [updatedChat] = await db .update(copilotChats) .set({ lastSeenAt: sql`GREATEST(${copilotChats.updatedAt}, NOW())` }) .where( @@ -38,6 +39,10 @@ export const POST = withRouteHandler(async (request: NextRequest) => { or(isNull(copilotChats.lastSeenAt), lt(copilotChats.lastSeenAt, copilotChats.updatedAt)) ) ) + .returning({ id: copilotChats.id }) + if (updatedChat && chat.type === 'mothership') { + publishChatStatusChanged(chat, { chatId, type: 'updated' }) + } return NextResponse.json({ success: true }) } catch (error) { diff --git a/apps/sim/app/api/mothership/events/route.test.ts b/apps/sim/app/api/mothership/events/route.test.ts new file mode 100644 index 00000000000..e4113c188a5 --- /dev/null +++ b/apps/sim/app/api/mothership/events/route.test.ts @@ -0,0 +1,201 @@ +/** @vitest-environment node */ +import { + authMockFns, + permissionsMock, + permissionsMockFns, + resetEnvFlagsMock, + setEnvFlags, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { ChatStatusEvent } from '@/lib/copilot/chat-status' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { HEARTBEAT_INTERVAL_MS } from '@/lib/events/sse-endpoint' +import { PermissionGroupCapabilityError } from '@/lib/permission-groups/capability-error' + +const { authorize, subscribe, unsubscribe } = vi.hoisted(() => ({ + authorize: vi.fn(), + subscribe: vi.fn(), + unsubscribe: vi.fn(), +})) +vi.mock('@/lib/copilot/chat/organization-chats', () => ({ + authorizeOrganizationChatEvents: { execute: authorize }, +})) +vi.mock('@/lib/copilot/chat-status', () => ({ chatPubSub: { onStatusChanged: subscribe } })) +vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock) + +import { GET } from '@/app/api/mothership/events/route' + +function request(query: string, signal?: AbortSignal) { + return new NextRequest(`http://localhost/api/mothership/events?${query}`, { signal }) +} + +function emit(event: ChatStatusEvent) { + const handler = subscribe.mock.calls[0][0] as (event: ChatStatusEvent) => void + handler(event) +} + +async function collect(body: ReadableStream, chunks: string[]) { + const reader = body.getReader() + const decoder = new TextDecoder() + while (true) { + const { done, value } = await reader.read() + if (done) return + chunks.push(decoder.decode(value)) + } +} + +describe('Mothership owner-scoped event stream', () => { + beforeEach(() => { + vi.clearAllMocks() + vi.useFakeTimers() + setEnvFlags({ isChatEnabled: true }) + authMockFns.mockGetSession.mockResolvedValue({ + user: { id: 'user-1' }, + session: { id: 'session-1' }, + }) + permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('read') + authorize.mockResolvedValue({ organizationId: 'org-1', userId: 'user-1', role: 'member' }) + subscribe.mockReturnValue(unsubscribe) + }) + afterEach(() => { + vi.useRealTimers() + resetEnvFlagsMock() + }) + + it('authenticates before validating scope', async () => { + authMockFns.mockGetSession.mockResolvedValue(null) + const response = await GET(request('organizationId=org-1&workspaceId=ws-1')) + expect(response.status).toBe(401) + expect(authorize).not.toHaveBeenCalled() + expect(subscribe).not.toHaveBeenCalled() + }) + + it.each(['', 'workspaceId=', 'organizationId=', 'organizationId=org-1&workspaceId=ws-1'])( + 'refuses absent, empty, or mixed owners: %s', + async (query) => { + expect((await GET(request(query))).status).toBe(400) + expect(subscribe).not.toHaveBeenCalled() + } + ) + + it('requires chat availability', async () => { + setEnvFlags({ isChatEnabled: false }) + expect((await GET(request('organizationId=org-1'))).status).toBe(404) + expect(subscribe).not.toHaveBeenCalled() + }) + + it('refuses a non-member or disabled organization before subscribing', async () => { + authorize.mockRejectedValue(new OrchestrationError('forbidden', 'Search is not enabled')) + expect((await GET(request('organizationId=org-1'))).status).toBe(403) + expect(subscribe).not.toHaveBeenCalled() + }) + + it('maps a permission group capability refusal to 403', async () => { + authorize.mockRejectedValueOnce( + new PermissionGroupCapabilityError( + 'copilot.use', + 'PERMISSION_GROUP_CAPABILITY_BLOCKED', + 'Chat is disabled' + ) + ) + expect((await GET(request('organizationId=org-1'))).status).toBe(403) + expect(subscribe).not.toHaveBeenCalled() + }) + + it('exposes only the current user’s organization events, without owner metadata', async () => { + const abort = new AbortController() + const response = await GET(request('organizationId=org-1', abort.signal)) + expect(response.status).toBe(200) + const chunks: string[] = [] + const collected = collect(response.body!, chunks) + emit({ + organizationId: 'org-1', + userId: 'other-user', + chatId: 'hidden-user-chat', + type: 'created', + }) + emit({ organizationId: 'org-2', userId: 'user-1', chatId: 'hidden-org-chat', type: 'created' }) + emit({ workspaceId: 'ws-1', chatId: 'hidden-workspace-chat', type: 'created' }) + emit({ + organizationId: 'org-1', + userId: 'user-1', + chatId: 'visible-chat', + type: 'completed', + streamId: 'stream-1', + }) + await vi.advanceTimersByTimeAsync(0) + abort.abort() + await collected + expect(chunks).toHaveLength(1) + expect(chunks[0]).toContain('visible-chat') + expect(chunks[0]).toContain('stream-1') + expect(chunks[0]).not.toMatch(/hidden|userId|organizationId|workspaceId/) + expect(authorize).toHaveBeenCalledTimes(2) + expect(authorize).toHaveBeenLastCalledWith({ + principal: { kind: 'session', sessionId: 'session-1', userId: 'user-1' }, + input: { organizationId: 'org-1' }, + }) + expect(unsubscribe).toHaveBeenCalledTimes(1) + }) + + it('drops a pending event and closes immediately when current authorization fails', async () => { + const response = await GET(request('organizationId=org-1')) + const chunks: string[] = [] + const collected = collect(response.body!, chunks) + authorize.mockRejectedValueOnce(new OrchestrationError('not_found', 'Organization not found')) + emit({ organizationId: 'org-1', userId: 'user-1', chatId: 'revoked-chat', type: 'renamed' }) + await collected + expect(chunks).toEqual([]) + expect(unsubscribe).toHaveBeenCalledTimes(1) + }) + + it('rechecks membership and rollout while idle and releases a revoked connection', async () => { + const response = await GET(request('organizationId=org-1')) + const chunks: string[] = [] + const collected = collect(response.body!, chunks) + authorize.mockRejectedValueOnce(new OrchestrationError('forbidden', 'Search is disabled')) + await vi.advanceTimersByTimeAsync(HEARTBEAT_INTERVAL_MS) + await collected + expect(chunks).toEqual([]) + expect(unsubscribe).toHaveBeenCalledTimes(1) + }) + + it('bounds publications waiting on slow authorization and reconciles through reconnect', async () => { + const response = await GET(request('organizationId=org-1')) + const chunks: string[] = [] + const collected = collect(response.body!, chunks) + let authorizeDone: (() => void) | undefined + authorize.mockReturnValueOnce( + new Promise((resolve) => { + authorizeDone = resolve + }) + ) + for (let index = 0; index < 17; index += 1) { + emit({ organizationId: 'org-1', userId: 'user-1', chatId: `chat-${index}`, type: 'updated' }) + } + await collected + expect(unsubscribe).toHaveBeenCalledTimes(1) + expect(authorize).toHaveBeenCalledTimes(2) + authorizeDone?.() + await vi.advanceTimersByTimeAsync(0) + expect(chunks).toEqual([]) + }) + + it('preserves workspace status events and excludes organization events', async () => { + const abort = new AbortController() + const response = await GET(request('workspaceId=ws-1', abort.signal)) + const chunks: string[] = [] + const collected = collect(response.body!, chunks) + emit({ organizationId: 'org-1', userId: 'user-1', chatId: 'org-chat', type: 'created' }) + emit({ workspaceId: 'ws-2', chatId: 'other-workspace-chat', type: 'created' }) + emit({ workspaceId: 'ws-1', chatId: 'workspace-chat', type: 'renamed' }) + abort.abort() + await collected + expect(chunks).toHaveLength(1) + expect(chunks[0]).toContain('workspace-chat') + expect(chunks[0]).not.toMatch(/org-chat|other-workspace-chat/) + expect(authorize).not.toHaveBeenCalled() + expect(authMockFns.mockGetSession).toHaveBeenCalledTimes(1) + }) +}) diff --git a/apps/sim/app/api/mothership/events/route.ts b/apps/sim/app/api/mothership/events/route.ts index c942c825665..c39f4068fc8 100644 --- a/apps/sim/app/api/mothership/events/route.ts +++ b/apps/sim/app/api/mothership/events/route.ts @@ -7,16 +7,25 @@ * Auth is handled via session cookies (EventSource sends cookies automatically). */ +import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { mothershipEventsQuerySchema } from '@/lib/api/contracts/mothership-chats' import { validationErrorResponse } from '@/lib/api/server' +import { + InternalUnauthenticatedError, + internalSessionAuth, +} from '@/lib/api/server/routes/internal-json-route' +import { authorizeOrganizationChatEvents } from '@/lib/copilot/chat/organization-chats' import { chatPubSub } from '@/lib/copilot/chat-status' import { isChatEnabled } from '@/lib/core/config/env-flags' +import { asOrchestrationError } from '@/lib/core/orchestration/types' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { createWorkspaceSSE } from '@/lib/events/sse-endpoint' +import { createSSEStream, createWorkspaceSSE } from '@/lib/events/sse-endpoint' export const dynamic = 'force-dynamic' +const logger = createLogger('MothershipEvents') + const mothershipEventsHandler = createWorkspaceSSE({ label: 'mothership-events', subscriptions: [ @@ -37,14 +46,50 @@ const mothershipEventsHandler = createWorkspaceSSE({ ], }) -export const GET = withRouteHandler((request: NextRequest) => { +export const GET = withRouteHandler(async (request: NextRequest) => { // Closes streams held by tabs that were open when Chat was turned off; the // client hook already declines to open new ones. if (!isChatEnabled) return new Response(null, { status: 404 }) - const validation = mothershipEventsQuerySchema.safeParse( - Object.fromEntries(request.nextUrl.searchParams.entries()) - ) - if (!validation.success) return validationErrorResponse(validation.error) - return mothershipEventsHandler(request) + try { + const principal = await internalSessionAuth.authenticate() + const validation = mothershipEventsQuerySchema.safeParse( + Object.fromEntries(request.nextUrl.searchParams.entries()) + ) + if (!validation.success) return validationErrorResponse(validation.error) + const { organizationId } = validation.data + if (!organizationId) return mothershipEventsHandler(request, principal) + + const revalidate = async () => { + await authorizeOrganizationChatEvents.execute({ principal, input: { organizationId } }) + } + await revalidate() + return createSSEStream(request, { + label: 'mothership-organization-events', + revalidate, + subscriptions: [ + { + subscribe: (send) => + chatPubSub?.onStatusChanged((event) => { + if (event.organizationId !== organizationId || event.userId !== principal.userId) + return + send('task_status', { + chatId: event.chatId, + type: event.type, + ...(event.streamId ? { streamId: event.streamId } : {}), + timestamp: Date.now(), + }) + }) ?? (() => {}), + }, + ], + }) + } catch (error) { + const code = asOrchestrationError(error)?.code + if (code === 'not_found' || code === 'forbidden') + return new Response('Organization access denied', { status: 403 }) + if (error instanceof InternalUnauthenticatedError) + return new Response('Unauthorized', { status: 401 }) + logger.error('Failed to subscribe to organization chats', error) + return new Response('Unable to subscribe to chats', { status: 500 }) + } }) diff --git a/apps/sim/app/api/v1/admin/credits/route.ts b/apps/sim/app/api/v1/admin/credits/route.ts index 2c41426d1f2..da245cdc7a4 100644 --- a/apps/sim/app/api/v1/admin/credits/route.ts +++ b/apps/sim/app/api/v1/admin/credits/route.ts @@ -25,7 +25,8 @@ import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { db } from '@sim/db' -import { organization, subscription, user, userStats } from '@sim/db/schema' +import { withInsertColumns } from '@sim/db/insert-columns' +import { organization, subscription, user, userStats, userStatsColumns } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateShortId } from '@sim/utils/id' import { normalizeEmail } from '@sim/utils/string' @@ -155,7 +156,7 @@ export const POST = withRouteHandler( .limit(1) if (!existingStats) { - await db.insert(userStats).values({ + await db.insert(withInsertColumns(userStats, userStatsColumns)).values({ id: generateShortId(), userId: entityId, }) diff --git a/apps/sim/app/api/v1/admin/users/[id]/billing/route.ts b/apps/sim/app/api/v1/admin/users/[id]/billing/route.ts index d3cdb3c5e9d..4ab52cb09e9 100644 --- a/apps/sim/app/api/v1/admin/users/[id]/billing/route.ts +++ b/apps/sim/app/api/v1/admin/users/[id]/billing/route.ts @@ -20,6 +20,7 @@ */ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { member, organization, @@ -247,7 +248,7 @@ export const PATCH = withRouteHandler( if (existingStats) { await db.update(userStats).set(updateData).where(eq(userStats.userId, userId)) } else { - await db.insert(userStats).values({ + await db.insert(withInsertColumns(userStats, userStatsColumns)).values({ id: generateShortId(), userId, ...updateData, diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.test.tsx index e68e55f599c..8e7d8d53d0d 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.test.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.test.tsx @@ -2,6 +2,8 @@ * @vitest-environment jsdom */ import { act } from 'react' +import { ToastProvider } from '@sim/emcn' +import { sleep } from '@sim/utils/helpers' import { QueryClient, QueryClientProvider } from '@tanstack/react-query' import { createRoot, type Root } from 'react-dom/client' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,6 +11,12 @@ import type { OrganizationChat } from '@/app/o/[organizationId]/components/organ const hoverState = vi.hoisted(() => ({ isOpen: false })) const mockRequestJson = vi.hoisted(() => vi.fn()) +const mockPush = vi.hoisted(() => vi.fn()) + +vi.mock('next/navigation', () => ({ + useRouter: () => ({ push: mockPush }), + usePathname: () => window.location.pathname, +})) vi.mock('@/lib/api/client/request', () => ({ requestJson: mockRequestJson })) @@ -64,7 +72,8 @@ beforeEach(() => { } ) vi.clearAllMocks() - mockRequestJson.mockResolvedValue({ success: true }) + mockRequestJson.mockReset().mockResolvedValue({ success: true }) + window.history.replaceState(null, '', '/o/org-1/home') hoverState.isOpen = false queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }) prefetchQuery = vi.spyOn(queryClient, 'prefetchQuery').mockResolvedValue() @@ -84,14 +93,16 @@ async function render(props: Partial[0]> = {}) { await act(async () => { root.render( - + + + ) }) @@ -239,6 +250,114 @@ describe('ChatsSection', () => { expect(prefetchQuery).not.toHaveBeenCalled() }) + async function openDelete(isCollapsed = false) { + hoverState.isOpen = isCollapsed + await render({ isCollapsed }) + const options = document.body.querySelector('[aria-label="Chat options"]')! + await act(async () => options.click()) + const action = Array.from( + document.body.querySelectorAll('[role="menuitem"]') + ).find((item) => item.textContent === 'Delete')! + expect(action).toBeDefined() + await act(async () => action.click()) + expect(document.body.querySelector('[role="dialog"]')?.textContent).toContain('Chat 1') + expect(mockRequestJson).not.toHaveBeenCalled() + } + + function modalButton(label: string) { + return Array.from( + document.body.querySelectorAll('[role="dialog"] button') + ).find((button) => button.textContent === label)! + } + + it.each([false, true])( + 'cancels deletion without a request with collapsed=%s', + async (isCollapsed) => { + await openDelete(isCollapsed) + await act(async () => modalButton('Cancel').click()) + expect(document.body.querySelector('[role="dialog"]')).toBeNull() + expect(mockRequestJson).not.toHaveBeenCalled() + expect(mockPush).not.toHaveBeenCalled() + } + ) + + it.each([false, true])( + 'deletes through the shared contract with collapsed=%s', + async (isCollapsed) => { + const invalidate = vi.spyOn(queryClient, 'invalidateQueries') + await openDelete(isCollapsed) + await act(async () => modalButton('Delete').click()) + expect(mockRequestJson).toHaveBeenCalledWith(expect.objectContaining({ method: 'DELETE' }), { + params: { chatId: 'chat-1' }, + }) + expect(invalidate).toHaveBeenCalledWith({ + queryKey: mothershipChatKeys.organizationLists('org-1'), + }) + expect(invalidate).not.toHaveBeenCalledWith({ + queryKey: mothershipChatKeys.workspaceLists('org-1'), + }) + expect(document.body.querySelector('[role="dialog"]')).toBeNull() + expect(mockPush).not.toHaveBeenCalled() + } + ) + + it('cancels delete confirmation with Escape without changing chats', async () => { + await openDelete() + await act(async () => { + document.body + .querySelector('[role="dialog"]')! + .dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true })) + }) + expect(document.body.querySelector('[role="dialog"]')).toBeNull() + expect(mockRequestJson).not.toHaveBeenCalled() + expect(mockPush).not.toHaveBeenCalled() + }) + + it('returns home when the deleted chat is still open', async () => { + await openDelete() + window.history.replaceState(null, '', CHATS[0].href) + await act(async () => modalButton('Delete').click()) + expect(mockPush).toHaveBeenCalledWith('/o/org-1/home') + }) + + it('keeps confirmation pending and does not override navigation after a slow delete', async () => { + const pending = Promise.withResolvers<{ success: boolean }>() + await openDelete() + mockRequestJson.mockReturnValueOnce(pending.promise) + window.history.replaceState(null, '', CHATS[0].href) + await act(async () => modalButton('Delete').click()) + await act(async () => sleep(1)) + expect(modalButton('Deleting...').disabled).toBe(true) + expect(modalButton('Cancel').disabled).toBe(true) + await act(async () => { + document.body + .querySelector('[role="dialog"]')! + .dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true })) + }) + expect(document.body.querySelector('[role="dialog"]')).not.toBeNull() + expect(mockPush).not.toHaveBeenCalled() + window.history.replaceState(null, '', CHATS[1].href) + await act(async () => pending.resolve({ success: true })) + expect(mockPush).not.toHaveBeenCalled() + expect(document.body.querySelector('[role="dialog"]')).toBeNull() + }) + + it('keeps the chat and confirmation available for retry after a failed delete', async () => { + await openDelete() + mockRequestJson.mockRejectedValueOnce(new Error('Delete rejected')) + window.history.replaceState(null, '', CHATS[0].href) + const key = mothershipChatKeys.detail('chat-1') + queryClient.setQueryData(key, { id: 'chat-1', messages: ['Preserved'] }) + await act(async () => modalButton('Delete').click()) + expect(mockPush).not.toHaveBeenCalled() + expect(document.body.querySelector('[role="dialog"]')).not.toBeNull() + expect(modalButton('Delete').disabled).toBe(false) + expect(queryClient.getQueryData(key)).toEqual({ id: 'chat-1', messages: ['Preserved'] }) + await act(async () => modalButton('Delete').click()) + expect(mockPush).toHaveBeenCalledWith('/o/org-1/home') + expect(document.body.querySelector('[role="dialog"]')).toBeNull() + }) + it('shows the empty state when there are no chats', async () => { await render({ chats: [] }) expect(container.textContent).toContain('No chats yet') diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.tsx index df2b36b4aa9..f83504cecae 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/chats-section/chats-section.tsx @@ -1,14 +1,6 @@ 'use client' -import { - ChipInput, - chipVariants, - cn, - DropdownMenuItem, - Loader, - OverflowText, - Skeleton, -} from '@sim/emcn' +import { chipVariants, cn, DropdownMenuItem, Loader, OverflowText, Skeleton } from '@sim/emcn' import { MoreHorizontal, Pin, Task } from '@sim/emcn/icons' import type { OrganizationChat } from '@/app/o/[organizationId]/components/organization-sidebar/hooks' import { useOrganizationChatActions } from '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chat-actions' @@ -18,7 +10,9 @@ import { CollapsedSidebarMenu, SidebarSection, } from '@/app/workspace/[workspaceId]/w/components/sidebar/components' +import { SidebarRenameRow } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row' import { ContextMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workflow-list/components/context-menu/context-menu' +import { DeleteModal } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workflow-list/components/delete-modal/delete-modal' import { SIDEBAR_ITEM_GAP_CLASS, SIDEBAR_SECTION_GAP_CLASS, @@ -186,7 +180,7 @@ export function ChatsSection({ )} {chats.map((chat) => rename.editingId === chat.id ? ( - ) : ( + ) } diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/index.ts b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/index.ts index 525cf120d1d..c7f48483337 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/index.ts +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/index.ts @@ -1,5 +1,4 @@ export { ChatsSection } from './chats-section' export { OrganizationFooter } from './organization-footer' export { OrganizationHeader } from './organization-header' -export { WorkspacesRailFlyout } from './workspaces-rail-flyout' export { WorkspacesSection } from './workspaces-section' diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.test.tsx new file mode 100644 index 00000000000..8a39e94a9fe --- /dev/null +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.test.tsx @@ -0,0 +1,130 @@ +/** + * @vitest-environment jsdom + */ +import { act, type ComponentProps } from 'react' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockNavigate, mockPush } = vi.hoisted(() => ({ + mockNavigate: vi.fn(), + mockPush: vi.fn(), +})) + +vi.mock('next/navigation', () => ({ + useRouter: () => ({ push: mockPush }), +})) +vi.mock('next/link', () => ({ + default: ({ + onNavigate, + ...props + }: ComponentProps<'a'> & { onNavigate?: (event: { preventDefault: () => void }) => void }) => ( + { + event.preventDefault() + let prevented = false + onNavigate?.({ + preventDefault: () => { + prevented = true + }, + }) + if (!prevented) mockNavigate(props.href) + }} + /> + ), +})) +vi.mock('@/lib/desktop', () => ({ getDesktopUpdates: () => null })) +vi.mock('@/hooks/use-desktop-update-state', () => ({ + useDesktopUpdateState: () => ({ status: 'idle' }), +})) +vi.mock('@/hooks/queries/user-profile', () => ({ + useUserProfile: () => ({ data: { id: 'user-1', name: 'Ada', email: 'ada@example.com' } }), +})) +vi.mock('@/app/o/[organizationId]/providers/organization-provider', () => ({ + useOrganizationContext: () => ({ organization: { id: 'org-1' } }), +})) +vi.mock('@/app/workspace/[workspaceId]/w/components/sidebar/components', () => ({ + SidebarTooltip: ({ children }: { children: React.ReactNode }) => children, +})) +vi.mock('@/components/icons', () => ({ SlackIcon: () => })) + +import { OrganizationFooter } from '@/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer' +import { useSettingsDirtyStore } from '@/stores/settings/dirty/store' + +let container: HTMLDivElement +let root: Root + +beforeEach(() => { + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + vi.clearAllMocks() + useSettingsDirtyStore.getState().reset() + container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) +}) + +afterEach(async () => { + await act(async () => root.unmount()) + container.remove() + useSettingsDirtyStore.getState().reset() + vi.unstubAllGlobals() +}) + +async function selectSettings() { + await act(async () => { + root.render( + {}} + onJoinSlack={() => {}} + /> + ) + }) + const trigger = container.querySelector('[data-item-id="profile"]') + if (!trigger) throw new Error('Profile menu is missing') + await act(async () => { + trigger.dispatchEvent(new MouseEvent('pointerdown', { bubbles: true, button: 0 })) + }) + const link = document.querySelector('a[href="/o/org-1/settings/general"]') + if (!link) throw new Error('Settings link is missing') + await act(async () => link.click()) +} + +describe('OrganizationFooter settings navigation', () => { + it('navigates immediately when settings are clean', async () => { + await selectSettings() + expect(mockNavigate).toHaveBeenCalledWith('/o/org-1/settings/general') + expect(useSettingsDirtyStore.getState().pendingLeave).toBeNull() + }) + + it('waits for discard confirmation before leaving a dirty form', async () => { + useSettingsDirtyStore.getState().setDirty(true) + await selectSettings() + expect(mockNavigate).not.toHaveBeenCalled() + expect(mockPush).not.toHaveBeenCalled() + expect(useSettingsDirtyStore.getState().pendingLeave).not.toBeNull() + + act(() => useSettingsDirtyStore.getState().confirmLeave()) + expect(mockPush).toHaveBeenCalledWith('/o/org-1/settings/general') + }) + + it('keeps the draft when leaving is cancelled', async () => { + useSettingsDirtyStore.getState().setDirty(true) + await selectSettings() + act(() => useSettingsDirtyStore.getState().cancelLeave()) + expect(mockNavigate).not.toHaveBeenCalled() + expect(mockPush).not.toHaveBeenCalled() + expect(useSettingsDirtyStore.getState().isDirty).toBe(true) + }) + + it('blocks navigation while saving without queuing a later redirect', async () => { + useSettingsDirtyStore.getState().setNavigationBlocked(true) + await selectSettings() + expect(mockNavigate).not.toHaveBeenCalled() + expect(mockPush).not.toHaveBeenCalled() + expect(useSettingsDirtyStore.getState().pendingLeave).toBeNull() + }) +}) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.tsx index 3abcc00a6f2..ecc37a12e01 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-footer/organization-footer.tsx @@ -17,8 +17,8 @@ import { Skeleton, } from '@sim/emcn' import { BookOpen, Download, HelpCircle, Settings } from '@sim/emcn/icons' -import Link from 'next/link' import { SlackIcon } from '@/components/icons' +import { SettingsGuardedLink } from '@/components/settings/settings-guarded-link' import { getDesktopUpdates } from '@/lib/desktop' import { organizationRoutes } from '@/lib/navigation/paths' import { getUserColor } from '@/lib/workspaces/colors' @@ -166,10 +166,12 @@ export function OrganizationFooter({ - + - + @@ -233,7 +235,7 @@ export function OrganizationFooter({ {/* Expanded, claims the row's free width so the help button lands hard right. `flex` makes the inline-flex chip a flex item, so the wrapper is exactly the chip's 30px rather than a line box padded by the strut's half-leading. */} -
{profileMenu}
+
{profileMenu}
{helpMenu} ) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.test.tsx new file mode 100644 index 00000000000..665eea639dc --- /dev/null +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.test.tsx @@ -0,0 +1,191 @@ +/** + * @vitest-environment jsdom + */ +import { act } from 'react' +import { ToastProvider } from '@sim/emcn' +import { QueryClient, QueryClientProvider } from '@tanstack/react-query' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ upload: vi.fn(), refresh: vi.fn() })) +vi.mock('@/lib/uploads/client/session-upload', () => ({ + uploadInternalFileSession: mocks.upload, +})) +vi.mock('next/navigation', () => ({ + useRouter: () => ({ refresh: mocks.refresh }), + usePathname: () => '/o/org-1/home', +})) + +import { OrganizationHeader } from '@/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header' +import { organizationKeys } from '@/hooks/queries/utils/organization-keys' + +const organization = { id: 'org-1', name: 'Design', slug: 'design', logo: null, memberCount: 2 } +let root: Root +let container: HTMLDivElement +let queryClient: QueryClient + +beforeEach(() => { + vi.clearAllMocks() + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + vi.stubGlobal( + 'ResizeObserver', + class { + observe() {} + unobserve() {} + disconnect() {} + } + ) + mocks.upload.mockResolvedValue({ path: '/api/files/serve/organization-logos/logo.png' }) + queryClient = new QueryClient({ defaultOptions: { mutations: { retry: false } } }) + container = document.createElement('div') + document.body.append(container) + root = createRoot(container) +}) + +afterEach(async () => { + await act(async () => root.unmount()) + container.remove() + queryClient.clear() + vi.unstubAllGlobals() +}) + +async function render(canEditLogo = true, isCollapsed = false, onExpandSidebar = vi.fn()) { + await act(async () => { + root.render( + + + + + + ) + }) +} + +async function openMenu() { + await act(async () => { + container + .querySelector('[aria-label="Organization menu"]')! + .dispatchEvent(new MouseEvent('pointerdown', { bubbles: true, button: 0 })) + }) +} + +function logoControl() { + return document.querySelector( + '[role="menuitem"][aria-label="Change organization logo"]' + ) +} + +async function pickFile(file: File) { + const input = container.querySelector('input[type="file"]')! + Object.defineProperty(input, 'files', { configurable: true, value: [file] }) + await act(async () => input.dispatchEvent(new Event('change', { bubbles: true }))) +} + +describe('OrganizationHeader logo upload', () => { + it('opens the native file picker by clicking the logo and keeps the menu open', async () => { + await render() + await openMenu() + const input = container.querySelector('input[type="file"]')! + const click = vi.spyOn(input, 'click').mockImplementation(() => {}) + expect(input.accept).toContain('image/png') + await act(async () => logoControl()!.click()) + expect(click).toHaveBeenCalledOnce() + expect(logoControl()).not.toBeNull() + expect(document.body.textContent).not.toContain('Upload logo') + expect(document.querySelector('[role="menu"]')?.textContent).toContain('Settings') + }) + + it.each(['Enter', ' '])('opens the file picker using the %j key', async (key) => { + await render() + await openMenu() + const input = container.querySelector('input[type="file"]')! + const click = vi.spyOn(input, 'click').mockImplementation(() => {}) + await act(async () => { + logoControl()!.focus() + logoControl()!.dispatchEvent(new KeyboardEvent('keydown', { key, bubbles: true })) + }) + expect(click).toHaveBeenCalledOnce() + }) + + it('does not offer logo changes to members', async () => { + await render(false) + await openMenu() + expect(logoControl()).toBeNull() + expect(container.querySelector('input[type="file"]')).toBeNull() + expect(document.querySelector('[role="menu"]')?.textContent).toContain('Design') + }) + + it('preserves the collapsed logo as the sidebar expand control', async () => { + const expand = vi.fn() + await render(true, true, expand) + await act(async () => { + container.querySelector('[aria-label="Expand sidebar"]')!.click() + }) + expect(expand).toHaveBeenCalledOnce() + expect(container.querySelector('input[type="file"]')).toBeNull() + expect(mocks.upload).not.toHaveBeenCalled() + }) + + it('disables logo changes while the upload is pending', async () => { + let completeUpload!: () => void + mocks.upload.mockImplementation( + () => + new Promise((resolve) => { + completeUpload = resolve + }) + ) + await render() + await openMenu() + await pickFile(new File(['image'], 'logo.png', { type: 'image/png' })) + await act(async () => { + await vi.waitFor(() => expect(logoControl()?.getAttribute('aria-disabled')).toBe('true')) + }) + expect(logoControl()?.getAttribute('aria-busy')).toBe('true') + expect(container.querySelector('input[type="file"]')!.disabled).toBe(true) + await act(async () => logoControl()!.click()) + expect(mocks.upload).toHaveBeenCalledOnce() + await act(async () => completeUpload()) + expect(mocks.refresh).toHaveBeenCalledOnce() + }) + + it('uploads under the organization scope and refreshes its identity after success', async () => { + await render() + const invalidate = vi.spyOn(queryClient, 'invalidateQueries') + const file = new File(['image'], 'logo.png', { type: 'image/png' }) + await pickFile(file) + expect(mocks.upload).toHaveBeenCalledWith({ + purpose: 'organization_logo', + organizationId: organization.id, + file, + }) + expect(invalidate).toHaveBeenCalledWith({ queryKey: organizationKeys.detail('org-1') }) + expect(invalidate).toHaveBeenCalledWith({ queryKey: organizationKeys.lists() }) + expect(mocks.refresh).toHaveBeenCalledOnce() + }) + + it('rejects unsupported files before uploading', async () => { + await render() + await pickFile(new File(['text'], 'notes.txt', { type: 'text/plain' })) + expect(mocks.upload).not.toHaveBeenCalled() + expect(mocks.refresh).not.toHaveBeenCalled() + expect(document.body.textContent).toContain('not a supported image format') + }) + + it('keeps the saved identity when upload fails and allows retrying the same file', async () => { + const file = new File(['image'], 'logo.png', { type: 'image/png' }) + mocks.upload.mockRejectedValueOnce(new Error('Upload failed')) + await render() + await pickFile(file) + expect(mocks.refresh).not.toHaveBeenCalled() + expect(document.body.textContent).toContain('Upload failed') + expect(container.querySelector('input[type="file"]')!.value).toBe('') + await pickFile(file) + expect(mocks.upload).toHaveBeenCalledTimes(2) + expect(mocks.refresh).toHaveBeenCalledOnce() + }) +}) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.tsx index c007e5d395d..19a580ae22b 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/organization-header/organization-header.tsx @@ -1,5 +1,6 @@ 'use client' +import { useRef } from 'react' import { Chip, ChipChevronDown, @@ -7,13 +8,19 @@ import { DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigger, + OverflowText, + Tooltip, + toast, } from '@sim/emcn' import { PanelLeft, Settings } from '@sim/emcn/icons' +import { useRouter } from 'next/navigation' import { IdentityTile } from '@/components/identity-tile/identity-tile' import { getOrganizationSettingsHref } from '@/components/settings/navigation' import { SettingsGuardedLink } from '@/components/settings/settings-guarded-link' import type { OrganizationSurfaceOrganization } from '@/lib/organizations/surface' +import { LOGO_ACCEPT_ATTRIBUTE } from '@/lib/uploads/client/logo-file' import { SIDEBAR_RAIL_CHIP_CLASS } from '@/app/workspace/[workspaceId]/w/components/sidebar/constants' +import { useUploadOrganizationLogo } from '@/hooks/queries/organization-logo' function getOrganizationInitial(name: string): string { return (name.trim()[0] || 'O').toUpperCase() @@ -21,6 +28,7 @@ function getOrganizationInitial(name: string): string { interface OrganizationHeaderProps { organization: OrganizationSurfaceOrganization + canEditLogo: boolean isCollapsed: boolean /** Expands the rail; the collapsed header is itself the expand control. */ onExpandSidebar: () => void @@ -35,9 +43,15 @@ interface OrganizationHeaderProps { */ export function OrganizationHeader({ organization, + canEditLogo, isCollapsed, onExpandSidebar, }: OrganizationHeaderProps) { + const fileInputRef = useRef(null) + const router = useRouter() + const { mutate: uploadLogo, isPending: isUploadingLogo } = useUploadOrganizationLogo( + organization.id + ) const initial = getOrganizationInitial(organization.name) if (isCollapsed) { @@ -67,9 +81,31 @@ export function OrganizationHeader({ } const { memberCount } = organization + const logo = ( + + ) return (
+ {canEditLogo && ( + { + const file = event.target.files?.[0] + event.target.value = '' + if (!file || isUploadingLogo) return + uploadLogo(file, { + onSuccess: () => router.refresh(), + onError: (error) => toast.error(error.message), + }) + }} + /> + )} - {/* The item rows' `px-2` and the rail chips' icon-to-label gap, so the card sits on the menu's own grid. */}
- + {canEditLogo ? ( + + + { + event.preventDefault() + fileInputRef.current?.click() + }} + > + {logo} + + + + {isUploadingLogo ? 'Uploading...' : 'Change logo'} + + + ) : ( + logo + )}
- - {organization.name} - + {memberCount} {memberCount === 1 ? 'member' : 'members'} diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/index.ts b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/index.ts deleted file mode 100644 index fe0024ab47c..00000000000 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/index.ts +++ /dev/null @@ -1 +0,0 @@ -export { WorkspacesRailFlyout } from './workspaces-rail-flyout' diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.test.tsx deleted file mode 100644 index b6729ecbf07..00000000000 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.test.tsx +++ /dev/null @@ -1,97 +0,0 @@ -/** - * @vitest-environment jsdom - */ -import { act } from 'react' -import { createRoot, type Root } from 'react-dom/client' -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' - -const workspacesState = vi.hoisted(() => ({ - workspaces: [] as { id: string; name: string }[], - isLoading: false, -})) - -vi.mock('next/link', () => ({ - default: ({ href, children, ...props }: { href: string; children: React.ReactNode }) => ( - - {children} - - ), -})) -vi.mock('@/app/o/[organizationId]/components/organization-sidebar/hooks', () => ({ - useOrganizationWorkspaces: () => workspacesState, -})) -vi.mock( - '@/app/workspace/[workspaceId]/w/components/sidebar/components/collapsed-sidebar-menu', - () => ({ - CollapsedResourceFlyout: ({ - entries, - isLoading, - emptyLabel, - }: { - entries: { id: string; name: string; href: string }[] - isLoading: boolean - emptyLabel: string - }) => - isLoading ? ( - Loading... - ) : entries.length === 0 ? ( - {emptyLabel} - ) : ( - entries.map((entry) => ( - - {entry.name} - - )) - ), - }) -) - -import { WorkspacesRailFlyout } from '@/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout' - -let container: HTMLDivElement -let root: Root - -beforeEach(() => { - ;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true - workspacesState.workspaces = [] - workspacesState.isLoading = false - container = document.createElement('div') - document.body.appendChild(container) - root = createRoot(container) -}) - -afterEach(async () => { - await act(async () => root.unmount()) - container.remove() -}) - -async function render() { - await act(async () => { - root.render() - }) -} - -describe('WorkspacesRailFlyout', () => { - it('lists every workspace as a link into it', async () => { - workspacesState.workspaces = [ - { id: 'ws-1', name: 'Design' }, - { id: 'ws-2', name: 'Ops' }, - ] - await render() - - const links = Array.from(container.querySelectorAll('a')).map((a) => a.getAttribute('href')) - expect(links).toEqual(['/workspace/ws-1', '/workspace/ws-2']) - expect(container.textContent).toContain('Design') - }) - - it('shows the empty label when the organization has no workspaces', async () => { - await render() - expect(container.textContent).toContain('No workspaces yet') - }) - - it('shows the loading row while the list resolves', async () => { - workspacesState.isLoading = true - await render() - expect(container.textContent).toContain('Loading...') - }) -}) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.tsx deleted file mode 100644 index b36e478cf14..00000000000 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout/workspaces-rail-flyout.tsx +++ /dev/null @@ -1,35 +0,0 @@ -'use client' - -import { useOrganizationWorkspaces } from '@/app/o/[organizationId]/components/organization-sidebar/hooks' -import type { FlyoutEntry } from '@/app/workspace/[workspaceId]/components/folders' -import { CollapsedResourceFlyout } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/collapsed-sidebar-menu' - -interface WorkspacesRailFlyoutProps { - organizationId: string -} - -/** - * Rail flyout body for the Workspaces tab: a jump list of the organization's - * workspaces, one row each, the way the workspace sidebar's Tables and Files tabs - * list theirs. Mounts only while the rail menu is open, so the workspace query - * runs only when someone hovers the chip. - */ -export function WorkspacesRailFlyout({ organizationId }: WorkspacesRailFlyoutProps) { - const { workspaces, isLoading } = useOrganizationWorkspaces(organizationId) - - const entries: FlyoutEntry[] = workspaces.map((workspace) => ({ - kind: 'item', - id: workspace.id, - name: workspace.name, - pinned: false, - href: `/workspace/${workspace.id}`, - })) - - return ( - - ) -} diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.test.tsx new file mode 100644 index 00000000000..e4ce1404ee2 --- /dev/null +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.test.tsx @@ -0,0 +1,126 @@ +/** + * @vitest-environment jsdom + */ +import { act } from 'react' +import { DropdownMenu, DropdownMenuContent, DropdownMenuTrigger } from '@sim/emcn' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const workspacesState = vi.hoisted(() => ({ + workspaces: [] as { id: string; name: string }[], + isLoading: false, + pinnedWorkspaceIds: new Set(), +})) + +vi.mock('next/link', () => ({ + default: ({ + href, + children, + onNavigate: _onNavigate, + ...props + }: { + href: string + children: React.ReactNode + onNavigate?: () => void + }) => ( + + {children} + + ), +})) +vi.mock( + '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces', + () => ({ + useOrganizationWorkspaces: () => workspacesState, + }) +) + +vi.mock('next/navigation', () => ({ + usePathname: () => '/o/org-1/home', + useRouter: () => ({ push: vi.fn() }), +})) +vi.mock('@/hooks/queries/workspace', () => ({ + useUpdateWorkspace: () => ({ mutateAsync: vi.fn() }), + useToggleWorkspacePin: () => ({ mutate: vi.fn() }), +})) + +import { WorkspaceList } from '@/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list' + +let container: HTMLDivElement +let root: Root + +beforeEach(() => { + ;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true + workspacesState.workspaces = [] + workspacesState.isLoading = false + container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) +}) + +afterEach(async () => { + await act(async () => root.unmount()) + container.remove() +}) + +async function render() { + await act(async () => { + root.render( + + Workspaces + + + + + ) + }) +} + +describe('WorkspaceList rail view', () => { + it('lists every workspace as a link into it', async () => { + workspacesState.workspaces = [ + { id: 'ws-1', name: 'Design' }, + { id: 'ws-2', name: 'Ops' }, + ] + await render() + + const links = Array.from(document.querySelectorAll('a')).map((a) => a.getAttribute('href')) + expect(links).toEqual(['/workspace/ws-1', '/workspace/ws-2']) + expect(document.body.textContent).toContain('Design') + }) + + it('shows the empty label when the organization has no workspaces', async () => { + await render() + expect(document.body.textContent).toContain('No workspaces yet') + }) + + it('keeps every workspace accessible in the flyout without a search field', async () => { + workspacesState.workspaces = Array.from({ length: 8 }, (_, index) => ({ + id: `ws-${index}`, + name: `Workspace ${index}`, + })) + await render() + expect(document.querySelectorAll('a')).toHaveLength(8) + expect(document.querySelector('input')).toBeNull() + }) + + it('shows the loading row while the list resolves', async () => { + workspacesState.isLoading = true + await render() + expect(document.body.textContent).toContain('Loading...') + }) +}) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.tsx new file mode 100644 index 00000000000..c0085b6b92c --- /dev/null +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list.tsx @@ -0,0 +1,237 @@ +'use client' + +import { useEffect, useState } from 'react' +import { + chipVariants, + cn, + DropdownMenuItem, + DropdownMenuItemAction, + Loader, + OverflowText, + toast, +} from '@sim/emcn' +import { MoreHorizontal, Pin } from '@sim/emcn/icons' +import { getErrorMessage } from '@sim/utils/errors' +import { IdentityTile } from '@/components/identity-tile/identity-tile' +import { SettingsGuardedLink } from '@/components/settings/settings-guarded-link' +import { WorkspaceContextMenu } from '@/components/workspaces/workspace-context-menu' +import { getWorkspaceInitial } from '@/lib/workspaces/initials' +import { useOrganizationWorkspaces } from '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces' +import { SidebarRenameRow } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row' +import { useFlyoutInlineRename } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename' +import type { useHoverMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-hover-menu' +import { useToggleWorkspacePin, useUpdateWorkspace } from '@/hooks/queries/workspace' +import { useContextMenu } from '@/hooks/use-context-menu' + +const PAGE_SIZE = 5 + +interface WorkspaceListProps { + organizationId: string + pathname?: string | null + /** The rail flyout uses the same actions and ordering as the expanded list. */ + flyout?: ReturnType +} + +export function WorkspaceList({ organizationId, pathname, flyout }: WorkspaceListProps) { + const { workspaces, pinnedWorkspaceIds, isLoading } = useOrganizationWorkspaces(organizationId) + const { mutate: togglePin } = useToggleWorkspacePin() + const { mutateAsync: updateWorkspace } = useUpdateWorkspace() + const menu = useContextMenu() + const [selectedId, setSelectedId] = useState(null) + const [visibleCount, setVisibleCount] = useState(PAGE_SIZE) + const selectedWorkspace = workspaces.find((workspace) => workspace.id === selectedId) + const rename = useFlyoutInlineRename({ + itemType: 'workspace', + onSave: async (workspaceId, name) => { + try { + await updateWorkspace({ workspaceId, name }) + } catch (error) { + toast.error(getErrorMessage(error, 'Failed to rename workspace')) + throw error + } + }, + }) + const lockFlyout = flyout?.setLocked + + useEffect(() => { + lockFlyout?.(menu.isOpen || rename.editingId !== null) + return () => lockFlyout?.(false) + }, [lockFlyout, menu.isOpen, rename.editingId]) + + const visibleWorkspaces = flyout ? workspaces : workspaces.slice(0, visibleCount) + const hasMore = workspaces.length > visibleCount + + const openMenu = (event: React.MouseEvent, workspaceId: string) => { + setSelectedId(workspaceId) + flyout?.setLocked(true) + menu.preventDismiss() + menu.handleContextMenu(event) + } + + return ( + <> + {isLoading && flyout && ( + + + Loading... + + )} + {!isLoading && workspaces.length === 0 && ( +
No workspaces yet
+ )} + {visibleWorkspaces.map((workspace) => { + const href = `/workspace/${workspace.id}` + const isActive = pathname === href || Boolean(pathname?.startsWith(`${href}/`)) + const isMenuOpen = menu.isOpen && selectedId === workspace.id + const isPinned = pinnedWorkspaceIds.has(workspace.id) + const label = ( + <> + + + + ) + const onMoreClick = (event: React.MouseEvent) => { + event.preventDefault() + event.stopPropagation() + if (isMenuOpen) { + menu.closeMenu() + return + } + setSelectedId(workspace.id) + flyout?.setLocked(true) + const rect = event.currentTarget.getBoundingClientRect() + menu.openMenuAt({ x: rect.right, y: rect.top }) + } + + if (rename.editingId === workspace.id) { + return ( + + } + aria-label={`Rename workspace ${workspace.name}`} + value={rename.value} + onChange={(event) => rename.setValue(event.target.value)} + onKeyDown={rename.handleKeyDown} + onBlur={() => void rename.saveRename()} + disabled={rename.isSaving} + /> + ) + } + + if (flyout) { + return ( + { + if (menu.isOpen || rename.editingId) event.preventDefault() + }} + action={ + menu.preventDismiss()} + onClick={onMoreClick} + > + + + } + > + openMenu(event, workspace.id)} + > + {label} + {isPinned && } + + + ) + } + + return ( + openMenu(event, workspace.id)} + > + {label} +
+ {isPinned && ( + + )} + +
+ + ) + })} + {!flyout && workspaces.length > PAGE_SIZE && ( + + )} + { + if (selectedWorkspace) + window.open(`/workspace/${selectedWorkspace.id}`, '_blank', 'noopener,noreferrer') + }} + onRename={() => { + if (selectedWorkspace?.permissions === 'admin') rename.startRename(selectedWorkspace) + }} + isPinned={Boolean(selectedId && pinnedWorkspaceIds.has(selectedId))} + onTogglePin={() => { + if (selectedWorkspace) { + togglePin( + { + workspaceId: selectedWorkspace.id, + pinned: !pinnedWorkspaceIds.has(selectedWorkspace.id), + }, + { onError: (error) => toast.error(error.message) } + ) + } + }} + /> + + ) +} diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.test.tsx index 7505e56a100..8f57c32eb84 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.test.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.test.tsx @@ -7,18 +7,34 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' const state = vi.hoisted(() => ({ isOpen: false, - workspaces: [] as { id: string; name: string; logoUrl: null }[], + workspaces: [] as { id: string; name: string; logoUrl: null; permissions: string }[], + pins: new Set(), + canCreate: true, + createOrganizationId: 'org-1', + mockCreate: vi.fn(), + mockRename: vi.fn(), + mockPin: vi.fn(), + mockPush: vi.fn(), isLoading: false, })) vi.mock('next/link', () => ({ - default: ({ href, children, ...props }: { href: string; children: React.ReactNode }) => ( + default: ({ + href, + children, + onNavigate: _onNavigate, + ...props + }: { + href: string + children: React.ReactNode + onNavigate?: () => void + }) => ( {children} ), })) -vi.mock('@/app/workspace/[workspaceId]/w/components/sidebar/hooks', () => ({ +vi.mock('@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-hover-menu', () => ({ useHoverMenu: () => ({ isOpen: state.isOpen, open: vi.fn(), @@ -28,8 +44,32 @@ vi.mock('@/app/workspace/[workspaceId]/w/components/sidebar/hooks', () => ({ contentProps: { onMouseEnter: vi.fn(), onMouseLeave: vi.fn(), onCloseAutoFocus: vi.fn() }, }), })) -vi.mock('@/app/o/[organizationId]/components/organization-sidebar/hooks', () => ({ - useOrganizationWorkspaces: () => ({ workspaces: state.workspaces, isLoading: state.isLoading }), +vi.mock( + '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces', + () => ({ + useOrganizationWorkspaces: () => ({ + workspaces: state.workspaces, + pinnedWorkspaceIds: state.pins, + isLoading: state.isLoading, + }), + }) +) + +vi.mock('next/navigation', () => ({ + usePathname: () => '/o/org-1/home', + useRouter: () => ({ push: state.mockPush }), +})) +vi.mock('@/hooks/queries/workspace', () => ({ + useUpdateWorkspace: () => ({ mutateAsync: state.mockRename }), + useToggleWorkspacePin: () => ({ mutate: state.mockPin }), + useCreateWorkspace: () => ({ mutateAsync: state.mockCreate, isPending: false }), + useWorkspaceCreationPolicy: () => ({ + data: { + canCreate: state.canCreate, + organizationId: state.createOrganizationId, + reason: 'Workspace limit reached', + }, + }), })) import { WorkspacesSection } from '@/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section' @@ -47,12 +87,19 @@ beforeEach(() => { disconnect() {} } ) + vi.clearAllMocks() + state.canCreate = true + state.createOrganizationId = 'org-1' + state.pins.clear() + state.mockCreate.mockResolvedValue({ id: 'ws-new' }) + state.mockRename.mockResolvedValue({}) state.isOpen = false state.isLoading = false state.workspaces = Array.from({ length: 8 }, (_, index) => ({ id: `ws-${index + 1}`, name: `Workspace ${index + 1}`, logoUrl: null, + permissions: 'admin', })) container = document.createElement('div') document.body.appendChild(container) @@ -68,13 +115,7 @@ afterEach(async () => { async function render(props: Partial[0]> = {}) { await act(async () => { root.render( - {}} - {...props} - /> + ) }) } @@ -90,7 +131,7 @@ function pager() { } describe('WorkspacesSection', () => { - it('shows the first page and pages the rest in like the sidebar chats', async () => { + it('shows the first page and expands the remaining workspaces', async () => { await render() expect(rows()).toHaveLength(5) expect(pager()?.textContent).toBe('See more') @@ -136,4 +177,96 @@ describe('WorkspacesSection', () => { await render({ isCollapsed: true }) expect(container.querySelector('[aria-label="Workspaces"]')).not.toBeNull() }) + it('does not add a workspace search field to the sidebar', async () => { + await render() + expect(container.querySelector('input')).toBeNull() + expect(rows()).toHaveLength(5) + }) + + it('uses the shared pin mutation and permission-aware rename action', async () => { + await render() + await act(async () => + container.querySelector('[aria-label="Options for Workspace 1"]')?.click() + ) + const pin = Array.from(document.querySelectorAll('[role="menuitem"]')).find( + (item) => item.textContent === 'Pin' + ) + expect(pin).toBeDefined() + await act(async () => pin?.click()) + expect(state.mockPin).toHaveBeenCalledWith( + { workspaceId: 'ws-1', pinned: true }, + expect.any(Object) + ) + await act(async () => + container.querySelector('[aria-label="Options for Workspace 1"]')?.click() + ) + const rename = Array.from(document.querySelectorAll('[role="menuitem"]')).find( + (item) => item.textContent === 'Rename' + ) + await act(async () => rename?.click()) + const input = container.querySelector( + '[aria-label="Rename workspace Workspace 1"]' + )! + expect(input).not.toBeNull() + expect(input.parentElement?.textContent).toBe('1') + expect(input.parentElement?.className).toContain('surface-active') + await act(async () => typeInto(input, 'Renamed workspace')) + await act(async () => + input.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true })) + ) + expect(state.mockRename).toHaveBeenCalledWith({ + workspaceId: 'ws-1', + name: 'Renamed workspace', + }) + }) + + it('prevents a read-only viewer from renaming while retaining pinning', async () => { + state.workspaces[0].permissions = 'read' + await render() + await act(async () => + container.querySelector('[aria-label="Options for Workspace 1"]')?.click() + ) + const rename = Array.from(document.querySelectorAll('[role="menuitem"]')).find( + (item) => item.textContent === 'Rename' + ) + expect(rename?.getAttribute('aria-disabled')).toBe('true') + await act(async () => rename?.click()) + expect(container.querySelector('[aria-label="Rename workspace Workspace 1"]')).toBeNull() + expect(state.mockRename).not.toHaveBeenCalled() + }) + + it('creates in the current organization through the existing modal and mutation', async () => { + await render() + await act(async () => + container.querySelector('[aria-label="New workspace"]')?.click() + ) + const input = document.querySelector('input[placeholder="Workspace name"]')! + expect(input).not.toBeNull() + await act(async () => typeInto(input, 'New team workspace')) + const create = Array.from(document.querySelectorAll('button')).find( + (button) => button.textContent === 'Create' + ) + await act(async () => create?.click()) + expect(state.mockCreate).toHaveBeenCalledWith({ name: 'New team workspace' }) + expect(state.mockPush).toHaveBeenCalledWith('/workspace/ws-new') + }) + + it.each([false, true])( + 'respects creation policy and never creates in a different organization (allowed=%s)', + async (allowed) => { + state.canCreate = allowed + state.createOrganizationId = allowed ? 'another-org' : 'org-1' + await render() + const create = container.querySelector('[aria-label="New workspace"]')! + expect(create.disabled).toBe(true) + await act(async () => create.click()) + expect(document.querySelector('input[placeholder="Workspace name"]')).toBeNull() + expect(state.mockCreate).not.toHaveBeenCalled() + } + ) }) + +function typeInto(input: HTMLInputElement, value: string) { + Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value')?.set?.call(input, value) + input.dispatchEvent(new Event('input', { bubbles: true })) +} diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.tsx index 5e64b48fa75..8c11625e2a2 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspaces-section.tsx @@ -1,98 +1,104 @@ 'use client' import { useState } from 'react' -import { chipVariants, cn, OverflowText } from '@sim/emcn' -import { Workspaces } from '@sim/emcn/icons' -import Link from 'next/link' -import { IdentityTile } from '@/components/identity-tile/identity-tile' -import { getWorkspaceInitial } from '@/lib/workspaces/initials' -import { WorkspacesRailFlyout } from '@/app/o/[organizationId]/components/organization-sidebar/components/workspaces-rail-flyout' -import { useOrganizationWorkspaces } from '@/app/o/[organizationId]/components/organization-sidebar/hooks' +import { Button, cn, Tooltip } from '@sim/emcn' +import { Plus, Workspaces } from '@sim/emcn/icons' +import { useRouter } from 'next/navigation' +import { WorkspaceList } from '@/app/o/[organizationId]/components/organization-sidebar/components/workspaces-section/workspace-list' import { CollapsedSidebarMenu, SidebarSection, } from '@/app/workspace/[workspaceId]/w/components/sidebar/components' +import { CreateWorkspaceModal } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/components/create-workspace-modal/create-workspace-modal' import { SIDEBAR_ITEM_GAP_CLASS } from '@/app/workspace/[workspaceId]/w/components/sidebar/constants' -import { useHoverMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks' - -/** Rows shown at first, and added per "See more" — the workspace sidebar's Chats paging. */ -const PAGE_SIZE = 5 +import { useHoverMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-hover-menu' +import { useCreateWorkspace, useWorkspaceCreationPolicy } from '@/hooks/queries/workspace' +import { useSettingsDirtyStore } from '@/stores/settings/dirty/store' interface WorkspacesSectionProps { organizationId: string isCollapsed: boolean pathname: string | null - onContextMenu: (e: React.MouseEvent, href: string) => void } -/** - * The organization's workspaces the viewer belongs to: the first section of the - * scroll region, so it carries no section gap — the divider padding above it is - * the whole distance, exactly as the workspace sidebar spaces its own Chats. - * Expanded, five rail chips and a muted "See more" that pages the rest in, the way - * the workspace sidebar pages its Chats; collapsed, a hover flyout off the rail glyph. - */ export function WorkspacesSection({ organizationId, isCollapsed, pathname, - onContextMenu, }: WorkspacesSectionProps) { + const router = useRouter() const hover = useHoverMenu() - const { workspaces, isLoading } = useOrganizationWorkspaces(organizationId) - const [visibleCount, setVisibleCount] = useState(PAGE_SIZE) - const hasMore = workspaces.length > visibleCount + const [isCreateOpen, setIsCreateOpen] = useState(false) + const { data: creationPolicy } = useWorkspaceCreationPolicy() + const { mutateAsync: createWorkspace, isPending: isCreating } = useCreateWorkspace() + const canCreate = creationPolicy?.canCreate && creationPolicy.organizationId === organizationId + const createDisabledReason = creationPolicy?.reason ?? 'Workspace creation is unavailable.' + + const openCreate = () => { + if (!canCreate || isCreating) return + useSettingsDirtyStore.getState().requestLeave(() => { + hover.close() + setIsCreateOpen(true) + }) + } return ( - - {isCollapsed ? ( -
- } - hover={hover} - ariaLabel='Workspaces' - > - - -
- ) : ( -
- {!isLoading && workspaces.length === 0 && ( -
- No workspaces yet -
- )} - {workspaces.slice(0, visibleCount).map((workspace) => { - const href = `/workspace/${workspace.id}` - return ( - onContextMenu(e, href)} - > - - - - ) - })} - {workspaces.length > PAGE_SIZE && ( - + + + {canCreate ? 'New workspace' : createDisabledReason} + + + ) + } + > + {isCollapsed ? ( +
+ } + hover={hover} + ariaLabel='Workspaces' + primaryAction={ + canCreate ? { label: 'New workspace', onSelect: openCreate } : undefined + } > - {hasMore ? 'See more' : 'See less'} - - )} -
- )} - + + +
+ ) : ( +
+ +
+ )} + + { + if (!canCreate) throw new Error(createDisabledReason) + const workspace = await createWorkspace({ name }) + setIsCreateOpen(false) + router.push(`/workspace/${workspace.id}`) + }} + /> + ) } diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chat-actions.ts b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chat-actions.ts index 9ba5070470c..1ad28686c39 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chat-actions.ts +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chat-actions.ts @@ -1,10 +1,13 @@ import { useCallback, useEffect, useState } from 'react' import { toast } from '@sim/emcn' import { getErrorMessage } from '@sim/utils/errors' +import { useRouter } from 'next/navigation' +import { organizationRoutes } from '@/lib/navigation/paths' import type { OrganizationChat } from '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-chats' import { useFlyoutInlineRename } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename' import { useHoverMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-hover-menu' import { + useDeleteMothershipChat, useMarkMothershipChatRead, useMarkMothershipChatUnread, useRenameMothershipChat, @@ -21,7 +24,9 @@ export function useOrganizationChatActions({ organizationId, chats, }: UseOrganizationChatActionsProps) { + const router = useRouter() const owner = { organizationId } + const { mutate: deleteChat, isPending: isDeleting } = useDeleteMothershipChat(owner) const { mutateAsync: renameChat } = useRenameMothershipChat(owner) const { mutate: pinChat } = useSetMothershipChatPinned(owner) const { mutate: readChat } = useMarkMothershipChatRead(owner) @@ -29,6 +34,7 @@ export function useOrganizationChatActions({ const menu = useContextMenu() const hover = useHoverMenu() const [selectedChatId, setSelectedChatId] = useState(null) + const [chatToDelete, setChatToDelete] = useState(null) const selectedChat = chats.find((chat) => chat.id === selectedChatId) const rename = useFlyoutInlineRename({ itemType: 'chat', @@ -80,6 +86,27 @@ export function useOrganizationChatActions({ const chatHref = selectedChat?.href const chatPinned = selectedChat?.isPinned + const startDelete = useCallback(() => { + if (selectedChat) setChatToDelete(selectedChat) + }, [selectedChat]) + + const cancelDelete = useCallback(() => { + if (!isDeleting) setChatToDelete(null) + }, [isDeleting]) + + const confirmDelete = useCallback(() => { + if (!chatToDelete || isDeleting) return + deleteChat(chatToDelete.id, { + onSuccess: () => { + setChatToDelete(null) + if (window.location.pathname === chatToDelete.href) { + router.push(organizationRoutes(organizationId).home) + } + }, + onError: (error) => toast.error(error.message), + }) + }, [chatToDelete, deleteChat, isDeleting, organizationId, router]) + const startRename = useCallback(() => { if (chatId && chatName !== undefined) rename.startRename({ id: chatId, name: chatName }) }, [chatId, chatName, rename.startRename]) @@ -119,6 +146,11 @@ export function useOrganizationChatActions({ onMorePointerDown, onMoreClick, startRename, + chatToDelete, + isDeleting, + startDelete, + cancelDelete, + confirmDelete, togglePin, markRead, markUnread, diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.test.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.test.tsx new file mode 100644 index 00000000000..6e639163a73 --- /dev/null +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.test.tsx @@ -0,0 +1,123 @@ +/** + * @vitest-environment jsdom + */ +import { act } from 'react' +import { createRoot, hydrateRoot, type Root } from 'react-dom/client' +import { renderToString } from 'react-dom/server' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { STORAGE_KEYS, WorkspaceRecencyStorage } from '@/lib/core/utils/browser-storage' + +const { mockUseWorkspacesQuery, pins } = vi.hoisted(() => ({ + pins: { current: new Set() }, + mockUseWorkspacesQuery: vi.fn(), +})) + +vi.mock('@/hooks/queries/workspace', () => ({ + useWorkspacesQuery: mockUseWorkspacesQuery, + EMPTY_PINNED_WORKSPACE_IDS: new Set(), + usePinnedWorkspaceIds: () => ({ data: pins.current }), +})) + +import { useOrganizationWorkspaces } from '@/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces' + +function Harness() { + const { workspaces } = useOrganizationWorkspaces('org-1') + return ( +
    + {workspaces.map((workspace) => ( +
  • {workspace.id}
    • + ))} +
+ ) +} + +let container: HTMLDivElement +let root: Root | undefined + +beforeEach(() => { + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + localStorage.clear() + pins.current = new Set() + mockUseWorkspacesQuery.mockReturnValue({ + data: [ + { id: 'newest', organizationId: 'org-1' }, + { id: 'other-org', organizationId: 'org-2' }, + { id: 'older', organizationId: 'org-1' }, + { id: 'oldest', organizationId: 'org-1' }, + ], + isLoading: false, + }) + container = document.createElement('div') + document.body.appendChild(container) +}) + +afterEach(async () => { + if (root) await act(async () => root?.unmount()) + root = undefined + container.remove() + localStorage.clear() + vi.unstubAllGlobals() +}) + +function workspaceIds() { + return Array.from(container.querySelectorAll('li'), (item) => item.textContent) +} + +describe('useOrganizationWorkspaces', () => { + it('hydrates the prefetched order before applying visit history without changing the query cache', async () => { + localStorage.setItem( + STORAGE_KEYS.WORKSPACE_RECENCY, + JSON.stringify({ oldest: 100, older: 200, 'other-org': 300 }) + ) + container.innerHTML = renderToString() + expect(workspaceIds()).toEqual(['newest', 'older', 'oldest']) + + const onRecoverableError = vi.fn() + await act(async () => { + root = hydrateRoot(container, , { onRecoverableError }) + }) + + expect(onRecoverableError).not.toHaveBeenCalled() + expect(workspaceIds()).toEqual(['older', 'oldest', 'newest']) + expect(mockUseWorkspacesQuery().data.map(({ id }: { id: string }) => id)).toEqual([ + 'newest', + 'other-org', + 'older', + 'oldest', + ]) + }) + + it('preserves creation-date order when the browser has no visit history', async () => { + await act(async () => { + root = createRoot(container) + root.render() + }) + + expect(workspaceIds()).toEqual(['newest', 'older', 'oldest']) + }) + it('keeps pins first and reacts to visits without mutating the query cache', async () => { + pins.current = new Set(['oldest']) + await act(async () => { + root = createRoot(container) + root.render() + }) + expect(workspaceIds()).toEqual(['oldest', 'newest', 'older']) + await act(async () => WorkspaceRecencyStorage.touch('older')) + expect(workspaceIds()).toEqual(['oldest', 'older', 'newest']) + pins.current = new Set() + await act(async () => root?.render()) + expect(workspaceIds()).toEqual(['older', 'newest', 'oldest']) + }) + + it('follows visit history changed by another tab', async () => { + await act(async () => { + root = createRoot(container) + root.render() + }) + await act(async () => { + localStorage.setItem(STORAGE_KEYS.WORKSPACE_RECENCY, JSON.stringify({ oldest: 300 })) + window.dispatchEvent(new StorageEvent('storage', { key: STORAGE_KEYS.WORKSPACE_RECENCY })) + }) + expect(workspaceIds()).toEqual(['oldest', 'newest', 'older']) + }) +}) diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.ts b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.ts index 6c54a696a36..761fddcd7c4 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.ts +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/hooks/use-organization-workspaces.ts @@ -1,4 +1,9 @@ -import { useWorkspacesQuery } from '@/hooks/queries/workspace' +import { + EMPTY_PINNED_WORKSPACE_IDS, + usePinnedWorkspaceIds, + useWorkspacesQuery, +} from '@/hooks/queries/workspace' +import { useWorkspaceOrder } from '@/hooks/use-workspace-order' /** * The organization's workspaces the viewer belongs to, for the sidebar's @@ -7,8 +12,15 @@ import { useWorkspacesQuery } from '@/hooks/queries/workspace' */ export function useOrganizationWorkspaces(organizationId: string) { const { data = [], isLoading } = useWorkspacesQuery() + const { data: pinnedWorkspaceIds = EMPTY_PINNED_WORKSPACE_IDS } = usePinnedWorkspaceIds() + const orderedWorkspaces = useWorkspaceOrder(data, pinnedWorkspaceIds) + const workspaces = orderedWorkspaces.filter( + (workspace) => workspace.organizationId === organizationId + ) - const workspaces = data.filter((workspace) => workspace.organizationId === organizationId) - - return { workspaces, isLoading } + return { + workspaces, + pinnedWorkspaceIds, + isLoading, + } } diff --git a/apps/sim/app/o/[organizationId]/components/organization-sidebar/organization-sidebar.tsx b/apps/sim/app/o/[organizationId]/components/organization-sidebar/organization-sidebar.tsx index 126004e808d..363e229f6ec 100644 --- a/apps/sim/app/o/[organizationId]/components/organization-sidebar/organization-sidebar.tsx +++ b/apps/sim/app/o/[organizationId]/components/organization-sidebar/organization-sidebar.tsx @@ -80,7 +80,7 @@ export const OrganizationSidebar = memo(function OrganizationSidebar() { const pathname = usePathname() const posthog = usePostHog() - const { organization, searchAccess } = useOrganizationContext() + const { organization, viewer, searchAccess } = useOrganizationContext() const toggleCollapsed = useSidebarStore((state) => state.toggleCollapsed) const { handlePointerDown } = useSidebarResize() const showCollapsedTooltips = useCollapsedTooltips(isCollapsed) @@ -181,6 +181,7 @@ export const OrganizationSidebar = memo(function OrganizationSidebar() { > @@ -262,7 +263,6 @@ export const OrganizationSidebar = memo(function OrganizationSidebar() { organizationId={organization.id} isCollapsed={isCollapsed} pathname={pathname} - onContextMenu={handleHrefContextMenu} /> {searchAccess.memberScoped && ( ({ toggleListening: vi.fn(), resetTranscript: vi.fn(), submit: vi.fn(), + upload: vi.fn(), })) vi.mock('@/hooks/use-speech-to-text', () => ({ useSpeechToText: mocks.speech })) +vi.mock('@/lib/uploads/client/session-upload', () => ({ uploadInternalFileSession: mocks.upload })) vi.mock('@/hooks/use-animated-placeholder', () => ({ useAnimatedPlaceholder: () => 'Ask Sim to' })) vi.mock('@/hooks/use-chat-input-focus', () => ({ useChatInputFocus: vi.fn() })) vi.mock('@/app/o/[organizationId]/providers/organization-provider', () => ({ @@ -19,6 +21,7 @@ vi.mock('@/app/o/[organizationId]/providers/organization-provider', () => ({ })) import { Composer } from '@/app/o/[organizationId]/home/components/composer/composer' +import { useFileAttachments } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments' let root: Root let container: HTMLDivElement @@ -26,6 +29,17 @@ let container: HTMLDivElement beforeEach(() => { vi.clearAllMocks() vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + vi.stubGlobal( + 'URL', + class extends URL { + static createObjectURL = vi.fn(() => 'blob:image-preview') + static revokeObjectURL = vi.fn() + } + ) + mocks.upload.mockResolvedValue({ + key: 'assistant/organization-a/user-a/image-a/screenshot.png', + path: '/api/files/serve/image-a?context=mothership', + }) vi.stubGlobal( 'matchMedia', vi.fn(() => ({ @@ -50,21 +64,25 @@ afterEach(async () => { await act(async () => root.unmount()) container.remove() vi.unstubAllGlobals() + vi.restoreAllMocks() }) -async function render(isInitialView: boolean) { +async function render(isInitialView: boolean, initialValue = 'Summarize') { function Harness() { - const [value, setValue] = useState('Summarize') + const [value, setValue] = useState(initialValue) + const files = useFileAttachments({ userId: 'user-a', organizationId: 'organization-a' }) return ( { - mocks.submit(value) + mocks.submit(value, files.attachedFiles) setValue('') + files.clearAttachedFiles() }} /> ) @@ -90,7 +108,7 @@ describe('organization voice composer', () => { await act(async () => { container.querySelector('button[aria-label="Send"]')!.click() }) - expect(mocks.submit).toHaveBeenCalledWith('Summarize the release') + expect(mocks.submit).toHaveBeenCalledWith('Summarize the release', []) expect(mocks.resetTranscript).toHaveBeenCalledOnce() await act(async () => mocks.speech.mock.calls.at(-1)![0].onTranscript('Next question')) expect(container.querySelector('textarea')!.value).toBe('Next question') @@ -109,3 +127,110 @@ describe('organization voice composer', () => { expect(container.querySelector('button[aria-label="Voice input"]')).toBeNull() }) }) + +function fileList(files: File[]): FileList { + return Object.assign(files, { item: (index: number) => files[index] ?? null }) +} + +async function paste(files: File[]) { + const event = new Event('paste', { bubbles: true, cancelable: true }) + Object.defineProperty(event, 'clipboardData', { value: { files: fileList(files) } }) + await act(async () => container.querySelector('textarea')!.dispatchEvent(event)) + return event +} + +describe('organization image composer', () => { + it.each([true, false])( + 'pastes and submits an image without text (initial: %s)', + async (initial) => { + await render(initial, '') + const image = new File(['image'], 'screenshot.png', { type: 'image/png' }) + const event = await paste([image]) + expect(event.defaultPrevented).toBe(true) + expect(mocks.upload).toHaveBeenCalledWith( + expect.objectContaining({ + purpose: 'mothership_attachment', + organizationId: 'organization-a', + file: image, + }) + ) + expect(container.querySelector('img')?.getAttribute('alt')).toBe('screenshot.png') + await act(async () => + container.querySelector('button[aria-label="Send"]')!.click() + ) + expect(mocks.submit).toHaveBeenCalledWith('', [ + expect.objectContaining({ + key: 'assistant/organization-a/user-a/image-a/screenshot.png', + uploading: false, + }), + ]) + expect(container.querySelector('img')).toBeNull() + } + ) + + it('leaves ordinary text paste to the textarea', async () => { + await render(true) + expect((await paste([])).defaultPrevented).toBe(false) + expect(mocks.upload).not.toHaveBeenCalled() + }) + + it('accepts dropped images through the same upload flow', async () => { + await render(false) + const image = new File(['image'], 'dropped.png', { type: 'image/png' }) + const drop = new Event('drop', { bubbles: true, cancelable: true }) + Object.defineProperty(drop, 'dataTransfer', { value: { files: fileList([image]) } }) + await act(async () => container.querySelector('textarea')!.dispatchEvent(drop)) + expect(drop.defaultPrevented).toBe(true) + expect(mocks.upload).toHaveBeenCalledWith(expect.objectContaining({ file: image })) + expect(container.querySelector('img')?.getAttribute('alt')).toBe('dropped.png') + }) + + it('blocks Send and Enter until an image upload finishes', async () => { + let finish!: (value: { key: string; path: string }) => void + mocks.upload.mockImplementation( + () => + new Promise((resolve) => { + finish = resolve + }) + ) + await render(true) + await paste([new File(['image'], 'screenshot.png', { type: 'image/png' })]) + expect(container.querySelector('button[aria-label="Send"]')!.disabled).toBe( + true + ) + await act(async () => + container + .querySelector('textarea')! + .dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true })) + ) + expect(mocks.submit).not.toHaveBeenCalled() + await act(async () => finish({ key: 'image-key', path: '/image-path' })) + expect(container.querySelector('button[aria-label="Send"]')!.disabled).toBe( + false + ) + }) + + it('uses the picker and lets an attachment be removed before sending', async () => { + await render(true, '') + const input = container.querySelector('input[type="file"]')! + const click = vi.spyOn(input, 'click') + await act(async () => + container.querySelector('button[aria-label="Attach images"]')!.click() + ) + expect(click).toHaveBeenCalledOnce() + expect(input.accept).toContain('image/png') + Object.defineProperty(input, 'files', { + value: fileList([new File(['image'], 'screenshot.png', { type: 'image/png' })]), + }) + await act(async () => input.dispatchEvent(new Event('change', { bubbles: true }))) + await act(async () => + container + .querySelector('button[aria-label="Remove screenshot.png"]')! + .click() + ) + expect(container.querySelector('img')).toBeNull() + expect(container.querySelector('button[aria-label="Send"]')!.disabled).toBe( + true + ) + }) +}) diff --git a/apps/sim/app/o/[organizationId]/home/components/composer/composer.tsx b/apps/sim/app/o/[organizationId]/home/components/composer/composer.tsx index 1c897d4d532..891af13473a 100644 --- a/apps/sim/app/o/[organizationId]/home/components/composer/composer.tsx +++ b/apps/sim/app/o/[organizationId]/home/components/composer/composer.tsx @@ -1,11 +1,15 @@ 'use client' import { useRef } from 'react' -import { Button, cn } from '@sim/emcn' -import { ArrowUp } from '@sim/emcn/icons' +import { Button, Chip, cn, Tooltip } from '@sim/emcn' +import { ArrowUp, Plus } from '@sim/emcn/icons' +import { ASSISTANT_IMAGE_ACCEPT_ATTRIBUTE } from '@/lib/uploads/shared/assistant-images' import { useOrganizationContext } from '@/app/o/[organizationId]/providers/organization-provider' +import { AttachedFilesList } from '@/app/workspace/[workspaceId]/home/components/user-input/components/attached-files-list/attached-files-list' +import { DropOverlay } from '@/app/workspace/[workspaceId]/home/components/user-input/components/drop-overlay/drop-overlay' import { MicButton } from '@/app/workspace/[workspaceId]/home/components/user-input/components/mic-button/mic-button' import { MicrophonePermissionHelp } from '@/app/workspace/[workspaceId]/home/components/user-input/components/microphone-permission-help/microphone-permission-help' +import type { useFileAttachments } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments' import { useAnimatedPlaceholder } from '@/hooks/use-animated-placeholder' import { useChatInputFocus } from '@/hooks/use-chat-input-focus' import { useVoiceInput } from '@/hooks/use-voice-input' @@ -17,6 +21,7 @@ const SEND_BUTTON_DISABLED = 'bg-[#808080] dark:bg-[#808080]' interface ComposerProps { value: string + files: ReturnType /** On the empty home the placeholder types itself and the field is taller; in a chat it is the plain footer input. */ isInitialView: boolean isSending: boolean @@ -32,6 +37,7 @@ interface ComposerProps { */ export function Composer({ value, + files, isInitialView, isSending, onChange, @@ -46,7 +52,9 @@ export function Composer({ getValue: () => value, onChange, }) - const canSubmit = value.trim().length > 0 + const canSubmit = + !files.attachedFiles.some((file) => file.uploading) && + (value.trim().length > 0 || files.attachedFiles.some((file) => file.key)) const animatedPlaceholder = useAnimatedPlaceholder(isInitialView) const placeholder = isInitialView ? animatedPlaceholder : 'Send message to Sim' @@ -58,11 +66,20 @@ export function Composer({ return (
+
onChange(event.target.value)} + onPaste={(event) => { + const pasted = event.clipboardData.files + if (!pasted.length) return + event.preventDefault() + void files.processFiles(pasted) + }} onKeyDown={(event) => { if (event.key === 'Enter' && !event.shiftKey && !event.nativeEvent.isComposing) { event.preventDefault() @@ -86,43 +109,68 @@ export function Composer({ />
-
- {voice.isSupported && ( - - )} - {isSending ? ( - - ) : ( - - )} + + + + + ) : ( + + )} +
+ + {files.isDragging && } ({ apiKeys: vi.fn(), authorizedApps: vi.fn(), fetchNextPage: vi.fn(), + upload: vi.fn(), })) vi.mock('@/lib/auth/auth-client', () => ({ useSession: () => ({ data: { user: { id: 'reader' } } }), })) +vi.mock('@/lib/uploads/client/session-upload', () => ({ uploadInternalFileSession: mocks.upload })) vi.mock('@/lib/core/utils/browser-storage', () => ({ MothershipHandoffStorage: { consume: mocks.consume }, })) @@ -46,6 +48,14 @@ let container: HTMLDivElement beforeEach(() => { vi.clearAllMocks() vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + vi.stubGlobal( + 'URL', + class extends URL { + static createObjectURL = vi.fn(() => 'blob:image-preview') + static revokeObjectURL = vi.fn() + } + ) + mocks.upload.mockResolvedValue({ key: 'image-key', path: '/image-path' }) mocks.context.mockReturnValue({ organization: { id: 'organization-a' }, searchAccess: { memberScoped: true }, @@ -290,6 +300,90 @@ describe('organization home', () => { await act(async () => composerProps().onSubmit()) expect(mocks.send).not.toHaveBeenCalled() }) + it('sends image-only turns with canonical attachment properties and clears the draft', async () => { + await act(async () => root.render()) + const files = [new File(['image'], 'screenshot.png', { type: 'image/png' })] + await act(async () => + composerProps().files.processFiles( + Object.assign(files, { item: (index: number) => files[index] ?? null }) + ) + ) + await act(async () => composerProps().onSubmit()) + expect(mocks.send).toHaveBeenCalledWith( + '', + [ + expect.objectContaining({ + id: expect.any(String), + key: 'image-key', + filename: 'screenshot.png', + media_type: 'image/png', + size: 5, + path: '/image-path', + }), + ], + undefined, + { requestMode: 'assistant' } + ) + expect(composerProps().files.attachedFiles).toEqual([]) + }) + + it('restores queued images when editing and includes them in the replacement turn', async () => { + mocks.renderer.mockImplementation(({ composer }: { composer: ReactNode }) => composer) + const attachments = [ + { + id: 'image-a', + key: 'image-key', + filename: 'screenshot.png', + media_type: 'image/png', + size: 5, + }, + ] + mocks.chat.mockReturnValue({ + messages: [], + sendMessage: mocks.send, + editQueuedMessage: () => ({ + id: 'queued-a', + content: 'Explain this', + fileAttachments: attachments, + }), + }) + await act(async () => root.render()) + await act(async () => mocks.renderer.mock.lastCall![0].onEditQueuedMessage('queued-a')) + expect(composerProps().files.attachedFiles[0]).toEqual( + expect.objectContaining({ + name: 'screenshot.png', + key: 'image-key', + uploading: false, + path: '/api/files/serve/image-key?context=mothership&preview=1', + }) + ) + await act(async () => composerProps().onSubmit()) + expect(mocks.send).toHaveBeenCalledWith( + 'Explain this', + [{ ...attachments[0], path: '/api/files/serve/image-key?context=mothership&preview=1' }], + undefined, + { + requestMode: 'assistant', + } + ) + }) + + it('resumes image-only handoffs without dropping their attachments', async () => { + const attachments = [ + { + id: 'image-a', + key: 'image-key', + filename: 'screenshot.png', + media_type: 'image/png', + size: 5, + }, + ] + mocks.consume.mockReturnValueOnce({ message: '', fileAttachments: attachments }) + await act(async () => root.render()) + expect(mocks.send).toHaveBeenCalledWith('', attachments, undefined, { + requestMode: 'assistant', + }) + }) it('resumes a scoped handoff with the original search filters', async () => { const assistantSearch = { documentIds: ['document-a'] } mocks.consume.mockReturnValueOnce({ message: 'Summarize', assistantSearch }) diff --git a/apps/sim/app/o/[organizationId]/home/organization-home.tsx b/apps/sim/app/o/[organizationId]/home/organization-home.tsx index 59d754b9117..34ec83488ae 100644 --- a/apps/sim/app/o/[organizationId]/home/organization-home.tsx +++ b/apps/sim/app/o/[organizationId]/home/organization-home.tsx @@ -2,6 +2,7 @@ import { useEffect, useState } from 'react' import { useSession } from '@/lib/auth/auth-client' +import { getMothershipAttachmentPreviewUrl } from '@/lib/copilot/chat/attachment-preview' import { MothershipHandoffStorage } from '@/lib/core/utils/browser-storage' import { Composer } from '@/app/o/[organizationId]/home/components/composer' import { GetStarted } from '@/app/o/[organizationId]/home/components/get-started' @@ -9,6 +10,8 @@ import { useOrganizationContext } from '@/app/o/[organizationId]/providers/organ import { SearchIntegrationConnection } from '@/app/workspace/[workspaceId]/home/components/message-content/components/special-tags/search-integration-connection' import { MothershipChat } from '@/app/workspace/[workspaceId]/home/components/mothership-chat' import { useChat } from '@/app/workspace/[workspaceId]/home/hooks/use-chat' +import type { FileAttachmentForApi } from '@/app/workspace/[workspaceId]/home/types' +import { useFileAttachments } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments' import { useMarkMothershipChatRead } from '@/hooks/queries/mothership-chats' interface OrganizationHomeProps { @@ -28,6 +31,7 @@ function OrganizationHomeContent({ userName, chatId }: OrganizationHomeProps) { const { data: session } = useSession() const [draft, setDraft] = useState('') const chat = useChat({ organizationId: organization.id }, chatId) + const files = useFileAttachments({ userId: session?.user?.id, organizationId: organization.id }) const { sendMessage } = chat const { mutate: markRead } = useMarkMothershipChatRead({ organizationId: organization.id }) const firstName = userName?.split(' ')[0] ?? '' @@ -40,8 +44,8 @@ function OrganizationHomeContent({ userName, chatId }: OrganizationHomeProps) { useEffect(() => { if (chatId) return const handoff = MothershipHandoffStorage.consume({ organizationId: organization.id }) - if (handoff?.message) { - void sendMessage(handoff.message, undefined, undefined, { + if (handoff && (handoff.message || handoff.fileAttachments?.length)) { + void sendMessage(handoff.message ?? '', handoff.fileAttachments, undefined, { requestMode: 'assistant', ...(handoff.resumeUserMessageId ? { resumeUserMessageId: handoff.resumeUserMessageId } @@ -51,21 +55,34 @@ function OrganizationHomeContent({ userName, chatId }: OrganizationHomeProps) { } }, [chatId, organization.id, sendMessage]) - const send = (message: string) => { - void sendMessage(message, undefined, undefined, { requestMode: 'assistant' }) + const send = (message: string, fileAttachments?: FileAttachmentForApi[]) => { + void sendMessage(message, fileAttachments, undefined, { requestMode: 'assistant' }) } const submit = () => { const message = draft.trim() - if (!message) return + if (files.attachedFiles.some((file) => file.uploading)) return + const attachments: FileAttachmentForApi[] = files.attachedFiles + .filter((file) => file.key) + .map((file) => ({ + id: file.id, + key: file.key!, + filename: file.name, + media_type: file.type, + size: file.size, + path: file.path, + })) + if (!message && !attachments.length) return setDraft('') - send(message) + send(message, attachments.length ? attachments : undefined) + files.clearAttachedFiles() } const hasChat = Boolean(chatId || chat.messages.length) const composer = ( { const queued = chat.editQueuedMessage(id) - if (queued) setDraft(queued.content) + if (queued) { + setDraft(queued.content) + files.restoreAttachedFiles( + (queued.fileAttachments ?? []).map((file) => ({ + id: file.id, + key: file.key, + name: file.filename, + type: file.media_type, + size: file.size, + path: file.path || getMothershipAttachmentPreviewUrl(file) || '', + previewUrl: getMothershipAttachmentPreviewUrl(file), + uploading: false, + })) + ) + } return queued }} onCancelQueueEdit={chat.cancelQueueEdit} diff --git a/apps/sim/app/o/[organizationId]/layout.test.tsx b/apps/sim/app/o/[organizationId]/layout.test.tsx index bcb6145d12f..2f16491f8b5 100644 --- a/apps/sim/app/o/[organizationId]/layout.test.tsx +++ b/apps/sim/app/o/[organizationId]/layout.test.tsx @@ -7,17 +7,24 @@ import { authMockFns } from '@sim/testing' import { dehydrate } from '@tanstack/react-query' import { renderToStaticMarkup } from 'react-dom/server' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { isChatEnabled } from '@/lib/core/config/env-flags' const { mockGetOrganizationSurfaceContext, mockWorkspaceChrome, mockPrefetchOrganizationSidebar, mockUseSession, + mockUseMothershipChatEvents, } = vi.hoisted(() => ({ mockGetOrganizationSurfaceContext: vi.fn(), mockWorkspaceChrome: vi.fn(({ children }: { children: ReactNode }) => children), mockPrefetchOrganizationSidebar: vi.fn(async () => undefined), mockUseSession: vi.fn(), + mockUseMothershipChatEvents: vi.fn(), +})) + +vi.mock('@/hooks/use-mothership-chat-events', () => ({ + useMothershipChatEvents: mockUseMothershipChatEvents, })) vi.mock('@/lib/auth/auth-client', () => ({ useSession: mockUseSession })) @@ -118,6 +125,10 @@ describe('OrganizationLayout', () => { 'active-org' ) expect(html).toContain('Organization child') + expect(mockUseMothershipChatEvents).toHaveBeenCalledWith( + { organizationId: 'org-1' }, + isChatEnabled + ) expect(html).not.toContain('Stop impersonating') expect(mockWorkspaceChrome).toHaveBeenCalledWith( expect.objectContaining({ initialSidebarCollapsed: true }), diff --git a/apps/sim/app/o/[organizationId]/layout.tsx b/apps/sim/app/o/[organizationId]/layout.tsx index fb4b66e85bd..41628908ca4 100644 --- a/apps/sim/app/o/[organizationId]/layout.tsx +++ b/apps/sim/app/o/[organizationId]/layout.tsx @@ -3,6 +3,7 @@ import { cookies } from 'next/headers' import { redirect } from 'next/navigation' import { getSession } from '@/lib/auth' import { getActiveOrganizationId } from '@/lib/auth/session-response' +import { isChatEnabled } from '@/lib/core/config/env-flags' import { organizationRoutes, WORKSPACE_SETTINGS_PATH } from '@/lib/navigation/paths' import { getOrganizationSurfaceContext } from '@/lib/organizations/surface' import { getQueryClient } from '@/app/_shell/providers/get-query-client' @@ -61,7 +62,7 @@ export default async function OrganizationLayout({ return ( - +
diff --git a/apps/sim/app/o/[organizationId]/providers/organization-provider.tsx b/apps/sim/app/o/[organizationId]/providers/organization-provider.tsx index 849ee65ffba..af1f23d4260 100644 --- a/apps/sim/app/o/[organizationId]/providers/organization-provider.tsx +++ b/apps/sim/app/o/[organizationId]/providers/organization-provider.tsx @@ -2,12 +2,14 @@ import { createContext, type ReactNode, useContext } from 'react' import type { OrganizationSurfaceContext } from '@/lib/organizations/surface' +import { useMothershipChatEvents } from '@/hooks/use-mothership-chat-events' const OrganizationContextValue = createContext(null) interface OrganizationProviderProps { children: ReactNode context: OrganizationSurfaceContext + chatEnabled: boolean } /** @@ -15,7 +17,15 @@ interface OrganizationProviderProps { * organization surface. The layout resolves both on the server, so the first paint * already knows the organization's name and logo. */ -export function OrganizationProvider({ children, context }: OrganizationProviderProps) { +export function OrganizationProvider({ + children, + context, + chatEnabled, +}: OrganizationProviderProps) { + useMothershipChatEvents( + context.searchAccess.memberScoped ? { organizationId: context.organization.id } : undefined, + chatEnabled + ) return ( {children} diff --git a/apps/sim/app/o/[organizationId]/settings/[section]/settings.tsx b/apps/sim/app/o/[organizationId]/settings/[section]/settings.tsx index f924714adc4..f5adfa63f19 100644 --- a/apps/sim/app/o/[organizationId]/settings/[section]/settings.tsx +++ b/apps/sim/app/o/[organizationId]/settings/[section]/settings.tsx @@ -9,6 +9,12 @@ import { import { SettingsSectionProvider } from '@/components/settings/settings-panel' import { useOrganizationContext } from '@/app/o/[organizationId]/providers/organization-provider' +const OrganizationRecentlyDeleted = dynamic(() => + import('@/app/o/[organizationId]/settings/components/organization-recently-deleted').then( + (m) => m.OrganizationRecentlyDeleted + ) +) + const OrganizationIntegrationsSettings = dynamic(() => import( '@/app/o/[organizationId]/settings/components/integrations/organization-integrations-settings' @@ -78,6 +84,9 @@ export function OrganizationSettings({ section }: OrganizationSettingsProps) { return ( + {section === 'recently-deleted' && ( + + )} {section === 'integrations' && } {section === 'connected-accounts' && ( diff --git a/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.test.tsx b/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.test.tsx new file mode 100644 index 00000000000..06bdd5ddafd --- /dev/null +++ b/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.test.tsx @@ -0,0 +1,178 @@ +/** @vitest-environment jsdom */ +import { act, type ReactNode } from 'react' +import { ToastProvider } from '@sim/emcn' +import { sleep } from '@sim/utils/helpers' +import { QueryClient, QueryClientProvider } from '@tanstack/react-query' +import { NuqsTestingAdapter } from 'nuqs/adapters/testing' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { SettingsHeaderSearch } from '@/components/settings/settings-header' + +const mocks = vi.hoisted(() => ({ request: vi.fn(), push: vi.fn() })) +vi.mock('@/lib/api/client/request', () => ({ requestJson: mocks.request })) +vi.mock('next/navigation', () => ({ + useRouter: () => ({ push: mocks.push }), + usePathname: () => '/o/org-1/settings/recently-deleted', +})) +vi.mock('@/components/settings/settings-panel', () => ({ + SettingsPanel: ({ children, search }: { children: ReactNode; search: SettingsHeaderSearch }) => ( + <> + search.onChange(event.target.value)} + /> + {children} + + ), +})) + +import { OrganizationRecentlyDeleted } from '@/app/o/[organizationId]/settings/components/organization-recently-deleted' +import { type MothershipChatMetadata, mothershipChatKeys } from '@/hooks/queries/mothership-chats' + +const CHATS: MothershipChatMetadata[] = [ + { + id: 'chat-1', + name: 'Older chat', + updatedAt: new Date('2026-09-10'), + deletedAt: new Date('2026-09-10'), + isActive: false, + isUnread: false, + isPinned: false, + }, + { + id: 'chat-2', + name: 'Recent chat', + updatedAt: new Date('2026-09-11'), + deletedAt: new Date('2026-09-11'), + isActive: false, + isUnread: false, + isPinned: false, + }, +] +const ARCHIVED_KEY = mothershipChatKeys.organizationList('org-1', 'archived') +let container: HTMLDivElement +let root: Root +let queryClient: QueryClient + +beforeEach(() => { + vi.clearAllMocks() + mocks.request.mockReset().mockResolvedValue({ success: true }) + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + vi.stubGlobal( + 'ResizeObserver', + class { + observe() {} + unobserve() {} + disconnect() {} + } + ) + queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }) + queryClient.setQueryData(ARCHIVED_KEY, CHATS) + container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) +}) + +afterEach(async () => { + await act(async () => root.unmount()) + container.remove() + queryClient.clear() + vi.unstubAllGlobals() +}) + +async function render(search = '') { + await act(async () => { + root.render( + + + + + + + + ) + }) +} + +function button(label: string) { + return Array.from(container.querySelectorAll('button')).find( + (button) => button.textContent === label + )! +} + +describe('OrganizationRecentlyDeleted', () => { + it('shows only this organization’s archived chats, newest deletion first', async () => { + queryClient.setQueryData(mothershipChatKeys.list('workspace-1', 'archived'), [ + { ...CHATS[0], name: 'Workspace chat' }, + ]) + await render() + const text = container.textContent ?? '' + expect(text.indexOf('Recent chat')).toBeLessThan(text.indexOf('Older chat')) + expect(text).not.toContain('Workspace chat') + expect(mocks.request).not.toHaveBeenCalled() + }) + + it('fetches the organization’s archived list when it is not cached', async () => { + queryClient.removeQueries({ queryKey: ARCHIVED_KEY }) + mocks.request.mockResolvedValueOnce({ data: [] }) + await render() + expect(mocks.request).toHaveBeenCalledWith( + expect.objectContaining({ method: 'GET' }), + expect.objectContaining({ + query: { organizationId: 'org-1', scope: 'archived' }, + signal: expect.any(AbortSignal), + }) + ) + }) + + it('filters through the shared settings search parameter', async () => { + await render('?search=recent') + expect(container.textContent).toContain('Recent chat') + expect(container.textContent).not.toContain('Older chat') + }) + + it('restores through the shared mutation and follows the authoritative archived list', async () => { + const invalidate = vi.spyOn(queryClient, 'invalidateQueries').mockResolvedValue() + await render() + await act(async () => button('Restore').click()) + expect(mocks.request).toHaveBeenCalledWith(expect.objectContaining({ method: 'POST' }), { + params: { chatId: 'chat-2' }, + }) + expect(invalidate).toHaveBeenCalledExactlyOnceWith({ + queryKey: mothershipChatKeys.organizationLists('org-1'), + }) + await act(async () => { + queryClient.setQueryData(ARCHIVED_KEY, [CHATS[0]]) + await sleep(1) + }) + expect(container.textContent).not.toContain('Recent chat') + await act(async () => { + queryClient.setQueryData(ARCHIVED_KEY, CHATS) + await sleep(1) + }) + expect(container.textContent).toContain('Recent chat') + expect(button('Restore').disabled).toBe(false) + expect(container.textContent).not.toContain('Restored') + }) + + it('disables repeat restoration while pending and leaves failures retryable', async () => { + vi.spyOn(queryClient, 'invalidateQueries').mockResolvedValue() + const pending = Promise.withResolvers<{ success: boolean }>() + mocks.request.mockReturnValueOnce(pending.promise) + await render() + await act(async () => button('Restore').click()) + await act(async () => sleep(1)) + expect(button('Restoring...').disabled).toBe(true) + await act(async () => button('Restoring...').click()) + expect(mocks.request).toHaveBeenCalledTimes(1) + await act(async () => { + pending.reject(new Error('Restore failed')) + await sleep(1) + }) + expect(button('Restore').disabled).toBe(false) + expect(button('View')).toBeUndefined() + expect(mocks.push).not.toHaveBeenCalled() + expect(queryClient.getQueryData(ARCHIVED_KEY)).toEqual(CHATS) + }) +}) diff --git a/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.tsx b/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.tsx new file mode 100644 index 00000000000..4976f123e40 --- /dev/null +++ b/apps/sim/app/o/[organizationId]/settings/components/organization-recently-deleted.tsx @@ -0,0 +1,84 @@ +'use client' + +import { Chip, toast } from '@sim/emcn' +import { Task } from '@sim/emcn/icons' +import { formatDate } from '@sim/utils/formatting' +import { SettingsPanel } from '@/components/settings/settings-panel' +import { SettingsEmptyState } from '@/app/workspace/[workspaceId]/settings/components/settings-empty-state' +import { + RESOURCE_LIST_STACK, + SettingsResourceRow, +} from '@/app/workspace/[workspaceId]/settings/components/settings-resource-row' +import { useSettingsSearch } from '@/app/workspace/[workspaceId]/settings/components/use-settings-search' +import { + type MothershipChatMetadata, + useOrganizationMothershipChats, + useRestoreMothershipChat, +} from '@/hooks/queries/mothership-chats' + +interface DeletedChatRowProps { + organizationId: string + chat: MothershipChatMetadata +} + +function DeletedChatRow({ organizationId, chat }: DeletedChatRowProps) { + const { mutate: restoreChat, isPending } = useRestoreMothershipChat({ organizationId }) + return ( + } + title={chat.name} + description={chat.deletedAt ? `Deleted ${formatDate(chat.deletedAt)}` : undefined} + trailing={ + restoreChat(chat.id, { onError: (error) => toast.error(error.message) })} + > + {isPending ? 'Restoring...' : 'Restore'} + + } + /> + ) +} + +interface OrganizationRecentlyDeletedProps { + organizationId: string +} + +export function OrganizationRecentlyDeleted({ organizationId }: OrganizationRecentlyDeletedProps) { + const [search, setSearch] = useSettingsSearch() + const { + data: chats = [], + isLoading, + error, + } = useOrganizationMothershipChats(organizationId, 'archived') + const searchTerm = search.trim().toLowerCase() + const filtered = chats + .filter((chat) => chat.name.toLowerCase().includes(searchTerm)) + .sort( + (a, b) => + (b.deletedAt?.getTime() ?? 0) - (a.deletedAt?.getTime() ?? 0) || + a.name.localeCompare(b.name) || + a.id.localeCompare(b.id) + ) + + return ( + + {error ? ( + {error.message} + ) : isLoading ? null : filtered.length === 0 ? ( + + {searchTerm && chats.length > 0 ? 'No chats match your search' : 'No deleted chats'} + + ) : ( +
+ {filtered.map((chat) => ( + + ))} +
+ )} + + ) +} diff --git a/apps/sim/app/o/[organizationId]/settings/navigation.test.ts b/apps/sim/app/o/[organizationId]/settings/navigation.test.ts index 2e7e3279110..36c23895f8c 100644 --- a/apps/sim/app/o/[organizationId]/settings/navigation.test.ts +++ b/apps/sim/app/o/[organizationId]/settings/navigation.test.ts @@ -28,7 +28,7 @@ describe('organization settings navigation', () => { it('exposes MCP setup and the read-only roster to an ordinary organization member', () => { expect( organizationSettingsNavigation(false, enterprise, available).map(({ id }) => id) - ).toEqual(['members', 'search-mcp']) + ).toEqual(['members', 'recently-deleted', 'search-mcp']) }) it('uses Sources for administration when Search is available', () => { @@ -49,7 +49,7 @@ describe('organization settings navigation', () => { { ...enterprise, hasEnterprisePlan: false }, available ).map(({ id }) => id) - ).toEqual(['billing', 'members', 'search-mcp']) + ).toEqual(['billing', 'members', 'recently-deleted', 'search-mcp']) }) it('honors individual self-hosted feature flags and hides billing when disabled', () => { @@ -64,7 +64,7 @@ describe('organization settings navigation', () => { }, available ).map(({ id }) => id) - ).toEqual(['members', 'sso', 'integrations', 'search-mcp', 'search-slack']) + ).toEqual(['members', 'recently-deleted', 'sso', 'integrations', 'search-mcp', 'search-slack']) }) it('normalizes old section names and does not expose unsupported routes', () => { @@ -88,6 +88,7 @@ describe('organization settings navigation', () => { 'organization:connected-accounts', 'organization:usage', 'organization:whitelabeling', + 'organization:recently-deleted', 'governance:audit-logs', 'governance:access-control', 'governance:sso', @@ -103,7 +104,7 @@ describe('organization settings navigation', () => { it('hosts the account General section ahead of the organization sections', () => { expect( organizationSurfaceSettingsNavigation(false, enterprise, available).map(({ id }) => id) - ).toEqual(['general', 'members', 'search-mcp']) + ).toEqual(['general', 'members', 'recently-deleted', 'search-mcp']) expect(ORGANIZATION_SETTINGS_GROUPS.map(({ key }) => key)).toEqual([ 'account', 'organization', @@ -123,6 +124,10 @@ describe('organization settings navigation', () => { section: 'billing', }) expect(resolveOrganizationSurfaceSection('skills')).toBeNull() + expect(resolveOrganizationSurfaceSection('recently-deleted')).toEqual({ + plane: 'organization', + section: 'recently-deleted', + }) }) it('hides gated sections while preserving ordinary organization navigation', () => { const sections = organizationSurfaceSettingsNavigation(true, enterprise, { diff --git a/apps/sim/app/workspace/[workspaceId]/home/components/user-input/components/drop-overlay/drop-overlay.tsx b/apps/sim/app/workspace/[workspaceId]/home/components/user-input/components/drop-overlay/drop-overlay.tsx index 03993db036e..08bd1e957d5 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/components/user-input/components/drop-overlay/drop-overlay.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/components/user-input/components/drop-overlay/drop-overlay.tsx @@ -1,6 +1,7 @@ 'use client' import { memo } from 'react' +import { ImageUp } from '@sim/emcn/icons' import { AudioIcon, CsvIcon, @@ -25,13 +26,19 @@ const DROP_OVERLAY_ICONS = [ VideoIcon, ] as const -export const DropOverlay = memo(function DropOverlay() { +interface DropOverlayProps { + imagesOnly?: boolean +} + +export const DropOverlay = memo(function DropOverlay({ imagesOnly = false }: DropOverlayProps) { return (
- Drop files + + {imagesOnly ? 'Drop images' : 'Drop files'} +
- {DROP_OVERLAY_ICONS.map((Icon, i) => ( + {(imagesOnly ? [ImageUp] : DROP_OVERLAY_ICONS).map((Icon, i) => ( ))}
diff --git a/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.mount-send.test.tsx b/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.mount-send.test.tsx index 7ec24fb8476..875dc709b53 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.mount-send.test.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.mount-send.test.tsx @@ -301,6 +301,37 @@ async function waitFor(predicate: () => boolean, budgetMs = 2000): Promise } describe('useChat remount send recovery', () => { + it('sends and recovers an image-only organization turn', async () => { + navigationMocks.usePathname.mockReturnValue('/o/org-1/home') + const { getResult, unmount } = renderUseChat({ organizationId: 'org-1' }) + const attachments = [ + { + id: 'image-a', + key: 'image-key', + filename: 'screenshot.png', + media_type: 'image/png', + size: 5, + }, + ] + await act(async () => { + void getResult().sendMessage('', attachments) + }) + await waitFor(() => state.postBodies.length === 1) + expect(state.postBodies[0]).toMatchObject({ + organizationId: 'org-1', + mode: 'assistant', + message: '', + fileAttachments: attachments, + }) + expect(state.postBodies[0]).not.toHaveProperty('workspaceId') + unmount() + await waitFor(() => window.localStorage.getItem('sim_mothership_handoff') !== null) + expect(MothershipHandoffStorage.consume({ organizationId: 'org-1' })).toMatchObject({ + message: '', + fileAttachments: attachments, + }) + }) + it('sends and recovers an organization turn without adding workspace scope', async () => { navigationMocks.usePathname.mockReturnValue('/o/org-1/home') const { getResult, unmount } = renderUseChat({ organizationId: 'org-1' }) diff --git a/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts b/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts index 6752c4cd6e3..0ac26689ac1 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts +++ b/apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts @@ -3811,7 +3811,7 @@ export function useChat( contexts?: ChatContext[], options?: StartSendMessageOptions ): Promise => { - if (!message.trim() || !scopeKey) return false + if ((!message.trim() && !fileAttachments?.length) || !scopeKey) return false const { onOptimisticSendApplied, queuedSendHandoff } = options ?? {} const pendingStop = options?.pendingStop ?? pendingStopPromiseRef.current const pendingStopStreamId = pendingStop @@ -4339,7 +4339,7 @@ export function useChat( contexts?: ChatContext[], options?: SendMessageOptions ) => { - if (!message.trim() || !scopeKey) return + if ((!message.trim() && !fileAttachments?.length) || !scopeKey) return const queueStore = useMothershipQueueStore.getState() const activeChatKey = chatKeyRef.current diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.test.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.test.tsx index 9d1a3d26d63..520c7fd8c8b 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.test.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.test.tsx @@ -16,6 +16,10 @@ vi.mock('@/lib/uploads/client/session-upload', () => ({ uploadInternalFileSession: mockUploadInternalFileSession, })) +import { + ASSISTANT_IMAGE_MAX_BYTES, + ASSISTANT_IMAGE_MAX_COUNT, +} from '@/lib/uploads/shared/assistant-images' import { MAX_WORKSPACE_FILE_SIZE } from '@/lib/uploads/shared/types' import { useFileAttachments } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments' @@ -24,13 +28,15 @@ interface HookHarness { unmount: () => void } -function renderFileAttachmentsHook(): HookHarness { +function renderFileAttachmentsHook( + owner: { workspaceId: string } | { organizationId: string } = { workspaceId: 'workspace-1' } +): HookHarness { ;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true const root: Root = createRoot(document.createElement('div')) let latest: ReturnType function Probe() { - latest = useFileAttachments({ userId: 'user-1', workspaceId: 'workspace-1' }) + latest = useFileAttachments({ userId: 'user-1', ...owner }) return null } @@ -115,4 +121,45 @@ describe('useFileAttachments admission', () => { unmount() }) + + it.each(['unsupported', 'oversized', 'too many'] as const)( + 'rejects %s organization images before allocating previews or sessions', + async (kind) => { + const { result, unmount } = renderFileAttachmentsHook({ organizationId: 'organization-1' }) + const files = + kind === 'unsupported' + ? [new File(['pdf'], 'document.pdf', { type: 'application/pdf' })] + : kind === 'oversized' + ? [sizedFile('large.png', ASSISTANT_IMAGE_MAX_BYTES + 1)] + : Array.from({ length: ASSISTANT_IMAGE_MAX_COUNT + 1 }, (_, index) => + sizedFile(`image-${index}.png`, 10) + ) + await act(async () => result().processFiles(asFileList(files))) + expect(mockToastError).toHaveBeenCalledOnce() + expect(createObjectUrl).not.toHaveBeenCalled() + expect(mockUploadInternalFileSession).not.toHaveBeenCalled() + expect(result().attachedFiles).toEqual([]) + unmount() + } + ) + + it('uses organization scope for images and removes a failed upload', async () => { + mockUploadInternalFileSession.mockRejectedValueOnce(new Error('Upload failed')) + const { result, unmount } = renderFileAttachmentsHook({ organizationId: 'organization-1' }) + const file = sizedFile('screenshot.png', 10) + await act(async () => result().processFiles(asFileList([file]))) + expect(mockUploadInternalFileSession).toHaveBeenCalledWith( + expect.objectContaining({ + purpose: 'mothership_attachment', + organizationId: 'organization-1', + file, + }) + ) + expect(mockUploadInternalFileSession.mock.calls[0][0]).not.toHaveProperty('workspaceId') + expect(result().attachedFiles).toEqual([]) + expect(mockToastError).toHaveBeenCalledWith('Couldn\'t upload "screenshot.png"', { + description: 'Upload failed', + }) + unmount() + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.ts index 13a5be86e4f..edf9c7aa24d 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/user-input/hooks/use-file-attachments.ts @@ -9,6 +9,12 @@ import { getMothershipAttachmentPreviewUrl } from '@/lib/copilot/chat/attachment import { assertMultiFileUploadAdmission } from '@/lib/uploads/client/admission' import { runWithConcurrency, WHOLE_FILE_PARALLEL_UPLOADS } from '@/lib/uploads/client/concurrency' import { uploadInternalFileSession } from '@/lib/uploads/client/session-upload' +import { + ASSISTANT_IMAGE_MAX_BYTES, + ASSISTANT_IMAGE_MAX_COUNT, + ASSISTANT_IMAGE_MAX_TOTAL_BYTES, + isAssistantImageType, +} from '@/lib/uploads/shared/assistant-images' import { MAX_WORKSPACE_FILE_SIZE } from '@/lib/uploads/shared/types' import { resolveFileType } from '@/lib/uploads/utils/file-utils' @@ -78,6 +84,7 @@ export interface MessageFileAttachment { interface UseFileAttachmentsProps { userId?: string workspaceId?: string + organizationId?: string disabled?: boolean isLoading?: boolean } @@ -90,7 +97,7 @@ interface UseFileAttachmentsProps { * @returns File attachment state and operations */ export function useFileAttachments(props: UseFileAttachmentsProps) { - const { userId, workspaceId, disabled, isLoading } = props + const { userId, workspaceId, organizationId, disabled, isLoading } = props const [attachedFiles, setAttachedFiles] = useState([]) const [dragCounter, setDragCounter] = useState(0) @@ -152,16 +159,29 @@ export function useFileAttachments(props: UseFileAttachmentsProps) { logger.error('User ID not available for file upload') return } - if (!workspaceId) { - logger.error('workspaceId required for mothership uploads') + if (!workspaceId && !organizationId) { + logger.error('Workspace or organization context required for attachments') return } if (fileList.length === 0) return try { + if ( + organizationId && + Array.from(fileList).some((file) => !isAssistantImageType(resolveFileType(file))) + ) { + toast.error('Attach PNG, JPEG, GIF, or WebP images.') + return + } assertMultiFileUploadAdmission(fileList, { existingFiles: attachedFilesRef.current, - maxFileBytes: MAX_WORKSPACE_FILE_SIZE, + maxFileBytes: organizationId ? ASSISTANT_IMAGE_MAX_BYTES : MAX_WORKSPACE_FILE_SIZE, + ...(organizationId + ? { + maxFiles: ASSISTANT_IMAGE_MAX_COUNT, + maxTotalBytes: ASSISTANT_IMAGE_MAX_TOTAL_BYTES, + } + : {}), }) } catch (error) { toast.error("Couldn't add files", { description: toError(error).message }) @@ -198,7 +218,7 @@ export function useFileAttachments(props: UseFileAttachmentsProps) { const result = await uploadInternalFileSession({ purpose: 'mothership_attachment', file, - workspaceId, + ...(organizationId ? { organizationId } : { workspaceId: workspaceId! }), signal: controller.signal, }) @@ -236,7 +256,7 @@ export function useFileAttachments(props: UseFileAttachmentsProps) { } }) }, - [userId, workspaceId, updateAttachedFiles] + [userId, workspaceId, organizationId, updateAttachedFiles] ) /** diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx index 3187bccd6fe..6963d707b56 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx @@ -394,7 +394,7 @@ export function SidebarFooter({ exactly the chip's 30px instead of a line box padded by the strut's half-leading, which would deepen the bar below the chip. Collapsed, it stretches to the rail on its own and the chip fills it. */} -
{profileMenu}
+
{profileMenu}
{helpMenu}
) diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row.tsx new file mode 100644 index 00000000000..97082de31be --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row.tsx @@ -0,0 +1,39 @@ +import type { ComponentPropsWithRef, ReactNode } from 'react' +import { chipVariants } from '@sim/emcn' + +interface SidebarRenameRowProps + extends Omit, 'children' | 'className' | 'type'> { + leadingAdornment?: ReactNode +} + +/** Keeps inline renaming on the existing sidebar row instead of introducing a form field. */ +export function SidebarRenameRow({ + leadingAdornment, + onKeyDown, + onClick, + ...props +}: SidebarRenameRowProps) { + return ( +
+ {leadingAdornment} + { + event.stopPropagation() + onKeyDown?.(event) + }} + onClick={(event) => { + event.stopPropagation() + onClick?.(event) + }} + /> +
+ ) +} diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/workspace-header.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/workspace-header.tsx index f6d35471dfb..c4bf57a223a 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/workspace-header.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/workspace-header.tsx @@ -29,11 +29,13 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { useQueryClient } from '@tanstack/react-query' import { IdentityTile } from '@/components/identity-tile/identity-tile' +import { WorkspaceContextMenu } from '@/components/workspaces/workspace-context-menu' import { useDeploymentShape } from '@/lib/core/config/deployment-shape' +import { WORKSPACE_SEARCH_THRESHOLD } from '@/lib/workspaces/constants' import { getWorkspaceInitial } from '@/lib/workspaces/initials' import { InviteModal } from '@/app/workspace/[workspaceId]/components/invite-modal' import { useWorkspacePermissionsContext } from '@/app/workspace/[workspaceId]/providers/workspace-permissions-provider' -import { ContextMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workflow-list/components/context-menu/context-menu' +import { SidebarRenameRow } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-rename-row' import { DeleteModal } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workflow-list/components/delete-modal/delete-modal' import { CreateWorkspaceModal } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/components/create-workspace-modal/create-workspace-modal' import { ViewInvitationsMenuItem } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workspace-header/components/pending-invitations/view-invitations-menu-item' @@ -50,20 +52,6 @@ import { useSettingsNavigation } from '@/hooks/use-settings-navigation' const logger = createLogger('WorkspaceHeader') -/** - * Show the search input once the workspace list reaches this count, and size the - * list viewport to exactly this many rows — so the sixth workspace is the one that - * both fills the viewport and brings in search. - * - * The viewport's `max-h-[200px]` is derived from it: 6 rows at `chipGeometryClass`'s - * 30px plus the 2px `gap-0.5` between them (6 * 30 + 5 * 2), plus the list's own - * `pt-1.5 pb-1` (6 + 4) — the gaps to the search field and the rule, carried as - * the scroll box's padding so rows scroll through them under the edge fade. - * Tailwind arbitrary values must be statically analyzable, so the arithmetic - * cannot live in the class — change them together. - */ -const WORKSPACE_SEARCH_THRESHOLD = 6 - interface DisabledReasonTooltipProps { reason: string | null children: ReactElement @@ -628,64 +616,54 @@ function WorkspaceHeaderImpl({ } > {editingWorkspaceId === workspace.id ? ( -
- - { - renameInputRef.current = el - if (el && !hasInputFocusedRef.current) { - hasInputFocusedRef.current = true - el.focus() - el.select() - } - }} - value={editingName} - onChange={(e) => setEditingName(e.target.value)} - onKeyDown={async (e) => { - e.stopPropagation() - if (e.key === 'Enter') { - e.preventDefault() - setIsListRenaming(true) - try { - await onRenameWorkspace(workspace.id, editingName.trim()) - setEditingWorkspaceId(null) - } finally { - setIsListRenaming(false) - } - } else if (e.key === 'Escape') { - e.preventDefault() + + } + ref={(el) => { + renameInputRef.current = el + if (el && !hasInputFocusedRef.current) { + hasInputFocusedRef.current = true + el.focus() + el.select() + } + }} + value={editingName} + onChange={(e) => setEditingName(e.target.value)} + onKeyDown={async (e) => { + if (e.key === 'Enter') { + e.preventDefault() + setIsListRenaming(true) + try { + await onRenameWorkspace(workspace.id, editingName.trim()) setEditingWorkspaceId(null) + } finally { + setIsListRenaming(false) } - }} - onBlur={async () => { - if (!editingWorkspaceId) return - const trimmedName = editingName.trim() - if (trimmedName && trimmedName !== workspace.name) { - setIsListRenaming(true) - try { - await onRenameWorkspace(workspace.id, trimmedName) - } finally { - setIsListRenaming(false) - } - } + } else if (e.key === 'Escape') { + e.preventDefault() setEditingWorkspaceId(null) - }} - className='w-full min-w-0 border-0 bg-transparent p-0 text-[var(--text-body)] text-sm outline-hidden focus:outline-hidden focus:ring-0 focus-visible:outline-hidden focus-visible:ring-0 focus-visible:ring-offset-0' - maxLength={100} - autoComplete='off' - autoCorrect='off' - autoCapitalize='off' - spellCheck='false' - disabled={isListRenaming} - onClick={(e) => { - e.stopPropagation() - }} - /> -
+ } + }} + onBlur={async () => { + if (!editingWorkspaceId) return + const trimmedName = editingName.trim() + if (trimmedName && trimmedName !== workspace.name) { + setIsListRenaming(true) + try { + await onRenameWorkspace(workspace.id, trimmedName) + } finally { + setIsListRenaming(false) + } + } + setEditingWorkspaceId(null) + }} + disabled={isListRenaming} + /> ) : (
)} - {(() => { - const capturedPermissions = capturedWorkspaceRef.current?.permissions - const contextCanAdmin = capturedPermissions === 'admin' - const capturedWorkspace = workspaces.find((w) => w.id === capturedWorkspaceRef.current?.id) - const isOwner = capturedWorkspace && sessionUserId === capturedWorkspace.ownerId - /** - * An organization admin holds this workspace through their org role, not - * a permission row, so there is nothing to give up and the removal - * endpoint refuses it. `permissions === 'admin'` cannot tell them apart - * from an explicit workspace admin, who may leave. This menu has no - * tooltip affordance to explain a greyed row, so the entry is withheld - * rather than shown dead. - */ - const canLeave = !isOwner && !capturedWorkspace?.isOrgAdmin && !!onLeaveWorkspace - - return ( - - ) - })()} + workspace.id === menuOpenWorkspaceId)} + workspaceCount={workspaces.length} + sessionUserId={sessionUserId} + isOpen={isContextMenuOpen} + position={contextMenuPosition} + menuRef={contextMenuRef} + onClose={closeContextMenu} + onRename={handleRenameAction} + renameInputRef={renameInputRef} + onDelete={handleDeleteAction} + onLeave={handleLeaveAction} + onTogglePin={handleTogglePinAction} + onUploadLogo={handleUploadLogoAction} + isPinned={Boolean(menuOpenWorkspaceId && pinnedWorkspaceIds.has(menuOpenWorkspaceId))} + /> { + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) +}) +afterEach(() => { + act(() => root.unmount()) + container.remove() + vi.unstubAllGlobals() +}) + +function renderRenameHook(onSave: (id: string, name: string) => Promise) { + const result = {} as { current: ReturnType } + function Harness() { + result.current = useFlyoutInlineRename({ itemType: 'workspace', onSave }) + return null + } + act(() => root.render()) + return { result } +} + +function deferred() { + let resolve!: () => void + let reject!: (reason: Error) => void + const promise = new Promise((done, fail) => { + resolve = done + reject = fail + }) + return { promise, resolve, reject } +} + +describe('useFlyoutInlineRename', () => { + it.each(['success', 'failure'] as const)( + 'keeps a newer rename intact when an older save finishes with %s', + async (outcome) => { + const oldSave = deferred() + const newSave = deferred() + const onSave = vi + .fn() + .mockReturnValueOnce(oldSave.promise) + .mockReturnValueOnce(newSave.promise) + const { result } = renderRenameHook(onSave) + act(() => result.current.startRename({ id: 'old', name: 'Old name' })) + act(() => result.current.setValue('Old renamed')) + let firstSave!: Promise + act(() => { + firstSave = result.current.saveRename() + }) + act(() => result.current.startRename({ id: 'new', name: 'New name' })) + act(() => result.current.setValue('New renamed')) + let secondSave!: Promise + act(() => { + secondSave = result.current.saveRename() + }) + await act(async () => { + if (outcome === 'success') oldSave.resolve() + else oldSave.reject(new Error('Save failed')) + await firstSave + }) + expect(result.current.editingId).toBe('new') + expect(result.current.value).toBe('New renamed') + expect(result.current.isSaving).toBe(true) + await act(async () => { + newSave.resolve() + await secondSave + }) + expect(result.current.editingId).toBeNull() + expect(result.current.isSaving).toBe(false) + } + ) + + it('allows retry after failure and prevents Enter plus blur from saving twice', async () => { + const pending = deferred() + const onSave = vi.fn().mockReturnValueOnce(pending.promise).mockResolvedValue(undefined) + const { result } = renderRenameHook(onSave) + act(() => result.current.startRename({ id: 'workspace', name: 'Original' })) + act(() => result.current.setValue('Renamed')) + let save!: Promise + act(() => { + save = result.current.saveRename() + void result.current.saveRename() + }) + expect(onSave).toHaveBeenCalledTimes(1) + await act(async () => { + pending.reject(new Error('Save failed')) + await save + }) + expect(result.current.editingId).toBe('workspace') + expect(result.current.value).toBe('Original') + expect(result.current.isSaving).toBe(false) + act(() => result.current.setValue('Retry')) + await act(async () => { + await result.current.saveRename() + }) + expect(onSave).toHaveBeenLastCalledWith('workspace', 'Retry') + expect(result.current.editingId).toBeNull() + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename.ts index a492918d789..f96613b173d 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-flyout-inline-rename.ts @@ -20,6 +20,7 @@ export function useFlyoutInlineRename({ itemType, onSave }: UseFlyoutInlineRenam const inputRef = useRef(null) const cancelRequestedRef = useRef(false) const isSavingRef = useRef(false) + const activeTargetRef = useRef(null) useEffect(() => { if (editingTarget && inputRef.current) { @@ -29,13 +30,20 @@ export function useFlyoutInlineRename({ itemType, onSave }: UseFlyoutInlineRenam }, [editingTarget]) const startRename = useCallback((target: RenameTarget) => { + const nextTarget = { ...target } + activeTargetRef.current = nextTarget cancelRequestedRef.current = false - setEditingTarget(target) + isSavingRef.current = false + setIsSaving(false) + setEditingTarget(nextTarget) setValue(target.name) }, []) const cancelRename = useCallback(() => { + activeTargetRef.current = null cancelRequestedRef.current = true + isSavingRef.current = false + setIsSaving(false) setEditingTarget(null) }, []) @@ -45,21 +53,24 @@ export function useFlyoutInlineRename({ itemType, onSave }: UseFlyoutInlineRenam return } - if (!editingTarget || isSavingRef.current) { + if (!editingTarget || activeTargetRef.current !== editingTarget || isSavingRef.current) { return } const trimmedValue = value.trim() if (!trimmedValue || trimmedValue === editingTarget.name) { + activeTargetRef.current = null setEditingTarget(null) return } isSavingRef.current = true setIsSaving(true) + let saved = false try { await onSave(editingTarget.id, trimmedValue) - setEditingTarget(null) + saved = true + if (activeTargetRef.current === editingTarget) setEditingTarget(null) } catch (error) { logger.error(`Failed to rename ${itemType}:`, { error, @@ -67,10 +78,14 @@ export function useFlyoutInlineRename({ itemType, onSave }: UseFlyoutInlineRenam oldName: editingTarget.name, newName: trimmedValue, }) - setValue(editingTarget.name) + if (activeTargetRef.current === editingTarget) setValue(editingTarget.name) } finally { - isSavingRef.current = false - setIsSaving(false) + /** A late save must not clear a newer row's rename session or pending state. */ + if (activeTargetRef.current === editingTarget) { + isSavingRef.current = false + setIsSaving(false) + if (saved) activeTargetRef.current = null + } } }, [editingTarget, itemType, onSave, value]) diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-logo-upload.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-logo-upload.ts index 2cc1e3c0ead..af435d16d10 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-logo-upload.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-logo-upload.ts @@ -1,8 +1,8 @@ import { useCallback, useEffect, useRef, useState } from 'react' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import { validateLogoFile } from '@/lib/uploads/client/logo-file' import { uploadInternalFileSession } from '@/lib/uploads/client/session-upload' -import { validateWorkspaceLogoFile } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file' const logger = createLogger('WorkspaceLogoUpload') @@ -67,7 +67,7 @@ export function useWorkspaceLogoUpload({ const processFile = useCallback( async (file: File) => { - const validationError = validateWorkspaceLogoFile(file) + const validationError = validateLogoFile(file) if (validationError) { onErrorRef.current?.(validationError) return diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-management.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-management.ts index 8a821c0243a..2b60f17bb2c 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-management.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-workspace-management.ts @@ -1,4 +1,4 @@ -import { useCallback, useEffect, useMemo, useRef, useState } from 'react' +import { useCallback, useEffect, useMemo, useRef } from 'react' import { createLogger } from '@sim/logger' import { usePathname, useRouter } from 'next/navigation' import { requestJson } from '@/lib/api/client/request' @@ -16,6 +16,7 @@ import { useWorkspacesQuery, type Workspace, } from '@/hooks/queries/workspace' +import { useWorkspaceOrder } from '@/hooks/use-workspace-order' import { useWorkflowRegistry } from '@/stores/workflows/registry/store' const logger = createLogger('useWorkspaceManagement') @@ -96,8 +97,6 @@ export function useWorkspaceManagement({ workspacesRef.current = workspaces routerRef.current = router - const [recencySortKey, setRecencySortKey] = useState(0) - useEffect(() => { return () => { if (syncTimerRef.current) clearTimeout(syncTimerRef.current) @@ -112,7 +111,6 @@ export function useWorkspaceManagement({ if (validIds.length > 0) { WorkspaceRecencyStorage.prune(new Set(validIds)) } - setRecencySortKey((k) => k + 1) if (syncTimerRef.current) clearTimeout(syncTimerRef.current) syncTimerRef.current = setTimeout(() => { @@ -122,23 +120,7 @@ export function useWorkspaceManagement({ }, 1000) }, []) - /** - * Pinned workspaces float to the top, recency ordering them within each group. - * Matches `resource-sort.ts`: pinning is a user-declared priority layered over - * the list's own sort, not a competing sort key. - */ - const sortedWorkspaces = useMemo(() => { - const byRecency = WorkspaceRecencyStorage.sortByRecency(workspaces) - if (pinnedWorkspaceIds.size === 0) return byRecency - const pinned: Workspace[] = [] - const unpinned: Workspace[] = [] - for (const workspace of byRecency) { - if (pinnedWorkspaceIds.has(workspace.id)) pinned.push(workspace) - else unpinned.push(workspace) - } - return [...pinned, ...unpinned] - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [workspaces, recencySortKey, pinnedWorkspaceIds]) + const sortedWorkspaces = useWorkspaceOrder(workspaces, pinnedWorkspaceIds) const toggleWorkspacePin = useCallback( (workspaceId: string) => { diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.ts deleted file mode 100644 index 30169ed911c..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.ts +++ /dev/null @@ -1,26 +0,0 @@ -const MAX_WORKSPACE_LOGO_SIZE = 5 * 1024 * 1024 - -const WORKSPACE_LOGO_IMAGE_TYPES = [ - 'image/png', - 'image/jpeg', - 'image/jpg', - 'image/gif', - 'image/svg+xml', - 'image/webp', -] as const - -const WORKSPACE_LOGO_IMAGE_TYPE_SET = new Set(WORKSPACE_LOGO_IMAGE_TYPES) - -export const WORKSPACE_LOGO_ACCEPT_ATTRIBUTE = WORKSPACE_LOGO_IMAGE_TYPES.join(',') - -export function validateWorkspaceLogoFile( - file: Pick -): string | null { - if (file.size > MAX_WORKSPACE_LOGO_SIZE) { - return `File "${file.name}" is too large. Maximum size is 5MB.` - } - if (!WORKSPACE_LOGO_IMAGE_TYPE_SET.has(file.type)) { - return `File "${file.name}" is not a supported image format. Please use PNG, JPEG, GIF, SVG, or WebP.` - } - return null -} diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx index 36b376cba74..878f915a1b1 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx @@ -47,6 +47,7 @@ import { isMacPlatform } from '@/lib/core/utils/platform' import { buildFolderTree, getFolderPathNames } from '@/lib/folders/tree' import { DOCS_URL, SLACK_COMMUNITY_URL } from '@/lib/help-links' import { captureEvent } from '@/lib/posthog/client' +import { LOGO_ACCEPT_ATTRIBUTE } from '@/lib/uploads/client/logo-file' import { useSidebarChrome } from '@/app/workspace/[workspaceId]/components/workspace-chrome' import { CONNECT_MODE } from '@/app/workspace/[workspaceId]/integrations/connect-route' import { useRegisterGlobalCommands } from '@/app/workspace/[workspaceId]/providers/global-commands-provider' @@ -102,7 +103,6 @@ import { useWorkspaceLogoUpload, useWorkspaceManagement, useWorkspaceWorkflowsRoom, - WORKSPACE_LOGO_ACCEPT_ATTRIBUTE, } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks' import { compareByOrder, @@ -830,7 +830,7 @@ export const Sidebar = memo(function Sidebar() { { enabled: chatEnabled } ) - useMothershipChatEvents(workspaceId) + useMothershipChatEvents(workspaceId, chatEnabled) /** * Stays empty when Chat is disabled, which also drops the command palette's @@ -1275,7 +1275,7 @@ export const Sidebar = memo(function Sidebar() { diff --git a/apps/sim/blocks/blocks/binary-download-versioning.test.ts b/apps/sim/blocks/blocks/binary-download-versioning.test.ts new file mode 100644 index 00000000000..88a5fe12b7f --- /dev/null +++ b/apps/sim/blocks/blocks/binary-download-versioning.test.ts @@ -0,0 +1,100 @@ +import { describe, expect, it } from 'vitest' +import { BoxBlock, BoxV2Block } from '@/blocks/blocks/box' +import { DropboxBlock, DropboxV2Block } from '@/blocks/blocks/dropbox' +import { DubBlock, DubV2Block } from '@/blocks/blocks/dub' +import { + MicrosoftDataverseBlock, + MicrosoftDataverseV2Block, +} from '@/blocks/blocks/microsoft_dataverse' +import { ServiceNowBlock, ServiceNowV2Block } from '@/blocks/blocks/servicenow' +import type { BlockConfig } from '@/blocks/types' + +const cases: { + legacy: BlockConfig + current: BlockConfig + operation: string + toolId: string + content: string +}[] = [ + { + legacy: BoxBlock, + current: BoxV2Block, + operation: 'download_file', + toolId: 'box_download_file', + content: 'content', + }, + { + legacy: DropboxBlock, + current: DropboxV2Block, + operation: 'dropbox_download', + toolId: 'dropbox_download', + content: 'content', + }, + { + legacy: DubBlock, + current: DubV2Block, + operation: 'get_qr_code', + toolId: 'dub_get_qr_code', + content: 'content', + }, + { + legacy: MicrosoftDataverseBlock, + current: MicrosoftDataverseV2Block, + operation: 'download_file', + toolId: 'microsoft_dataverse_download_file', + content: 'fileContent', + }, + { + legacy: ServiceNowBlock, + current: ServiceNowV2Block, + operation: 'servicenow_download_attachment', + toolId: 'servicenow_download_attachment', + content: 'content', + }, +] + +describe.each(cases)( + '$current.type download versioning', + ({ legacy, current, operation, toolId, content }) => { + it('preserves saved blocks and changes only the download tool for new blocks', () => { + expect(legacy.hideFromToolbar).toBe(true) + expect(legacy.sunset).toEqual({ status: 'legacy', replacedBy: current.type }) + expect(current.hideFromToolbar).toBe(false) + expect(current.sunset).toBeUndefined() + expect(current.subBlocks).toBe(legacy.subBlocks) + expect(current.tools.config?.params).toBe(legacy.tools.config?.params) + expect(legacy.tools.config?.tool({ operation })).toBe(toolId) + expect(current.tools.config?.tool({ operation })).toBe(`${toolId}_v2`) + expect(current.tools.access).toEqual( + legacy.tools.access.map((id) => (id === toolId ? `${id}_v2` : id)) + ) + }) + + it('removes the download content output while preserving unrelated outputs', () => { + expect(legacy.outputs).toHaveProperty(content) + if (current.type === 'servicenow_v2') { + expect(current.outputs.content).toEqual({ + type: 'string', + description: 'HTML body of a knowledge article', + }) + } else { + expect(current.outputs).not.toHaveProperty(content) + } + expect(current.outputs.file).toEqual(legacy.outputs.file) + }) + } +) + +it('keeps Dataverse upload metadata separate from canonical download files', () => { + expect(MicrosoftDataverseV2Block.outputs).not.toHaveProperty('fileSize') + expect(MicrosoftDataverseV2Block.outputs).not.toHaveProperty('mimeType') + expect(MicrosoftDataverseV2Block.outputs.fileName.condition).toEqual({ + field: 'operation', + value: 'upload_file', + }) + expect(MicrosoftDataverseV2Block.outputs.success.condition).toEqual({ + field: 'operation', + value: 'download_file', + not: true, + }) +}) diff --git a/apps/sim/blocks/blocks/box.ts b/apps/sim/blocks/blocks/box.ts index 6d3b03f157e..3b81fc5a25b 100644 --- a/apps/sim/blocks/blocks/box.ts +++ b/apps/sim/blocks/blocks/box.ts @@ -1,3 +1,4 @@ +import { omit } from '@sim/utils/object' import { BoxCompanyIcon } from '@/components/icons' import { getScopesForService } from '@/lib/oauth/utils' import type { BlockConfig, BlockMeta } from '@/blocks/types' @@ -7,9 +8,11 @@ import { normalizeFileInput } from '@/blocks/utils' /** Canonical pair for the upload payload: file picker in basic mode, file reference in advanced. */ const UPLOAD_FILE_FIELD = ['uploadFile', 'fileRef'] as const -export const BoxBlock: BlockConfig = { +export const BoxBlock = { type: 'box', - name: 'Box', + name: 'Box (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'box_v2' }, description: 'Manage files, folders, and e-signatures with Box', longDescription: 'Integrate Box into your workflow to manage files, folders, and e-signatures. Upload and download files, search content, create folders, send documents for e-signature, track signing status, and more.', @@ -657,6 +660,28 @@ export const BoxBlock: BlockConfig = { count: 'number', nextMarker: 'string', }, +} satisfies BlockConfig + +export const BoxV2Block: BlockConfig = { + ...BoxBlock, + type: 'box_v2', + name: 'Box', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...BoxBlock.tools, + access: BoxBlock.tools.access.map((toolId) => + toolId === 'box_download_file' ? 'box_download_file_v2' : toolId + ), + config: { + ...BoxBlock.tools.config, + tool: (params) => { + const toolId = BoxBlock.tools.config.tool(params) + return toolId === 'box_download_file' ? 'box_download_file_v2' : toolId + }, + }, + }, + outputs: omit(BoxBlock.outputs, ['content']), } export const BoxBlockMeta = { diff --git a/apps/sim/blocks/blocks/dropbox.ts b/apps/sim/blocks/blocks/dropbox.ts index 3047df56821..572c47047d0 100644 --- a/apps/sim/blocks/blocks/dropbox.ts +++ b/apps/sim/blocks/blocks/dropbox.ts @@ -1,3 +1,4 @@ +import { omit } from '@sim/utils/object' import { DropboxIcon } from '@/components/icons' import { getScopesForService } from '@/lib/oauth/utils' import type { BlockConfig, BlockMeta } from '@/blocks/types' @@ -12,9 +13,11 @@ import type { DropboxResponse } from '@/tools/dropbox/types' */ const UPLOAD_FILE_FIELD = ['uploadFile', 'fileRef'] as const -export const DropboxBlock: BlockConfig = { +export const DropboxBlock = { type: 'dropbox', - name: 'Dropbox', + name: 'Dropbox (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'dropbox_v2' }, description: 'Upload, download, share, and manage files in Dropbox', authMode: AuthMode.OAuth, longDescription: @@ -544,6 +547,28 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`, // List revisions output isDeleted: { type: 'boolean', description: 'Whether the latest revision is deleted or moved' }, }, +} satisfies BlockConfig + +export const DropboxV2Block: BlockConfig = { + ...DropboxBlock, + type: 'dropbox_v2', + name: 'Dropbox', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...DropboxBlock.tools, + access: DropboxBlock.tools.access.map((toolId) => + toolId === 'dropbox_download' ? 'dropbox_download_v2' : toolId + ), + config: { + ...DropboxBlock.tools.config, + tool: (params) => { + const toolId = DropboxBlock.tools.config.tool(params) + return toolId === 'dropbox_download' ? 'dropbox_download_v2' : toolId + }, + }, + }, + outputs: omit(DropboxBlock.outputs, ['content']), } export const DropboxBlockMeta = { diff --git a/apps/sim/blocks/blocks/dub.ts b/apps/sim/blocks/blocks/dub.ts index 1b2e213be6a..3361f171254 100644 --- a/apps/sim/blocks/blocks/dub.ts +++ b/apps/sim/blocks/blocks/dub.ts @@ -1,3 +1,4 @@ +import { omit } from '@sim/utils/object' import { DubIcon } from '@/components/icons' import type { BlockConfig, BlockMeta } from '@/blocks/types' import { AuthMode, IntegrationType } from '@/blocks/types' @@ -8,9 +9,11 @@ const BULK_UPDATE_TARGET_FIELD = ['bulkUpdateLinkIds', 'bulkUpdateExternalIds'] const ANALYTICS_LINK_FIELD = ['analyticsLinkId', 'analyticsExternalId'] as const const EVENTS_LINK_FIELD = ['eventsLinkId', 'eventsExternalId'] as const -export const DubBlock: BlockConfig = { +export const DubBlock = { type: 'dub', - name: 'Dub', + name: 'Dub (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'dub_v2' }, description: 'Link management with Dub', authMode: AuthMode.ApiKey, longDescription: @@ -1277,6 +1280,28 @@ export const DubBlock: BlockConfig = { condition: { field: 'operation', value: 'create_tag' }, }, }, +} satisfies BlockConfig + +export const DubV2Block: BlockConfig = { + ...DubBlock, + type: 'dub_v2', + name: 'Dub', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...DubBlock.tools, + access: DubBlock.tools.access.map((toolId) => + toolId === 'dub_get_qr_code' ? 'dub_get_qr_code_v2' : toolId + ), + config: { + ...DubBlock.tools.config, + tool: (params) => { + const toolId = DubBlock.tools.config.tool(params) + return toolId === 'dub_get_qr_code' ? 'dub_get_qr_code_v2' : toolId + }, + }, + }, + outputs: omit(DubBlock.outputs, ['content']), } export const DubBlockMeta = { diff --git a/apps/sim/blocks/blocks/file.ts b/apps/sim/blocks/blocks/file.ts index cff055af97e..947d09c1477 100644 --- a/apps/sim/blocks/blocks/file.ts +++ b/apps/sim/blocks/blocks/file.ts @@ -1960,20 +1960,16 @@ export const FileV5Block: BlockConfig = { if (operation === 'file_write') { // Writing stores one file, so the single form. const fileInput = normalizeFileInput(params.writeFileInput, { single: true }) - // The contract counts any defined `content` as "text was provided", and - // an untouched Content box serializes as an empty string — so sending it - // unconditionally would make every file write collide with its own empty - // text box. The selected file is what disambiguates: with one present, - // an empty Content box means "not used" and is dropped, while a - // non-empty one is still forwarded so the contract can report that both - // were filled. With no file, `content` always goes through, which keeps - // writing a deliberately empty text file possible. - const contentText = typeof params.content === 'string' ? params.content : undefined - const omitContent = Boolean(fileInput) && !contentText + /** + * Explicitly clear unused Content because the executor merges these params + * over the original inputs. Preserve empty text when no file is selected. + */ + const omitContent = + Boolean(fileInput) && (params.content == null || params.content === '') return { fileName: params.fileName, folderPath: optionalText(params.writeFolderRef), - ...(omitContent ? {} : { content: params.content }), + content: omitContent ? undefined : params.content, ...(fileInput ? { fileInput } : {}), contentType: params.contentType, overwrite: params.overwrite === true || params.overwrite === 'true', diff --git a/apps/sim/blocks/blocks/jupyter.test.ts b/apps/sim/blocks/blocks/jupyter.test.ts new file mode 100644 index 00000000000..5c5c284b7e9 --- /dev/null +++ b/apps/sim/blocks/blocks/jupyter.test.ts @@ -0,0 +1,35 @@ +import { describe, expect, it } from 'vitest' +import { JupyterBlock, JupyterV2Block } from '@/blocks/blocks/jupyter' + +describe('Jupyter block versions', () => { + it('offers v2 for new blocks and preserves the legacy block', () => { + expect(JupyterBlock.hideFromToolbar).toBe(true) + expect(JupyterBlock.sunset).toEqual({ status: 'legacy', replacedBy: 'jupyter_v2' }) + expect(JupyterV2Block.hideFromToolbar).toBe(false) + expect(JupyterV2Block.sunset).toBeUndefined() + expect(JupyterV2Block.canvasPresentation).toBe(JupyterBlock.canvasPresentation) + expect(JupyterV2Block.subBlocks).toBe(JupyterBlock.subBlocks) + }) + + it.each(JupyterBlock.tools.access)('versions only Get Content: %s', (operation) => { + const id = operation === 'jupyter_get_content' ? 'jupyter_get_content_v2' : operation + expect(JupyterBlock.tools.config.tool({ operation })).toBe(operation) + expect(JupyterV2Block.tools.config?.tool({ operation })).toBe(id) + expect(JupyterV2Block.tools.access).toContain(id) + }) + + it('preserves the path and credentials for the versioned read', () => { + expect( + JupyterV2Block.tools.config?.params?.({ + operation: 'jupyter_get_content', + serverUrl: 'https://jupyter.example.com', + token: 'token', + path: 'reports/book.xlsx', + }) + ).toEqual({ + serverUrl: 'https://jupyter.example.com', + token: 'token', + path: 'reports/book.xlsx', + }) + }) +}) diff --git a/apps/sim/blocks/blocks/jupyter.ts b/apps/sim/blocks/blocks/jupyter.ts index e251c738abc..55d6c0dd03c 100644 --- a/apps/sim/blocks/blocks/jupyter.ts +++ b/apps/sim/blocks/blocks/jupyter.ts @@ -1,7 +1,7 @@ import { JupyterIcon } from '@/components/icons' import type { BlockConfig, BlockMeta } from '@/blocks/types' import { AuthMode, IntegrationType } from '@/blocks/types' -import { normalizeFileInput } from '@/blocks/utils' +import { createVersionedToolSelector, normalizeFileInput } from '@/blocks/utils' const PATH_OPERATIONS = [ 'jupyter_list_contents', @@ -31,9 +31,11 @@ const KERNEL_ID_OPERATIONS = [ /** Both members of the `file` canonical group — advanced mode fills only `fileRef`. */ const UPLOAD_FILE_FIELD = ['uploadFile', 'fileRef'] as const -export const JupyterBlock: BlockConfig = { +export const JupyterBlock = { type: 'jupyter', - name: 'Jupyter', + name: 'Jupyter (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'jupyter_v2' }, description: 'Manage files, notebooks, kernels, and sessions on a Jupyter server', longDescription: 'Integrate a self-hosted Jupyter server into the workflow. Browse, read, create, upload, rename, copy, and delete files and notebooks; start, stop, restart, and interrupt kernels; and manage sessions that bind notebooks to kernels.', @@ -392,6 +394,33 @@ export const JupyterBlock: BlockConfig = { kernelId: 'string', sessionId: 'string', }, +} satisfies BlockConfig + +const selectJupyterV2Tool = createVersionedToolSelector({ + baseToolSelector: JupyterBlock.tools.config.tool, + suffix: '_v2', + fallbackToolId: 'jupyter_get_content_v2', +}) + +export const JupyterV2Block: BlockConfig = { + ...JupyterBlock, + type: 'jupyter_v2', + name: 'Jupyter', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...JupyterBlock.tools, + access: JupyterBlock.tools.access.map((toolId) => + toolId === 'jupyter_get_content' ? 'jupyter_get_content_v2' : toolId + ), + config: { + ...JupyterBlock.tools.config, + tool: (params) => + params.operation === 'jupyter_get_content' + ? selectJupyterV2Tool(params) + : JupyterBlock.tools.config.tool(params), + }, + }, } export const JupyterBlockMeta = { diff --git a/apps/sim/blocks/blocks/microsoft_dataverse.ts b/apps/sim/blocks/blocks/microsoft_dataverse.ts index 2c51cfe7ad2..748859694a0 100644 --- a/apps/sim/blocks/blocks/microsoft_dataverse.ts +++ b/apps/sim/blocks/blocks/microsoft_dataverse.ts @@ -1,3 +1,4 @@ +import { omit } from '@sim/utils/object' import { MicrosoftDataverseIcon } from '@/components/icons' import { getScopesForService } from '@/lib/oauth/utils' import type { BlockConfig, BlockMeta } from '@/blocks/types' @@ -8,9 +9,11 @@ import type { DataverseResponse } from '@/tools/microsoft_dataverse/types' /** Canonical upload pair for the file column payload, basic then advanced. */ const FILE_FIELD = ['uploadFile', 'fileReference'] as const -export const MicrosoftDataverseBlock: BlockConfig = { +export const MicrosoftDataverseBlock = { type: 'microsoft_dataverse', - name: 'Microsoft Dataverse', + name: 'Microsoft Dataverse (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'microsoft_dataverse_v2' }, description: 'Manage records in Microsoft Dataverse tables', authMode: AuthMode.OAuth, longDescription: @@ -788,6 +791,43 @@ Return ONLY the expand expression - no $expand= prefix, no explanations.`, description: 'Full raw table metadata response (get table metadata)', }, }, +} satisfies BlockConfig + +export const MicrosoftDataverseV2Block: BlockConfig = { + ...MicrosoftDataverseBlock, + type: 'microsoft_dataverse_v2', + name: 'Microsoft Dataverse', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...MicrosoftDataverseBlock.tools, + access: MicrosoftDataverseBlock.tools.access.map((toolId) => + toolId === 'microsoft_dataverse_download_file' + ? 'microsoft_dataverse_download_file_v2' + : toolId + ), + config: { + ...MicrosoftDataverseBlock.tools.config, + tool: (params) => { + const toolId = MicrosoftDataverseBlock.tools.config.tool(params) + return toolId === 'microsoft_dataverse_download_file' + ? 'microsoft_dataverse_download_file_v2' + : toolId + }, + }, + }, + outputs: { + ...omit(MicrosoftDataverseBlock.outputs, ['fileContent', 'fileSize', 'mimeType']), + fileName: { + type: 'string', + description: 'Name of the uploaded file', + condition: { field: 'operation', value: 'upload_file' }, + }, + success: { + ...MicrosoftDataverseBlock.outputs.success, + condition: { field: 'operation', value: 'download_file', not: true }, + }, + }, } export const MicrosoftDataverseBlockMeta = { diff --git a/apps/sim/blocks/blocks/quiver.test.ts b/apps/sim/blocks/blocks/quiver.test.ts new file mode 100644 index 00000000000..c1953d8dfc6 --- /dev/null +++ b/apps/sim/blocks/blocks/quiver.test.ts @@ -0,0 +1,34 @@ +import { describe, expect, it } from 'vitest' +import { QuiverBlock, QuiverV2Block } from '@/blocks/blocks/quiver' + +describe('Quiver block versions', () => { + it('keeps the legacy block executable and offers v2 for new blocks', () => { + expect(QuiverBlock.hideFromToolbar).toBe(true) + expect(QuiverBlock.sunset).toEqual({ status: 'legacy', replacedBy: 'quiver_v2' }) + expect(QuiverV2Block.name).toBe('Quiver') + expect(QuiverV2Block.hideFromToolbar).toBe(false) + expect(QuiverV2Block.sunset).toBeUndefined() + expect(QuiverV2Block.subBlocks).toBe(QuiverBlock.subBlocks) + expect(QuiverV2Block.tools.config?.params).toBe(QuiverBlock.tools.config?.params) + }) + + it.each([ + ['text_to_svg', 'quiver_text_to_svg', 'quiver_text_to_svg_v2'], + ['image_to_svg', 'quiver_image_to_svg', 'quiver_image_to_svg_v2'], + ['list_models', 'quiver_list_models', 'quiver_list_models'], + ])('routes %s to the versioned response contract', (operation, legacyId, currentId) => { + expect(QuiverBlock.tools.config?.tool({ operation })).toBe(legacyId) + expect(QuiverV2Block.tools.config?.tool({ operation })).toBe(currentId) + expect(QuiverBlock.tools.access).toContain(legacyId) + expect(QuiverV2Block.tools.access).toContain(currentId) + }) + + it('defaults to SVG generation and exposes all generated files in v2', () => { + expect(QuiverV2Block.tools.config?.tool({})).toBe('quiver_text_to_svg_v2') + expect(QuiverV2Block.outputs.files).toMatchObject({ type: 'file[]' }) + expect(QuiverBlock.outputs.files).toMatchObject({ type: 'json' }) + expect(QuiverV2Block.outputs).not.toHaveProperty('svgContent') + expect(QuiverV2Block.outputs).not.toHaveProperty('file') + expect(QuiverBlock.outputs.svgContent).toMatchObject({ type: 'string' }) + }) +}) diff --git a/apps/sim/blocks/blocks/quiver.ts b/apps/sim/blocks/blocks/quiver.ts index 879c9fa90df..c87e691642a 100644 --- a/apps/sim/blocks/blocks/quiver.ts +++ b/apps/sim/blocks/blocks/quiver.ts @@ -1,15 +1,18 @@ +import { omit } from '@sim/utils/object' import { QuiverIcon } from '@/components/icons' import type { BlockConfig, BlockMeta } from '@/blocks/types' import { AuthMode, IntegrationType } from '@/blocks/types' -import { normalizeFileInput } from '@/blocks/utils' -import type { QuiverSvgResponse } from '@/tools/quiver/types' +import { createVersionedToolSelector, normalizeFileInput } from '@/blocks/utils' +import type { QuiverSvgResponse, QuiverSvgV2Response } from '@/tools/quiver/types' const REFERENCE_IMAGES_FIELD = ['referenceFiles', 'referenceInput'] as const const IMAGE_FIELD = ['imageFile', 'imageInput'] as const -export const QuiverBlock: BlockConfig = { +export const QuiverBlock = { type: 'quiver', - name: 'Quiver', + name: 'Quiver (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'quiver_v2' }, description: 'Generate and vectorize SVGs', longDescription: 'Generate SVG images from text prompts or vectorize raster images into SVGs using QuiverAI. Supports reference images, style instructions, and multiple output generation.', @@ -255,6 +258,34 @@ export const QuiverBlock: BlockConfig = { description: 'List of available models (list_models operation only)', }, }, +} satisfies BlockConfig + +const selectQuiverV2Tool = createVersionedToolSelector({ + baseToolSelector: (params: Record) => + `quiver_${params.operation || 'text_to_svg'}`, + suffix: '_v2', + fallbackToolId: 'quiver_text_to_svg_v2', +}) + +export const QuiverV2Block: BlockConfig = { + ...QuiverBlock, + type: 'quiver_v2', + name: 'Quiver', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...QuiverBlock.tools, + access: ['quiver_text_to_svg_v2', 'quiver_image_to_svg_v2', 'quiver_list_models'], + config: { + ...QuiverBlock.tools.config, + tool: (params: Record) => + params.operation === 'list_models' ? 'quiver_list_models' : selectQuiverV2Tool(params), + }, + }, + outputs: { + ...omit(QuiverBlock.outputs, ['file', 'svgContent']), + files: { type: 'file[]', description: 'All generated SVG files' }, + }, } export const QuiverBlockMeta = { @@ -294,13 +325,13 @@ export const QuiverBlockMeta = { name: 'generate-brand-icon', description: 'Generate a clean SVG icon from a text prompt and save it to the files store.', content: - '# Generate Brand Icon\n\nTurn a text description into a production-ready SVG icon using Quiver text-to-SVG.\n\n## Steps\n1. Collect the icon concept (for example, a product name plus a brand color and style cues).\n2. Run the text_to_svg operation with a focused prompt that names the subject, color palette, and visual style (flat, line, filled).\n3. Optionally set n greater than 1 to generate several variations to choose from.\n4. Save the returned SVG file to the files store, or pass svgContent downstream for embedding.\n\n## Output\nReport the saved file location and the request id. When multiple variations are generated, list each so the user can pick one.', + '# Generate Brand Icon\n\nTurn a text description into a production-ready SVG icon using Quiver text-to-SVG.\n\n## Steps\n1. Collect the icon concept (for example, a product name plus a brand color and style cues).\n2. Run the text_to_svg operation with a focused prompt that names the subject, color palette, and visual style (flat, line, filled).\n3. Optionally set n greater than 1 to generate several variations to choose from.\n4. Save the returned SVG file to the files store, or pass the file downstream.\n\n## Output\nReport the saved file location and the request id. When multiple variations are generated, list each so the user can pick one.', }, { name: 'vectorize-raster-image', description: 'Convert an uploaded raster image (PNG or JPG) into a clean editable SVG.', content: - '# Vectorize Raster Image\n\nConvert a bitmap logo or graphic into a scalable SVG with Quiver image-to-SVG.\n\n## Steps\n1. Accept the raster image upload and pass it as the image input.\n2. Run the image_to_svg operation, optionally setting auto_crop and a target_size to tighten the output.\n3. Inspect svgContent for fidelity; rerun with adjusted instructions if details are lost.\n4. Save the SVG file for use in presentations, exports, or the web.\n\n## Output\nReturn the vectorized SVG file and confirm dimensions. Note any visual elements that did not vectorize cleanly.', + '# Vectorize Raster Image\n\nConvert a bitmap logo or graphic into a scalable SVG with Quiver image-to-SVG.\n\n## Steps\n1. Accept the raster image upload and pass it as the image input.\n2. Run the image_to_svg operation, optionally setting auto_crop and a target_size to tighten the output.\n3. Inspect the generated SVG file for fidelity; rerun with adjusted instructions if details are lost.\n4. Save the SVG file for use in presentations, exports, or the web.\n\n## Output\nReturn the vectorized SVG file and confirm dimensions. Note any visual elements that did not vectorize cleanly.', }, { name: 'create-data-diagram', diff --git a/apps/sim/blocks/blocks/servicenow.ts b/apps/sim/blocks/blocks/servicenow.ts index 4b4eb33cac9..6ac69083260 100644 --- a/apps/sim/blocks/blocks/servicenow.ts +++ b/apps/sim/blocks/blocks/servicenow.ts @@ -162,9 +162,11 @@ const optionalChoices = (options: reado ...options, ] -export const ServiceNowBlock: BlockConfig = { +export const ServiceNowBlock = { type: 'servicenow', - name: 'ServiceNow', + name: 'ServiceNow (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'servicenow_v2' }, description: 'Create, read, update, and delete ServiceNow records', longDescription: 'Integrate ServiceNow into your workflow. Create, read, update, and delete records in any ServiceNow table including incidents, tasks, change requests, users, and more.', @@ -1921,6 +1923,33 @@ Output: {"state": "2", "assigned_to": "john.doe", "work_notes": "Assigned and st 'servicenow_webhook', ], }, +} satisfies BlockConfig + +export const ServiceNowV2Block: BlockConfig = { + ...ServiceNowBlock, + type: 'servicenow_v2', + name: 'ServiceNow', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...ServiceNowBlock.tools, + access: ServiceNowBlock.tools.access.map((toolId) => + toolId === 'servicenow_download_attachment' ? 'servicenow_download_attachment_v2' : toolId + ), + config: { + ...ServiceNowBlock.tools.config, + tool: (params) => { + const toolId = ServiceNowBlock.tools.config.tool(params) + return toolId === 'servicenow_download_attachment' + ? 'servicenow_download_attachment_v2' + : toolId + }, + }, + }, + outputs: { + ...ServiceNowBlock.outputs, + content: { type: 'string', description: 'HTML body of a knowledge article' }, + }, } export const ServiceNowBlockMeta = { diff --git a/apps/sim/blocks/blocks/sftp.test.ts b/apps/sim/blocks/blocks/sftp.test.ts new file mode 100644 index 00000000000..6d8570a2308 --- /dev/null +++ b/apps/sim/blocks/blocks/sftp.test.ts @@ -0,0 +1,59 @@ +import { describe, expect, it } from 'vitest' +import { SftpBlock, SftpV2Block } from '@/blocks/blocks/sftp' + +describe('SFTP block versions', () => { + it('preserves legacy blocks and offers v2 for new blocks', () => { + expect(SftpBlock.hideFromToolbar).toBe(true) + expect(SftpBlock.sunset).toEqual({ status: 'legacy', replacedBy: 'sftp_v2' }) + expect(SftpV2Block.hideFromToolbar).toBe(false) + expect(SftpV2Block.sunset).toBeUndefined() + expect(SftpV2Block.name).toBe('SFTP') + expect(SftpV2Block.canvasPresentation).toBe(SftpBlock.canvasPresentation) + }) + + it.each(SftpBlock.tools.access)('versions only the download operation: %s', (operation) => { + const currentId = operation === 'sftp_download' ? 'sftp_download_v2' : operation + expect(SftpBlock.tools.config.tool({ operation })).toBe(operation) + expect(SftpV2Block.tools.config?.tool({ operation })).toBe(currentId) + expect(SftpV2Block.tools.access).toContain(currentId) + }) + + it('preserves defaults and the create-file alias', () => { + expect(SftpV2Block.tools.config?.tool({})).toBe('sftp_upload') + expect(SftpV2Block.tools.config?.tool({ operation: 'sftp_create' })).toBe('sftp_upload') + }) + + it('removes download encoding and inline content from v2', () => { + expect(SftpBlock.subBlocks.some((subBlock) => subBlock.id === 'encoding')).toBe(true) + expect(SftpV2Block.subBlocks.some((subBlock) => subBlock.id === 'encoding')).toBe(false) + expect(SftpV2Block.inputs).not.toHaveProperty('encoding') + expect(SftpV2Block.outputs).not.toHaveProperty('content') + expect(SftpV2Block.outputs).not.toHaveProperty('fileName') + expect(SftpV2Block.outputs).not.toHaveProperty('size') + for (const key of ['success', 'message']) { + expect(SftpV2Block.outputs[key].condition).toEqual({ + field: 'operation', + value: 'sftp_download', + not: true, + }) + } + expect(SftpV2Block.outputs.file.type).toBe('file') + const params = { + operation: 'sftp_download', + host: 'sftp.example.com', + port: '22', + username: 'user', + password: 'test-password', + remotePath: '/file.txt', + encoding: 'base64', + } + expect(SftpBlock.tools.config.params(params)).toHaveProperty('encoding', 'base64') + expect(SftpV2Block.tools.config?.params?.(params)).toEqual({ + host: 'sftp.example.com', + port: 22, + username: 'user', + password: 'test-password', + remotePath: '/file.txt', + }) + }) +}) diff --git a/apps/sim/blocks/blocks/sftp.ts b/apps/sim/blocks/blocks/sftp.ts index 62181bdaabb..a4132d27565 100644 --- a/apps/sim/blocks/blocks/sftp.ts +++ b/apps/sim/blocks/blocks/sftp.ts @@ -1,13 +1,16 @@ import { ClipboardList, Download, File, Search, Server, Trash, Upload } from '@sim/emcn/icons' +import { omit } from '@sim/utils/object' import { SftpIcon } from '@/components/icons' import type { BlockConfig, BlockMeta } from '@/blocks/types' import { AuthMode, IntegrationType } from '@/blocks/types' -import { normalizeFileInput } from '@/blocks/utils' +import { createVersionedToolSelector, normalizeFileInput } from '@/blocks/utils' import type { SftpUploadResult } from '@/tools/sftp/types' -export const SftpBlock: BlockConfig = { +export const SftpBlock = { type: 'sftp', - name: 'SFTP', + name: 'SFTP (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'sftp_v2' }, description: 'Transfer files via SFTP (SSH File Transfer Protocol)', longDescription: 'Upload, download, list, and manage files on remote servers via SFTP. Supports both password and private key authentication for secure file transfers.', @@ -332,6 +335,50 @@ export const SftpBlock: BlockConfig = { message: { type: 'string', description: 'Operation status message' }, error: { type: 'string', description: 'Error message if operation failed' }, }, +} satisfies BlockConfig + +const selectSftpV2Tool = createVersionedToolSelector({ + baseToolSelector: SftpBlock.tools.config.tool, + suffix: '_v2', + fallbackToolId: 'sftp_download_v2', +}) + +export const SftpV2Block: BlockConfig = { + ...SftpBlock, + type: 'sftp_v2', + name: 'SFTP', + hideFromToolbar: false, + sunset: undefined, + subBlocks: SftpBlock.subBlocks.filter((subBlock) => subBlock.id !== 'encoding'), + tools: { + ...SftpBlock.tools, + access: SftpBlock.tools.access.map((toolId) => + toolId === 'sftp_download' ? 'sftp_download_v2' : toolId + ), + config: { + ...SftpBlock.tools.config, + tool: (params) => + params.operation === 'sftp_download' + ? selectSftpV2Tool(params) + : SftpBlock.tools.config.tool(params), + params: (params) => { + const input: Record = SftpBlock.tools.config.params(params) + return omit(input, ['encoding']) + }, + }, + }, + inputs: omit(SftpBlock.inputs, ['encoding']), + outputs: { + ...omit(SftpBlock.outputs, ['content', 'fileName', 'size']), + success: { + ...SftpBlock.outputs.success, + condition: { field: 'operation', value: 'sftp_download', not: true }, + }, + message: { + ...SftpBlock.outputs.message, + condition: { field: 'operation', value: 'sftp_download', not: true }, + }, + }, } export const SftpBlockMeta = { @@ -416,7 +463,7 @@ export const SftpBlockMeta = { name: 'pull-remote-drop-folder', description: 'Poll a remote SFTP drop folder on a schedule and ingest any new files.', content: - '# Pull Remote Drop Folder\n\nPeriodically fetch newly arrived files from a remote SFTP directory into a workflow.\n\n## Steps\n1. Use the List Directory operation to read the remote drop folder and inspect `entries`.\n2. Filter for files newer than the last processed timestamp.\n3. For each new file, use the Download File operation and read `file`/`content`.\n4. Hand the contents to downstream blocks for parsing.\n\n## Output\nNew remote files are downloaded and their contents are available for processing each run.', + '# Pull Remote Drop Folder\n\nPeriodically fetch newly arrived files from a remote SFTP directory into a workflow.\n\n## Steps\n1. Use the List Directory operation to read the remote drop folder and inspect `entries`.\n2. Filter for files newer than the last processed timestamp.\n3. For each new file, use the Download File operation and read `file`.\n4. Pass the file to downstream blocks for parsing.\n\n## Output\nNew remote files are downloaded and available for processing each run.', }, { name: 'push-report-to-partner', diff --git a/apps/sim/blocks/blocks/ssh.test.ts b/apps/sim/blocks/blocks/ssh.test.ts new file mode 100644 index 00000000000..a49753c3c08 --- /dev/null +++ b/apps/sim/blocks/blocks/ssh.test.ts @@ -0,0 +1,39 @@ +import { describe, expect, it } from 'vitest' +import { SSHBlock, SSHV2Block } from '@/blocks/blocks/ssh' + +describe('SSH block versions', () => { + it('preserves legacy blocks and offers v2 for new blocks', () => { + expect(SSHBlock.hideFromToolbar).toBe(true) + expect(SSHBlock.sunset).toEqual({ status: 'legacy', replacedBy: 'ssh_v2' }) + expect(SSHV2Block.hideFromToolbar).toBe(false) + expect(SSHV2Block.sunset).toBeUndefined() + expect(SSHV2Block.name).toBe('SSH') + expect(SSHV2Block.subBlocks).toBe(SSHBlock.subBlocks) + expect(SSHV2Block.tools.config?.params).toBe(SSHBlock.tools.config.params) + expect(SSHV2Block.canvasPresentation).toBe(SSHBlock.canvasPresentation) + }) + + it.each(SSHBlock.tools.access)('versions only the download operation: %s', (operation) => { + const currentId = operation === 'ssh_download_file' ? 'ssh_download_file_v2' : operation + expect(SSHBlock.tools.config.tool({ operation })).toBe(operation) + expect(SSHV2Block.tools.config?.tool({ operation })).toBe(currentId) + expect(SSHV2Block.tools.access).toContain(currentId) + }) + + it('preserves the command default and explicit read-content operation', () => { + expect(SSHV2Block.tools.config?.tool({})).toBe('ssh_execute_command') + expect(SSHV2Block.outputs).not.toHaveProperty('fileContent') + expect(SSHV2Block.outputs.file.type).toBe('file') + expect(SSHV2Block.outputs.content.condition).toEqual({ + field: 'operation', + value: 'ssh_read_file_content', + }) + for (const key of ['success', 'message']) { + expect(SSHV2Block.outputs[key].condition).toEqual({ + field: 'operation', + value: 'ssh_download_file', + not: true, + }) + } + }) +}) diff --git a/apps/sim/blocks/blocks/ssh.ts b/apps/sim/blocks/blocks/ssh.ts index a991a0ea6c8..c49a3a4de0c 100644 --- a/apps/sim/blocks/blocks/ssh.ts +++ b/apps/sim/blocks/blocks/ssh.ts @@ -1,12 +1,16 @@ import { ClipboardList, Download, File, Search, Server, Wrench } from '@sim/emcn/icons' +import { omit } from '@sim/utils/object' import { SshIcon, SshTerminalIcon } from '@/components/icons' import type { BlockConfig, BlockMeta } from '@/blocks/types' import { AuthMode, IntegrationType } from '@/blocks/types' +import { createVersionedToolSelector } from '@/blocks/utils' import type { SSHResponse } from '@/tools/ssh/types' -export const SSHBlock: BlockConfig = { +export const SSHBlock = { type: 'ssh', - name: 'SSH', + name: 'SSH (Legacy)', + hideFromToolbar: true, + sunset: { status: 'legacy', replacedBy: 'ssh_v2' }, description: 'Connect to remote servers via SSH', authMode: AuthMode.ApiKey, longDescription: @@ -661,6 +665,48 @@ Examples: os: { type: 'string', description: 'Operating system' }, message: { type: 'string', description: 'Operation status message' }, }, +} satisfies BlockConfig + +const selectSshV2Tool = createVersionedToolSelector({ + baseToolSelector: SSHBlock.tools.config.tool, + suffix: '_v2', + fallbackToolId: 'ssh_download_file_v2', +}) + +export const SSHV2Block: BlockConfig = { + ...SSHBlock, + type: 'ssh_v2', + name: 'SSH', + hideFromToolbar: false, + sunset: undefined, + tools: { + ...SSHBlock.tools, + access: SSHBlock.tools.access.map((toolId) => + toolId === 'ssh_download_file' ? 'ssh_download_file_v2' : toolId + ), + config: { + ...SSHBlock.tools.config, + tool: (params) => + params.operation === 'ssh_download_file' + ? selectSshV2Tool(params) + : SSHBlock.tools.config.tool(params), + }, + }, + outputs: { + ...omit(SSHBlock.outputs, ['fileContent']), + success: { + ...SSHBlock.outputs.success, + condition: { field: 'operation', value: 'ssh_download_file', not: true }, + }, + message: { + ...SSHBlock.outputs.message, + condition: { field: 'operation', value: 'ssh_download_file', not: true }, + }, + content: { + ...SSHBlock.outputs.content, + condition: { field: 'operation', value: 'ssh_read_file_content' }, + }, + }, } export const SSHBlockMeta = { diff --git a/apps/sim/blocks/registry-maps.ts b/apps/sim/blocks/registry-maps.ts index 672ebb5e0e1..b75dc96d948 100644 --- a/apps/sim/blocks/registry-maps.ts +++ b/apps/sim/blocks/registry-maps.ts @@ -25,7 +25,7 @@ import { } from '@/blocks/blocks/azure_data_explorer' import { AzureDevOpsBlock, AzureDevOpsBlockMeta } from '@/blocks/blocks/azure_devops' import { BitbucketBlock, BitbucketBlockMeta } from '@/blocks/blocks/bitbucket' -import { BoxBlock, BoxBlockMeta } from '@/blocks/blocks/box' +import { BoxBlock, BoxBlockMeta, BoxV2Block } from '@/blocks/blocks/box' import { BrandfetchBlock, BrandfetchBlockMeta } from '@/blocks/blocks/brandfetch' import { BrexBlock, BrexBlockMeta } from '@/blocks/blocks/brex' import { BrightDataBlock, BrightDataBlockMeta } from '@/blocks/blocks/brightdata' @@ -64,10 +64,10 @@ import { DevinBlock, DevinBlockMeta } from '@/blocks/blocks/devin' import { DiscordBlock, DiscordBlockMeta } from '@/blocks/blocks/discord' import { DocuSignBlock, DocuSignBlockMeta } from '@/blocks/blocks/docusign' import { DowndetectorBlock, DowndetectorBlockMeta } from '@/blocks/blocks/downdetector' -import { DropboxBlock, DropboxBlockMeta } from '@/blocks/blocks/dropbox' +import { DropboxBlock, DropboxBlockMeta, DropboxV2Block } from '@/blocks/blocks/dropbox' import { DropcontactBlock, DropcontactBlockMeta } from '@/blocks/blocks/dropcontact' import { DSPyBlock, DSPyBlockMeta } from '@/blocks/blocks/dspy' -import { DubBlock, DubBlockMeta } from '@/blocks/blocks/dub' +import { DubBlock, DubBlockMeta, DubV2Block } from '@/blocks/blocks/dub' import { DuckDuckGoBlock, DuckDuckGoBlockMeta } from '@/blocks/blocks/duckduckgo' import { DynamoDBBlock, DynamoDBBlockMeta } from '@/blocks/blocks/dynamodb' import { DynatraceBlock, DynatraceBlockMeta } from '@/blocks/blocks/dynatrace' @@ -173,7 +173,7 @@ import { JiraServiceManagementBlockMeta, } from '@/blocks/blocks/jira_service_management' import { JotformBlock, JotformBlockMeta } from '@/blocks/blocks/jotform' -import { JupyterBlock, JupyterBlockMeta } from '@/blocks/blocks/jupyter' +import { JupyterBlock, JupyterBlockMeta, JupyterV2Block } from '@/blocks/blocks/jupyter' import { KalshiBlock, KalshiBlockMeta, @@ -209,6 +209,7 @@ import { MicrosoftAdBlock, MicrosoftAdBlockMeta } from '@/blocks/blocks/microsof import { MicrosoftDataverseBlock, MicrosoftDataverseBlockMeta, + MicrosoftDataverseV2Block, } from '@/blocks/blocks/microsoft_dataverse' import { MicrosoftDynamics365Block, @@ -272,7 +273,7 @@ import { PulseBlock, PulseBlockMeta, PulseV2Block } from '@/blocks/blocks/pulse' import { QdrantBlock, QdrantBlockMeta } from '@/blocks/blocks/qdrant' import { QuartrBlock, QuartrBlockMeta } from '@/blocks/blocks/quartr' import { QuickBooksBlock, QuickBooksBlockMeta } from '@/blocks/blocks/quickbooks' -import { QuiverBlock, QuiverBlockMeta } from '@/blocks/blocks/quiver' +import { QuiverBlock, QuiverBlockMeta, QuiverV2Block } from '@/blocks/blocks/quiver' import { RabbitmqBlock, RabbitmqBlockMeta } from '@/blocks/blocks/rabbitmq' import { RailwayBlock, RailwayBlockMeta } from '@/blocks/blocks/railway' import { RB2BBlock, RB2BBlockMeta } from '@/blocks/blocks/rb2b' @@ -301,9 +302,9 @@ import { SendblueBlock, SendblueBlockMeta } from '@/blocks/blocks/sendblue' import { SendGridBlock, SendGridBlockMeta } from '@/blocks/blocks/sendgrid' import { SentryBlock, SentryBlockMeta } from '@/blocks/blocks/sentry' import { SerperBlock, SerperBlockMeta } from '@/blocks/blocks/serper' -import { ServiceNowBlock, ServiceNowBlockMeta } from '@/blocks/blocks/servicenow' +import { ServiceNowBlock, ServiceNowBlockMeta, ServiceNowV2Block } from '@/blocks/blocks/servicenow' import { SESBlock, SESBlockMeta } from '@/blocks/blocks/ses' -import { SftpBlock, SftpBlockMeta } from '@/blocks/blocks/sftp' +import { SftpBlock, SftpBlockMeta, SftpV2Block } from '@/blocks/blocks/sftp' import { SharepointBlock, SharepointBlockMeta, SharepointV2Block } from '@/blocks/blocks/sharepoint' import { ShopifyBlock, ShopifyBlockMeta } from '@/blocks/blocks/shopify' import { SimWorkspaceEventBlock } from '@/blocks/blocks/sim_workspace_event' @@ -318,7 +319,7 @@ import { SportmonksBlock, SportmonksBlockMeta } from '@/blocks/blocks/sportmonks import { SpotifyBlock, SpotifyBlockMeta } from '@/blocks/blocks/spotify' import { SQSBlock, SQSBlockMeta } from '@/blocks/blocks/sqs' import { SquareBlock, SquareBlockMeta } from '@/blocks/blocks/square' -import { SSHBlock, SSHBlockMeta } from '@/blocks/blocks/ssh' +import { SSHBlock, SSHBlockMeta, SSHV2Block } from '@/blocks/blocks/ssh' import { SSMBlock, SSMBlockMeta } from '@/blocks/blocks/ssm' import { StagehandBlock, StagehandBlockMeta } from '@/blocks/blocks/stagehand' import { StartTriggerBlock } from '@/blocks/blocks/start_trigger' @@ -406,6 +407,7 @@ export const BLOCK_REGISTRY: Record = { azure_devops: AzureDevOpsBlock, bitbucket: BitbucketBlock, box: BoxBlock, + box_v2: BoxV2Block, brandfetch: BrandfetchBlock, brex: BrexBlock, brightdata: BrightDataBlock, @@ -447,9 +449,11 @@ export const BLOCK_REGISTRY: Record = { docusign: DocuSignBlock, downdetector: DowndetectorBlock, dropbox: DropboxBlock, + dropbox_v2: DropboxV2Block, dropcontact: DropcontactBlock, dspy: DSPyBlock, dub: DubBlock, + dub_v2: DubV2Block, duckduckgo: DuckDuckGoBlock, dynamodb: DynamoDBBlock, dynatrace: DynatraceBlock, @@ -541,6 +545,7 @@ export const BLOCK_REGISTRY: Record = { jira_service_management: JiraServiceManagementBlock, jotform: JotformBlock, jupyter: JupyterBlock, + jupyter_v2: JupyterV2Block, kalshi: KalshiBlock, kalshi_v2: KalshiV2Block, ketch: KetchBlock, @@ -572,6 +577,7 @@ export const BLOCK_REGISTRY: Record = { memory: MemoryBlock, microsoft_ad: MicrosoftAdBlock, microsoft_dataverse: MicrosoftDataverseBlock, + microsoft_dataverse_v2: MicrosoftDataverseV2Block, microsoft_dynamics_365: MicrosoftDynamics365Block, microsoft_excel: MicrosoftExcelBlock, microsoft_excel_v2: MicrosoftExcelV2Block, @@ -620,6 +626,7 @@ export const BLOCK_REGISTRY: Record = { quartr: QuartrBlock, quickbooks: QuickBooksBlock, quiver: QuiverBlock, + quiver_v2: QuiverV2Block, rabbitmq: RabbitmqBlock, railway: RailwayBlock, rb2b: RB2BBlock, @@ -651,8 +658,10 @@ export const BLOCK_REGISTRY: Record = { sentry: SentryBlock, serper: SerperBlock, servicenow: ServiceNowBlock, + servicenow_v2: ServiceNowV2Block, ses: SESBlock, sftp: SftpBlock, + sftp_v2: SftpV2Block, sharepoint: SharepointBlock, sharepoint_v2: SharepointV2Block, shopify: ShopifyBlock, @@ -670,6 +679,7 @@ export const BLOCK_REGISTRY: Record = { sqs: SQSBlock, square: SquareBlock, ssh: SSHBlock, + ssh_v2: SSHV2Block, ssm: SSMBlock, stagehand: StagehandBlock, start_trigger: StartTriggerBlock, @@ -875,6 +885,7 @@ export const BLOCK_META_REGISTRY: Record = { jira_service_management: JiraServiceManagementBlockMeta, jotform: JotformBlockMeta, jupyter: JupyterBlockMeta, + jupyter_v2: JupyterBlockMeta, kalshi: KalshiBlockMeta, kalshi_v2: KalshiV2BlockMeta, ketch: KetchBlockMeta, @@ -970,6 +981,7 @@ export const BLOCK_META_REGISTRY: Record = { servicenow: ServiceNowBlockMeta, ses: SESBlockMeta, sftp: SftpBlockMeta, + sftp_v2: SftpBlockMeta, sharepoint: SharepointBlockMeta, shopify: ShopifyBlockMeta, similarweb: SimilarwebBlockMeta, @@ -985,6 +997,7 @@ export const BLOCK_META_REGISTRY: Record = { sqs: SQSBlockMeta, square: SquareBlockMeta, ssh: SSHBlockMeta, + ssh_v2: SSHBlockMeta, ssm: SSMBlockMeta, stagehand: StagehandBlockMeta, stripe: StripeBlockMeta, diff --git a/apps/sim/components/settings/navigation.test.ts b/apps/sim/components/settings/navigation.test.ts index 37ded7d6390..7e29f8ca48c 100644 --- a/apps/sim/components/settings/navigation.test.ts +++ b/apps/sim/components/settings/navigation.test.ts @@ -470,6 +470,24 @@ describe('settings navigation boundaries', () => { ).toBe('manage') }) + it('allows members to recover their own organization chats without changing workspace settings ownership', () => { + expect( + resolveOrganizationSectionAccess({ + section: 'recently-deleted', + isTargetOrganizationMember: true, + isTargetOrganizationAdmin: false, + }) + ).toBe('view') + expect( + resolveOrganizationSectionAccess({ + section: 'recently-deleted', + isTargetOrganizationMember: false, + isTargetOrganizationAdmin: false, + }) + ).toBe('unavailable') + expect(ORGANIZATION_PLANE_UNIFIED_SECTIONS.has('recently-deleted')).toBe(false) + }) + it('gates organization control-plane sections by the target organization plan', () => { const hostedFree = { billingEnabled: true, @@ -478,6 +496,7 @@ describe('settings navigation boundaries', () => { selfHosted: {}, } expect(isOrganizationSettingsSectionAvailable('members', hostedFree)).toBe(true) + expect(isOrganizationSettingsSectionAvailable('recently-deleted', hostedFree)).toBe(true) expect(isOrganizationSettingsSectionAvailable('billing', hostedFree)).toBe(true) expect(isOrganizationSettingsSectionAvailable('sso', hostedFree)).toBe(false) expect( diff --git a/apps/sim/components/settings/navigation.ts b/apps/sim/components/settings/navigation.ts index d935ce3e92b..2665b09e256 100644 --- a/apps/sim/components/settings/navigation.ts +++ b/apps/sim/components/settings/navigation.ts @@ -44,6 +44,7 @@ export type AccountSettingsSection = 'general' | 'billing' | 'api-keys' | 'admin export type SelfHostSettingsSection = 'general' | 'billing' | 'chat-keys' export type OrganizationSettingsSection = + | 'recently-deleted' | 'integrations' | 'connected-accounts' | 'search-mcp' @@ -900,6 +901,7 @@ const ORGANIZATION_SECTION_GROUPS: Record { const group = ORGANIZATION_SECTION_GROUPS[id] + if (id === 'recently-deleted') { + return { + id, + label: 'Recently deleted', + description: 'Restore your deleted chats.', + icon: Trash, + group, + } + } if (id === 'connected-accounts') { return { id, @@ -1011,7 +1022,7 @@ export function resolveOrganizationSectionAccess({ isTargetOrganizationAdmin, }: ResolveOrganizationSectionAccessOptions): OrganizationSectionAccess { if (!isTargetOrganizationMember) return 'unavailable' - if (section === 'search-mcp') return 'view' + if (section === 'search-mcp' || section === 'recently-deleted') return 'view' if (section === 'members') return isTargetOrganizationAdmin ? 'manage' : 'view' return isTargetOrganizationAdmin ? 'manage' : 'unavailable' } @@ -1054,7 +1065,8 @@ export function isOrganizationSettingsSectionAvailable( section: OrganizationSettingsSection, features: OrganizationSettingsFeatures ): boolean { - if (section === 'members' || section === 'search-mcp') return true + if (section === 'members' || section === 'search-mcp' || section === 'recently-deleted') + return true if (section === 'billing') return features.billingEnabled /* Sim Search itself is enterprise on the hosted product; self-hosted gates it by flag, not by section. */ if (section === 'integrations' || section === 'search-slack') diff --git a/apps/sim/components/workspaces/workspace-context-menu.test.tsx b/apps/sim/components/workspaces/workspace-context-menu.test.tsx new file mode 100644 index 00000000000..db5cb098883 --- /dev/null +++ b/apps/sim/components/workspaces/workspace-context-menu.test.tsx @@ -0,0 +1,114 @@ +/** + * @vitest-environment jsdom + */ +import { act, createRef } from 'react' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { WorkspaceContextMenu } from '@/components/workspaces/workspace-context-menu' +import type { Workspace } from '@/lib/api/contracts/workspaces' + +const workspace: Workspace = { + id: 'workspace', + name: 'Workspace', + organizationId: 'org', + workspaceMode: 'organization', + ownerId: 'owner', + permissions: 'admin', +} + +let container: HTMLDivElement +let root: Root +beforeEach(() => { + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + container = document.createElement('div') + document.body.appendChild(container) + root = createRoot(container) +}) +afterEach(() => { + act(() => root.unmount()) + container.remove() + vi.unstubAllGlobals() +}) +function menuItem(name: string) { + return Array.from(document.querySelectorAll('[role="menuitem"]')).find( + (item) => item.textContent === name + ) +} + +function showMenu( + overrides: Partial = {}, + sessionUserId = 'member', + workspaceCount = 2 +) { + act(() => + root.render( + ()} + onClose={vi.fn()} + onTogglePin={vi.fn()} + onRename={vi.fn()} + onDelete={vi.fn()} + onLeave={vi.fn()} + onUploadLogo={vi.fn()} + /> + ) + ) +} + +describe('WorkspaceContextMenu', () => { + it.each(['read', 'write'] as const)( + 'keeps personal pinning available for %s access but disables admin actions', + (permissions) => { + showMenu({ permissions }) + expect(menuItem('Pin')).not.toHaveAttribute('aria-disabled', 'true') + expect(menuItem('Rename')).toHaveAttribute('aria-disabled', 'true') + expect(menuItem('Delete')).toHaveAttribute('aria-disabled', 'true') + expect(menuItem('Upload logo')).toHaveAttribute('aria-disabled', 'true') + } + ) + + it('allows an explicit admin to rename and leave', () => { + showMenu() + expect(menuItem('Rename')).not.toHaveAttribute('aria-disabled', 'true') + expect(menuItem('Leave')).toBeDefined() + }) + + it('does not offer leaving access inherited from the organization role', () => { + showMenu({ isOrgAdmin: true }) + expect(menuItem('Leave')).toBeUndefined() + expect(menuItem('Rename')).not.toHaveAttribute('aria-disabled', 'true') + }) + + it('does not offer leaving the owned workspace or deleting the final workspace', () => { + showMenu({}, 'owner', 1) + expect(menuItem('Leave')).toBeUndefined() + expect(menuItem('Delete')).toHaveAttribute('aria-disabled', 'true') + }) + + it('omits unavailable actions even when the viewer has admin permissions', () => { + act(() => + root.render( + ()} + onClose={vi.fn()} + onRename={vi.fn()} + onTogglePin={vi.fn()} + /> + ) + ) + expect(menuItem('Upload logo')).toBeUndefined() + expect(menuItem('Leave')).toBeUndefined() + expect(menuItem('Delete')).toBeUndefined() + expect(menuItem('Rename')).toBeDefined() + }) +}) diff --git a/apps/sim/components/workspaces/workspace-context-menu.tsx b/apps/sim/components/workspaces/workspace-context-menu.tsx new file mode 100644 index 00000000000..9b1bb88ce77 --- /dev/null +++ b/apps/sim/components/workspaces/workspace-context-menu.tsx @@ -0,0 +1,47 @@ +'use client' + +import type { ComponentProps } from 'react' +import type { Workspace } from '@/lib/api/contracts/workspaces' +import { ContextMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/components/workflow-list/components/context-menu/context-menu' + +interface WorkspaceContextMenuProps + extends Omit< + ComponentProps, + | 'disableRename' + | 'disableDelete' + | 'disableUploadLogo' + | 'showLeave' + | 'showUploadLogo' + | 'showPin' + | 'showRename' + > { + workspace?: Workspace | null + workspaceCount: number + sessionUserId?: string +} + +/** Uses the same workspace role policy in the switcher and organization sidebar. */ +export function WorkspaceContextMenu({ + workspace, + workspaceCount, + sessionUserId, + ...props +}: WorkspaceContextMenuProps) { + const canAdmin = workspace?.permissions === 'admin' + const canLeave = Boolean( + workspace && sessionUserId && sessionUserId !== workspace.ownerId && !workspace.isOrgAdmin + ) + + return ( + + ) +} diff --git a/apps/sim/content/library/ai-workflow-automation-platform-buyers-checklist/index.mdx b/apps/sim/content/library/ai-workflow-automation-platform-buyers-checklist/index.mdx new file mode 100644 index 00000000000..5ec2b8b9a33 --- /dev/null +++ b/apps/sim/content/library/ai-workflow-automation-platform-buyers-checklist/index.mdx @@ -0,0 +1,134 @@ +--- +slug: ai-workflow-automation-platform-buyers-checklist +title: 'What to Look for in an AI Workflow Automation Platform: Buyer''s Checklist' +description: 'A six-criteria buyer''s checklist for evaluating AI workflow automation platforms, covering agent support, orchestration, model flexibility, integrations, governance, and pricing.' +date: 2026-09-11 +updated: 2026-09-11 +authors: + - andrew +readingTime: 9 +tags: [AI Agents, Workflow Automation, Enterprise AI, Sim] +ogImage: /library/ai-workflow-automation-platform-buyers-checklist/cover.jpg +canonical: https://www.sim.ai/library/ai-workflow-automation-platform-buyers-checklist +draft: false +faq: + - q: "Is an AI agent platform the same as workflow automation software?" + a: "The categories overlap. Workflow automation software primarily executes predefined steps, while an AI agent platform lets models choose tools and actions based on context. Some products, including Sim, support both structured workflows and tool-using agents." + - q: "Do I need BYOK support, or is hosted model access enough?" + a: "Hosted access works when the platform offers suitable models, predictable limits, and acceptable data handling. Bring your own key becomes useful when you need direct provider billing, specific model access, or greater control over credentials." + - q: "What is the difference between self-hosted and cloud-hosted governance?" + a: "Cloud hosting places infrastructure maintenance and security operations with the vendor. Self-hosting gives you more control over data location, networking, and model access, but you must manage updates, scaling, monitoring, and hardening. Enterprise controls such as SSO and audit logs may still require a paid license." + - q: "How do credit-based and task-based pricing compare at high volume?" + a: "Task-based plans usually charge for each successful action, so workflows with many actions consume more quota. Credit-based plans may assign different costs to AI calls, enrichment, and other nodes. You should price several representative workflows rather than compare headline monthly allowances." + - q: "Can I switch platforms later without rebuilding everything?" + a: "Most migrations require some rebuilding because platforms define triggers, branches, credentials, and data mappings differently. Standard APIs, webhooks, MCP tools, portable prompts, and external data stores can reduce the work. Before buying, ask whether you can export workflow definitions, logs, and stored data." +--- + +## TL;DR + +- **AI agent support.** Can agents reason, use tools, access context, and complete tasks with limited supervision? +- **Multi-step orchestration.** Can workflows branch, loop, run parallel steps, and pause for human approval? +- **Model flexibility.** Can you change or bring your own language model without rebuilding workflows? +- **Integration depth.** Does each connector support the actions you need, with API, webhook, or MCP fallbacks? +- **Security and governance.** Can you control access, audit activity, manage data retention, and meet compliance requirements? +- **Pricing model.** Can you predict costs as tasks, model usage, and workflow volume increase? + +The comparison table evaluates Sim, Zapier, n8n, Make, and Gumloop against these six criteria. + +## Why the buying criteria have changed + +Workflow automation software once centered on predictable trigger-action sequences. An event in one app started a fixed action in another. AI agents add reasoning, tool selection, context, and variable execution paths, so buyers now need to assess how a platform controls decisions as well as how it connects applications. + +Established automation vendors have expanded accordingly. Zapier now combines structured Zaps with [goal-driven Agents](https://zapier.com/agents). n8n documents [LangChain-based AI components](https://docs.n8n.io/build/integrate-ai/langchain-in-n8n) for models, memory, tools, and retrieval. Connector count alone does not show how well a product can build and operate agent workflows. + +Use the six criteria in this guide to weight each platform against your deployment requirements. Agent execution and orchestration determine what workflows can do, while model choice and integration depth determine technical fit. Governance controls establish whether you can deploy those workflows under your security requirements, and pricing determines whether production volume fits your budget. For broader market context, see the guides to [AI agent platforms](https://www.sim.ai/library/best-ai-agent-platforms-2026) and [AI automation tools](https://www.sim.ai/library/best-ai-automation-tools-2026). + +## AI agent support: can the platform run autonomous, tool-using agents? + +AI agent support requires a reasoning loop that lets a model choose and call tools until it completes a goal. The agent should also retain relevant context during the run and retrieve stored knowledge when needed. A workflow that sends one prompt to an LLM and passes the response onward provides an AI step, but it does not let an agent choose and use tools in a reasoning loop. + +Sim provides an Agent block within its workflow builder. Each [Sim Agent block](https://docs.sim.ai/workflows/blocks/agent) reasons with a selected model and can act through connected tools. Sim also supports [MCP tools](https://docs.sim.ai/agents/mcp) for services without a built-in integration. Knowledge bases give agents retrievable information, while Sim Tables store structured information that later workflow runs can read and update. + +Platforms package these capabilities differently. Zapier offers [Agents](https://zapier.com/agents) that can use company knowledge and act across connected apps. n8n lets builders configure the agent, LLM, memory, and other components through its [LangChain integration](https://docs.n8n.io/build/integrate-ai/langchain-in-n8n). Its [Call n8n Workflow Tool](https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolworkflow) lets an agent invoke another workflow and retrieve its output, supporting delegated tasks. + +Gumloop documents [tool-using, context-aware Agents](https://docs.gumloop.com/core-concepts/agents) that choose actions based on a goal. Make describes [AI Agents](https://www.make.com/en/ai-agents) that orchestrate processes across its app catalog while exposing reasoning and tool use. These differences make a live test more useful than a feature label. + +Ask each vendor to run an agent against a realistic support request. The agent should consult your knowledge base, call a customer-record tool, and pause before making a consequential update. Then inspect whether the platform records its tool calls and preserves enough context to explain the decision. + +## Multi-step and multi-app orchestration: how workflows branch, loop, and hand off + +Orchestration depth measures how well workflow automation software handles changing paths and repeated work. A capable platform can route records based on conditions, process independent steps in parallel, and iterate over collections. It should also delegate reusable work to sub-workflows and pause for human approval before consequential actions. The guide to [AI agent orchestration frameworks](https://www.sim.ai/library/ai-agent-orchestration-frameworks-explained) explains these patterns in more depth. + +Sim provides Condition and Router blocks for branching, while Parallel and Loop blocks handle concurrent work and iteration. The Sim Workflow Block delegates a task to another workflow. Its [Human in the Loop block](https://docs.sim.ai/workflows/blocks/human-in-the-loop) can pause a run until a reviewer responds and notify a reviewer. The workflow resumes after the reviewer approves, rejects, or supplies requested input. + +Deployment options determine whether you can reuse the same workflow in different contexts. Sim can expose a workflow through an API or a chat interface. You can also [deploy it as an MCP server](https://docs.sim.ai/workflows/deployment/mcp), which lets compatible AI clients call the workflow as a tool. + +Other platforms organize complex work differently. n8n can expose another workflow to an agent through its Workflow Tool, and it documents [modular sub-workflows](https://docs.n8n.io/build/flow-logic/break-workflows-into-smaller-parts). Zapier provides [Paths for conditional branches](https://help.zapier.com/hc/en-us/articles/8496288555917-Add-branching-logic-to-Zap-workflows-with-Paths) plus [Looping and Sub-Zaps](https://help.zapier.com/hc/en-us/sections/16075022072077-Sub-Zap-Looping). Make uses [routers](https://help.make.com/router), [iterators](https://help.make.com/iterator), and [aggregators](https://help.make.com/aggregator) to route and combine data. Gumloop offers [reusable subflows](https://docs.gumloop.com/core-concepts/subflows) and conditional routing. + +During a vendor demo, ask the platform to build one realistic process with an exception path, a loop over multiple records, and an approval that may sit pending for several days. Then inspect how the platform retries failed branches, preserves state during the pause, and traces delegated work. A two-action demo cannot reveal those limits. + +## Model flexibility: locked into one LLM vendor or free to choose? + +Model flexibility lets you choose an LLM for each agent or workflow node, then replace the provider without rebuilding the surrounding logic. A flexible platform may also support bring your own key, or BYOK, so you can use direct provider billing or an existing provider agreement. + +Sim connects to major model providers and separates hosted access from BYOK. Confirm the models, included credits, workflow charges, and plan limits on [Sim's pricing page](https://www.sim.ai/pricing) before estimating production costs. Enterprise deployments can also use self-hosted infrastructure. The [BYOK and multi-model guide](https://www.sim.ai/library/byok-multi-model-ai-agent-builder) covers the architectural tradeoffs. + +Competing platforms expose model choice differently. n8n's configurable language-model components include provider-specific nodes such as its [Anthropic Chat Model](https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatanthropic) and [Ollama Chat Model](https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama). Gumloop documents [model selection and presets](https://docs.gumloop.com/core-concepts/ai_models), while its [pricing page](https://www.gumloop.com/pricing) identifies BYOK as a plan feature. Make's [AI Agent credit documentation](https://help.make.com/credit-usage-for-ai-agents) distinguishes its own provider from custom provider connections on paid plans. + +During a vendor demo, replace the model inside an existing agent and reconnect nothing else. Then confirm which providers, keys, and local deployment options your intended plan includes. + +## Integration depth: raw connector count versus usable depth per app + +Integration depth measures whether a connector supports the operations your workflow needs. A platform may list an app but expose only a few triggers and actions. Check support for the specific records, events, searches, and updates in your workflow. For any missing operation, confirm that the platform provides a workable fallback through a webhook, a generic API request, or a Model Context Protocol tool. + +Published directory totals change frequently. Zapier's [pricing page](https://zapier.com/pricing) describes access to thousands of apps, while Make's [pricing page](https://www.make.com/en/pricing) lists more than 3,000 apps. Sim's current [pricing page](https://www.sim.ai/pricing) describes more than 1,000 integrations, and Sim supports MCP tools for services without native coverage. Rather than compare these changing totals with estimates from secondary reviews, inspect each vendor's live directory and test the operations you require. + +Your evaluation should use a sample workflow rather than the directory total. Ask each vendor to build the same flow with your actual apps, including one uncommon action and one unsupported service. A smaller catalog may still meet your requirements if its connectors expose the operations you need and an API or MCP tool covers the unsupported service in your test. The [Zapier alternatives guide](https://www.sim.ai/library/best-zapier-alternatives) provides another way to frame this comparison. + +## Security and governance: what enterprise buyers actually need to check + +Verify the platform's identity controls, permissions, audit records, data handling, compliance evidence, and deployment options before approval. + +- **Identity management.** Confirm that SSO centralizes sign-in and SCIM automatically provisions or removes users through your identity provider. +- **Access control.** Check whether role-based access control can restrict specific models, integrations, credentials, and administrative actions. +- **Auditability.** Require searchable audit logs for configuration changes, user activity, and workflow runs. Confirm how long the vendor retains those records. +- **Data control.** Ask where prompts, outputs, credentials, and logs reside. Review retention settings, deletion procedures, and export options. +- **Compliance.** Request current certification reports or trust-center documents for standards such as SOC 2 Type II and ISO 27001. A logo on a sales page provides limited evidence. +- **Deployment.** Determine whether self-hosting or an on-premises option covers the full product. Some vendors reserve identity and access controls for paid enterprise licenses. + +[Sim Enterprise documentation](https://docs.sim.ai/platform/enterprise) describes permission groups, SSO, audit logs, usage tracking, data retention, and data drains. Sim separately documents [SCIM provisioning](https://docs.sim.ai/platform/enterprise/scim) and [self-hosted Enterprise configuration](https://docs.sim.ai/platform/enterprise/self-hosted). Request the applicable reports and confirm that their scope covers the Sim services you plan to use. + +Compare vendors at the plan level because security controls and deployment options may be limited to specific editions. n8n documents [SAML availability](https://docs.n8n.io/administer/manage-users-and-access/verify-user-identity/use-saml) and [role-based permissions](https://docs.n8n.io/administer/manage-users-and-access/set-permissions-and-roles-rbac), including plan restrictions. Zapier documents [SAML SSO](https://help.zapier.com/hc/en-us/articles/8496279747085-Set-up-single-sign-on-with-SAML) on Team and Enterprise plans. Gumloop documents [Enterprise SSO, SAML, and SCIM](https://docs.gumloop.com/enterprise-features/sso_saml_scim) as well as [audit logging](https://docs.gumloop.com/enterprise-features/audit_logging). Ask each vendor to confirm current scope in writing. + +## Pricing model: task, operation, or credit — and what that means at scale + +Pricing units determine which workflow patterns become expensive, so compare the cost of a complete production run rather than the advertised monthly fee. Under task-based pricing, successful actions consume part of your allowance. Zapier's [pricing documentation](https://zapier.com/pricing) explains its task allowance and usage model. + +Credit-based pricing requires closer inspection because different nodes may consume different amounts. Make's [pricing page](https://www.make.com/en/pricing) says each module action in a scenario generally counts as one credit, while its AI documentation explains additional model-related usage. Gumloop's [credit documentation](https://docs.gumloop.com/core-concepts/credits) says agent costs vary with the model, tools, and run length. + +Self-hosting may reduce variable platform charges, but it adds infrastructure and operating costs. n8n prices its hosted and self-hosted offerings around workflow execution allowances; its [current pricing page](https://n8n.io/pricing/) explains that executions include unlimited steps. You still need to account for servers, monitoring, upgrades, backups, and staff time when operating a deployment yourself. + +Sim calculates usage from a base run charge, billable model usage, and hosted tool usage, as detailed in its [cost calculation documentation](https://docs.sim.ai/platform/costs). Verify current allowances and plan features on [Sim's pricing page](https://www.sim.ai/pricing). For every vendor, price a representative workflow with expected records, loops, retries, and AI-node choices. + +## Comparison table: Sim vs. Zapier vs. n8n vs. Make vs. Gumloop + +The table compares the five platforms across the six buying criteria. Plan availability and pricing can change, so verify each entry with the linked vendor documentation. + +| Criterion | [Sim](https://docs.sim.ai/introduction) | [Zapier](https://zapier.com/agents) | [n8n](https://docs.n8n.io/build/integrate-ai/langchain-in-n8n) | [Make](https://www.make.com/en/ai-agents) | [Gumloop](https://docs.gumloop.com/core-concepts/agents) | +| --- | --- | --- | --- | --- | --- | +| AI agents | [Native tool-using Agent blocks](https://docs.sim.ai/workflows/blocks/agent) | [Separate Agents product alongside Zaps](https://zapier.com/agents) | [Configurable LangChain agents, models, tools, and memory](https://docs.n8n.io/build/integrate-ai/langchain-in-n8n) | [AI Agents with visible reasoning and tool use](https://www.make.com/en/ai-agents) | [Tool-using, context-aware Agents](https://docs.gumloop.com/core-concepts/agents) | +| Orchestration | [Human review](https://docs.sim.ai/workflows/blocks/human-in-the-loop) plus branches, loops, and parallel runs | [Paths](https://help.zapier.com/hc/en-us/articles/8496288555917-Add-branching-logic-to-Zap-workflows-with-Paths), Looping, and Sub-Zaps | [Modular sub-workflows](https://docs.n8n.io/build/flow-logic/break-workflows-into-smaller-parts) plus flow-control nodes | [Routers](https://help.make.com/router), iterators, and aggregators | [Reusable subflows](https://docs.gumloop.com/core-concepts/subflows) and routing | +| Models | Multiple hosted providers and BYOK options | Model availability varies by Zapier product | [Multiple provider-specific model nodes](https://docs.n8n.io/build/integrate-ai/langchain-in-n8n) | [Make provider plus custom providers on eligible plans](https://help.make.com/credit-usage-for-ai-agents) | [Model catalog and presets](https://docs.gumloop.com/core-concepts/ai_models), with BYOK listed in pricing | +| Integrations | [1,000+ listed on the current pricing page](https://www.sim.ai/pricing), with MCP fallback | [Thousands listed on the current pricing page](https://zapier.com/pricing) | Connector coverage should be tested against the required operations | [3,000+ listed on the current pricing page](https://www.make.com/en/pricing) | Connector coverage should be tested against the required operations | +| Governance | [Permission groups, SSO, audit logs, and retention](https://docs.sim.ai/platform/enterprise), plus self-hosting options | [SAML SSO on eligible plans](https://help.zapier.com/hc/en-us/articles/8496279747085-Set-up-single-sign-on-with-SAML) | [Plan-dependent SSO](https://docs.n8n.io/administer/manage-users-and-access/verify-user-identity/use-saml) and RBAC, with self-hosting options | Verify plan-specific controls with the vendor | [Enterprise SSO and SCIM](https://docs.gumloop.com/enterprise-features/sso_saml_scim), model controls, and audit logs | +| Pricing | [Credits based on runs, model usage, and hosted tools](https://docs.sim.ai/platform/costs) | [Task-based allowances](https://zapier.com/pricing) | [Workflow execution allowances](https://n8n.io/pricing/) | [Credits based on module and AI usage](https://www.make.com/en/pricing) | [Variable credits based on models, tools, and run length](https://docs.gumloop.com/core-concepts/credits) | + +## Matching the checklist to your buying scenario + +If you are evaluating a platform's technical architecture, prioritize agent support and orchestration depth, then verify model flexibility against your provider and deployment requirements. During a demo, ask each vendor to build an agent that selects tools, switches models, handles a failed step, and pauses for human approval. + +If you own the operating process, start with integration depth and orchestration because those criteria determine whether the workflow can perform the required actions. Test a real process that crosses your core applications, then calculate its cost at the expected monthly volume. A large connector catalog offers little value if the required connectors lack the actions your process needs. + +If you approve enterprise software, verify security and governance before assessing integration coverage and pricing predictability. Ask vendors to demonstrate access controls, audit records, retention settings, and deployment options rather than accepting a security summary. + +Sim is a strong candidate when your evaluation prioritizes tool-using agents, model choice, and the documented enterprise governance controls above. Use the six criteria as rows in a vendor scorecard, weight them for your buying scenario, and require vendors to prove each score during the same demo workflow. diff --git a/apps/sim/content/library/why-no-code-ai-agents-need-live-web-access/index.mdx b/apps/sim/content/library/why-no-code-ai-agents-need-live-web-access/index.mdx new file mode 100644 index 00000000000..4d870a63143 --- /dev/null +++ b/apps/sim/content/library/why-no-code-ai-agents-need-live-web-access/index.mdx @@ -0,0 +1,118 @@ +--- +slug: why-no-code-ai-agents-need-live-web-access +title: 'Why No-Code AI Agents Need Live Web Access (And How to Wire It Up)' +description: 'How to pair a no-code AI agent builder with a dedicated live-web layer—TinyFish Search, Fetch, Browser, and Agent—and wire it into Sim workflows.' +date: 2026-09-11 +updated: 2026-09-11 +authors: + - andrew +readingTime: 9 +tags: [AI Agents, No-Code, Web Automation, TinyFish, Sim] +ogImage: /library/why-no-code-ai-agents-need-live-web-access/cover.jpg +canonical: https://www.sim.ai/library/why-no-code-ai-agents-need-live-web-access +draft: false +faq: + - q: "What happens when a site rate-limits or blocks a request?" + a: "Honor any Retry-After response, and use bounded exponential backoff when the site does not provide retry timing. If Fetch cannot retrieve a protected page, route the task through Browser or Agent for browser rendering and navigation. Keep a failure branch in the workflow so repeated blocks do not stall later steps." + - q: "Can I use Agent without calling Browser separately?" + a: "Yes. Agent manages multi-step navigation and extraction through its own API. Use Browser directly when you need explicit control over sessions, browser profiles, or individual page interactions." + - q: "How should I handle portal credentials in Vault?" + a: "Store credentials in Vault and reference the Vault item from the TinyFish block instead of placing secrets in prompts or workflow fields. Scope workflow and workspace access to the people and runs that need those credentials. Review your platform’s access and retention controls before using production accounts." + - q: "Do I need to migrate off my current no-code tool?" + a: "No. TinyFish provides a web access layer through an API key. You can call it from an existing builder through a native integration, an HTTP block, or custom code. The Sim integration (https://sim.ai/integrations/tinyfish) provides one working example." +--- + +## TL;DR + +- No-code AI agent builders handle prompts, branches, and integrations, but their workflows often depend on cached search results or brittle scrapers for web access. +- [TinyFish](https://www.tinyfish.ai/) supplies the missing live-web layer through Search, Fetch, Browser, and Agent. Together, these tools support browser-rendered discovery, content retrieval, protected or authenticated sites, and multi-step navigation. +- In TinyFish's published [Online-Mind2Web evaluation](https://www.tinyfish.ai/benchmarks), Agent achieved an 89.9% overall success rate. TinyFish describes the evaluation as [300 tasks across 136 live websites](https://www.tinyfish.ai/blog/mind2web). +- Sim's TinyFish integration shows how a no-code AI agent builder can connect visual workflow logic to a dedicated live-web layer. + +## The wiring problem no-code builders don't solve + +A no-code AI agent builder lets you compose workflows without writing the orchestration code. You can arrange prompts, branches, approval steps, and integrations without writing the orchestration code yourself. However, each workflow still depends on the web tools available to its underlying model. A polished visual flow cannot make a cached search result current or give a basic HTTP request an authenticated browser session. + +Cached search indexes create problems when a workflow depends on changing information. A pricing monitor may retrieve last week’s plan page, while an inventory check may report an item that has already sold out. Point-in-time scrapers can fail when a site renders content in the browser, changes its page structure, or loads data only after user interaction. + +Authenticated portals create a different failure mode. Without persistent cookies, stored credentials, and browser state, an agent sees the logged-out version of a supplier portal rather than its invoices or order records. Some portals return a normal response code with a login shell, so the workflow may continue without recognizing that extraction failed. + +Bot protection can stop retrieval before the model receives any useful content. Modern challenges inspect browser behavior and session signals that generic request tools often lack. Repeated retries rarely fix that mismatch because the site keeps rejecting the same type of client. + +Visual branches can respond to a known retrieval error, but they cannot recover data that the web tool never retrieved. A workflow may even finish successfully while working with stale search results or a logged-out page. Automations that rely on current public data or authenticated portals need a web-access layer that can retrieve that data before the workflow evaluates it. + +For a broader look at how these pieces fit together, see [AI agent orchestration frameworks explained](https://www.sim.ai/library/ai-agent-orchestration-frameworks-explained). + +## Live web access as a missing infrastructure layer + +You should provision workflow logic and live web access as separate infrastructure. A no-code builder handles decisions and application integrations. A dedicated web layer retrieves current pages through real browser sessions and converts them into usable data. + +When a builder delegates retrieval to the selected model or a basic scraper, it may work for stable public pages. It can break down when a workflow needs current data, an authenticated session, or access through bot controls. Purpose-built web infrastructure fills the capability gap without requiring each builder to operate its own browser platform. + +Separating the two layers also makes each one easier to change. You can replace a model or revise workflow logic without rebuilding browser access, and you can update browsing or extraction without moving the workflow. [TinyFish provides Search, Fetch, Browser, and Agent](https://docs.tinyfish.ai/) through one live-web platform, while the no-code builder remains responsible for orchestration and downstream actions. + +## TinyFish Search, Fetch, Browser, and Agent + +[One TinyFish API key](https://docs.tinyfish.ai/) gives a workflow four distinct ways to reach the live web. You choose the primitive based on whether the workflow needs discovery, extraction, direct browser control, or autonomous navigation. + +[Search](https://www.tinyfish.ai/search) finds current information from the live web and returns structured results. A no-code AI agent builder can pass titles, URLs, snippets, and metadata into later blocks without parsing a conventional results page. Search fits workflows that need to discover recent reviews, product listings, or newly published pages. + +[Fetch](https://www.tinyfish.ai/fetch) loads a known URL in a real browser and converts it into clean content. A workflow can send that output directly to a model for classification, summarization, or field extraction. Search and Fetch are free. Use them first for public pages that do not require interaction or authentication. + +[Browser](https://www.tinyfish.ai/browser) provides cloud browser sessions with standard CDP connections and anti-bot handling. Your workflow controls the browser when it needs to click interface elements, maintain a login, or inspect content unavailable in a basic HTTP response. Browser usage consumes TinyFish credits. + +[Agent](https://www.tinyfish.ai/agent) handles goals that require multiple browser actions and decisions. You provide the objective and desired output, and Agent navigates the site, adapts its next action, and extracts the requested data. TinyFish's published [Online-Mind2Web benchmark](https://www.tinyfish.ai/benchmarks) reports an 89.9% overall success rate. Online-Mind2Web measures multi-step tasks on live websites, making the evaluation relevant to workflows that navigate and extract data across several actions. Agent also consumes credits. + +## How pricing works: free APIs and usage credits + +According to [TinyFish pricing](https://www.tinyfish.ai/pricing), Search and Fetch consume zero credits, while Agent and Browser consume credits based on usage. TinyFish offers pay-as-you-go access as well as Starter, Pro, and custom options, so teams can choose between usage-based billing and a recurring credit allocation. + +Estimate spend by running a representative workflow against the sites you expect to access, recording its credit usage, and multiplying that amount by the scheduled run volume. Include retries and unusually long browser sessions in the estimate. Because page complexity, blocking, and task frequency affect usage, test representative workflows before setting a budget. + +## Wiring TinyFish into Sim + +Sim provides a working example of this two-layer setup. Its [TinyFish integration](https://sim.ai/integrations/tinyfish) adds live web capabilities through one workflow block. If you are new to visual agents, start with [how to create an AI agent](https://www.sim.ai/library/how-to-create-an-ai-agent) or compare the [best no-code AI agent builders](https://www.sim.ai/library/best-no-code-ai-agent-builders-2026). + +The block exposes nine tools that cover agent runs, web retrieval, Vault items, and browser profiles. Run Agent and Start Agent Run launch work, while Get Run, Cancel Run, and List Runs manage execution. Search finds current web results, and Fetch URLs retrieves page content. List Vault Items and List Browser Profiles expose the stored resources available to the workflow. + +### Adding the TinyFish block and authenticating + +Add the TinyFish block where the workflow first needs live web data. Create a TinyFish connection in Sim, paste the API key into the connection field, and save it. Run a simple Search or Fetch URLs call to confirm that Sim can authenticate and pass the returned JSON to the next block. + +Configure each TinyFish block to call one operation. Use Search for discovery and Fetch URLs for known pages. Run Agent handles work that can finish within the current execution. Longer jobs can use Start Agent Run, followed by Get Run to poll for completion. Cancel Run and List Runs provide control over active or previous jobs. + +Authenticated portal workflows should reference stored resources instead of placing credentials in prompts. Use List Vault Items to identify the stored credential resource required by the portal workflow. Use List Browser Profiles when the workflow needs a configured browser identity or session. Configure the agent run with the supported resource references, target URL, and extraction instructions. + +Map the block output into later Sim nodes after the call works in isolation. For example, a returned JSON object can feed a table step, while run status can control a branch that waits, retries, or reports an error. + +### Template: competitor pricing watch + +A weekly trigger in Sim starts the pricing check and passes each competitor URL to [TinyFish Agent](https://www.tinyfish.ai/agent). Agent navigates pricing pages, including pages that render prices with JavaScript or apply bot protection. The workflow returns each plan as a structured record with its price, billing period, included limits, and source URL. + +A storage step preserves the current records as a dated snapshot. On the next run, a comparison step loads the previous snapshot and matches plans by a stable identifier. It detects plan additions and removals, along with changes to prices or terms. Sim then posts the relevant differences to Slack with the old value, new value, and source page. + +[TinyFish Agent](https://www.tinyfish.ai/agent) reduces the workflow's dependence on fixed selectors by navigating the rendered page and extracting the requested fields. A renamed class or redesigned pricing table can stop a selector-based scraper, while Agent can navigate the rendered page and extract the requested pricing fields. Site changes can still require review, but they are less likely to break the workflow than a selector-only scraper. + +### Template: supplier portal collector + +A supplier portal collector uses [TinyFish Vault credentials](https://docs.tinyfish.ai/key-concepts/credentials) to reach invoice data that public scrapers cannot access. In Sim, configure the TinyFish block with Run Agent or Start Agent Run, then reference the Vault item containing the portal credentials. The agent opens the supplier portal in a browser session, signs in, and navigates to the outstanding invoices page. + +The extraction prompt should define a strict output schema for downstream steps. Ask the agent to return one JSON record per invoice with the invoice ID and amount, plus fields such as due date and payment status. A downstream Sim step can map those records into a table for reconciliation or approval. + +For longer portal sessions, [Start Agent Run](https://docs.tinyfish.ai/agent-api) lets Sim launch the task asynchronously. Get Run can check its status and retrieve the completed output. The asynchronous pattern lets Sim track a slow login or multi-page invoice task without holding one synchronous TinyFish call open until completion. + +### Template: review monitor + +A review monitor assigns discovery and retrieval to TinyFish, then sends the retrieved content to a model for classification. [TinyFish Search](https://www.tinyfish.ai/search) finds recent review pages and returns structured results with their URLs. The Sim workflow compares those URLs with its stored history, discards pages it has already processed, and sends each new URL to [Fetch](https://www.tinyfish.ai/fetch). Fetch converts the page into clean content so later steps do not need to parse navigation menus, ads, or raw HTML. + +A model step in Sim then classifies the retrieved review by sentiment and can extract details such as the product mentioned or the reason for a complaint. Sim can route the structured output into a table or Slack based on that classification. If Fetch cannot retrieve a page because the site requires interaction or blocks direct requests, route the URL to [TinyFish Agent](https://www.tinyfish.ai/agent) for browser-based navigation and extraction. Each TinyFish primitive handles one part of the pipeline, while Sim controls scheduling, state, and downstream actions. For more extraction patterns, see [the best AI agents for data extraction and RAG](https://www.sim.ai/library/best-ai-agents-for-data-extraction-and-rag-in-2026). + +## Pair any no-code framework with one live-web layer + +The Sim block is one working example, not a special case. Any builder that can call an API and consume structured output can pair with [TinyFish](https://docs.tinyfish.ai/) the same way, sending a query, URL, or task and getting back JSON its own nodes evaluate or store. + +You can swap the visual builder without rebuilding web access or add TinyFish to an existing workflow through a native integration, an HTTP block, or custom code. + +## Where to start + +Add a dedicated live-web layer when your workflow needs current data, authenticated sessions, or browser interaction. Try the TinyFish block in [Sim](https://sim.ai) for a visual setup, or connect a [TinyFish API key](https://docs.tinyfish.ai/) directly to your existing agent framework. Choose Search, Fetch, Browser, or Agent based on the pages your workflow must reach. diff --git a/apps/sim/ee/workspace-forking/lib/copy/copy-files.ts b/apps/sim/ee/workspace-forking/lib/copy/copy-files.ts index e5bbfd5069a..b2fc0597106 100644 --- a/apps/sim/ee/workspace-forking/lib/copy/copy-files.ts +++ b/apps/sim/ee/workspace-forking/lib/copy/copy-files.ts @@ -1,4 +1,5 @@ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { workspaceFileColumns, workspaceFiles } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' @@ -362,7 +363,7 @@ export async function executeForkFileBlobCopies( await db.transaction(async (tx) => { assertForkCopyActive(control) const [inserted] = await tx - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ id: task.targetFileId, key: task.targetKey, diff --git a/apps/sim/executor/handlers/generic/file-write.test.ts b/apps/sim/executor/handlers/generic/file-write.test.ts new file mode 100644 index 00000000000..27f78599226 --- /dev/null +++ b/apps/sim/executor/handlers/generic/file-write.test.ts @@ -0,0 +1,95 @@ +/** + * @vitest-environment node + */ +import { createExecutorContext, createSerializedBlock } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { fileManageWriteBodySchema } from '@/lib/api/contracts/tools/file' +import { FileV5Block } from '@/blocks/blocks/file' +import { getBlock } from '@/blocks/index' +import { GenericBlockHandler } from '@/executor/handlers/generic/generic-handler' +import { executeTool } from '@/tools' +import { fileWriteTool } from '@/tools/file/write' +import { getTool } from '@/tools/utils' + +vi.mock('@/blocks/index', () => ({ getBlock: vi.fn() })) +vi.mock('@/tools', () => ({ executeTool: vi.fn() })) +vi.mock('@/tools/utils', () => ({ getTool: vi.fn() })) + +const generatedFile = { + id: 'generated-file', + name: 'report.xlsx', + url: 'https://example.com/report.xlsx', + size: 16978, + type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', +} + +async function executeWrite(inputs: Record) { + const handler = new GenericBlockHandler() + await handler.execute( + createExecutorContext(), + createSerializedBlock({ type: 'file_v5', tool: 'file_write' }), + { operation: 'file_write', fileName: 'report.xlsx', ...inputs } + ) + const [, params] = vi.mocked(executeTool).mock.calls[0] + return fileManageWriteBodySchema.safeParse(fileWriteTool.operation.input(params)) +} + +describe('File Write executor inputs', () => { + beforeEach(() => { + vi.clearAllMocks() + vi.mocked(getBlock).mockReturnValue(FileV5Block) + vi.mocked(getTool).mockReturnValue(fileWriteTool) + vi.mocked(executeTool).mockResolvedValue({ success: true, output: {} }) + }) + + it.each([ + ['empty', { content: '' }], + ['null', { content: null }], + ['omitted', {}], + ])('clears %s Content when storing a generated file', async (_label, contentInput) => { + const result = await executeWrite({ ...contentInput, writeFileInput: generatedFile }) + + expect(result.success).toBe(true) + if (!result.success) throw result.error + expect(result.data.content).toBeUndefined() + expect(result.data.fileInput).toEqual(generatedFile) + }) + + it.each(['text', ' '])('rejects file and populated Content %j', async (content) => { + const result = await executeWrite({ content, writeFileInput: generatedFile }) + + expect(result.success).toBe(false) + if (result.success) throw new Error('Expected conflicting inputs to fail validation') + expect(result.error.issues).toEqual([ + expect.objectContaining({ + path: ['content'], + message: + 'Provide exactly one of content (text to write) or fileInput (an existing file to store).', + }), + ]) + }) + + it.each([0, false, { text: 'invalid' }, ['invalid']])( + 'rejects non-string Content %j alongside a file', + async (content) => { + const result = await executeWrite({ content, writeFileInput: generatedFile }) + + expect(result.success).toBe(false) + if (result.success) throw new Error('Expected malformed Content to fail validation') + expect(result.error.issues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ path: ['content'], code: 'invalid_type' }), + ]) + ) + } + ) + + it.each(['', 'text'])('preserves text-only Content %j', async (content) => { + const result = await executeWrite({ content }) + + expect(result.success).toBe(true) + if (!result.success) throw result.error + expect(result.data.content).toBe(content) + expect(result.data.fileInput).toBeUndefined() + }) +}) diff --git a/apps/sim/executor/utils/file-tool-processor.aliases.test.ts b/apps/sim/executor/utils/file-tool-processor.aliases.test.ts new file mode 100644 index 00000000000..040c338e00c --- /dev/null +++ b/apps/sim/executor/utils/file-tool-processor.aliases.test.ts @@ -0,0 +1,107 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it, vi } from 'vitest' +import type { UserFile } from '@/executor/types' +import type { ToolDefinition } from '@/tools/types' + +vi.mock('@/lib/internal/tool-operations/file-result.server', () => ({ + storeInternalToolFileResult: vi.fn(), +})) +vi.mock('@/lib/uploads/utils/file-utils.server', () => ({ downloadFileFromUrl: vi.fn() })) + +import { FileToolProcessor } from '@/executor/utils/file-tool-processor' + +const STORED_FILE: UserFile = { + id: 'file-1', + key: 'execution/file-1', + url: '/api/files/serve/execution/file-1', + name: 'notes.txt', + type: 'text/plain', + size: 5, + base64: 'aGVsbG8=', +} +const TOOL: ToolDefinition = { + id: 'alias-test', + name: 'Alias Test', + description: 'File aliases', + version: '1.0.0', + params: {}, + outputs: { file: { type: 'file', description: 'Stored file' } }, +} +const CONTEXT = { workflowId: 'workflow-1', executionId: 'execution-1', workspaceId: 'workspace-1' } + +describe('file output alias replacement', () => { + it('handles deeply nested, small JSON metadata without recursive stack growth', async () => { + const depth = 20_000 + const json = `${'{"child":'.repeat(depth)}null${'}'.repeat(depth)}` + const metadata: Record = JSON.parse(json) + const result = await FileToolProcessor.processToolOutputs( + { file: STORED_FILE, metadata }, + TOOL, + CONTEXT + ) + + expect(json.length).toBeLessThan(1024 * 1024) + expect(result.file).not.toHaveProperty('base64') + expect(result.metadata === metadata).toBe(false) + let cursor: unknown = result.metadata + let actualDepth = 0 + while (cursor && typeof cursor === 'object' && 'child' in cursor) { + actualDepth++ + cursor = cursor.child + } + expect(actualDepth).toBe(depth) + expect(cursor).toBeNull() + }) + + it('preserves cycles and shared aliases while replacing every reference to the file', async () => { + const shared = { file: STORED_FILE } + const input: Record = { + file: STORED_FILE, + messages: [shared, shared], + } + input.self = input + const result = await FileToolProcessor.processToolOutputs(input, TOOL, CONTEXT) + expect(result.self).toBe(result) + const messages = result.messages as Array<{ file: UserFile }> + expect(messages[0]).toBe(messages[1]) + expect(messages[0]?.file).toBe(result.file) + expect(result.file).not.toHaveProperty('base64') + expect(STORED_FILE.base64).toBe('aGVsbG8=') + expect(input.self).toBe(input) + }) + + it('keeps buffers, stored files, and non-plain objects as leaves', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024) + const date = new Date('2026-01-01') + const existing = { ...STORED_FILE, id: 'existing-file', base64: undefined } + const result = await FileToolProcessor.processToolOutputs( + { file: STORED_FILE, metadata: { buffer, date, existing } }, + TOOL, + CONTEXT + ) + const metadata = result.metadata as Record + expect(metadata.buffer).toBe(buffer) + expect(metadata.date).toBe(date) + expect(metadata.existing).toBe(existing) + }) + + it('preserves null prototypes and own __proto__ keys without modifying prototypes', async () => { + const metadata: Record = Object.create(null) + metadata.file = STORED_FILE + const keys = JSON.parse('{"__proto__":{"file":null}}') + keys.__proto__.file = STORED_FILE + const result = await FileToolProcessor.processToolOutputs( + { file: STORED_FILE, metadata, keys }, + TOOL, + CONTEXT + ) + expect(Object.getPrototypeOf(result.metadata)).toBeNull() + const copiedKeys = result.keys as Record + expect(Object.getPrototypeOf(copiedKeys)).toBe(Object.prototype) + expect(Object.hasOwn(copiedKeys, '__proto__')).toBe(true) + expect((copiedKeys.__proto__ as Record).file).toBe(result.file) + expect({}).not.toHaveProperty('file') + }) +}) diff --git a/apps/sim/executor/utils/file-tool-processor.context.test.ts b/apps/sim/executor/utils/file-tool-processor.context.test.ts new file mode 100644 index 00000000000..84d36d284a9 --- /dev/null +++ b/apps/sim/executor/utils/file-tool-processor.context.test.ts @@ -0,0 +1,167 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { InternalToolOperationContext } from '@/lib/internal/tool-operations/types' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' +import type { UserFile } from '@/executor/types' +import type { ToolConfig } from '@/tools/types' + +const mocks = vi.hoisted(() => ({ + download: vi.fn(), + uploadExecution: vi.fn(), + uploadCopilot: vi.fn(), + deleteFile: vi.fn(), + deleteMetadata: vi.fn(), +})) + +vi.mock('@/lib/uploads/utils/file-utils.server', () => ({ downloadFileFromUrl: mocks.download })) +vi.mock('@/lib/uploads/contexts/execution', () => ({ uploadExecutionFile: mocks.uploadExecution })) +vi.mock('@/lib/uploads/contexts/copilot', () => ({ uploadCopilotFile: mocks.uploadCopilot })) +vi.mock('@/lib/uploads/core/storage-service', () => ({ deleteFile: mocks.deleteFile })) +vi.mock('@/lib/uploads/server/metadata', () => ({ deleteFileMetadata: mocks.deleteMetadata })) + +import { FileToolProcessor } from '@/executor/utils/file-tool-processor' + +const context: InternalToolOperationContext = { + workflowId: '', + userId: 'actor-1', + workspaceId: 'workspace-1', + copilotToolExecution: true, +} +const tool = { + id: 'test_attachments', + name: 'Test attachments', + description: 'Downloads message attachments', + version: '1.0.0', + params: {}, + request: { url: 'https://example.com/messages', method: 'GET' }, + outputs: { files: { type: 'file[]' } }, +} satisfies ToolConfig + +const stored: UserFile = { + id: 'file-1', + key: 'copilot/actor-1/file-1/workbook.xlsx', + name: 'workbook.xlsx', + type: 'application/octet-stream', + size: 12 * 1024 * 1024, + url: 'https://storage.example/workbook.xlsx', + context: 'copilot', +} + +describe('file output processing across trusted contexts', () => { + beforeEach(() => { + vi.resetAllMocks() + mocks.uploadCopilot.mockResolvedValue(stored) + }) + + it('stores a large late attachment once and replaces every nested alias for Copilot', async () => { + const bytes = Buffer.alloc(12 * 1024 * 1024) + const attachment = { name: 'workbook.xlsx', contentType: stored.type, data: bytes } + const input = { files: [attachment, attachment], results: [{ attachments: [attachment] }] } + + const result = await FileToolProcessor.processToolOutputs(input, tool, context) + + expect(result).toEqual({ files: [stored, stored], results: [{ attachments: [stored] }] }) + expect((result.files as UserFile[])[0]).toBe(stored) + expect(mocks.uploadCopilot).toHaveBeenCalledOnce() + expect(mocks.uploadCopilot.mock.calls[0]?.[0].buffer).toBe(bytes) + expect(mocks.uploadCopilot.mock.calls[0]?.[0].userId).toBe('actor-1') + expect(mocks.uploadExecution).not.toHaveBeenCalled() + expect(JSON.stringify(result).length).toBeLessThan(2048) + expect(input.results[0]?.attachments[0]?.data).toBe(bytes) + }) + + it('rejects an aggregate over budget before uploading any file', async () => { + const first = Buffer.alloc(1) + Object.defineProperty(first, 'length', { value: MAX_FILE_SIZE }) + await expect( + FileToolProcessor.processToolOutputs( + { + files: [ + { name: 'first.bin', data: first }, + { name: 'second.bin', data: Buffer.alloc(1) }, + ], + }, + tool, + context + ) + ).rejects.toThrow('exceeds the maximum allowed size') + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('validates all files before creating storage objects', async () => { + await expect( + FileToolProcessor.processToolOutputs( + { + files: [ + { name: 'valid.txt', data: Buffer.from('valid') }, + { name: 'invalid.txt', data: '?' }, + ], + }, + tool, + context + ) + ).rejects.toThrow('invalid base64') + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('passes the remaining aggregate budget and cancellation signal to URL downloads', async () => { + const controller = new AbortController() + mocks.download.mockImplementation(async () => { + controller.abort(new Error('Download cancelled')) + return Buffer.alloc(1) + }) + await expect( + FileToolProcessor.processToolOutputs( + { + files: [ + { name: 'first.txt', data: Buffer.alloc(3) }, + { name: 'second.txt', url: 'https://example.com/file' }, + ], + }, + tool, + context, + controller.signal + ) + ).rejects.toThrow('Download cancelled') + expect(mocks.download).toHaveBeenCalledWith('https://example.com/file', { + userId: 'actor-1', + maxBytes: MAX_FILE_SIZE - 3, + signal: controller.signal, + }) + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('removes materialized base64 from existing references and their nested aliases', async () => { + const materialized = { ...stored, base64: 'c2VjcmV0' } + const result = await FileToolProcessor.processToolOutputs( + { files: [materialized], messages: [{ file: materialized }] }, + tool, + context + ) + expect(result).toEqual({ files: [stored], messages: [{ file: stored }] }) + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('rolls back an unpublished attachment if a later upload fails', async () => { + mocks.uploadCopilot + .mockResolvedValueOnce(stored) + .mockRejectedValueOnce(new Error('Storage down')) + await expect( + FileToolProcessor.processToolOutputs( + { + files: [ + { name: 'first.txt', data: Buffer.from('a') }, + { name: 'second.txt', data: Buffer.from('b') }, + ], + }, + tool, + context + ) + ).rejects.toThrow('Storage down') + expect(mocks.deleteFile).toHaveBeenCalledWith({ key: stored.key, context: 'copilot' }) + expect(mocks.deleteMetadata).toHaveBeenCalledWith(stored.key) + }) +}) diff --git a/apps/sim/executor/utils/file-tool-processor.test.ts b/apps/sim/executor/utils/file-tool-processor.test.ts index 549dbedf2a2..a32cee604bf 100644 --- a/apps/sim/executor/utils/file-tool-processor.test.ts +++ b/apps/sim/executor/utils/file-tool-processor.test.ts @@ -57,6 +57,28 @@ describe('FileToolProcessor', () => { } satisfies UserFile) }) + it('passes stored file descriptors through without downloading or uploading again', async () => { + const stored: UserFile = { + id: 'file-1', + key: 'execution/workspace-1/workflow-1/execution-1/file-1/workbook.xlsx', + name: 'workbook.xlsx', + size: 12 * 1024 * 1024, + type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', + url: 'https://storage.example/workbook.xlsx', + context: 'execution', + } + + const result = await FileToolProcessor.processToolOutputs( + { file: stored }, + toolConfig, + executionContext + ) + + expect(result.file).toBe(stored) + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + expect(mockDownloadFileFromUrl).not.toHaveBeenCalled() + }) + it('caps URL downloads and stores raster images using byte-derived metadata', async () => { const png = Buffer.concat([ Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]), diff --git a/apps/sim/executor/utils/file-tool-processor.ts b/apps/sim/executor/utils/file-tool-processor.ts index d66dea14763..e81a2134539 100644 --- a/apps/sim/executor/utils/file-tool-processor.ts +++ b/apps/sim/executor/utils/file-tool-processor.ts @@ -1,265 +1,219 @@ import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { omit } from '@sim/utils/object' import { isCanonicalBase64 } from '@/lib/api/contracts/primitives' -import { isUserFile } from '@/lib/core/utils/user-file' -import { uploadExecutionFile, uploadFileFromRawData } from '@/lib/uploads/contexts/execution' +import { isUserFile, type UserFileLike } from '@/lib/core/utils/user-file' +import { + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' +import { storeInternalToolFileResult } from '@/lib/internal/tool-operations/file-result.server' +import type { InternalToolOperationContext } from '@/lib/internal/tool-operations/types' import { downloadFileFromUrl } from '@/lib/uploads/utils/file-utils.server' -import { MAX_FILE_SIZE, sniffImageContentType } from '@/lib/uploads/utils/validation' -import type { ExecutionContext, UserFile } from '@/executor/types' -import type { ToolDefinition, ToolFileData } from '@/tools/types' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' +import type { UserFile } from '@/executor/types' +import type { ToolDefinition } from '@/tools/types' const logger = createLogger('FileToolProcessor') -const IMAGE_FILE_EXTENSIONS: Record = { - 'image/gif': 'gif', - 'image/jpeg': 'jpg', - 'image/png': 'png', - 'image/webp': 'webp', +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) } -/** - * Strip a base64 `data:` URI prefix, leaving the encoded payload. An empty payload is - * a legitimate zero-byte file; a payload that only looks empty after normalization is - * not, so callers compare against what this returns rather than the raw value. - */ +/** Strip a data URI prefix while preserving legitimate zero-byte payloads. */ function stripBase64DataUri(value: string): string { return /^data:[^,]*;base64,/i.test(value) ? value.slice(value.indexOf(',') + 1) : value } -/** - * Normalize a base64 payload to canonical RFC 4648 form so it can be validated: drop - * the line wrapping MIME encoders emit, translate the base64url alphabet, and restore - * the padding unpadded encoders omit. - */ +/** Normalize wrapped or unpadded base64url into canonical RFC 4648 form. */ function normalizeBase64(payload: string): string { const compact = payload.replace(/\s/g, '').replace(/-/g, '+').replace(/_/g, '/') const remainder = compact.length % 4 return remainder === 0 ? compact : compact + '='.repeat(4 - remainder) } -function assertFileSize(size: number, fileName: string): void { - if (size > MAX_FILE_SIZE) { - throw new Error(`File '${fileName}' exceeds the maximum allowed size of ${MAX_FILE_SIZE} bytes`) +function assertFileSize(size: number, name: string, remainingBytes: number): void { + if (size > remainingBytes) { + throw new Error(`File '${name}' exceeds the maximum allowed size of ${remainingBytes} bytes`) } } -function resolveStoredFileMetadata( - fileName: string, - declaredMimeType: string, - buffer: Buffer -): { fileName: string; mimeType: string } { - if (!declaredMimeType.startsWith('image/')) { - return { fileName, mimeType: declaredMimeType } - } - - const mimeType = sniffImageContentType(buffer) - if (!mimeType) { - return { - fileName: `${fileName.replace(/\.[^.]+$/, '')}.bin`, - mimeType: 'application/octet-stream', +/** Replaces aliases by identity, preserving message-to-file associations without inline bytes. */ +function replaceFileReferences( + value: unknown, + replacements: ReadonlyMap, + visited = new WeakMap() +): unknown { + type PendingCopy = + | { kind: 'array'; source: unknown[]; target: unknown[] } + | { kind: 'object'; source: object; target: Record } + const pending: PendingCopy[] = [] + + function copyOrReplace(item: unknown): unknown { + if (typeof item !== 'object' || item === null) return item + const replacement = replacements.get(item) + if (replacement) return replacement + if (isUserFile(item) || Buffer.isBuffer(item)) return item + if (visited.has(item)) return visited.get(item) + if (Array.isArray(item)) { + const target: unknown[] = [] + visited.set(item, target) + pending.push({ kind: 'array', source: item, target }) + return target } + const prototype = Object.getPrototypeOf(item) + if (prototype !== Object.prototype && prototype !== null) return item + const target: Record = Object.create(prototype) + visited.set(item, target) + pending.push({ kind: 'object', source: item, target }) + return target } - const extension = IMAGE_FILE_EXTENSIONS[mimeType] - return { - fileName: extension ? `${fileName.replace(/\.[^.]+$/, '')}.${extension}` : fileName, - mimeType, + const result = copyOrReplace(value) + while (pending.length > 0) { + const copy = pending.pop()! + if (copy.kind === 'array') { + for (const item of copy.source) copy.target.push(copyOrReplace(item)) + } else { + for (const [key, item] of Object.entries(copy.source)) { + Object.defineProperty(copy.target, key, { + value: copyOrReplace(item), + enumerable: true, + writable: true, + configurable: true, + }) + } + } } + return result } -/** - * Processes tool outputs and converts file-typed outputs to UserFile objects. - * This enables tools to return file data that gets automatically stored in the - * execution filesystem and made available as UserFile objects for workflow use. - */ +/** Stores declared file outputs once, for both workflow and Copilot callers. */ export class FileToolProcessor { - /** - * Process tool outputs and convert file-typed outputs to UserFile objects - */ static async processToolOutputs( - toolOutput: any, + toolOutput: Record, toolConfig: ToolDefinition, - executionContext: ExecutionContext - ): Promise { - if (!toolConfig.outputs) { - return toolOutput - } - - const processedOutput = { ...toolOutput } + context: InternalToolOperationContext, + signal?: AbortSignal + ): Promise> { + if (!toolConfig.outputs) return toolOutput + signal?.throwIfAborted() + const pendingFiles = new Map() + const replacements = new Map() + let remainingBytes = MAX_FILE_SIZE for (const [outputKey, outputDef] of Object.entries(toolConfig.outputs)) { - if (!FileToolProcessor.isFileOutput(outputDef.type)) { - continue - } - - const fileData = processedOutput[outputKey] - if (!fileData) { - logger.warn(`File-typed output '${outputKey}' is missing from tool result`) - continue - } - + if (outputDef.type !== 'file' && outputDef.type !== 'file[]') continue + const value = toolOutput[outputKey] + if (value === undefined || value === null) continue try { - processedOutput[outputKey] = await FileToolProcessor.processFileOutput( - fileData, - outputDef.type, - outputKey, - executionContext - ) + if (outputDef.type === 'file[]' && !Array.isArray(value)) { + throw new Error(`Output '${outputKey}' is marked as file[] but is not an array`) + } + const files = outputDef.type === 'file[]' && Array.isArray(value) ? value : [value] + for (const file of files) { + signal?.throwIfAborted() + if (!isRecord(file)) throw new Error('File output must be a file object') + if (isUserFile(file)) { + if (file.base64 !== undefined) replacements.set(file, omit(file, ['base64'])) + continue + } + if (pendingFiles.has(file)) continue + const buffered = await FileToolProcessor.readFile(file, context, remainingBytes, signal) + remainingBytes -= buffered.buffer.length + pendingFiles.set(file, buffered) + } } catch (error) { + signal?.throwIfAborted() logger.error(`Error processing file output '${outputKey}':`, error) - const errorMessage = toError(error).message - throw new Error(`Failed to process file output '${outputKey}': ${errorMessage}`) + throw new Error(`Failed to process file output '${outputKey}': ${toError(error).message}`) } } - return processedOutput - } - - /** - * Check if an output type is file-related - */ - private static isFileOutput(type: string): boolean { - return type === 'file' || type === 'file[]' - } - - /** - * Process a single file output (either single file or array of files) - */ - private static async processFileOutput( - fileData: any, - outputType: string, - outputKey: string, - executionContext: ExecutionContext - ): Promise { - if (outputType === 'file[]') { - return FileToolProcessor.processFileArray(fileData, outputKey, executionContext) - } - return FileToolProcessor.processFileData(fileData, executionContext) - } - - /** - * Process an array of files - */ - private static async processFileArray( - fileData: any, - outputKey: string, - executionContext: ExecutionContext - ): Promise { - if (!Array.isArray(fileData)) { - throw new Error(`Output '${outputKey}' is marked as file[] but is not an array`) + const originals = [...pendingFiles.keys()] + const present = (files: readonly UserFile[]) => { + originals.forEach((original, index) => { + replacements.set(original, files[index]!) + }) + if (replacements.size === 0) return toolOutput + const output = replaceFileReferences(toolOutput, replacements) + if (!isRecord(output)) throw new Error('Tool file output must be an object') + return output } - - const files: UserFile[] = [] - for (const file of fileData) { - files.push(await FileToolProcessor.processFileData(file, executionContext)) - } - return files + if (pendingFiles.size === 0) return present([]) + return storeInternalToolFileResult( + createInternalToolFilesResult([...pendingFiles.values()], present), + context, + (output) => { + if (!isRecord(output)) throw new Error('Tool file output must be an object') + return output + }, + signal + ) } - /** - * Convert various file data formats to UserFile by storing in execution filesystem. - * If the input is already a UserFile, returns it unchanged. - */ - private static async processFileData( - fileData: ToolFileData | UserFile, - context: ExecutionContext - ): Promise { - // If already a UserFile (e.g., from tools that handle their own file storage), - // return it directly without re-processing - if (isUserFile(fileData)) { - return fileData as UserFile + private static async readFile( + file: Record, + context: InternalToolOperationContext, + remainingBytes: number, + signal?: AbortSignal + ): Promise { + if (typeof file.name !== 'string' || !file.name.trim()) { + throw new Error('File output requires a filename') } - - const data = fileData as ToolFileData - try { - let buffer: Buffer | null = null - - if (Buffer.isBuffer(data.data)) { - assertFileSize(data.data.length, data.name) - buffer = data.data - } else if ( - data.data && - typeof data.data === 'object' && - 'type' in data.data && - 'data' in data.data - ) { - const serializedBuffer = data.data as { type: string; data: number[] } - if (serializedBuffer.type === 'Buffer' && Array.isArray(serializedBuffer.data)) { - assertFileSize(serializedBuffer.data.length, data.name) - buffer = Buffer.from(serializedBuffer.data) - } else { - throw new Error(`Invalid serialized buffer format for ${data.name}`) - } - } else if (typeof data.data === 'string') { - const payload = stripBase64DataUri(data.data) - const base64Data = normalizeBase64(payload) - - const paddingBytes = base64Data.endsWith('==') ? 2 : base64Data.endsWith('=') ? 1 : 0 - assertFileSize(Math.floor((base64Data.length * 3) / 4) - paddingBytes, data.name) - if (!isCanonicalBase64(base64Data) || (payload.length > 0 && base64Data.length === 0)) { - throw new Error(`File '${data.name}' has invalid base64 data`) - } - buffer = Buffer.from(base64Data, 'base64') + const name = file.name + const mimeType = + (typeof file.mimeType === 'string' && file.mimeType) || + (typeof file.contentType === 'string' && file.contentType) || + 'application/octet-stream' + let buffer: Buffer | undefined + const data = file.data + + if (Buffer.isBuffer(data)) { + assertFileSize(data.length, name, remainingBytes) + buffer = data + } else if (data instanceof ArrayBuffer || ArrayBuffer.isView(data)) { + assertFileSize(data.byteLength, name, remainingBytes) + buffer = + data instanceof ArrayBuffer + ? Buffer.from(data) + : Buffer.from(data.buffer, data.byteOffset, data.byteLength) + } else if (Array.isArray(data) || (isRecord(data) && data.type === 'Buffer')) { + const bytes = Array.isArray(data) ? data : data.data + if (!Array.isArray(bytes)) throw new Error(`Invalid serialized buffer format for ${name}`) + assertFileSize(bytes.length, name, remainingBytes) + if (!bytes.every((byte) => Number.isInteger(byte) && byte >= 0 && byte <= 255)) { + throw new Error(`Invalid serialized buffer format for ${name}`) } - - if ((!buffer || buffer.length === 0) && data.url) { - buffer = await downloadFileFromUrl(data.url, { - maxBytes: MAX_FILE_SIZE, - userId: context.userId, - }) - } - - if (buffer) { - assertFileSize(buffer.length, data.name) - const storedMetadata = resolveStoredFileMetadata(data.name, data.mimeType, buffer) - - return await uploadExecutionFile( - { - workspaceId: context.workspaceId || '', - workflowId: context.workflowId, - executionId: context.executionId || '', - }, - buffer, - storedMetadata.fileName, - storedMetadata.mimeType, - context.userId - ) - } - - if (!data.data) { - throw new Error( - `File data for '${data.name}' must have either 'data' (Buffer/base64) or 'url' property` - ) + buffer = Buffer.from(bytes) + } else if (typeof data === 'string') { + const payload = stripBase64DataUri(data) + const base64 = normalizeBase64(payload) + const padding = base64.endsWith('==') ? 2 : base64.endsWith('=') ? 1 : 0 + assertFileSize(Math.floor((base64.length * 3) / 4) - padding, name, remainingBytes) + if (!isCanonicalBase64(base64) || (payload.length > 0 && base64.length === 0)) { + throw new Error(`File '${name}' has invalid base64 data`) } + buffer = Buffer.from(base64, 'base64') + } - return uploadFileFromRawData( - { - name: data.name, - data: data.data, - mimeType: data.mimeType, - }, - { - workspaceId: context.workspaceId || '', - workflowId: context.workflowId, - executionId: context.executionId || '', - }, - context.userId - ) - } catch (error) { - logger.error(`Error processing file data for '${data.name}':`, error) - throw error + if ((!buffer || buffer.length === 0) && typeof file.url === 'string' && file.url) { + buffer = await downloadFileFromUrl(file.url, { + maxBytes: remainingBytes, + userId: context.userId, + ...(signal ? { signal } : {}), + }) + } + signal?.throwIfAborted() + if (!buffer) { + throw new Error(`File data for '${name}' must have either 'data' (Buffer/base64) or 'url'`) } + assertFileSize(buffer.length, name, remainingBytes) + return { buffer, name, mimeType } } - /** - * Check if a tool has any file-typed outputs - */ static hasFileOutputs(toolConfig: ToolDefinition): boolean { - if (!toolConfig.outputs) { - return false - } - - return Object.values(toolConfig.outputs).some( + return Object.values(toolConfig.outputs ?? {}).some( (output) => output.type === 'file' || output.type === 'file[]' ) } diff --git a/apps/sim/hooks/queries/mothership-chats.test.ts b/apps/sim/hooks/queries/mothership-chats.test.ts index 755367b1040..1953cc79fbc 100644 --- a/apps/sim/hooks/queries/mothership-chats.test.ts +++ b/apps/sim/hooks/queries/mothership-chats.test.ts @@ -2,10 +2,12 @@ * @vitest-environment node */ +import { sleep } from '@sim/utils/helpers' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import type { MothershipResource } from '@/lib/copilot/resources/types' -const { queryClient, suspendBrowserScope, suspendTerminalScope } = vi.hoisted(() => ({ +const { queryClient, suspendBrowserScope, suspendTerminalScope, clearChat } = vi.hoisted(() => ({ + clearChat: vi.fn(), queryClient: { cancelQueries: vi.fn().mockResolvedValue(undefined), invalidateQueries: vi.fn().mockResolvedValue(undefined), @@ -17,6 +19,10 @@ const { queryClient, suspendBrowserScope, suspendTerminalScope } = vi.hoisted(() suspendTerminalScope: vi.fn(async () => true), })) +vi.mock('@/stores/mothership-queue/store', () => ({ + useMothershipQueueStore: { getState: () => ({ clearChat }) }, +})) + vi.mock('@tanstack/react-query', () => ({ keepPreviousData: {}, queryOptions: (options: unknown) => options, @@ -232,10 +238,16 @@ describe('tasks query boundary parsing', () => { mutation.onSettled(undefined, new Error('delete failed'), 'chat-failed') expect(suspendBrowserScope).not.toHaveBeenCalled() expect(suspendTerminalScope).not.toHaveBeenCalled() + expect(clearChat).not.toHaveBeenCalled() + expect(queryClient.removeQueries).not.toHaveBeenCalled() await mutation.onSuccess(undefined, 'chat-deleted') expect(suspendBrowserScope).toHaveBeenCalledWith('chat-deleted') expect(suspendTerminalScope).toHaveBeenCalledWith('chat-deleted') + expect(clearChat).toHaveBeenCalledWith('chat-deleted') + expect(queryClient.removeQueries).toHaveBeenCalledWith({ + queryKey: ['mothership-chats', 'detail', 'chat-deleted'], + }) }) it('suspends every native resource group after a successful bulk delete', async () => { @@ -254,6 +266,44 @@ describe('tasks query boundary parsing', () => { expect(suspendTerminalScope).toHaveBeenCalledWith('chat-b') }) + it('waits for slower successful deletions before reconciling a failed batch', async () => { + const pending = Promise.withResolvers() + const mutation = useDeleteMothershipChats({ organizationId: 'org-1' }) as unknown as { + mutationFn: (chatIds: string[]) => Promise + onSettled: () => void + } + vi.mocked(fetch) + .mockResolvedValueOnce(new Response('delete failed', { status: 500 })) + .mockReturnValueOnce(pending.promise) + const result = mutation.mutationFn(['chat-failed', 'chat-slow']) + const reconciled = vi.fn() + const observed = result.then( + () => { + mutation.onSettled() + reconciled() + }, + () => { + mutation.onSettled() + reconciled() + } + ) + await sleep(1) + expect(fetch).toHaveBeenCalledTimes(2) + expect(reconciled).not.toHaveBeenCalled() + expect(queryClient.invalidateQueries).not.toHaveBeenCalled() + expect(clearChat).not.toHaveBeenCalled() + pending.resolve(jsonResponse({ success: true })) + await observed + await expect(result).rejects.toThrow() + expect(queryClient.invalidateQueries).toHaveBeenCalledExactlyOnceWith({ + queryKey: ['mothership-chats', 'list', 'organization', 'org-1'], + }) + expect(clearChat).toHaveBeenCalledExactlyOnceWith('chat-slow') + expect(queryClient.removeQueries).toHaveBeenCalledExactlyOnceWith({ + queryKey: ['mothership-chats', 'detail', 'chat-slow'], + }) + }) + it('suspends each successful bulk delete even when a sibling delete fails', async () => { const mutation = useDeleteMothershipChats('workspace-1') as unknown as { mutationFn: (chatIds: string[]) => Promise @@ -268,5 +318,13 @@ describe('tasks query boundary parsing', () => { expect(suspendTerminalScope).toHaveBeenCalledWith('chat-a') expect(suspendBrowserScope).not.toHaveBeenCalledWith('chat-b') expect(suspendTerminalScope).not.toHaveBeenCalledWith('chat-b') + expect(clearChat).toHaveBeenCalledWith('chat-a') + expect(clearChat).not.toHaveBeenCalledWith('chat-b') + expect(queryClient.removeQueries).toHaveBeenCalledWith({ + queryKey: ['mothership-chats', 'detail', 'chat-a'], + }) + expect(queryClient.removeQueries).not.toHaveBeenCalledWith({ + queryKey: ['mothership-chats', 'detail', 'chat-b'], + }) }) }) diff --git a/apps/sim/hooks/queries/mothership-chats.ts b/apps/sim/hooks/queries/mothership-chats.ts index dcaadc0f2ee..f076f468bce 100644 --- a/apps/sim/hooks/queries/mothership-chats.ts +++ b/apps/sim/hooks/queries/mothership-chats.ts @@ -273,7 +273,6 @@ export function useOrganizationMothershipChats( return data.data.map(mapChat) }, staleTime: MOTHERSHIP_CHAT_LIST_STALE_TIME, - refetchInterval: (query) => (query.state.data?.some((chat) => chat.isActive) ? 5_000 : false), }) } @@ -337,12 +336,12 @@ export function useDeleteMothershipChat(owner?: MothershipChatOwner) { mutationFn: deleteChat, onSuccess: async (_data, chatId) => { await suspendDesktopChatScopes(chatId) - }, - onSettled: (_data, _error, chatId) => { - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.ownerLists(owner) }) queryClient.removeQueries({ queryKey: mothershipChatKeys.detail(chatId) }) useMothershipQueueStore.getState().clearChat(chatId) }, + onSettled: () => { + queryClient.invalidateQueries({ queryKey: mothershipChatKeys.ownerLists(owner) }) + }, }) } @@ -374,24 +373,20 @@ export function useDeleteMothershipChats(owner?: MothershipChatOwner) { const queryClient = useQueryClient() return useMutation({ mutationFn: async (chatIds: string[]) => { - // Couple each successful DELETE to its own native suspension. If one - // sibling request fails, Promise.all rejects but the independently - // successful tasks still stop their pages and PTYs instead of being - // stranded live behind the aggregate onSuccess callback. - await Promise.all( + /** Reconcile only after every request settles, while cleaning up only deleted chats. */ + const results = await Promise.allSettled( chatIds.map(async (chatId) => { await deleteChat(chatId) await suspendDesktopChatScopes(chatId) + queryClient.removeQueries({ queryKey: mothershipChatKeys.detail(chatId) }) + useMothershipQueueStore.getState().clearChat(chatId) }) ) + const failed = results.find((result) => result.status === 'rejected') + if (failed) throw failed.reason }, - onSettled: (_data, _error, chatIds) => { + onSettled: () => { queryClient.invalidateQueries({ queryKey: mothershipChatKeys.ownerLists(owner) }) - const queueStore = useMothershipQueueStore.getState() - for (const chatId of chatIds) { - queryClient.removeQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - queueStore.clearChat(chatId) - } }, }) } diff --git a/apps/sim/hooks/queries/organization-logo.ts b/apps/sim/hooks/queries/organization-logo.ts new file mode 100644 index 00000000000..f3b15fd6251 --- /dev/null +++ b/apps/sim/hooks/queries/organization-logo.ts @@ -0,0 +1,20 @@ +import { useMutation, useQueryClient } from '@tanstack/react-query' +import { validateLogoFile } from '@/lib/uploads/client/logo-file' +import { uploadInternalFileSession } from '@/lib/uploads/client/session-upload' +import { organizationKeys } from '@/hooks/queries/utils/organization-keys' + +export function useUploadOrganizationLogo(organizationId: string) { + const queryClient = useQueryClient() + + return useMutation({ + mutationFn: (file: File) => { + const validationError = validateLogoFile(file) + if (validationError) throw new Error(validationError) + return uploadInternalFileSession({ purpose: 'organization_logo', organizationId, file }) + }, + onSuccess: () => { + queryClient.invalidateQueries({ queryKey: organizationKeys.detail(organizationId) }) + queryClient.invalidateQueries({ queryKey: organizationKeys.lists() }) + }, + }) +} diff --git a/apps/sim/hooks/use-mothership-chat-events-lifecycle.test.tsx b/apps/sim/hooks/use-mothership-chat-events-lifecycle.test.tsx new file mode 100644 index 00000000000..bcf078e3583 --- /dev/null +++ b/apps/sim/hooks/use-mothership-chat-events-lifecycle.test.tsx @@ -0,0 +1,115 @@ +/** @vitest-environment jsdom */ + +import { act } from 'react' +import { QueryClient, QueryClientProvider } from '@tanstack/react-query' +import { createRoot } from 'react-dom/client' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { MothershipChatOwner } from '@/hooks/queries/mothership-chats' +import { mothershipChatKeys } from '@/hooks/queries/mothership-chats' + +const { connect, close, deployment } = vi.hoisted(() => ({ + connect: vi.fn(), + close: vi.fn(), + deployment: { chatEnabled: true }, +})) +vi.mock('@/lib/events/rotating-event-source', () => ({ createRotatingEventSource: connect })) + +import { useMothershipChatEvents } from '@/hooks/use-mothership-chat-events' + +function EventSubscriber({ owner }: { owner: MothershipChatOwner | undefined }) { + useMothershipChatEvents(owner, deployment.chatEnabled) + return null +} + +function renderEvents(owner: MothershipChatOwner | undefined) { + const client = new QueryClient({ defaultOptions: { queries: { retry: false } } }) + const invalidate = vi.spyOn(client, 'invalidateQueries') + const container = document.createElement('div') + document.body.appendChild(container) + const root = createRoot(container) + const rerender = ({ owner }: { owner: MothershipChatOwner | undefined }) => { + act(() => + root.render( + + + + ) + ) + } + rerender({ owner }) + return { + invalidate, + client, + rerender, + unmount: () => { + act(() => root.unmount()) + container.remove() + }, + } +} + +describe('chat event subscription lifecycle', () => { + beforeEach(() => { + vi.clearAllMocks() + vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true) + deployment.chatEnabled = true + connect.mockReturnValue({ close }) + }) + afterEach(() => { + vi.restoreAllMocks() + vi.unstubAllGlobals() + }) + + it('subscribes once for a stable organization, including new owner objects on rerender', () => { + const view = renderEvents({ organizationId: 'org-lifecycle-1' }) + expect(connect).toHaveBeenCalledWith( + expect.objectContaining({ url: '/api/mothership/events?organizationId=org-lifecycle-1' }) + ) + view.rerender({ owner: { organizationId: 'org-lifecycle-1' } }) + expect(connect).toHaveBeenCalledTimes(1) + view.unmount() + expect(close).toHaveBeenCalledTimes(1) + view.client.clear() + }) + + it('reconciles missed changes on reconnect while leaving seamless rotation alone', () => { + const view = renderEvents({ organizationId: 'org-lifecycle-2' }) + const connection = connect.mock.calls[0][0] + act(() => connection.onOpen('initial')) + expect(view.invalidate).not.toHaveBeenCalled() + act(() => connection.onOpen('rotation')) + expect(view.invalidate).not.toHaveBeenCalled() + act(() => connection.onOpen('reconnect')) + expect(view.invalidate).toHaveBeenCalledExactlyOnceWith({ + queryKey: mothershipChatKeys.organizationLists('org-lifecycle-2'), + }) + view.unmount() + view.client.clear() + }) + + it('closes the old scope and reconciles when returning to a previously visited organization', () => { + const view = renderEvents({ organizationId: 'org-lifecycle-3' }) + view.rerender({ owner: 'ws-lifecycle-3' }) + expect(close).toHaveBeenCalledTimes(1) + expect(connect).toHaveBeenLastCalledWith( + expect.objectContaining({ url: '/api/mothership/events?workspaceId=ws-lifecycle-3' }) + ) + view.rerender({ owner: { organizationId: 'org-lifecycle-3' } }) + act(() => connect.mock.calls[2][0].onOpen('initial')) + expect(view.invalidate).toHaveBeenCalledExactlyOnceWith({ + queryKey: mothershipChatKeys.organizationLists('org-lifecycle-3'), + }) + view.unmount() + view.client.clear() + }) + + it('does not subscribe without an owner or when chat is disabled', () => { + const view = renderEvents(undefined) + expect(connect).not.toHaveBeenCalled() + deployment.chatEnabled = false + view.rerender({ owner: { organizationId: 'org-disabled' } }) + expect(connect).not.toHaveBeenCalled() + view.unmount() + view.client.clear() + }) +}) diff --git a/apps/sim/hooks/use-mothership-chat-events.test.ts b/apps/sim/hooks/use-mothership-chat-events.test.ts index fb01186b06c..4ccfdf50148 100644 --- a/apps/sim/hooks/use-mothership-chat-events.test.ts +++ b/apps/sim/hooks/use-mothership-chat-events.test.ts @@ -415,6 +415,27 @@ describe('handleMothershipChatStatusEvent', () => { expect(queryClient.removeQueries).not.toHaveBeenCalled() }) + it.each(['created', 'updated', 'renamed', 'started', 'completed', 'deleted'])( + 'invalidates only organization lists for organization %s events', + (type) => { + handleMothershipChatStatusEvent( + queryClient, + { organizationId: 'org-1' }, + { chatId: 'chat-1', type } + ) + expect(queryClient.invalidateQueries).toHaveBeenCalledWith({ + queryKey: mothershipChatKeys.organizationLists('org-1'), + }) + expect(queryClient.invalidateQueries).not.toHaveBeenCalledWith({ + queryKey: mothershipChatKeys.workspaceLists('org-1'), + }) + if (type === 'deleted') + expect(queryClient.removeQueries).toHaveBeenCalledWith({ + queryKey: mothershipChatKeys.detail('chat-1'), + }) + } + ) + it('does not invalidate when task event payload is invalid', () => { handleMothershipChatStatusEvent(queryClient, 'ws-1', '{') @@ -441,6 +462,13 @@ describe('resyncMothershipChatCaches', () => { }) }) + it('reconciles active and archived organization lists after reconnect', () => { + resyncMothershipChatCaches(queryClient, { organizationId: 'org-1' }) + expect(queryClient.invalidateQueries).toHaveBeenCalledExactlyOnceWith({ + queryKey: mothershipChatKeys.organizationLists('org-1'), + }) + }) + it('leaves chat details untouched so a mounted stream cannot be refetched mid-turn', () => { resyncMothershipChatCaches(queryClient, 'ws-1') diff --git a/apps/sim/hooks/use-mothership-chat-events.ts b/apps/sim/hooks/use-mothership-chat-events.ts index 2d1a9d91717..6d2a17ecda0 100644 --- a/apps/sim/hooks/use-mothership-chat-events.ts +++ b/apps/sim/hooks/use-mothership-chat-events.ts @@ -3,17 +3,27 @@ import { createLogger } from '@sim/logger' import type { QueryClient } from '@tanstack/react-query' import { useQueryClient } from '@tanstack/react-query' import { getLiveAssistantMessageId } from '@/lib/copilot/chat/effective-transcript' -import { useDeploymentShape } from '@/lib/core/config/deployment-shape' import { suspendDesktopChatScopes } from '@/lib/desktop/chat-scope' import { createRotatingEventSource } from '@/lib/events/rotating-event-source' -import { type MothershipChatHistory, mothershipChatKeys } from '@/hooks/queries/mothership-chats' +import { + type MothershipChatHistory, + type MothershipChatOwner, + mothershipChatKeys, +} from '@/hooks/queries/mothership-chats' const logger = createLogger('MothershipChatEvents') -/** Workspaces this process has subscribed to before, so a re-subscribe can be told from a first one. */ +/** Owner scopes this process subscribed to, so returning to a scope reconciles missed events. */ const everSubscribed = new Set() -const CHAT_STATUS_TYPES = ['started', 'completed', 'created', 'deleted', 'renamed'] as const +const CHAT_STATUS_TYPES = [ + 'started', + 'completed', + 'created', + 'deleted', + 'renamed', + 'updated', +] as const type ChatStatusEventType = (typeof CHAT_STATUS_TYPES)[number] const CHAT_STATUS_TYPE_SET = new Set(CHAT_STATUS_TYPES) @@ -98,7 +108,7 @@ function parseChatStatusEventPayload(data: unknown): ChatStatusEventPayload | nu export function handleMothershipChatStatusEvent( queryClient: Pick, - workspaceId: string, + owner: MothershipChatOwner, data: unknown ): void { const payload = parseChatStatusEventPayload(data) @@ -107,9 +117,8 @@ export function handleMothershipChatStatusEvent( return } - // workspaceLists covers both the active and archived (Recently Deleted) - // lists: delete/restore events move chats between the two scopes. - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.workspaceLists(workspaceId) }) + /** Delete and restore move chats between active and archived owner lists. */ + queryClient.invalidateQueries({ queryKey: mothershipChatKeys.ownerLists(owner) }) if (!payload.chatId) return if (payload.type === 'deleted') { // A task may be deleted from another window, browser, or device. Stop its @@ -149,9 +158,9 @@ export function handleMothershipChatStatusEvent( */ export function resyncMothershipChatCaches( queryClient: Pick, - workspaceId: string + owner: MothershipChatOwner ): void { - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.workspaceLists(workspaceId) }) + queryClient.invalidateQueries({ queryKey: mothershipChatKeys.ownerLists(owner) }) } /** @@ -162,38 +171,46 @@ export function resyncMothershipChatCaches( * without the guard every session would hold an open connection to an endpoint * that cannot serve it. */ -export function useMothershipChatEvents(workspaceId: string | undefined) { +export function useMothershipChatEvents( + owner: MothershipChatOwner | undefined, + chatEnabled: boolean +) { const queryClient = useQueryClient() - const { chatEnabled } = useDeploymentShape() + const workspaceId = typeof owner === 'string' ? owner : undefined + const organizationId = typeof owner === 'object' ? owner.organizationId : undefined useEffect(() => { - if (!workspaceId || !chatEnabled) return - - const isResubscribe = everSubscribed.has(workspaceId) - everSubscribed.add(workspaceId) + if ((!workspaceId && !organizationId) || !chatEnabled) return + + const eventOwner = organizationId ? { organizationId } : workspaceId! + const ownerParam = organizationId + ? `organizationId=${encodeURIComponent(organizationId)}` + : `workspaceId=${encodeURIComponent(workspaceId!)}` + const isResubscribe = everSubscribed.has(ownerParam) + everSubscribed.add(ownerParam) const connection = createRotatingEventSource({ - url: `/api/mothership/events?workspaceId=${encodeURIComponent(workspaceId)}`, + url: `/api/mothership/events?${ownerParam}`, events: { task_status: (event) => { handleMothershipChatStatusEvent( queryClient, - workspaceId, + eventOwner, event instanceof MessageEvent ? event.data : undefined ) }, }, onOpen: (reason) => { if (reason === 'reconnect' || (reason === 'initial' && isResubscribe)) { - resyncMothershipChatCaches(queryClient, workspaceId) + resyncMothershipChatCaches(queryClient, eventOwner) } }, onError: () => { - logger.warn(`SSE connection error for workspace ${workspaceId}`) + logger.warn('Chat status SSE connection error') }, }) return () => { connection.close() } - }, [workspaceId, queryClient, chatEnabled]) + }, [workspaceId, organizationId, queryClient, chatEnabled]) } diff --git a/apps/sim/hooks/use-workspace-order.ts b/apps/sim/hooks/use-workspace-order.ts new file mode 100644 index 00000000000..6d3438607ec --- /dev/null +++ b/apps/sim/hooks/use-workspace-order.ts @@ -0,0 +1,22 @@ +'use client' + +import { useMemo, useSyncExternalStore } from 'react' +import { WorkspaceRecencyStorage } from '@/lib/core/utils/browser-storage' +import type { Workspace } from '@/hooks/queries/workspace' + +const serverSnapshot = () => null + +/** Layers the viewer's pins and visit history over the server's newest-first list. */ +export function useWorkspaceOrder(workspaces: Workspace[], pinnedIds: ReadonlySet) { + const recencySnapshot = useSyncExternalStore( + WorkspaceRecencyStorage.subscribe, + WorkspaceRecencyStorage.getSnapshot, + serverSnapshot + ) + return useMemo(() => { + const byRecency = recencySnapshot + ? WorkspaceRecencyStorage.sortByRecency(workspaces) + : workspaces + return [...byRecency].sort((a, b) => Number(pinnedIds.has(b.id)) - Number(pinnedIds.has(a.id))) + }, [workspaces, pinnedIds, recencySnapshot]) +} diff --git a/apps/sim/instrumentation-node.ts b/apps/sim/instrumentation-node.ts index b35aa21c931..d9a3668d304 100644 --- a/apps/sim/instrumentation-node.ts +++ b/apps/sim/instrumentation-node.ts @@ -407,6 +407,6 @@ export async function register() { // Not awaited: the connection is warmed in the background so the first request // that needs Redis does not pay the handshake inside its own command deadline, // but boot never waits on Redis to serve requests that do not touch it. - const { warmRedisConnection } = await import('./lib/core/config/redis') + const { warmRedisConnection } = await import('@/lib/core/config/redis') void warmRedisConnection() } diff --git a/apps/sim/lib/admin/dashboard.ts b/apps/sim/lib/admin/dashboard.ts index 2ae89882bfa..b66d94f6ae1 100644 --- a/apps/sim/lib/admin/dashboard.ts +++ b/apps/sim/lib/admin/dashboard.ts @@ -1,5 +1,6 @@ import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { member, organization, @@ -11,6 +12,7 @@ import { usageLog, user, userStats, + userStatsColumns, workspace, } from '@sim/db/schema' import { generateId } from '@sim/utils/id' @@ -1524,7 +1526,7 @@ export async function grantDashboardUserBalance( ? null : getPerUserMinimumLimit(initialSubscription).toString() await tx - .insert(userStats) + .insert(withInsertColumns(userStats, userStatsColumns)) .values({ id: generateId(), userId, diff --git a/apps/sim/lib/api/contracts/mothership-chats.ts b/apps/sim/lib/api/contracts/mothership-chats.ts index 8320cb26be5..4c48e8b37b3 100644 --- a/apps/sim/lib/api/contracts/mothership-chats.ts +++ b/apps/sim/lib/api/contracts/mothership-chats.ts @@ -139,11 +139,7 @@ export const mothershipExecuteBodySchema = z.object({ }) export type MothershipExecuteBody = z.input -export const mothershipEventsQuerySchema = z - .object({ - workspaceId: z.string().optional(), - }) - .passthrough() +export const mothershipEventsQuerySchema = mothershipChatOwnerSchema export const mothershipChatGetQuerySchema = z .object({ diff --git a/apps/sim/lib/api/contracts/storage-transfer.ts b/apps/sim/lib/api/contracts/storage-transfer.ts index 099ced3fdbd..d584e03df74 100644 --- a/apps/sim/lib/api/contracts/storage-transfer.ts +++ b/apps/sim/lib/api/contracts/storage-transfer.ts @@ -157,6 +157,7 @@ export const storageContextSchema = z.enum([ 'og-images', 'logs', 'workspace-logos', + 'organization-logos', ]) export const fileParseBodySchema = z diff --git a/apps/sim/lib/api/contracts/upload-sessions.ts b/apps/sim/lib/api/contracts/upload-sessions.ts index a60749ac704..600d62ab6db 100644 --- a/apps/sim/lib/api/contracts/upload-sessions.ts +++ b/apps/sim/lib/api/contracts/upload-sessions.ts @@ -2,6 +2,7 @@ import { z } from 'zod' import { folderIdSchema, noInputSchema, + organizationIdSchema, workflowIdSchema, workspaceIdSchema, } from '@/lib/api/contracts/primitives' @@ -16,6 +17,10 @@ import { v2UploadTransferSchema, } from '@/lib/api/contracts/v2/uploads' import { executionIdSchema } from '@/lib/api/contracts/workflows' +import { + ASSISTANT_IMAGE_CONTENT_TYPES, + ASSISTANT_IMAGE_MAX_BYTES, +} from '@/lib/uploads/shared/assistant-images' import { MAX_WORKSPACE_FILE_SIZE, MAX_WORKSPACE_FORMDATA_FILE_SIZE, @@ -57,14 +62,43 @@ export const createInternalFileUploadBodySchema = z.discriminatedUnion('purpose' workspaceId: workspaceIdSchema, }) .strict(), + z + .object({ + purpose: z.literal('organization_logo'), + ...internalFileUploadBaseShape, + size: z.number().int().min(1).max(MAX_ASSET_FILE_SIZE), + organizationId: organizationIdSchema, + }) + .strict(), z .object({ purpose: z.literal('mothership_attachment'), ...internalFileUploadBaseShape, size: z.number().int().min(1).max(MAX_WORKSPACE_FILE_SIZE), - workspaceId: workspaceIdSchema, + workspaceId: workspaceIdSchema.optional(), + organizationId: organizationIdSchema.optional(), }) - .strict(), + .strict() + .superRefine((body, ctx) => { + if (Boolean(body.workspaceId) === Boolean(body.organizationId)) { + ctx.addIssue({ + code: 'custom', + path: ['workspaceId'], + message: 'Provide exactly one workspaceId or organizationId', + }) + } + if ( + body.organizationId && + (body.size > ASSISTANT_IMAGE_MAX_BYTES || + !ASSISTANT_IMAGE_CONTENT_TYPES.some((type) => type === body.contentType)) + ) { + ctx.addIssue({ + code: 'custom', + path: ['contentType'], + message: 'Assistant attachments must be PNG, JPEG, GIF, or WebP images up to 5 MB', + }) + } + }), z .object({ purpose: z.literal('execution_attachment'), @@ -138,6 +172,14 @@ export const internalFileUploadSessionSchema = z.discriminatedUnion('purpose', [ result: internalUploadedAssetSchema.nullable(), }) .strict(), + z + .object({ + ...internalFileUploadSessionBaseShape, + purpose: z.literal('organization_logo'), + size: z.number().int().positive(), + result: internalUploadedAssetSchema.nullable(), + }) + .strict(), z .object({ ...internalFileUploadSessionBaseShape, diff --git a/apps/sim/lib/auth/anonymous.ts b/apps/sim/lib/auth/anonymous.ts index c4be061bea0..465992f6ddd 100644 --- a/apps/sim/lib/auth/anonymous.ts +++ b/apps/sim/lib/auth/anonymous.ts @@ -1,4 +1,5 @@ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import * as schema from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' @@ -37,7 +38,7 @@ export async function ensureAnonymousUserExists(): Promise { }) if (!existingStats) { - await db.insert(schema.userStats).values({ + await db.insert(withInsertColumns(schema.userStats, schema.userStatsColumns)).values({ id: generateId(), userId: ANONYMOUS_USER_ID, currentUsageLimit: '10000000000', diff --git a/apps/sim/lib/billing/core/usage.ts b/apps/sim/lib/billing/core/usage.ts index 7038083ec79..1a9ad247645 100644 --- a/apps/sim/lib/billing/core/usage.ts +++ b/apps/sim/lib/billing/core/usage.ts @@ -1,4 +1,5 @@ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { member, organization, settings, user, userStats, userStatsColumns } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { isOrgAdminRole } from '@sim/platform-authz/workspace' @@ -155,7 +156,7 @@ export async function getOrgUsageLimit( */ export async function handleNewUser(userId: string): Promise { try { - await db.insert(userStats).values({ + await db.insert(withInsertColumns(userStats, userStatsColumns)).values({ id: generateId(), userId: userId, currentUsageLimit: getFreeTierLimit().toString(), @@ -182,7 +183,7 @@ export async function handleNewUser(userId: string): Promise { */ export async function ensureUserStatsExists(userId: string): Promise { await db - .insert(userStats) + .insert(withInsertColumns(userStats, userStatsColumns)) .values({ id: generateId(), userId: userId, diff --git a/apps/sim/lib/billing/enterprise-provisioning.ts b/apps/sim/lib/billing/enterprise-provisioning.ts index 6a5f1152037..50918ed50d6 100644 --- a/apps/sim/lib/billing/enterprise-provisioning.ts +++ b/apps/sim/lib/billing/enterprise-provisioning.ts @@ -1,10 +1,12 @@ import { AuditAction, AuditResourceType, recordAudit, recordAuditOnce } from '@sim/audit' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { invitation, invitationWorkspaceGrant, member, organization, + organizationColumns, outboxEvent, permissions, subscription, @@ -1638,7 +1640,7 @@ export async function issueEnterpriseProvisioning( if (organizationToCreate) { const now = new Date() - await tx.insert(organization).values({ + await tx.insert(withInsertColumns(organization, organizationColumns)).values({ id: organizationToCreate.id, name: organizationToCreate.name, slug: slugifyOrganizationName(organizationToCreate.name, organizationToCreate.id), diff --git a/apps/sim/lib/billing/organization.ts b/apps/sim/lib/billing/organization.ts index fc6219b250c..e1e3e3d2951 100644 --- a/apps/sim/lib/billing/organization.ts +++ b/apps/sim/lib/billing/organization.ts @@ -1,5 +1,12 @@ import { db } from '@sim/db' -import { member, organization, subscription as subscriptionTable, user } from '@sim/db/schema' +import { withInsertColumns } from '@sim/db/insert-columns' +import { + member, + organization, + organizationColumns, + subscription as subscriptionTable, + user, +} from '@sim/db/schema' import { createLogger } from '@sim/logger' import { isOrgAdminRole } from '@sim/platform-authz/workspace' import { generateId } from '@sim/utils/id' @@ -442,7 +449,7 @@ export async function ensureOrganizationForTeamSubscriptionTx( organizationId = `org_${generateId()}` const now = new Date() - await tx.insert(organization).values({ + await tx.insert(withInsertColumns(organization, organizationColumns)).values({ id: organizationId, name: userData.name || `${userData.email || 'User'}'s Team`, slug: `${userId}-team-${generateId()}` diff --git a/apps/sim/lib/billing/organizations/create-organization.ts b/apps/sim/lib/billing/organizations/create-organization.ts index 1947a333745..9fb27bffb74 100644 --- a/apps/sim/lib/billing/organizations/create-organization.ts +++ b/apps/sim/lib/billing/organizations/create-organization.ts @@ -1,5 +1,6 @@ import { db } from '@sim/db' -import { member, organization } from '@sim/db/schema' +import { withInsertColumns } from '@sim/db/insert-columns' +import { member, organization, organizationColumns } from '@sim/db/schema' import { generateId } from '@sim/utils/id' import { and, eq, ne } from 'drizzle-orm' import { acquireUserBillingIdentityLock } from '@/lib/billing/organizations/billing-identity-lock' @@ -100,7 +101,7 @@ export async function createOrganizationWithOwnerTx( throw new OrganizationSlugTakenError(slug) } - await tx.insert(organization).values({ + await tx.insert(withInsertColumns(organization, organizationColumns)).values({ id: organizationId, name, slug, diff --git a/apps/sim/lib/billing/organizations/membership.ts b/apps/sim/lib/billing/organizations/membership.ts index d4bd323b3cb..7687d32db23 100644 --- a/apps/sim/lib/billing/organizations/membership.ts +++ b/apps/sim/lib/billing/organizations/membership.ts @@ -6,6 +6,7 @@ */ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { account, credential, @@ -18,6 +19,7 @@ import { subscription as subscriptionTable, user, userStats, + userStatsColumns, workspace, workspaceFiles, } from '@sim/db/schema' @@ -1837,7 +1839,7 @@ export async function transferOrganizationOwnership( if (oldStats) { await tx - .insert(userStats) + .insert(withInsertColumns(userStats, userStatsColumns)) .values({ id: generateId(), userId: newOwnerUserId, diff --git a/apps/sim/lib/billing/storage/tracking.ts b/apps/sim/lib/billing/storage/tracking.ts index 7ae702a94c3..7f416fe0180 100644 --- a/apps/sim/lib/billing/storage/tracking.ts +++ b/apps/sim/lib/billing/storage/tracking.ts @@ -17,7 +17,8 @@ * writes any of them or deletes a locked row. */ -import { organization, userStats, workspace } from '@sim/db/schema' +import { withInsertColumns } from '@sim/db/insert-columns' +import { organization, userStats, userStatsColumns, workspace } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' import { isRecordLike } from '@sim/utils/object' @@ -631,7 +632,7 @@ export async function checkAndIncrementStorageUsageInTx( if (!orgScoped) { await tx - .insert(userStats) + .insert(withInsertColumns(userStats, userStatsColumns)) .values({ id: generateId(), userId, diff --git a/apps/sim/lib/consent/constants.ts b/apps/sim/lib/consent/constants.ts index b8a168f8a77..5a0f1e34950 100644 --- a/apps/sim/lib/consent/constants.ts +++ b/apps/sim/lib/consent/constants.ts @@ -14,6 +14,19 @@ * Sim's consent instance. Public by construction — the browser calls it * directly, so it is a client-visible origin like the GTM and GA container IDs * in the root layout, not a credential. + * + * The browser must keep calling it directly. c15t documents a same-origin + * rewrite (`/api/c15t/:path*`) as an optimization, and it would also sidestep + * the bot challenge this origin sometimes answers with — but it resolves the + * jurisdiction from the address the request arrives from, and proxying makes + * every visitor arrive from our servers. Measured against the live instance: + * `x-forwarded-for`, `x-real-ip`, `true-client-ip`, `cf-connecting-ip` and + * `x-vercel-ip-country` are all ignored, and only c15t's own `x-c15t-country` + * override is honored. Sim has no edge that supplies a country header for us to + * forward into it, so behind a proxy every visitor would resolve to our region + * and no one in the EU would be asked for consent at all. The same dependency + * rules out the SSR prefetch, which reads those headers through + * `extractRelevantHeaders`. Revisit only alongside an edge that provides geo. */ export const CONSENT_BACKEND_URL = 'https://sim-sim.inth.app' diff --git a/apps/sim/lib/copilot/chat-status.test.ts b/apps/sim/lib/copilot/chat-status.test.ts new file mode 100644 index 00000000000..1d559dc966a --- /dev/null +++ b/apps/sim/lib/copilot/chat-status.test.ts @@ -0,0 +1,54 @@ +/** @vitest-environment node */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { publish } = vi.hoisted(() => ({ publish: vi.fn() })) +vi.mock('@/lib/events/pubsub', () => ({ + createPubSubChannel: () => ({ publish, subscribe: vi.fn(), dispose: vi.fn() }), +})) + +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' + +describe('chat status ownership', () => { + beforeEach(() => vi.clearAllMocks()) + + it('preserves the workspace event shape', () => { + publishChatStatusChanged( + { workspaceId: 'ws-1', userId: 'user-1' }, + { chatId: 'chat-1', type: 'renamed' } + ) + expect(publish).toHaveBeenCalledWith({ workspaceId: 'ws-1', chatId: 'chat-1', type: 'renamed' }) + }) + + it.each(['created', 'updated', 'renamed', 'deleted', 'started', 'completed'] as const)( + 'binds %s events to the organization and private chat owner', + (type) => { + publishChatStatusChanged( + { organizationId: 'org-1', userId: 'user-1' }, + { chatId: 'chat-1', type } + ) + expect(publish).toHaveBeenCalledWith({ + organizationId: 'org-1', + userId: 'user-1', + chatId: 'chat-1', + type, + }) + } + ) + + it('does not broadcast an organization event without its private owner', () => { + expect(() => + publishChatStatusChanged({ organizationId: 'org-1' }, { chatId: 'chat-1', type: 'created' }) + ).toThrow('Invalid organization chat owner') + expect(publish).not.toHaveBeenCalled() + }) + + it('refuses ambiguous workspace and organization ownership', () => { + expect(() => + publishChatStatusChanged( + { workspaceId: 'ws-1', organizationId: 'org-1', userId: 'user-1' }, + { chatId: 'chat-1', type: 'created' } + ) + ).toThrow('Invalid organization chat owner') + expect(publish).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/copilot/chat-status.ts b/apps/sim/lib/copilot/chat-status.ts index 221eb20ac9f..3fb5edf163c 100644 --- a/apps/sim/lib/copilot/chat-status.ts +++ b/apps/sim/lib/copilot/chat-status.ts @@ -11,10 +11,13 @@ import { createPubSubChannel, type PubSubChannel } from '@/lib/events/pubsub' -interface ChatStatusEvent { - workspaceId: string +export type ChatStatusOwner = + | { workspaceId: string; organizationId?: never; userId?: never } + | { organizationId: string; userId: string; workspaceId?: never } + +export type ChatStatusEvent = ChatStatusOwner & { chatId: string - type: 'started' | 'completed' | 'created' | 'deleted' | 'renamed' + type: 'started' | 'completed' | 'created' | 'deleted' | 'renamed' | 'updated' streamId?: string } @@ -40,3 +43,20 @@ export const chatPubSub = channel dispose: () => channel.dispose(), } : null + +/** Projects canonical chat ownership into the same status channel for both surfaces. */ +export function publishChatStatusChanged( + chat: { workspaceId?: string | null; organizationId?: string | null; userId?: string | null }, + event: Pick +): void { + if (chat.organizationId) { + if (!chat.userId || chat.workspaceId) throw new Error('Invalid organization chat owner') + chatPubSub?.publishStatusChanged({ + organizationId: chat.organizationId, + userId: chat.userId, + ...event, + }) + } else if (chat.workspaceId) { + chatPubSub?.publishStatusChanged({ workspaceId: chat.workspaceId, ...event }) + } +} diff --git a/apps/sim/lib/copilot/chat/assistant-images.test.ts b/apps/sim/lib/copilot/chat/assistant-images.test.ts new file mode 100644 index 00000000000..5a11b4905de --- /dev/null +++ b/apps/sim/lib/copilot/chat/assistant-images.test.ts @@ -0,0 +1,104 @@ +/** @vitest-environment node */ +import type { SessionPrincipal } from '@sim/auth/principal' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import type { readOrganizationAssistantImage } from '@/lib/uploads/contexts/organization-assistant/application' +import { ASSISTANT_IMAGE_MAX_COUNT } from '@/lib/uploads/shared/assistant-images' + +const { readImage } = vi.hoisted(() => ({ + readImage: vi.fn(), +})) +vi.mock('@/lib/uploads/contexts/organization-assistant/application', () => ({ + readOrganizationAssistantImage: readImage, +})) + +import { prepareAssistantImages } from '@/lib/copilot/chat/assistant-images' +import { getMothershipAttachmentPreviewUrl } from '@/lib/copilot/chat/attachment-preview' + +const principal: SessionPrincipal = { kind: 'session', userId: 'user-1', sessionId: 'session-1' } +const key = 'assistant/org-1/user-1/upload-1/image.png' +const image = { + id: 'upload-1', + key, + name: 'image.png', + contentType: 'image/png', + size: 5, + buffer: Buffer.from('image'), +} + +describe('Assistant image preparation', () => { + beforeEach(() => { + vi.clearAllMocks() + readImage.mockResolvedValue(image) + }) + + it('uses canonical metadata and model content from the authorized reader', async () => { + const signal = new AbortController().signal + const result = await prepareAssistantImages({ + principal, + organizationId: 'org-1', + attachments: [{ key }], + signal, + }) + expect(readImage).toHaveBeenCalledWith({ principal, organizationId: 'org-1', key, signal }) + expect(result).toEqual({ + attachments: [ + { id: 'upload-1', key, filename: 'image.png', media_type: 'image/png', size: 5 }, + ], + content: [ + { + type: 'image', + filename: 'image.png', + source: { type: 'base64', media_type: 'image/png', data: 'aW1hZ2U=' }, + }, + ], + }) + expect(getMothershipAttachmentPreviewUrl(result.attachments[0])).toBe( + `/api/files/serve/${encodeURIComponent(key)}?context=mothership&preview=1` + ) + }) + + it('rejects an oversized batch before reading any image', async () => { + await expect( + prepareAssistantImages({ + principal, + organizationId: 'org-1', + attachments: Array.from({ length: ASSISTANT_IMAGE_MAX_COUNT + 1 }, () => ({ key })), + }) + ).rejects.toMatchObject({ code: 'validation' }) + expect(readImage).not.toHaveBeenCalled() + }) + + it('fails the entire turn if any image is inaccessible', async () => { + readImage.mockRejectedValueOnce(new OrchestrationError('not_found', 'Image not found')) + await expect( + prepareAssistantImages({ + principal, + organizationId: 'org-1', + attachments: [{ key }, { key: 'another-image' }], + }) + ).rejects.toMatchObject({ code: 'not_found' }) + expect(readImage).toHaveBeenCalledOnce() + }) + + it('rejects non-image content even if an upstream reader returns it', async () => { + readImage.mockResolvedValueOnce({ ...image, contentType: 'application/pdf' }) + await expect( + prepareAssistantImages({ principal, organizationId: 'org-1', attachments: [{ key }] }) + ).rejects.toMatchObject({ code: 'validation' }) + }) + + it('does not read images after the request is aborted', async () => { + const controller = new AbortController() + controller.abort() + await expect( + prepareAssistantImages({ + principal, + organizationId: 'org-1', + attachments: [{ key }], + signal: controller.signal, + }) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(readImage).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/copilot/chat/assistant-images.ts b/apps/sim/lib/copilot/chat/assistant-images.ts new file mode 100644 index 00000000000..c2f6f643743 --- /dev/null +++ b/apps/sim/lib/copilot/chat/assistant-images.ts @@ -0,0 +1,68 @@ +import type { SessionPrincipal } from '@sim/auth/principal' +import type { PersistedFileAttachment } from '@/lib/copilot/chat/persisted-message' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { readOrganizationAssistantImage } from '@/lib/uploads/contexts/organization-assistant/application' +import { + ASSISTANT_IMAGE_MAX_COUNT, + ASSISTANT_IMAGE_MAX_TOTAL_BYTES, +} from '@/lib/uploads/shared/assistant-images' +import { createFileContent, type MessageContent } from '@/lib/uploads/utils/file-utils' + +export interface AssistantImageContent extends MessageContent { + type: 'image' + filename: string +} + +interface PreparedAssistantImages { + attachments: PersistedFileAttachment[] + content: AssistantImageContent[] +} + +/** Resolves private uploads before any attachment metadata or bytes enter a chat turn. */ +export async function prepareAssistantImages({ + principal, + organizationId, + attachments, + signal, +}: { + principal: SessionPrincipal + organizationId: string + attachments: readonly { key: string }[] + signal?: AbortSignal +}): Promise { + if (attachments.length > ASSISTANT_IMAGE_MAX_COUNT) { + throw new OrchestrationError( + 'validation', + `Attach up to ${ASSISTANT_IMAGE_MAX_COUNT} images per message` + ) + } + + const prepared: PreparedAssistantImages = { attachments: [], content: [] } + let totalBytes = 0 + for (const attachment of attachments) { + signal?.throwIfAborted() + const image = await readOrganizationAssistantImage({ + principal, + organizationId, + key: attachment.key, + signal, + }) + totalBytes += image.buffer.length + if (totalBytes > ASSISTANT_IMAGE_MAX_TOTAL_BYTES) { + throw new OrchestrationError('payload_too_large', 'Attached images are too large') + } + const content = createFileContent(image.buffer, image.contentType) + if (content?.type !== 'image') { + throw new OrchestrationError('validation', 'Assistant attachments must be supported images') + } + prepared.attachments.push({ + id: image.id, + key: image.key, + filename: image.name, + media_type: image.contentType, + size: image.size, + }) + prepared.content.push({ ...content, type: 'image', filename: image.name }) + } + return prepared +} diff --git a/apps/sim/lib/copilot/chat/fork-chat-files.ts b/apps/sim/lib/copilot/chat/fork-chat-files.ts index 719b77d867e..084bc682cf5 100644 --- a/apps/sim/lib/copilot/chat/fork-chat-files.ts +++ b/apps/sim/lib/copilot/chat/fork-chat-files.ts @@ -1,3 +1,4 @@ +import { withInsertColumns } from '@sim/db/insert-columns' import { type WorkspaceFileRow, workspaceFileColumns, workspaceFiles } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' @@ -145,7 +146,7 @@ export async function planChatFileCopies(params: { // Ids and keys are generated client-side, so one multi-row insert suffices — // no per-row round trips while the fork transaction is held open. if (copyRows.length > 0) { - await tx.insert(workspaceFiles).values(copyRows) + await tx.insert(withInsertColumns(workspaceFiles, workspaceFileColumns)).values(copyRows) for (const source of rows) { const targetId = idMap.get(source.id) if (!targetId) continue diff --git a/apps/sim/lib/copilot/chat/organization-chats.test.ts b/apps/sim/lib/copilot/chat/organization-chats.test.ts index 72194d9cec4..f5540d89d0e 100644 --- a/apps/sim/lib/copilot/chat/organization-chats.test.ts +++ b/apps/sim/lib/copilot/chat/organization-chats.test.ts @@ -2,10 +2,22 @@ import { dbChainMockFns, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' import { createTrustedOrganizationCopilotPrincipal } from '@/lib/copilot/auth/application-delegation' -import { authorizeOrganizationChatDelegation } from '@/lib/copilot/chat/organization-chats' +import { + authorizeOrganizationChatDelegation, + authorizeOrganizationChatEvents, + createOrganizationChat, +} from '@/lib/copilot/chat/organization-chats' import { OrchestrationError } from '@/lib/core/orchestration/types' -const { authorize } = vi.hoisted(() => ({ authorize: vi.fn() })) +const { authorize, requireSearch, publish } = vi.hoisted(() => ({ + authorize: vi.fn(), + requireSearch: vi.fn(), + publish: vi.fn(), +})) +vi.mock('@/lib/knowledge/access/availability', () => ({ + requireOrganizationSearchAvailable: requireSearch, +})) +vi.mock('@/lib/copilot/chat-status', () => ({ publishChatStatusChanged: publish })) vi.mock('@/lib/core/application/organization-authorization', () => ({ authorizeOrganizationOperation: authorize, })) @@ -64,3 +76,62 @@ describe('private organization chat delegation', () => { expect(authorize).not.toHaveBeenCalled() }) }) + +describe('organization chat events application boundary', () => { + const principal = { kind: 'session', userId: 'member-1', sessionId: 'session-1' } as const + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + authorize.mockResolvedValue({ userId: 'member-1', organizationId: 'org-1', role: 'member' }) + requireSearch.mockResolvedValue(undefined) + }) + + it('authorizes current membership before reading the feature rollout', async () => { + await authorizeOrganizationChatEvents.execute({ principal, input: { organizationId: 'org-1' } }) + expect(authorize).toHaveBeenCalledWith( + principal, + expect.objectContaining({ + id: 'organization.chats.subscribe', + principalKinds: ['session'], + minimumRole: 'member', + capability: 'copilot.use', + }), + { organizationId: 'org-1' } + ) + expect(requireSearch).toHaveBeenCalledWith('org-1') + expect(authorize.mock.invocationCallOrder[0]).toBeLessThan( + requireSearch.mock.invocationCallOrder[0] + ) + }) + + it('does not examine rollout state for a non-member', async () => { + authorize.mockRejectedValueOnce(new OrchestrationError('not_found', 'Organization not found')) + await expect( + authorizeOrganizationChatEvents.execute({ principal, input: { organizationId: 'org-1' } }) + ).rejects.toThrow('Organization not found') + expect(requireSearch).not.toHaveBeenCalled() + }) + + it('propagates rollout revocation and infrastructure failures', async () => { + requireSearch.mockRejectedValueOnce(new OrchestrationError('forbidden', 'Search is disabled')) + await expect( + authorizeOrganizationChatEvents.execute({ principal, input: { organizationId: 'org-1' } }) + ).rejects.toThrow('Search is disabled') + authorize.mockRejectedValueOnce(new Error('database unavailable')) + await expect( + authorizeOrganizationChatEvents.execute({ principal, input: { organizationId: 'org-1' } }) + ).rejects.toThrow('database unavailable') + }) + + it('publishes newly created chats only after persistence and under the canonical owner', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([{ id: 'new-chat' }]) + await createOrganizationChat.execute({ principal, input: { organizationId: 'org-1' } }) + expect(publish).toHaveBeenCalledWith( + { organizationId: 'org-1', userId: 'member-1', role: 'member' }, + { chatId: 'new-chat', type: 'created' } + ) + expect(dbChainMockFns.returning.mock.invocationCallOrder[0]).toBeLessThan( + publish.mock.invocationCallOrder[0] + ) + }) +}) diff --git a/apps/sim/lib/copilot/chat/organization-chats.ts b/apps/sim/lib/copilot/chat/organization-chats.ts index 4cb3d7dcb3c..242f3ab68c8 100644 --- a/apps/sim/lib/copilot/chat/organization-chats.ts +++ b/apps/sim/lib/copilot/chat/organization-chats.ts @@ -4,12 +4,20 @@ import { copilotChats } from '@sim/db/schema' import { and, eq, isNull } from 'drizzle-orm' import type { MothershipChatScope } from '@/lib/api/contracts/mothership-chats' import { listMothershipChats } from '@/lib/copilot/chat/list-mothership-chats' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { MOTHERSHIP_CHAT_DEFAULT_MODEL } from '@/lib/copilot/constants' import { authorizeOrganizationOperation } from '@/lib/core/application/organization-authorization' import { defineOrganizationOperation } from '@/lib/core/application/organization-operation' import { OrchestrationError } from '@/lib/core/orchestration/types' +import { requireOrganizationSearchAvailable } from '@/lib/knowledge/access/availability' export const organizationChatOperations = { + subscribe: defineOrganizationOperation({ + id: 'organization.chats.subscribe', + minimumRole: 'member', + principalKinds: ['session'], + capability: 'copilot.use', + }), read: defineOrganizationOperation({ id: 'organization.chats.read', minimumRole: 'member', @@ -42,6 +50,20 @@ export const authorizeOrganizationChat = { }, } +/** Revalidates the organization surface's rollout and private-chat membership for live updates. */ +export const authorizeOrganizationChatEvents = { + operation: organizationChatOperations.subscribe, + async execute({ principal, input }: { principal: Principal; input: OrganizationChatInput }) { + const context = await authorizeOrganizationOperation( + principal, + organizationChatOperations.subscribe, + input + ) + await requireOrganizationSearchAvailable(context.organizationId) + return context + }, +} + export const listOrganizationChats = { operation: organizationChatOperations.list, async execute({ @@ -83,6 +105,7 @@ export const createOrganizationChat = { }) .returning({ id: copilotChats.id }) if (!chat) throw new Error('Failed to create organization conversation') + publishChatStatusChanged(context, { chatId: chat.id, type: 'created' }) return chat }, } diff --git a/apps/sim/lib/copilot/chat/payload.test.ts b/apps/sim/lib/copilot/chat/payload.test.ts index 5c504effebe..93a86f0b825 100644 --- a/apps/sim/lib/copilot/chat/payload.test.ts +++ b/apps/sim/lib/copilot/chat/payload.test.ts @@ -647,6 +647,33 @@ describe('Assistant payload', () => { mockIsIntegrationDeploymentAvailable.mockReturnValue(true) mockCreateUserToolSchema.mockReturnValue({ type: 'object', properties: {} }) }) + it('sends prepared organization images as model-readable attachments without workspace tracking', async () => { + mockTrackChatUpload.mockClear() + const image = { + type: 'image' as const, + filename: 'image.png', + source: { type: 'base64' as const, media_type: 'image/png', data: 'aW1hZ2U=' }, + } + const payload = await buildCopilotRequestPayload( + { + message: '', + userId: 'user-1', + userMessageId: 'message-1', + organizationId: 'org-1', + mode: 'assistant', + model: '', + assistantImages: [image], + fileAttachments: [{ id: 'image', key: 'private-upload', size: 5 }], + }, + { selectedModel: '' } + ) + expect(payload.message).toBe('') + expect(payload.fileAttachments).toEqual([image]) + expect(payload).not.toHaveProperty('context') + expect(payload).not.toHaveProperty('workspaceId') + expect(mockTrackChatUpload).not.toHaveBeenCalled() + }) + it('forwards organization scope without workspace, integration, or desktop authority', async () => { const payload = await buildCopilotRequestPayload( { diff --git a/apps/sim/lib/copilot/chat/payload.ts b/apps/sim/lib/copilot/chat/payload.ts index ff76023488c..35a48a938a7 100644 --- a/apps/sim/lib/copilot/chat/payload.ts +++ b/apps/sim/lib/copilot/chat/payload.ts @@ -10,6 +10,7 @@ import { isAssistantIntegrationTool, } from '@/lib/copilot/assistant/tool-policy' import { getBlockVisibilityForCopilot, visibilitySignature } from '@/lib/copilot/block-visibility' +import type { AssistantImageContent } from '@/lib/copilot/chat/assistant-images' import type { VfsSnapshotV1 } from '@/lib/copilot/generated/vfs-snapshot-v1' import { type IntegrationGateConfig, @@ -52,6 +53,7 @@ interface BuildPayloadParams { */ mcpServerIds?: string[] fileAttachments?: Array<{ id: string; key: string; size: number; [key: string]: unknown }> + assistantImages?: AssistantImageContent[] commands?: string[] chatId?: string prefetch?: boolean @@ -412,6 +414,9 @@ export async function buildCopilotRequestPayload( ...(provider ? { provider } : {}), mode: transportMode, ...(isAssistant && params.assistantSearch ? { assistantSearch: params.assistantSearch } : {}), + ...(isAssistant && params.organizationId && params.assistantImages?.length + ? { fileAttachments: params.assistantImages } + : {}), messageId: userMessageId, ...(allContexts.length > 0 ? { context: allContexts } : {}), ...(chatId ? { chatId } : {}), diff --git a/apps/sim/lib/copilot/chat/post.test.ts b/apps/sim/lib/copilot/chat/post.test.ts index 6a0d6914736..e2a2e6fcd68 100644 --- a/apps/sim/lib/copilot/chat/post.test.ts +++ b/apps/sim/lib/copilot/chat/post.test.ts @@ -40,6 +40,7 @@ const { resolveBillingAttribution, resolveOrganizationBillingAttribution, authorizeOrganizationChat, + readOrganizationAssistantImage, finalizeAssistantTurn, appendCopilotChatMessages, persistChatResources, @@ -62,6 +63,7 @@ const { resolveBillingAttribution: vi.fn(), resolveOrganizationBillingAttribution: vi.fn(), authorizeOrganizationChat: vi.fn(), + readOrganizationAssistantImage: vi.fn(), finalizeAssistantTurn: vi.fn(), appendCopilotChatMessages: vi.fn(), persistChatResources: vi.fn(), @@ -135,6 +137,10 @@ vi.mock('@/lib/copilot/chat/organization-chats', () => ({ authorizeOrganizationChat: { execute: authorizeOrganizationChat }, })) +vi.mock('@/lib/uploads/contexts/organization-assistant/application', () => ({ + readOrganizationAssistantImage, +})) + vi.mock('@/lib/credentials/application/personal-credentials', () => ({ listPersonalCredentials: { execute: listPersonal }, })) @@ -192,9 +198,7 @@ vi.mock('@/lib/copilot/resources/persistence', () => ({ vi.mock('@/lib/permission-groups/config-scope.server', () => permissionGroupScopeMock) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: { - publishStatusChanged: mockPublishStatusChanged, - }, + publishChatStatusChanged: mockPublishStatusChanged, })) import { chatOperations } from '@/lib/copilot/application/operations' @@ -237,6 +241,14 @@ describe('handleUnifiedChatPost', () => { userId: 'user-1', role: 'member', }) + readOrganizationAssistantImage.mockResolvedValue({ + id: 'upload-1', + key: 'assistant/org-1/user-1/upload-1/image.png', + name: 'image.png', + contentType: 'image/png', + size: 5, + buffer: Buffer.from('image'), + }) getEffectiveEnvironmentSnapshot.mockResolvedValue({ personalEncrypted: { API_KEY: 'encrypted-secret' }, workspaceEncrypted: {}, @@ -341,6 +353,170 @@ describe('handleUnifiedChatPost', () => { ) }) + it.each(['Describe this image', ''])( + 'prepares organization image bytes and persists canonical metadata (message: %s)', + async (message) => { + getSession.mockResolvedValue({ user: { id: 'user-1' }, session: { id: 'session-1' } }) + dbChainMockFns.returning.mockResolvedValueOnce([{ model: null }]) + const key = 'assistant/org-1/user-1/upload-1/image.png' + const response = await handleUnifiedChatPost( + new NextRequest('http://localhost/api/mothership/chat', { + method: 'POST', + body: JSON.stringify({ + message, + organizationId: 'org-1', + mode: 'assistant', + fileAttachments: [ + { id: 'forged-id', key, filename: 'forged.txt', media_type: 'text/plain', size: 0 }, + ], + }), + }) + ) + expect(response.status).toBe(200) + expect(readOrganizationAssistantImage).toHaveBeenCalledWith({ + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + organizationId: 'org-1', + key, + signal: expect.any(AbortSignal), + }) + expect(buildCopilotRequestPayload).toHaveBeenCalledWith( + expect.objectContaining({ + message, + assistantImages: [ + { + type: 'image', + filename: 'image.png', + source: { type: 'base64', media_type: 'image/png', data: 'aW1hZ2U=' }, + }, + ], + }), + expect.anything() + ) + expect(appendCopilotChatMessages).toHaveBeenCalledWith( + 'chat-1', + [ + expect.objectContaining({ + content: message, + fileAttachments: [ + { id: 'upload-1', key, filename: 'image.png', media_type: 'image/png', size: 5 }, + ], + }), + ], + expect.anything(), + expect.anything() + ) + expect(getUserEntityPermissions).not.toHaveBeenCalled() + expect(generateWorkspaceSnapshot).not.toHaveBeenCalled() + } + ) + + it('rejects inaccessible images before creating or persisting a conversation', async () => { + getSession.mockResolvedValue({ user: { id: 'user-1' }, session: { id: 'session-1' } }) + readOrganizationAssistantImage.mockRejectedValueOnce( + new OrchestrationError('not_found', 'Image not found') + ) + const response = await handleUnifiedChatPost( + new NextRequest('http://localhost/api/mothership/chat', { + method: 'POST', + body: JSON.stringify({ + message: '', + organizationId: 'org-1', + mode: 'assistant', + fileAttachments: [ + { + id: 'image', + key: 'other-user-image', + filename: 'image.png', + media_type: 'image/png', + size: 5, + }, + ], + }), + }) + ) + expect(response.status).toBe(403) + expect(resolveOrCreateChat).not.toHaveBeenCalled() + expect(appendCopilotChatMessages).not.toHaveBeenCalled() + expect(createSSEStream).not.toHaveBeenCalled() + }) + + it('continues rejecting empty messages without organization images', async () => { + const response = await handleUnifiedChatPost( + new NextRequest('http://localhost/api/mothership/chat', { + method: 'POST', + body: JSON.stringify({ message: '', organizationId: 'org-1', mode: 'assistant' }), + }) + ) + expect(response.status).toBe(400) + expect(readOrganizationAssistantImage).not.toHaveBeenCalled() + expect(resolveOrCreateChat).not.toHaveBeenCalled() + }) + + it('keeps workspace files unavailable in workspace Assistant mode', async () => { + const response = await handleUnifiedChatPost( + new NextRequest('http://localhost/api/mothership/chat', { + method: 'POST', + body: JSON.stringify({ + message: 'Read this file', + workspaceId: 'ws-1', + mode: 'assistant', + fileAttachments: [ + { + id: 'file-1', + key: 'workspace/file.png', + filename: 'file.png', + media_type: 'image/png', + size: 5, + }, + ], + }), + }) + ) + expect(response.status).toBe(400) + expect(readOrganizationAssistantImage).not.toHaveBeenCalled() + expect(resolveOrCreateChat).not.toHaveBeenCalled() + }) + + it('broadcasts organization turn start, completion, and failure under its private owner', async () => { + getSession.mockResolvedValue({ user: { id: 'user-1' }, session: { id: 'session-1' } }) + dbChainMockFns.returning.mockResolvedValueOnce([{ model: 'mothership' }]) + const response = await handleUnifiedChatPost( + new NextRequest('http://localhost/api/mothership/chat', { + method: 'POST', + body: JSON.stringify({ + message: 'Find the policy', + organizationId: 'org-1', + mode: 'assistant', + }), + }) + ) + expect(response.status).toBe(200) + const args = createSSEStream.mock.calls[0][0] + const owner = { organizationId: 'org-1', userId: 'user-1', workspaceId: undefined } + expect(mockPublishStatusChanged).toHaveBeenCalledWith(owner, { + chatId: 'chat-1', + type: 'started', + streamId: args.streamId, + }) + await args.orchestrateOptions.onComplete({ + success: true, + content: 'Answer', + contentBlocks: [], + toolCalls: [], + }) + expect(mockPublishStatusChanged).toHaveBeenLastCalledWith(owner, { + chatId: 'chat-1', + type: 'completed', + streamId: args.streamId, + }) + await args.orchestrateOptions.onError(new Error('provider failed')) + expect(mockPublishStatusChanged).toHaveBeenLastCalledWith(owner, { + chatId: 'chat-1', + type: 'completed', + streamId: args.streamId, + }) + }) + it.each([{ workspaceId: 'ws-1' }, { workflowId: 'wf-1' }, { mode: 'agent' }])( 'rejects mixed organization scope before persistence: %j', async (extra) => { @@ -1069,12 +1245,14 @@ describe('handleUnifiedChatPost', () => { requestId: 'request-1', }) - expect(mockPublishStatusChanged).toHaveBeenCalledWith({ - workspaceId: 'ws-1', - chatId: 'chat-1', - type: 'completed', - streamId: streamArgs?.streamId, - }) + expect(mockPublishStatusChanged).toHaveBeenCalledWith( + expect.objectContaining({ workspaceId: 'ws-1' }), + { + chatId: 'chat-1', + type: 'completed', + streamId: streamArgs?.streamId, + } + ) }) it('rejects requests that have neither workflow nor workspace attachment', async () => { diff --git a/apps/sim/lib/copilot/chat/post.ts b/apps/sim/lib/copilot/chat/post.ts index ba2f04f7afe..282393fa1cd 100644 --- a/apps/sim/lib/copilot/chat/post.ts +++ b/apps/sim/lib/copilot/chat/post.ts @@ -19,6 +19,10 @@ import { resolveOrganizationBillingAttribution, } from '@/lib/billing/core/billing-attribution' import { chatOperations } from '@/lib/copilot/application/operations' +import { + type AssistantImageContent, + prepareAssistantImages, +} from '@/lib/copilot/chat/assistant-images' import { DESKTOP_TERMINAL_HINT_ID_MAX_LENGTH, DESKTOP_TERMINAL_HINT_TEXT_MAX_LENGTH, @@ -44,7 +48,7 @@ import { } from '@/lib/copilot/chat/selection-context' import { finalizeAssistantTurn } from '@/lib/copilot/chat/terminal-state' import { generateWorkspaceSnapshot } from '@/lib/copilot/chat/workspace-context' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { COPILOT_REQUEST_MODES } from '@/lib/copilot/constants' import { computeWorkspaceEntitlements } from '@/lib/copilot/entitlements' import { prepareCopilotEnvironmentContext } from '@/lib/copilot/environment-context' @@ -277,63 +281,70 @@ const ChatContextSchema = z } }) -const ChatMessageSchema = z.object({ - message: z.string().min(1, 'Message is required'), - /* Bounded because it becomes part of a Postgres key in `chatSendIdempotency`; +const ChatMessageSchema = z + .object({ + message: z.string(), + /* Bounded because it becomes part of a Postgres key in `chatSendIdempotency`; a client-supplied id longer than the btree entry limit would throw there. A generated id is 36 chars. */ - userMessageId: z.string().max(128).optional(), - chatId: z.string().optional(), - workflowId: z.string().optional(), - workspaceId: z.string().optional(), - organizationId: z.string().min(1).max(200).optional(), - workflowName: z.string().optional(), - model: z.string().optional().default(DEFAULT_MODEL), - mode: z.enum(COPILOT_REQUEST_MODES).optional().default('agent'), - assistantSearch: workspaceSearchFiltersSchema.optional(), - prefetch: z.boolean().optional(), - createNewChat: z.boolean().optional().default(false), - implicitFeedback: z.string().optional(), - fileAttachments: z.array(FileAttachmentSchema).optional(), - resourceAttachments: z - .preprocess(dropUnaddressableAttachments, z.array(ResourceAttachmentSchema)) - .optional(), - provider: z.string().optional(), - contexts: z.array(ChatContextSchema).optional(), - commands: z.array(z.string()).optional(), - userTimezone: z.string().optional(), - desktopCapabilities: z - .object({ - localFilesystem: z.boolean().optional(), - browser: z.boolean().optional(), - terminal: z.boolean().optional(), - terminals: z - .array( - z.object({ - id: z.string().max(DESKTOP_TERMINAL_HINT_ID_MAX_LENGTH), - cwd: z.string().max(DESKTOP_TERMINAL_HINT_TEXT_MAX_LENGTH).optional(), - running: z.string().max(DESKTOP_TERMINAL_HINT_TEXT_MAX_LENGTH).optional(), - interactive: z.boolean().optional(), - active: z.boolean().optional(), - }) - ) - .optional(), - browserSessions: z - .array( - z.object({ - hostname: z - .string() - .max(253) - .regex(/^[a-z0-9.-]+$/), - evidence: z.enum(['sign-in-completed', 'cookies']), - lastObservedAt: z.string().datetime(), - }) - ) - .max(20) - .optional(), - }) - .optional(), -}) + userMessageId: z.string().max(128).optional(), + chatId: z.string().optional(), + workflowId: z.string().optional(), + workspaceId: z.string().optional(), + organizationId: z.string().min(1).max(200).optional(), + workflowName: z.string().optional(), + model: z.string().optional().default(DEFAULT_MODEL), + mode: z.enum(COPILOT_REQUEST_MODES).optional().default('agent'), + assistantSearch: workspaceSearchFiltersSchema.optional(), + prefetch: z.boolean().optional(), + createNewChat: z.boolean().optional().default(false), + implicitFeedback: z.string().optional(), + fileAttachments: z.array(FileAttachmentSchema).optional(), + resourceAttachments: z + .preprocess(dropUnaddressableAttachments, z.array(ResourceAttachmentSchema)) + .optional(), + provider: z.string().optional(), + contexts: z.array(ChatContextSchema).optional(), + commands: z.array(z.string()).optional(), + userTimezone: z.string().optional(), + desktopCapabilities: z + .object({ + localFilesystem: z.boolean().optional(), + browser: z.boolean().optional(), + terminal: z.boolean().optional(), + terminals: z + .array( + z.object({ + id: z.string().max(DESKTOP_TERMINAL_HINT_ID_MAX_LENGTH), + cwd: z.string().max(DESKTOP_TERMINAL_HINT_TEXT_MAX_LENGTH).optional(), + running: z.string().max(DESKTOP_TERMINAL_HINT_TEXT_MAX_LENGTH).optional(), + interactive: z.boolean().optional(), + active: z.boolean().optional(), + }) + ) + .optional(), + browserSessions: z + .array( + z.object({ + hostname: z + .string() + .max(253) + .regex(/^[a-z0-9.-]+$/), + evidence: z.enum(['sign-in-completed', 'cookies']), + lastObservedAt: z.string().datetime(), + }) + ) + .max(20) + .optional(), + }) + .optional(), + }) + .refine( + (body) => + body.message.length > 0 || + (body.mode === 'assistant' && !!body.organizationId && !!body.fileAttachments?.length), + { message: 'Message is required', path: ['message'] } + ) type UnifiedChatRequest = z.infer type BrowserSessions = NonNullable['browserSessions'] @@ -351,7 +362,7 @@ type UnifiedChatBranch = goRoute: '/api/copilot' titleModel: string titleProvider?: string - notifyWorkspaceStatus: false + notifyChatStatus: false buildPayload: (params: { message: string userId: string @@ -397,7 +408,7 @@ type UnifiedChatBranch = goRoute: '/api/mothership' titleModel: string titleProvider?: undefined - notifyWorkspaceStatus: boolean + notifyChatStatus: boolean buildPayload: (params: { message: string userId: string @@ -406,6 +417,7 @@ type UnifiedChatBranch = contexts: Array<{ type: string; content: string; tag?: string; path?: string }> mcpServerIds?: string[] fileAttachments?: UnifiedChatRequest['fileAttachments'] + assistantImages?: AssistantImageContent[] userPermission?: string entitlements?: string[] userTimezone?: string @@ -566,7 +578,9 @@ async function persistUserMessage(params: { fileAttachments?: UnifiedChatRequest['fileAttachments'] contexts?: UnifiedChatRequest['contexts'] workspaceId?: string - notifyWorkspaceStatus: boolean + notifyChatStatus: boolean + organizationId?: string + userId?: string requestMode?: 'assistant' | 'agent' /** * Root context for the mothership request. When present the persist @@ -585,7 +599,9 @@ async function persistUserMessage(params: { fileAttachments, contexts, workspaceId, - notifyWorkspaceStatus, + organizationId, + userId, + notifyChatStatus, parentOtelContext, } = params if (!chatId) return @@ -637,13 +653,15 @@ async function persistUserMessage(params: { updated ? CopilotChatPersistOutcome.Appended : CopilotChatPersistOutcome.ChatNotFound ) - if (notifyWorkspaceStatus && updated && workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId, - chatId, - type: 'started', - streamId: userMessageId, - }) + if (notifyChatStatus && updated) { + publishChatStatusChanged( + { workspaceId, organizationId, userId }, + { + chatId, + type: 'started', + streamId: userMessageId, + } + ) } }, parentOtelContext @@ -712,7 +730,9 @@ function buildOnComplete(params: { userMessageId: string requestId: string workspaceId?: string - notifyWorkspaceStatus: boolean + notifyChatStatus: boolean + organizationId?: string + userId?: string requestMode?: 'assistant' | 'agent' /** * Root agent span for this request. When present, the final @@ -728,7 +748,16 @@ function buildOnComplete(params: { }) => void } }) { - const { chatId, userMessageId, requestId, workspaceId, notifyWorkspaceStatus, otelRoot } = params + const { + chatId, + userMessageId, + requestId, + workspaceId, + organizationId, + userId, + notifyChatStatus, + otelRoot, + } = params return async (result: OrchestratorResult) => { if (otelRoot && result.success) { @@ -758,13 +787,15 @@ function buildOnComplete(params: { finalization.updated || finalization.outcome === CopilotChatFinalizeOutcome.AssistantAlreadyPersisted - if (notifyWorkspaceStatus && workspaceId && shouldPublishCompletion) { - chatPubSub?.publishStatusChanged({ - workspaceId, - chatId, - type: 'completed', - streamId: userMessageId, - }) + if (notifyChatStatus && shouldPublishCompletion) { + publishChatStatusChanged( + { workspaceId, organizationId, userId }, + { + chatId, + type: 'completed', + streamId: userMessageId, + } + ) } return } @@ -784,13 +815,15 @@ function buildOnComplete(params: { ...(result.success ? {} : { streamMarkerPolicy: 'active-or-cleared' as const }), }) - if (notifyWorkspaceStatus && workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId, - chatId, - type: 'completed', - streamId: userMessageId, - }) + if (notifyChatStatus) { + publishChatStatusChanged( + { workspaceId, organizationId, userId }, + { + chatId, + type: 'completed', + streamId: userMessageId, + } + ) } } catch (error) { logger.error(`[${requestId}] Failed to persist chat messages`, { @@ -806,10 +839,20 @@ function buildOnError(params: { userMessageId: string requestId: string workspaceId?: string - notifyWorkspaceStatus: boolean + notifyChatStatus: boolean + organizationId?: string + userId?: string requestMode?: 'assistant' | 'agent' }) { - const { chatId, userMessageId, requestId, workspaceId, notifyWorkspaceStatus } = params + const { + chatId, + userMessageId, + requestId, + workspaceId, + organizationId, + userId, + notifyChatStatus, + } = params return async (_error: Error, result?: OrchestratorResult) => { if (!chatId) return @@ -831,13 +874,15 @@ function buildOnError(params: { streamMarkerPolicy: 'active-or-cleared', }) - if (notifyWorkspaceStatus && workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId, - chatId, - type: 'completed', - streamId: userMessageId, - }) + if (notifyChatStatus) { + publishChatStatusChanged( + { workspaceId, organizationId, userId }, + { + chatId, + type: 'completed', + streamId: userMessageId, + } + ) } } catch (error) { logger.error(`[${requestId}] Failed to finalize errored chat stream`, { @@ -886,7 +931,7 @@ async function resolveBranch(params: { effectiveModel: DEFAULT_MODEL, goRoute: '/api/mothership', titleModel: DEFAULT_MODEL, - notifyWorkspaceStatus: false, + notifyChatStatus: true, buildPayload: async (payloadParams) => buildCopilotRequestPayload( { @@ -936,7 +981,7 @@ async function resolveBranch(params: { goRoute: '/api/copilot', titleModel: selectedModel, titleProvider: provider, - notifyWorkspaceStatus: false, + notifyChatStatus: false, buildPayload: async (payloadParams) => buildCopilotRequestPayload( { @@ -1005,7 +1050,7 @@ async function resolveBranch(params: { effectiveModel: DEFAULT_MODEL, goRoute: '/api/mothership', titleModel: DEFAULT_MODEL, - notifyWorkspaceStatus: true, + notifyChatStatus: true, buildPayload: async (payloadParams) => buildCopilotRequestPayload( { @@ -1138,7 +1183,7 @@ export async function handleUnifiedChatPost(req: NextRequest) { body.mode === 'assistant' && (body.workflowId || body.workflowName || - body.fileAttachments?.length || + (body.fileAttachments?.length && !body.organizationId) || body.contexts?.length) ) { return createBadRequestResponse( @@ -1251,6 +1296,21 @@ export async function handleUnifiedChatPost(req: NextRequest) { return capabilityRefusalResponse(chatCapability) } + const assistantImages = + branch.kind === 'organization' && body.fileAttachments?.length + ? await prepareAssistantImages({ + principal: { + kind: 'session', + userId: authenticatedUserId, + sessionId: session.session.id, + }, + organizationId: branch.organizationId, + attachments: body.fileAttachments, + signal: req.signal, + }) + : undefined + const fileAttachments = assistantImages?.attachments ?? body.fileAttachments + /* Prompt content is captured only once the turn is going to run. Both calls are internally gated on OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, but the gate is on @@ -1478,10 +1538,12 @@ export async function handleUnifiedChatPost(req: NextRequest) { chatId: actualChatId, userMessageId, message: body.message, - fileAttachments: body.fileAttachments, + fileAttachments, contexts: normalizedContexts, workspaceId, - notifyWorkspaceStatus: branch.notifyWorkspaceStatus, + notifyChatStatus: branch.notifyChatStatus, + organizationId: branch.kind === 'organization' ? branch.organizationId : undefined, + userId: authenticatedUserId, requestMode: body.mode === 'assistant' ? 'assistant' : 'agent', parentOtelContext: activeOtelRoot.context, }) @@ -1543,7 +1605,7 @@ export async function handleUnifiedChatPost(req: NextRequest) { contexts: turnContexts, assistantSearch: body.mode === 'assistant' ? body.assistantSearch : undefined, mcpServerIds, - fileAttachments: body.fileAttachments, + fileAttachments, userPermission: userPermission ?? undefined, entitlements, userTimezone: body.userTimezone, @@ -1572,7 +1634,8 @@ export async function handleUnifiedChatPost(req: NextRequest) { contexts: turnContexts, assistantSearch: body.mode === 'assistant' ? body.assistantSearch : undefined, mcpServerIds, - fileAttachments: body.fileAttachments, + fileAttachments, + assistantImages: assistantImages?.content, userPermission: userPermission ?? undefined, entitlements, userTimezone: body.userTimezone, @@ -1630,7 +1693,9 @@ export async function handleUnifiedChatPost(req: NextRequest) { userMessageId, requestId, workspaceId, - notifyWorkspaceStatus: branch.notifyWorkspaceStatus, + notifyChatStatus: branch.notifyChatStatus, + organizationId: branch.kind === 'organization' ? branch.organizationId : undefined, + userId: authenticatedUserId, requestMode: body.mode === 'assistant' ? 'assistant' : 'agent', otelRoot, }), @@ -1639,7 +1704,9 @@ export async function handleUnifiedChatPost(req: NextRequest) { userMessageId, requestId, workspaceId, - notifyWorkspaceStatus: branch.notifyWorkspaceStatus, + notifyChatStatus: branch.notifyChatStatus, + organizationId: branch.kind === 'organization' ? branch.organizationId : undefined, + userId: authenticatedUserId, requestMode: body.mode === 'assistant' ? 'assistant' : 'agent', }), }, @@ -1702,6 +1769,12 @@ export async function handleUnifiedChatPost(req: NextRequest) { if (applicationError?.code === 'forbidden' || applicationError?.code === 'not_found') { return NextResponse.json({ error: 'Conversation access denied' }, { status: 403 }) } + if (applicationError?.code === 'validation' || applicationError?.code === 'payload_too_large') { + return NextResponse.json( + { error: applicationError.message }, + { status: applicationError.code === 'validation' ? 400 : 413 } + ) + } if (isWorkspaceAccessDeniedError(error)) { return NextResponse.json({ error: 'Workspace access denied' }, { status: 403 }) } diff --git a/apps/sim/lib/copilot/request/lifecycle/start.test.ts b/apps/sim/lib/copilot/request/lifecycle/start.test.ts index e50486707e6..e02f5c2d8ce 100644 --- a/apps/sim/lib/copilot/request/lifecycle/start.test.ts +++ b/apps/sim/lib/copilot/request/lifecycle/start.test.ts @@ -119,7 +119,7 @@ vi.mock('@/lib/copilot/request/session/sse', () => ({ })) vi.mock('@/lib/copilot/chat-status', () => ({ - chatPubSub: null, + publishChatStatusChanged: vi.fn(), })) vi.mock('@/lib/copilot/request/go/fetch', () => ({ diff --git a/apps/sim/lib/copilot/request/lifecycle/start.ts b/apps/sim/lib/copilot/request/lifecycle/start.ts index b5ee96d37a6..43ccf62da93 100644 --- a/apps/sim/lib/copilot/request/lifecycle/start.ts +++ b/apps/sim/lib/copilot/request/lifecycle/start.ts @@ -12,7 +12,7 @@ import { resolveOrganizationBillingAttribution, } from '@/lib/billing/core/billing-attribution' import { createRunSegment } from '@/lib/copilot/async-runs/repository' -import { chatPubSub } from '@/lib/copilot/chat-status' +import { publishChatStatusChanged } from '@/lib/copilot/chat-status' import { MothershipStreamV1EventType, MothershipStreamV1SessionKind, @@ -512,13 +512,7 @@ function fireTitleGeneration(params: { type: MothershipStreamV1EventType.session, payload: { kind: MothershipStreamV1SessionKind.title, title }, }) - if (workspaceId) { - chatPubSub?.publishStatusChanged({ - workspaceId, - chatId, - type: 'renamed', - }) - } + publishChatStatusChanged({ workspaceId, organizationId, userId }, { chatId, type: 'renamed' }) }) .catch((error) => { logger.error(`[${requestId}] Title generation failed:`, error) diff --git a/apps/sim/lib/copilot/request/tools/executor.test.ts b/apps/sim/lib/copilot/request/tools/executor.test.ts index c30ca83db03..8b5415a44b8 100644 --- a/apps/sim/lib/copilot/request/tools/executor.test.ts +++ b/apps/sim/lib/copilot/request/tools/executor.test.ts @@ -146,6 +146,42 @@ function buildPendingToolCall(): ToolCallState { } } +describe('tool result size diagnostics', () => { + beforeEach(() => { + vi.clearAllMocks() + completeAsyncToolCall.mockResolvedValue(null) + markAsyncToolRunning.mockResolvedValue(null) + upsertAsyncToolCall.mockResolvedValue(null) + }) + + it.each(['é🔎', { content: 'é🔎' }])( + 'records UTF-8 bytes after result projection for %j', + async (output) => { + executeTool.mockResolvedValueOnce({ success: true, output }) + const toolCall = buildPendingToolCall() + const context = buildStreamingContext(toolCall) + const endSpan = vi.spyOn(context.trace, 'endSpan') + + const completion = await executeToolAndReport(toolCall.id, context, { + userId: 'user-1', + workflowId: 'workflow-1', + resolvedSecretTraceRegistry: new ResolvedSecretTraceRegistry(), + }) + + expect(completion.status).toBe(MothershipStreamV1ToolOutcome.success) + const serialized = + typeof completion.data === 'string' ? completion.data : JSON.stringify(completion.data) + expect(endSpan).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'tool.execute', + attributes: expect.objectContaining({ outputBytes: Buffer.byteLength(serialized) }), + }), + 'ok' + ) + } + ) +}) + describe('toolWatchdogTimeoutMs', () => { it('gives request-scoped MCP tools the long-running watchdog', () => { expect(toolWatchdogTimeoutMs('mcp-363de040-web_search_exa')).toBe(TOOL_WATCHDOG_LONG_RUNNING_MS) diff --git a/apps/sim/lib/copilot/request/tools/executor.ts b/apps/sim/lib/copilot/request/tools/executor.ts index d9c6b523e9d..e7c03b320e0 100644 --- a/apps/sim/lib/copilot/request/tools/executor.ts +++ b/apps/sim/lib/copilot/request/tools/executor.ts @@ -125,11 +125,11 @@ function summarizeToolResultForSpan(result: { const output = (result as { output: unknown }).output if (typeof output === 'string') { summary.outputKind = 'string' - summary.outputBytes = output.length + summary.outputBytes = Buffer.byteLength(output) } else if (output && typeof output === 'object') { summary.outputKind = Array.isArray(output) ? 'array' : 'object' try { - summary.outputBytes = JSON.stringify(output).length + summary.outputBytes = Buffer.byteLength(JSON.stringify(output)) } catch { summary.outputBytes = 0 } @@ -143,7 +143,7 @@ function summarizeToolResultForSpan(result: { } } else if (output !== undefined && output !== null) { summary.outputKind = typeof output - summary.outputBytes = String(output).length + summary.outputBytes = Buffer.byteLength(String(output)) } return summary } diff --git a/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.test.ts b/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.test.ts index 91747f3629a..1e669a7b45b 100644 --- a/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.test.ts +++ b/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.test.ts @@ -1,7 +1,15 @@ /** @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' -const mocks = vi.hoisted(() => ({ search: vi.fn(), read: vi.fn(), authorizeChat: vi.fn() })) +const mocks = vi.hoisted(() => ({ + search: vi.fn(), + read: vi.fn(), + authorizeChat: vi.fn(), + info: vi.fn(), +})) +vi.mock('@sim/logger', () => ({ + createLogger: () => ({ info: mocks.info, error: vi.fn(), warn: vi.fn() }), +})) vi.mock('@/lib/copilot/chat/organization-chats', () => ({ authorizeOrganizationChatDelegation: { execute: mocks.authorizeChat }, })) @@ -179,6 +187,48 @@ describe('Assistant retrieval tools', () => { }) ) }) + it.each([0, 20, 50])( + 'measures UTF-8 bytes for %i passages without logging their content', + async (count) => { + const content = 'Confidential passage é🔎'.repeat(100) + mocks.search.mockResolvedValueOnce({ + knowledgeBases: [{ id: 'index', name: 'Enterprise Search' }], + results: Array.from({ length: count }, (_, index) => ({ + knowledgeBaseId: 'index', + documentId: `doc-${index % 4}`, + documentName: 'Private title', + sourceUrl: null, + sourceModifiedAt: null, + metadata: {}, + content, + chunkIndex: index, + similarity: 1, + })), + }) + + const output = await searchWorkspaceServerTool.execute( + { query: 'Private query', ...(count === 50 ? { topK: 50 } : {}) }, + context + ) + + expect(output.success).toBe(true) + expect(mocks.info).toHaveBeenCalledWith( + 'Knowledge search completed', + expect.objectContaining({ + toolCallId: 'call', + toolResultBytes: Buffer.byteLength(JSON.stringify(output)), + passageBytes: count * Buffer.byteLength(content), + maxPassageBytes: count ? Buffer.byteLength(content) : 0, + uniqueDocumentCount: Math.min(count, 4), + }) + ) + const logged = JSON.stringify(mocks.info.mock.calls) + expect(logged).not.toContain('Confidential passage') + expect(logged).not.toContain('Private title') + expect(logged).not.toContain('Private query') + } + ) + it('returns stable citation IDs with internal links for uploaded documents', async () => { const result = await searchWorkspaceServerTool.execute({ query: 'orion' }, context) expect(result).toMatchObject({ diff --git a/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.ts b/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.ts index 4fad2478865..01f6cb833f5 100644 --- a/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.ts +++ b/apps/sim/lib/copilot/tools/server/knowledge/workspace-search.ts @@ -16,6 +16,12 @@ import { } from '@/lib/knowledge/application/workspace-search' import { sourceAuthor } from '@/lib/knowledge/search/author' import { createKnowledgeDocumentCitation } from '@/lib/knowledge/search/citation' +import { + annotateSearchDiagnostics, + measureSearchStage, + recordSearchStageDuration, + withSearchDiagnostics, +} from '@/lib/knowledge/search/diagnostics' import { intersectWorkspaceSearchFilters } from '@/lib/knowledge/search/filters' import { connectorDisplayName } from '@/lib/sim-search/connectors' import { projectResolvedSecretModelContent } from '@/executor/utils/resolved-secret-content-projection' @@ -37,77 +43,99 @@ const CITATION_INSTRUCTION = export const searchWorkspaceServerTool: BaseServerTool = { name: 'search_workspace', async execute(raw, context?: ServerToolContext) { - try { - const scope = requireCopilotKnowledgeScope(context) - const { query, topK, ...requestedFilters } = searchInputSchema.parse(raw) - const registry = context?.resolvedSecretTraceRegistry - if (!registry) throw new Error('Knowledge result provenance is unavailable') - const projected = projectResolvedSecretModelContent(query, registry) - if (!projected.safe || typeof projected.value !== 'string') { - return { - success: false, - message: 'Search query contains protected content. Rephrase the query.', - } - } - const input = { - query: projected.value, - topK, - filters: intersectWorkspaceSearchFilters(requestedFilters, context?.assistantSearch), + return withSearchDiagnostics( + { surface: context?.searchSurface ?? 'copilot', - resultSecretRegistry: registry, - signal: context?.abortSignal, - } as const - const result = - scope.kind === 'organization' - ? await executeCopilotOrganizationKnowledgeUseCase(context, searchOrganizationKnowledge, { - ...input, - organizationId: scope.organizationId, - }) - : await executeCopilotKnowledgeUseCase(context, searchWorkspaceKnowledge, { - ...input, - workspaceId: scope.workspaceId, + toolCallId: context?.toolCallId, + executionId: context?.executionId, + }, + async () => { + try { + const inputStarted = performance.now() + const scope = requireCopilotKnowledgeScope(context) + const { query, topK, ...requestedFilters } = searchInputSchema.parse(raw) + const registry = context?.resolvedSecretTraceRegistry + if (!registry) throw new Error('Knowledge result provenance is unavailable') + const projected = projectResolvedSecretModelContent(query, registry) + if (!projected.safe || typeof projected.value !== 'string') { + return { + success: false, + message: 'Search query contains protected content. Rephrase the query.', + } + } + const input = { + query: projected.value, + topK, + filters: intersectWorkspaceSearchFilters(requestedFilters, context?.assistantSearch), + surface: context?.searchSurface ?? 'copilot', + resultSecretRegistry: registry, + signal: context?.abortSignal, + } as const + recordSearchStageDuration('tool_input', performance.now() - inputStarted) + const result = await measureSearchStage('tool_application', () => + scope.kind === 'organization' + ? executeCopilotOrganizationKnowledgeUseCase(context, searchOrganizationKnowledge, { + ...input, + organizationId: scope.organizationId, + }) + : executeCopilotKnowledgeUseCase(context, searchWorkspaceKnowledge, { + ...input, + workspaceId: scope.workspaceId, + }) + ) + return await measureSearchStage('tool_presentation', () => { + const names = new Map(result.knowledgeBases.map((base) => [base.id, base.name])) + const output = { + success: true, + message: `Found ${result.results.length} passages. ${CITATION_INSTRUCTION}`, + data: { + query, + results: result.results.map((item) => ({ + documentId: item.documentId, + knowledgeBaseId: item.knowledgeBaseId, + knowledgeBaseName: names.get(item.knowledgeBaseId) ?? '', + siteName: item.connectorType + ? connectorDisplayName(item.connectorType) + : names.get(item.knowledgeBaseId), + documentName: item.documentName, + sourceUrl: item.sourceUrl, + connectorType: item.connectorType, + sourceModifiedAt: item.sourceModifiedAt?.toISOString() ?? null, + author: sourceAuthor(item.metadata), + content: item.content, + chunkIndex: item.chunkIndex, + similarity: item.similarity, + ...createKnowledgeDocumentCitation({ + scope, + knowledgeBaseId: item.knowledgeBaseId, + documentId: item.documentId, + sourceUrl: item.sourceUrl, + baseUrl: getBaseUrl(), + }), + })), + }, + } + const passageBytes = output.data.results.map((item) => Buffer.byteLength(item.content)) + annotateSearchDiagnostics({ + toolResultBytes: Buffer.byteLength(JSON.stringify(output)), + passageBytes: passageBytes.reduce((total, bytes) => total + bytes, 0), + maxPassageBytes: Math.max(0, ...passageBytes), + uniqueDocumentCount: new Set(output.data.results.map((item) => item.documentId)).size, }) - const names = new Map(result.knowledgeBases.map((base) => [base.id, base.name])) - return { - success: true, - message: `Found ${result.results.length} passages. ${CITATION_INSTRUCTION}`, - data: { - query, - results: result.results.map((item) => ({ - documentId: item.documentId, - knowledgeBaseId: item.knowledgeBaseId, - knowledgeBaseName: names.get(item.knowledgeBaseId) ?? '', - siteName: item.connectorType - ? connectorDisplayName(item.connectorType) - : names.get(item.knowledgeBaseId), - documentName: item.documentName, - sourceUrl: item.sourceUrl, - connectorType: item.connectorType, - sourceModifiedAt: item.sourceModifiedAt?.toISOString() ?? null, - author: sourceAuthor(item.metadata), - content: item.content, - chunkIndex: item.chunkIndex, - similarity: item.similarity, - ...createKnowledgeDocumentCitation({ - scope, - knowledgeBaseId: item.knowledgeBaseId, - documentId: item.documentId, - sourceUrl: item.sourceUrl, - baseUrl: getBaseUrl(), - }), - })), - }, - } - } catch (error) { - logger.error('Workspace search failed', { error }) - return { - success: false, - message: - error instanceof z.ZodError - ? 'Invalid search arguments' - : messageForCopilotKnowledgeError(error), + return output + }) + } catch (error) { + logger.error('Workspace search failed', { error }) + return { + success: false, + message: + error instanceof z.ZodError + ? 'Invalid search arguments' + : messageForCopilotKnowledgeError(error), + } + } } - } + ) }, } diff --git a/apps/sim/lib/core/config/env.ts b/apps/sim/lib/core/config/env.ts index a7bee9c48b4..efece380bdb 100644 --- a/apps/sim/lib/core/config/env.ts +++ b/apps/sim/lib/core/config/env.ts @@ -548,6 +548,9 @@ export const env = createEnv({ DROPBOX_CLIENT_SECRET: z.string().optional(), // Dropbox OAuth client secret SLACK_CLIENT_ID: z.string().optional(), // Slack OAuth client ID SLACK_SEARCH_APP_ID: z.string().optional(), + SLACK_SEARCH_CLIENT_ID: z.string().optional(), + SLACK_SEARCH_CLIENT_SECRET: z.string().optional(), + SLACK_SEARCH_SIGNING_SECRET: z.string().optional(), SLACK_SEARCH_SHARED_APP: z.boolean().optional(), SLACK_CLIENT_SECRET: z.string().optional(), // Slack OAuth client secret SLACK_SIGNING_SECRET: z.string().optional(), // Official Sim Slack app signing secret (verifies inbound events for the native OAuth trigger) diff --git a/apps/sim/lib/core/config/redis-budget.test.ts b/apps/sim/lib/core/config/redis-budget.test.ts new file mode 100644 index 00000000000..55c786d3514 --- /dev/null +++ b/apps/sim/lib/core/config/redis-budget.test.ts @@ -0,0 +1,44 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { coldConnectionBudgetMs } from '@/lib/core/config/redis-budget' + +describe('coldConnectionBudgetMs', () => { + it('charges an unanswered handshake at two command deadlines plus the half-close wait', () => { + // Unauthenticated: SETNAME/SETINFO time out, then INFO times out, then the + // half-closed socket waits for a FIN a wedged peer never sends. + expect( + coldConnectionBudgetMs({ + connectTimeoutMs: 1_000, + commandTimeoutMs: 5_000, + disconnectTimeoutMs: 2_000, + reconnectDelayMs: 500, + }) + ).toBe(2 * 5_000 + 2_000 + 500 + 1_000) + }) + + it('charges the connect deadline when a connection that never completes is the longer case', () => { + // Lowering the command deadline must not shrink the budget below what a + // connect that never completes costs before retryStrategy can fire. + expect( + coldConnectionBudgetMs({ + connectTimeoutMs: 10_000, + commandTimeoutMs: 3_000, + disconnectTimeoutMs: 2_000, + reconnectDelayMs: 500, + }) + ).toBe(10_000 + 500 + 1_000) + }) + + it('always leaves room for the healthy attempt that follows the reconnect', () => { + expect( + coldConnectionBudgetMs({ + connectTimeoutMs: 0, + commandTimeoutMs: 0, + disconnectTimeoutMs: 0, + reconnectDelayMs: 0, + }) + ).toBe(1_000) + }) +}) diff --git a/apps/sim/lib/core/config/redis-budget.ts b/apps/sim/lib/core/config/redis-budget.ts new file mode 100644 index 00000000000..f9743251a4b --- /dev/null +++ b/apps/sim/lib/core/config/redis-budget.ts @@ -0,0 +1,45 @@ +/** Generous room for the healthy handshake that follows a recovered stall; a real one takes tens of milliseconds. */ +const HEALTHY_HANDSHAKE_ALLOWANCE_MS = 1_000 + +export interface ColdConnectionBudgetOptions { + /** The client's TCP connect deadline: bounds a connection that never completes at all. */ + connectTimeoutMs: number + /** The client's per-command deadline: bounds a handshake command the server never answers. */ + commandTimeoutMs: number + /** How long ioredis waits after half-closing a dead socket for the peer's FIN before destroying it. */ + disconnectTimeoutMs: number + /** What the client's `retryStrategy` returns for its first reconnect. */ + reconnectDelayMs: number +} + +/** + * How long a wait for a cold ioredis connection must allow before giving up, + * if it is to survive one dead attempt and still see a healthy one land. + * + * A dead attempt is diagnosed by whichever deadline governs the phase it + * stalls in. A connect that never completes costs `connectTimeoutMs`, and the + * socket is destroyed outright. A connection that opens but whose handshake + * is never answered costs command deadlines — one with a password, because a + * timed-out `AUTH` is fatal; two without, because `CLIENT SETNAME`/`SETINFO` + * must settle, by timing out, before the `INFO` ready check starts its own — + * and then `disconnectTimeoutMs` more, because ioredis half-closes the socket + * and a peer that is wedged never answers with a FIN. The budget takes the + * larger phase so it holds for either URL shape without parsing it. Only then + * does `retryStrategy` run and a fresh attempt begin. + * + * A wait sized to a single deadline expires while the first attempt is still + * being diagnosed, so a configured retry can never be the thing that saves it. + * + * The guarantee is one dead attempt from a fresh or previously-ready client. + * ioredis feeds `retryStrategy` its running attempt count, so a wait that + * begins while the client is already deep in a reconnect loop faces larger + * delays this does not model. The handshake sequence is ioredis 5's; keep this + * beside the pinned client, not in a generic package. + */ +export function coldConnectionBudgetMs(options: ColdConnectionBudgetOptions): number { + const deadAttemptMs = Math.max( + options.connectTimeoutMs, + 2 * options.commandTimeoutMs + options.disconnectTimeoutMs + ) + return deadAttemptMs + options.reconnectDelayMs + HEALTHY_HANDSHAKE_ALLOWANCE_MS +} diff --git a/apps/sim/lib/core/config/redis.test.ts b/apps/sim/lib/core/config/redis.test.ts index bf02a57e48d..933fceae3ec 100644 --- a/apps/sim/lib/core/config/redis.test.ts +++ b/apps/sim/lib/core/config/redis.test.ts @@ -47,14 +47,19 @@ vi.mock('ioredis', () => ({ import { acquireLock, + CONNECT_TIMEOUT_MS, closeRedisConnection, + DISCONNECT_TIMEOUT_MS, describeRedisConnection, extendLock, getRedisClient, onRedisReconnect, resetForTesting, + SHARED_COMMAND_TIMEOUT_MS, + sharedReconnectDelayMs, warmRedisConnection, } from '@/lib/core/config/redis' +import { coldConnectionBudgetMs } from '@/lib/core/config/redis-budget' describe('redis config', () => { beforeEach(() => { @@ -471,7 +476,50 @@ describe('redis config', () => { }) }) + describe('sharedReconnectDelayMs', () => { + it('grows exponentially from the base and caps, with upward-only jitter', () => { + expect(sharedReconnectDelayMs(1, 0)).toBe(1_000) + expect(sharedReconnectDelayMs(2, 0)).toBe(2_000) + expect(sharedReconnectDelayMs(5, 0)).toBe(10_000) + expect(sharedReconnectDelayMs(6, 0)).toBe(10_000) + expect(sharedReconnectDelayMs(1, 1)).toBe(1_300) + }) + }) + describe('warmRedisConnection', () => { + it('outlasts one dead handshake so the reconnect can be what warms it', async () => { + mockRedisInstance.status = 'connecting' + const warm = warmRedisConnection() + + // The dead attempt's own diagnosis and half-close, then the longest first + // reconnect delay: the moment a healthy second attempt can begin. + await vi.advanceTimersByTimeAsync( + Math.max(CONNECT_TIMEOUT_MS, 2 * SHARED_COMMAND_TIMEOUT_MS + DISCONNECT_TIMEOUT_MS) + + sharedReconnectDelayMs(1, 1) + ) + const client = getRedisClient() + Object.assign(client ?? {}, { status: 'ready' }) + client?.emit('ready') + + await expect(warm).resolves.toBe(true) + }) + + it('still gives up once the budget is spent', async () => { + mockRedisInstance.status = 'connecting' + const warm = warmRedisConnection() + + await vi.advanceTimersByTimeAsync( + coldConnectionBudgetMs({ + connectTimeoutMs: CONNECT_TIMEOUT_MS, + commandTimeoutMs: SHARED_COMMAND_TIMEOUT_MS, + disconnectTimeoutMs: DISCONNECT_TIMEOUT_MS, + reconnectDelayMs: sharedReconnectDelayMs(1, 1), + }) + ) + + await expect(warm).resolves.toBe(false) + }) + it('resolves immediately when the connection is already usable', async () => { mockRedisInstance.status = 'ready' diff --git a/apps/sim/lib/core/config/redis.ts b/apps/sim/lib/core/config/redis.ts index 2c11595df32..7ca7889952d 100644 --- a/apps/sim/lib/core/config/redis.ts +++ b/apps/sim/lib/core/config/redis.ts @@ -2,9 +2,10 @@ import { isIP } from 'node:net' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { randomFloat } from '@sim/utils/random' -import Redis, { type RedisOptions } from 'ioredis' +import Redis from 'ioredis' import { env } from '@/lib/core/config/env' import { getConfiguredCacheProvider } from '@/lib/core/config/env-capabilities.server' +import { coldConnectionBudgetMs } from '@/lib/core/config/redis-budget' const logger = createLogger('Redis') @@ -42,13 +43,24 @@ function resolveRedisTlsOptions(url: string | undefined): { servername: string } * and TLS SNI when REDIS_URL targets an IP. Every Redis client we open should * spread this; callers add their own retry / timeout policy on top. */ -export function getRedisConnectionDefaults( - url: string | undefined -): Pick { +export interface RedisConnectionDefaults { + keepAlive: number + connectTimeout: number + disconnectTimeout: number + enableOfflineQueue: boolean + tls?: { servername: string } +} + +export const CONNECT_TIMEOUT_MS = 10_000 +/** ioredis's own default, stated so readiness budgets can be derived from it. */ +export const DISCONNECT_TIMEOUT_MS = 2_000 + +export function getRedisConnectionDefaults(url: string | undefined): RedisConnectionDefaults { const tls = resolveRedisTlsOptions(url) return { keepAlive: 1000, - connectTimeout: 10000, + connectTimeout: CONNECT_TIMEOUT_MS, + disconnectTimeout: DISCONNECT_TIMEOUT_MS, enableOfflineQueue: true, ...(tls ? { tls } : {}), } @@ -203,12 +215,34 @@ export function describeRedisConnection( const PING_INTERVAL_MS = 15_000 const MAX_PING_FAILURES = 2 +export const SHARED_COMMAND_TIMEOUT_MS = 5_000 +const RECONNECT_BASE_MS = 1_000 +const RECONNECT_MAX_BASE_MS = 10_000 +const RECONNECT_JITTER_RATIO = 0.3 + +/** + * The shared client's reconnect delay for attempt `times`, with `jitter` in + * `[0, 1]` scaling the upward-only jitter band. Pure so the same formula can + * be evaluated for a budget — `sharedReconnectDelayMs(1, 1)` is the longest + * possible first reconnect — as well as from `retryStrategy`, which adds the + * bookkeeping around it. + */ +export function sharedReconnectDelayMs(times: number, jitter: number): number { + const base = Math.min(RECONNECT_BASE_MS * 2 ** (times - 1), RECONNECT_MAX_BASE_MS) + return Math.round(base + jitter * base * RECONNECT_JITTER_RATIO) +} + /** - * Warm-up budget. Sized to outlast a slow handshake rather than a fast one, - * because giving up early just returns the handshake to the first command's - * deadline, which is the thing this exists to avoid. + * Warm-up budget: one dead attempt, its longest possible first reconnect + * delay, then a healthy attempt. Giving up sooner returns the handshake to the + * first command's deadline, which is the thing warming exists to avoid. */ -const REDIS_WARMUP_TIMEOUT_MS = 10_000 +const REDIS_WARMUP_TIMEOUT_MS = coldConnectionBudgetMs({ + connectTimeoutMs: CONNECT_TIMEOUT_MS, + commandTimeoutMs: SHARED_COMMAND_TIMEOUT_MS, + disconnectTimeoutMs: DISCONNECT_TIMEOUT_MS, + reconnectDelayMs: sharedReconnectDelayMs(1, 1), +}) export function getConfiguredRedisUrl(): string | null { if (getConfiguredCacheProvider() === 'database') return null @@ -296,7 +330,7 @@ export function getRedisClient(): Redis | null { state.client = new Redis(redisUrl, { ...defaults, - commandTimeout: 5000, + commandTimeout: SHARED_COMMAND_TIMEOUT_MS, maxRetriesPerRequest: 5, retryStrategy: (times) => { @@ -304,9 +338,7 @@ export function getRedisClient(): Redis | null { logger.error(`Redis reconnection attempt ${times}`, { nextRetryMs: 30000 }) return 30000 } - const base = Math.min(1000 * 2 ** (times - 1), 10000) - const jitter = randomFloat() * base * 0.3 - const delay = Math.round(base + jitter) + const delay = sharedReconnectDelayMs(times, randomFloat()) state.reconnects++ logger.warn('Redis reconnecting', { attempt: times, nextRetryMs: delay }) return delay diff --git a/apps/sim/lib/core/utils/browser-storage.ts b/apps/sim/lib/core/utils/browser-storage.ts index 32109dcfb58..fd5c46fde9a 100644 --- a/apps/sim/lib/core/utils/browser-storage.ts +++ b/apps/sim/lib/core/utils/browser-storage.ts @@ -117,11 +117,39 @@ export const STORAGE_KEYS = { export class WorkspaceRecencyStorage { private static readonly KEY = STORAGE_KEYS.WORKSPACE_RECENCY + private static readonly CHANGE_EVENT = 'workspace-recency-changed' + + static subscribe(onChange: () => void): () => void { + const onStorage = (event: StorageEvent) => { + if (event.key === WorkspaceRecencyStorage.KEY || event.key === null) onChange() + } + window.addEventListener('storage', onStorage) + window.addEventListener(WorkspaceRecencyStorage.CHANGE_EVENT, onChange) + return () => { + window.removeEventListener('storage', onStorage) + window.removeEventListener(WorkspaceRecencyStorage.CHANGE_EVENT, onChange) + } + } + + /** A stable snapshot lets both sidebars follow visits without render-time writes. */ + static getSnapshot(): string | null { + try { + return window.localStorage.getItem(WorkspaceRecencyStorage.KEY) + } catch { + return null + } + } + + private static save(map: Record): void { + if (BrowserStorage.setItem(WorkspaceRecencyStorage.KEY, map)) { + window.dispatchEvent(new Event(WorkspaceRecencyStorage.CHANGE_EVENT)) + } + } static touch(workspaceId: string): void { const map = WorkspaceRecencyStorage.getAll() map[workspaceId] = Date.now() - BrowserStorage.setItem(WorkspaceRecencyStorage.KEY, map) + WorkspaceRecencyStorage.save(map) } static getAll(): Record { @@ -139,7 +167,7 @@ export class WorkspaceRecencyStorage { static remove(workspaceId: string): void { const map = WorkspaceRecencyStorage.getAll() delete map[workspaceId] - BrowserStorage.setItem(WorkspaceRecencyStorage.KEY, map) + WorkspaceRecencyStorage.save(map) } /** @@ -156,7 +184,7 @@ export class WorkspaceRecencyStorage { } } if (pruned) { - BrowserStorage.setItem(WorkspaceRecencyStorage.KEY, map) + WorkspaceRecencyStorage.save(map) } } @@ -362,22 +390,27 @@ export class MothershipHandoffStorage { * accumulate — "Add to chat" can fire twice before the route swap completes, * and the second write must not drop the first. * @returns True if stored, false when the workspace is empty or the handoff - * carries neither a message nor a context. + * carries no message, context, or attachment. */ static store(handoff: MothershipHandoff, owner: MothershipHandoffOwner): boolean { const workspaceId = typeof owner === 'string' ? owner : undefined const organizationId = typeof owner === 'string' ? undefined : owner.organizationId const message = handoff.message?.trim() + const hasAttachments = Boolean(handoff.fileAttachments?.length) const contexts = handoff.contexts ?? [] - if (!(workspaceId || organizationId) || (!message && contexts.length === 0)) { + if ( + !(workspaceId || organizationId) || + (!message && !hasAttachments && contexts.length === 0) + ) { return false } return BrowserStorage.setItem(MothershipHandoffStorage.KEY, { - ...(message ? { message } : {}), - contexts: message - ? contexts - : [...MothershipHandoffStorage.pendingContexts(owner), ...contexts], + ...(message || hasAttachments ? { message: message ?? '' } : {}), + contexts: + message || hasAttachments + ? contexts + : [...MothershipHandoffStorage.pendingContexts(owner), ...contexts], ...(handoff.fileAttachments?.length ? { fileAttachments: handoff.fileAttachments } : {}), ...(handoff.resumeUserMessageId ? { resumeUserMessageId: handoff.resumeUserMessageId } : {}), ...(handoff.requestMode ? { requestMode: handoff.requestMode } : {}), @@ -398,7 +431,13 @@ export class MothershipHandoffStorage { */ private static pendingContexts(owner: MothershipHandoffOwner): ChatContext[] { const data = BrowserStorage.getItem(MothershipHandoffStorage.KEY, null) - if (!data || data.message || !MothershipHandoffStorage.belongsTo(data, owner)) return [] + if ( + !data || + data.message || + data.fileAttachments?.length || + !MothershipHandoffStorage.belongsTo(data, owner) + ) + return [] if (!data.timestamp || Date.now() - data.timestamp > MothershipHandoffStorage.MAX_AGE_MS) { return [] } @@ -433,10 +472,11 @@ export class MothershipHandoffStorage { MothershipHandoffStorage.clear() const contexts = Array.isArray(data.contexts) ? data.contexts : [] + const hasAttachments = Array.isArray(data.fileAttachments) && data.fileAttachments.length > 0 if ( !(data.workspaceId || data.organizationId) || Boolean(data.workspaceId && data.organizationId) || - (!data.message && contexts.length === 0) || + (!data.message && !hasAttachments && contexts.length === 0) || !data.timestamp || Date.now() - data.timestamp > maxAge ) { @@ -447,7 +487,7 @@ export class MothershipHandoffStorage { if (!assistantSearch.success) return null return { - ...(data.message ? { message: data.message } : {}), + ...(data.message || hasAttachments ? { message: data.message ?? '' } : {}), contexts, ...(data.requestMode === 'assistant' ? { requestMode: 'assistant' as const } : {}), ...(data.assistantSearch ? { assistantSearch: assistantSearch.data } : {}), diff --git a/apps/sim/lib/credential-groups/provider-configuration.test.ts b/apps/sim/lib/credential-groups/provider-configuration.test.ts index 603d360acba..bd833a7f285 100644 --- a/apps/sim/lib/credential-groups/provider-configuration.test.ts +++ b/apps/sim/lib/credential-groups/provider-configuration.test.ts @@ -2,6 +2,18 @@ import { dbChainMockFns, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +const shared = vi.hoisted(() => ({ + env: { + SLACK_SEARCH_APP_ID: '', + SLACK_SEARCH_CLIENT_ID: 'environment-client', + SLACK_SEARCH_CLIENT_SECRET: 'environment-secret', + SLACK_SEARCH_SIGNING_SECRET: 'environment-signing', + }, + flag: vi.fn(), +})) +vi.mock('@/lib/core/config/env', () => ({ env: shared.env })) +vi.mock('@/lib/core/config/feature-flags', () => ({ isFeatureEnabled: shared.flag })) + vi.mock('@/lib/core/security/encryption', () => ({ encryptSecret: async (value: string) => ({ encrypted: `encrypted:${value}` }), decryptSecret: async (value: string) => ({ decrypted: value.replace(/^encrypted:/, '') }), @@ -26,6 +38,8 @@ const configuration = { } beforeEach(() => { resetDbChainMock() + shared.env.SLACK_SEARCH_APP_ID = '' + shared.flag.mockResolvedValue(true) }) describe('organization Slack app references', () => { @@ -36,7 +50,12 @@ describe('organization Slack app references', () => { dbChainMockFns.limit .mockResolvedValueOnce([{ encryptedProviderConfiguration }]) .mockResolvedValueOnce([ - { id: 'A1', clientId: 'client-1', encryptedClientSecret: 'encrypted:current-secret' }, + { + id: 'A1', + clientId: 'client-1', + encryptedClientSecret: 'encrypted:current-secret', + encryptedSigningSecret: 'encrypted:signing', + }, ]) expect( await getSlackCredentialGroupConfiguration({ @@ -50,6 +69,42 @@ describe('organization Slack app references', () => { clientSecret: 'current-secret', }) }) + it.each([true, false])( + 'resolves environment credentials only for an active shared installation (active=%s)', + async (active) => { + shared.env.SLACK_SEARCH_APP_ID = 'A1' + dbChainMockFns.limit + .mockResolvedValueOnce([ + { + encryptedProviderConfiguration: + await encryptCredentialGroupProviderConfiguration(configuration), + }, + ]) + .mockResolvedValueOnce([ + { + id: 'A1', + kind: 'shared', + organizationId: null, + clientId: null, + encryptedClientSecret: null, + encryptedSigningSecret: null, + }, + ]) + .mockResolvedValueOnce(active ? [{ id: 'installation' }] : []) + const result = getSlackCredentialGroupConfiguration({ + organizationId: 'org-1', + credentialGroupId: 'group-1', + }) + if (active) + await expect(result).resolves.toMatchObject({ + clientId: 'environment-client', + clientSecret: 'environment-secret', + appId: 'A1', + teamId: 'T1', + }) + else await expect(result).rejects.toThrow('disabled or removed') + } + ) it('fails when the referenced app is absent from the owning organization', async () => { dbChainMockFns.limit .mockResolvedValueOnce([ diff --git a/apps/sim/lib/credential-groups/provider-configuration.ts b/apps/sim/lib/credential-groups/provider-configuration.ts index 13249edb58c..fd4bde8c0d5 100644 --- a/apps/sim/lib/credential-groups/provider-configuration.ts +++ b/apps/sim/lib/credential-groups/provider-configuration.ts @@ -6,6 +6,7 @@ import { resourceScopeFromOwner } from '@/lib/core/resource-scope' import { resourceScopeCondition } from '@/lib/core/resource-scope.server' import { decryptSecret, encryptSecret } from '@/lib/core/security/encryption' import type { DbOrTx } from '@/lib/db/types' +import { resolveSlackAppCredentials } from '@/lib/slack-search/app-configuration' import { requireSlackSearchAppAvailable } from '@/lib/slack-search/shared-app' const CREDENTIAL_GROUP_PROVIDER_CONFIGURATION_TYPE = @@ -173,12 +174,12 @@ async function resolveSlackConfiguration( .limit(1) if (!installation) throw new Error('The shared Slack installation is disabled or removed') } - const { decrypted: clientSecret } = await decryptSecret(app.encryptedClientSecret) + const resolved = await resolveSlackAppCredentials(app) return { appId: app.id, teamId: configuration.teamId, - clientId: app.clientId, - clientSecret, + clientId: resolved.clientId, + clientSecret: resolved.clientSecret, scopes: configuration.scopes, verifiedAt: configuration.verifiedAt, } diff --git a/apps/sim/lib/credential-groups/slack-managed-users.test.ts b/apps/sim/lib/credential-groups/slack-managed-users.test.ts index b24795083aa..3e32054c599 100644 --- a/apps/sim/lib/credential-groups/slack-managed-users.test.ts +++ b/apps/sim/lib/credential-groups/slack-managed-users.test.ts @@ -24,6 +24,18 @@ const { attempts, redis } = vi.hoisted(() => { } }) +const shared = vi.hoisted(() => ({ + env: { + SLACK_SEARCH_APP_ID: '', + SLACK_SEARCH_CLIENT_ID: 'environment-client', + SLACK_SEARCH_CLIENT_SECRET: 'environment-secret', + SLACK_SEARCH_SIGNING_SECRET: 'environment-signing', + }, + flag: vi.fn(), +})) +vi.mock('@/lib/core/config/env', () => ({ env: shared.env })) +vi.mock('@/lib/core/config/feature-flags', () => ({ isFeatureEnabled: shared.flag })) + vi.mock('@/lib/core/config/redis', () => ({ getRedisClient: () => redis })) vi.mock('@/lib/core/security/encryption', () => ({ encryptSecret: vi.fn(async (value: string) => ({ @@ -70,6 +82,9 @@ describe('Slack managed-user authorization', () => { vi.clearAllMocks() resetDbChainMock() attempts.clear() + shared.env.SLACK_SEARCH_APP_ID = '' + shared.env.SLACK_SEARCH_CLIENT_SECRET = 'environment-secret' + shared.flag.mockResolvedValue(true) }) afterEach(() => { @@ -84,6 +99,7 @@ describe('Slack managed-user authorization', () => { app: { id: 'A123', clientId: 'client-1', + encryptedSigningSecret: `encrypted:${Buffer.from('signing-secret').toString('base64')}`, encryptedClientSecret: `encrypted:${Buffer.from('private-client-secret').toString('base64')}`, revision: 'app-revision', }, @@ -125,6 +141,54 @@ describe('Slack managed-user authorization', () => { await expect(loadSlackManagedUsersAttempt(created.state)).rejects.toThrow('malformed') }) + it.each(['rotation', 'disabled', 'success'] as const)( + 'keeps shared setup state secret-free and rechecks configuration on %s', + async (outcome) => { + shared.env.SLACK_SEARCH_APP_ID = 'ASHARED' + dbChainMockFns.limit + .mockResolvedValueOnce([{ id: 'group-1', updatedAt: new Date(1), options: [] }]) + .mockResolvedValueOnce([ + { + app: { + id: 'ASHARED', + kind: 'shared', + organizationId: null, + clientId: null, + encryptedClientSecret: null, + encryptedSigningSecret: null, + revision: 'old-revision', + }, + teamId: 'T123', + }, + ]) + const created = await createSlackManagedUsersAttempt({ + organizationId: 'org-1', + userId: 'user-1', + credentialGroupId: 'group-1', + appId: 'ASHARED', + teamId: 'T123', + }) + const stored = JSON.parse([...attempts.values()][0]) + expect(stored).toMatchObject({ credentialSource: 'environment', expectedAppId: 'ASHARED' }) + expect(stored).not.toHaveProperty('encryptedClientSecret') + expect(JSON.stringify(stored)).not.toContain('environment-secret') + if (outcome === 'rotation') { + shared.env.SLACK_SEARCH_CLIENT_SECRET = 'rotated' + await expect(consumeSlackManagedUsersAttempt(created.state)).rejects.toThrow('changed') + } else if (outcome === 'disabled') { + shared.flag.mockResolvedValue(false) + await expect(consumeSlackManagedUsersAttempt(created.state)).rejects.toThrow('unavailable') + } else { + await expect(consumeSlackManagedUsersAttempt(created.state)).resolves.toMatchObject({ + clientId: 'environment-client', + clientSecret: 'environment-secret', + organizationId: 'org-1', + }) + await expect(consumeSlackManagedUsersAttempt(created.state)).resolves.toBeNull() + } + } + ) + it('binds the bot token to Slack app and workspace identities', async () => { const fetchMock = vi .fn() @@ -202,6 +266,7 @@ describe('Slack managed-user authorization', () => { const app = { id: 'A123', clientId: 'client-1', + encryptedSigningSecret: `encrypted:${Buffer.from('signing-secret').toString('base64')}`, encryptedClientSecret: `encrypted:${Buffer.from('client-secret').toString('base64')}`, revision: 'app-revision', } diff --git a/apps/sim/lib/credential-groups/slack-managed-users.ts b/apps/sim/lib/credential-groups/slack-managed-users.ts index 5b2863c9d7a..5bd4b689c59 100644 --- a/apps/sim/lib/credential-groups/slack-managed-users.ts +++ b/apps/sim/lib/credential-groups/slack-managed-users.ts @@ -29,7 +29,9 @@ import { } from '@/lib/credential-groups/slack-managed-user-scopes' import type { DbOrTx } from '@/lib/db/types' import { SLACK_CUSTOM_BOT_PROVIDER_ID, SLACK_CUSTOM_BOT_SECRET_TYPE } from '@/lib/oauth/types' +import { resolveSlackAppCredentials } from '@/lib/slack-search/app-configuration' import { requireSlackSearchAppAvailable } from '@/lib/slack-search/shared-app' +import { getSharedSlackSearchAppConfiguration } from '@/lib/slack-search/shared-app-env' const logger = createLogger('SlackManagedUsers') const SLACK_MANAGED_USERS_ATTEMPT_TTL_MS = 10 * 60 * 1000 @@ -54,7 +56,7 @@ interface SlackCustomBotSecret { metadata?: Record } -interface StoredSlackManagedUsersAttempt { +type StoredSlackManagedUsersAttempt = { appRevision?: string version: typeof SLACK_MANAGED_USERS_ATTEMPT_VERSION workspaceId?: string @@ -67,11 +69,13 @@ interface StoredSlackManagedUsersAttempt { expectedAppId: string expectedTeamId: string clientId: string - encryptedClientSecret: string redirectUri: string requiredScopes: string[] createdAt: number -} +} & ( + | { credentialSource: 'environment'; encryptedClientSecret?: never } + | { credentialSource?: undefined; encryptedClientSecret: string } +) export interface SlackManagedUsersAttempt { appRevision?: string @@ -162,7 +166,11 @@ function isStoredAttempt(value: unknown): value is StoredSlackManagedUsersAttemp typeof candidate.expectedAppId === 'string' && typeof candidate.expectedTeamId === 'string' && typeof candidate.clientId === 'string' && - typeof candidate.encryptedClientSecret === 'string' && + (candidate.credentialSource === 'environment' + ? typeof candidate.organizationId === 'string' && + candidate.encryptedClientSecret === undefined + : candidate.credentialSource === undefined && + typeof candidate.encryptedClientSecret === 'string') && typeof candidate.redirectUri === 'string' && Array.isArray(candidate.requiredScopes) && candidate.requiredScopes.length > 0 && @@ -518,10 +526,11 @@ export async function createSlackManagedUsersAttempt(params: { 'invalid_response' ) await requireSlackSearchAppAvailable(configured.app.id) + const app = await resolveSlackAppCredentials(configured.app) identity = { appId: configured.app.id, teamId: configured.teamId } - clientId = configured.app.clientId - clientSecret = (await decryptSecret(configured.app.encryptedClientSecret)).decrypted - appRevision = configured.app.revision + clientId = app.clientId + clientSecret = app.clientSecret + appRevision = app.revision requiredScopes = resolveSlackManagedUserScopes( existingOption ? existingOption.requiredScopes : SLACK_SEARCH_USER_SCOPES ) @@ -545,7 +554,8 @@ export async function createSlackManagedUsersAttempt(params: { const redis = requireRedis() const state = generateId() const redirectUri = getSlackManagedUsersRedirectUri() - const encryptedClientSecret = await encryptSecret(clientSecret) + const sharedApp = + scope.kind === 'organization' ? getSharedSlackSearchAppConfiguration(identity.appId) : null const attempt: StoredSlackManagedUsersAttempt = { version: SLACK_MANAGED_USERS_ATTEMPT_VERSION, ...resourceScopeFields(scope), @@ -559,7 +569,9 @@ export async function createSlackManagedUsersAttempt(params: { expectedTeamId: identity.teamId, clientId, ...(appRevision ? { appRevision } : {}), - encryptedClientSecret: encryptedClientSecret.encrypted, + ...(sharedApp + ? { credentialSource: 'environment' as const } + : { encryptedClientSecret: (await encryptSecret(clientSecret)).encrypted }), requiredScopes, redirectUri, createdAt: Date.now(), @@ -606,7 +618,19 @@ async function parseSlackManagedUsersAttempt( const parsed: unknown = JSON.parse(raw) if (!isStoredAttempt(parsed)) throw new Error('Slack managed-user state is malformed') if (Date.now() - parsed.createdAt > SLACK_MANAGED_USERS_ATTEMPT_TTL_MS) return null - const clientSecret = await decryptSecret(parsed.encryptedClientSecret) + let clientSecret: string + if (parsed.credentialSource === 'environment') { + const app = getSharedSlackSearchAppConfiguration(parsed.expectedAppId) + if (!app || app.revision !== parsed.appRevision || app.clientId !== parsed.clientId) + throw new SlackManagedUsersError( + 'The shared Slack app changed. Start again.', + 'invalid_state' + ) + await requireSlackSearchAppAvailable(app.id) + clientSecret = app.clientSecret + } else { + clientSecret = (await decryptSecret(parsed.encryptedClientSecret)).decrypted + } return { ...resourceScopeFields(resourceScopeFromOwner(parsed)), userId: parsed.userId, @@ -622,7 +646,7 @@ async function parseSlackManagedUsersAttempt( expectedTeamId: parsed.expectedTeamId, clientId: parsed.clientId, ...(parsed.appRevision ? { appRevision: parsed.appRevision } : {}), - clientSecret: clientSecret.decrypted, + clientSecret, redirectUri: parsed.redirectUri, requiredScopes: parsed.requiredScopes, createdAt: parsed.createdAt, @@ -706,11 +730,12 @@ export async function exchangeAndConfigureSlackManagedUsers(params: { .limit(1) .for('update') if (app?.kind === 'shared') await requireSlackSearchAppAvailable(app.id) + const resolved = app ? await resolveSlackAppCredentials(app) : null if ( - !app || + !resolved || !params.attempt.appRevision || - app.revision !== params.attempt.appRevision || - app.clientId !== params.attempt.clientId + resolved.revision !== params.attempt.appRevision || + resolved.clientId !== params.attempt.clientId ) throw new SlackManagedUsersError( 'The Slack app changed during authorization. Start again.', diff --git a/apps/sim/lib/credentials/__integration__/organization-personal-tokens.integration.ts b/apps/sim/lib/credentials/__integration__/organization-personal-tokens.integration.ts index fb1f2ffdd7c..9a86b0f3415 100644 --- a/apps/sim/lib/credentials/__integration__/organization-personal-tokens.integration.ts +++ b/apps/sim/lib/credentials/__integration__/organization-personal-tokens.integration.ts @@ -1,11 +1,13 @@ /** Real storage, encryption, migration, and authorization; no external GitLab calls. */ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { credential, credentialGroup, credentialGroupEnrollment, member, organization, + organizationColumns, permissions, resourcePolicy, user, @@ -95,7 +97,7 @@ describe('organization personal tokens', () => { updatedAt: now, })) ) - await db.insert(organization).values( + await db.insert(withInsertColumns(organization, organizationColumns)).values( [ids.org, ids.foreignOrg].map((id) => ({ id, name: 'Token fixture organization', diff --git a/apps/sim/lib/events/sse-endpoint.ts b/apps/sim/lib/events/sse-endpoint.ts index 63a0111c8d6..9f110e58f08 100644 --- a/apps/sim/lib/events/sse-endpoint.ts +++ b/apps/sim/lib/events/sse-endpoint.ts @@ -5,6 +5,7 @@ * and streams Server-Sent Events with heartbeats and cleanup. */ +import type { SessionPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { randomFloat } from '@sim/utils/random' @@ -58,11 +59,12 @@ export const ROTATION_GRACE_MS = 30_000 export const MAX_UNDRAINED_CHUNKS = 16 export function createWorkspaceSSE(config: WorkspaceSSEConfig) { - const logger = createLogger(`${config.label}-SSE`) - - return async function GET(request: NextRequest): Promise { - const session = await getSession() - if (!session?.user?.id) { + return async function GET( + request: NextRequest, + authenticatedPrincipal?: SessionPrincipal + ): Promise { + const userId = authenticatedPrincipal?.userId ?? (await getSession())?.user?.id + if (!userId) { return new Response('Unauthorized', { status: 401 }) } @@ -72,115 +74,171 @@ export function createWorkspaceSSE(config: WorkspaceSSEConfig) { return new Response('Missing workspaceId query parameter', { status: 400 }) } - const permissions = await getUserEntityPermissions(session.user.id, 'workspace', workspaceId) + const permissions = await getUserEntityPermissions(userId, 'workspace', workspaceId) if (!permissions) { return new Response('Access denied to workspace', { status: 403 }) } - const teardowns: Array<() => void> = [] - let cleaned = false + return createSSEStream(request, { + label: `${config.label}:workspace:${workspaceId}`, + subscriptions: config.subscriptions.map((subscription) => ({ + subscribe: (send) => subscription.subscribe(workspaceId, send), + })), + }) + } +} + +interface SSEStreamConfig { + label: string + subscriptions: Array<{ + subscribe(send: (eventName: string, data: Record) => void): () => void + }> + /** Rechecks a long-lived authorization before each publication and on heartbeats. */ + revalidate?: () => Promise +} - const cleanup = (reason: string) => { - if (cleaned) return - cleaned = true - for (const teardown of teardowns.splice(0)) { - try { - teardown() - } catch (error) { - logger.warn(`SSE teardown failed for workspace ${workspaceId}`, { - reason, - error: getErrorMessage(error), - }) - } +/** Shared SSE transport; callers authorize their scope before opening the stream. */ +export function createSSEStream(request: NextRequest, config: SSEStreamConfig): Response { + const logger = createLogger(`${config.label}-SSE`) + const teardowns: Array<() => void> = [] + let cleaned = false + + const cleanup = (reason: string) => { + if (cleaned) return + cleaned = true + for (const teardown of teardowns.splice(0)) { + try { + teardown() + } catch (error) { + logger.warn(`SSE teardown failed for ${config.label}`, { + reason, + error: getErrorMessage(error), + }) } - logger.info(`SSE connection closed for workspace ${workspaceId}`, { reason }) } + logger.info(`SSE connection closed for ${config.label}`, { reason }) + } - const stream = new ReadableStream({ - start(controller) { - const close = (reason: string) => { - cleanup(reason) - try { - controller.close() - } catch { - // Already closed - } + const stream = new ReadableStream({ + start(controller) { + const close = (reason: string) => { + cleanup(reason) + try { + controller.close() + } catch { + // Already closed } + } - const enqueue = (payload: string): boolean => { - if (cleaned) return false - try { - controller.enqueue(encoder.encode(payload)) - return true - } catch { - close('errored') - return false - } + const enqueue = (payload: string): boolean => { + if (cleaned) return false + try { + controller.enqueue(encoder.encode(payload)) + return true + } catch { + close('errored') + return false } + } - const send = (eventName: string, data: Record) => { - enqueue(`event: ${eventName}\ndata: ${JSON.stringify(data)}\n\n`) + let authorization: Promise | undefined + const revalidate = (): Promise => { + if (!config.revalidate) return Promise.resolve() + authorization ??= config.revalidate().finally(() => { + authorization = undefined + }) + return authorization + } + let pendingEvents = 0 + const send = (eventName: string, data: Record) => { + if (cleaned) return + const payload = `event: ${eventName}\ndata: ${JSON.stringify(data)}\n\n` + if (!config.revalidate) { + enqueue(payload) + return } - - try { - for (const subscription of config.subscriptions) { - teardowns.push(subscription.subscribe(workspaceId, send)) + if (pendingEvents >= MAX_UNDRAINED_CHUNKS) { + close('authorization_backpressure') + return + } + pendingEvents += 1 + void revalidate().then( + () => { + pendingEvents -= 1 + enqueue(payload) + }, + () => { + pendingEvents -= 1 + close('authorization_lost') } + ) + } - const rotationDeadline = - Date.now() + MAX_CONNECTION_MS + randomFloat() * MAX_CONNECTION_JITTER_MS - let rotationStartedAt: number | null = null + try { + for (const subscription of config.subscriptions) { + teardowns.push(subscription.subscribe(send)) + } - const heartbeat = setInterval(() => { - if (cleaned) { - clearInterval(heartbeat) - return - } + const rotationDeadline = + Date.now() + MAX_CONNECTION_MS + randomFloat() * MAX_CONNECTION_JITTER_MS + let rotationStartedAt: number | null = null - const now = Date.now() - if (rotationStartedAt !== null && now - rotationStartedAt >= ROTATION_GRACE_MS) { - close('rotated') - return - } - if (rotationStartedAt === null && now >= rotationDeadline) { - if (enqueue('event: rotate\ndata: {}\n\n')) { - rotationStartedAt = now - } - return - } + const heartbeat = setInterval(() => { + if (cleaned) { + clearInterval(heartbeat) + return + } - const desiredSize = controller.desiredSize - if (desiredSize !== null && desiredSize <= -MAX_UNDRAINED_CHUNKS) { - close('unread') - return + const now = Date.now() + if (rotationStartedAt !== null && now - rotationStartedAt >= ROTATION_GRACE_MS) { + close('rotated') + return + } + if (rotationStartedAt === null && now >= rotationDeadline) { + if (enqueue('event: rotate\ndata: {}\n\n')) { + rotationStartedAt = now } + return + } + + const desiredSize = controller.desiredSize + if (desiredSize !== null && desiredSize <= -MAX_UNDRAINED_CHUNKS) { + close('unread') + return + } + if (config.revalidate) { + void revalidate().then( + () => enqueue(': heartbeat\n\n'), + () => close('authorization_lost') + ) + } else { enqueue(': heartbeat\n\n') - }, HEARTBEAT_INTERVAL_MS) - teardowns.push(() => clearInterval(heartbeat)) - - const listenerScope = new AbortController() - request.signal.addEventListener('abort', () => close('aborted'), { - once: true, - signal: listenerScope.signal, - }) - teardowns.push(() => listenerScope.abort()) - - logger.info(`SSE connection opened for workspace ${workspaceId}`) - } catch (error) { - cleanup('setup_failed') - logger.error(`Failed to open SSE connection for workspace ${workspaceId}`, { - error: getErrorMessage(error), - }) - try { - controller.error(error) - } catch {} - } - }, - cancel() { - cleanup('cancelled') - }, - }) + } + }, HEARTBEAT_INTERVAL_MS) + teardowns.push(() => clearInterval(heartbeat)) + + const listenerScope = new AbortController() + request.signal.addEventListener('abort', () => close('aborted'), { + once: true, + signal: listenerScope.signal, + }) + teardowns.push(() => listenerScope.abort()) + + logger.info(`SSE connection opened for ${config.label}`) + } catch (error) { + cleanup('setup_failed') + logger.error(`Failed to open SSE connection for ${config.label}`, { + error: getErrorMessage(error), + }) + try { + controller.error(error) + } catch {} + } + }, + cancel() { + cleanup('cancelled') + }, + }) - return new Response(stream, { headers: SSE_HEADERS }) - } + return new Response(stream, { headers: SSE_HEADERS }) } diff --git a/apps/sim/lib/execution/execution-signal.test.ts b/apps/sim/lib/execution/execution-signal.test.ts index 4b73887d294..e29f92c1a04 100644 --- a/apps/sim/lib/execution/execution-signal.test.ts +++ b/apps/sim/lib/execution/execution-signal.test.ts @@ -8,17 +8,22 @@ const { connection, mockRedisUrl, mockSubscribe, mockUnsubscribe } = vi.hoisted( connection: { status: 'ready', client: undefined as EventEmitter | undefined, + options: undefined as Record | undefined, + }, + mockRedisUrl: { + value: 'redis://localhost:6379' as string | undefined, + error: undefined as Error | undefined, }, - mockRedisUrl: { value: 'redis://localhost:6379' as string | undefined }, mockSubscribe: vi.fn(), mockUnsubscribe: vi.fn(), })) vi.mock('ioredis', () => ({ default: class extends EventEmitter { - constructor() { + constructor(_url: string, options: Record) { super() connection.client = this + connection.options = options } get status() { @@ -31,15 +36,38 @@ vi.mock('ioredis', () => ({ })) vi.mock('@/lib/core/config/redis', () => ({ - getConfiguredRedisUrl: () => mockRedisUrl.value, - getRedisConnectionDefaults: () => ({}), + getConfiguredRedisUrl: () => { + if (mockRedisUrl.error) throw mockRedisUrl.error + return mockRedisUrl.value + }, + // Realistic defaults: the readiness budget derives from these. The literals + // are inherent to mocking the module that exports the real constants. + getRedisConnectionDefaults: () => ({ connectTimeout: 10_000, disconnectTimeout: 2_000 }), })) +import { coldConnectionBudgetMs } from '@/lib/core/config/redis-budget' import { + connectExecutionSignalHub, getExecutionSignalHub, publishLocalExecutionSignal, } from '@/lib/execution/execution-signal' +/** The readiness budget production derives from the options the subscriber was built with. */ +function readyBudgetMs(): number { + const options = connection.options as { + connectTimeout: number + commandTimeout: number + disconnectTimeout: number + retryStrategy: (attempt: number) => number + } + return coldConnectionBudgetMs({ + connectTimeoutMs: options.connectTimeout, + commandTimeoutMs: options.commandTimeout, + disconnectTimeoutMs: options.disconnectTimeout, + reconnectDelayMs: options.retryStrategy(1), + }) +} + describe('ExecutionSignalHub', () => { beforeEach(() => { vi.clearAllMocks() @@ -48,6 +76,7 @@ describe('ExecutionSignalHub', () => { mockSubscribe.mockResolvedValue(1) mockUnsubscribe.mockResolvedValue(0) mockRedisUrl.value = 'redis://localhost:6379' + mockRedisUrl.error = undefined const signalGlobal = globalThis as typeof globalThis & { _executionSignalHub?: unknown } signalGlobal._executionSignalHub = undefined }) @@ -241,7 +270,7 @@ describe('ExecutionSignalHub', () => { 'Timed out waiting for Redis subscriber readiness' ) - const timeout = vi.advanceTimersByTimeAsync(4000).then(() => { + const timeout = vi.advanceTimersByTimeAsync(readyBudgetMs() - 1000).then(() => { connection.client?.emit('error', new Error('ECONNREFUSED')) return vi.advanceTimersByTimeAsync(1000) }) @@ -356,6 +385,119 @@ describe('ExecutionSignalHub', () => { expect(replacement).not.toHaveBeenCalledWith('unavailable') }) + it('waits exactly the budget derived from the options the subscriber is built with', async () => { + vi.useFakeTimers() + try { + connection.status = 'connect' + const hub = getExecutionSignalHub() + const subscription = hub.subscribe('execution-1', vi.fn()) + const settled = vi.fn() + void subscription.then(settled, settled) + + await vi.advanceTimersByTimeAsync(readyBudgetMs() - 1) + expect(settled).not.toHaveBeenCalled() + + await vi.advanceTimersByTimeAsync(1) + await expect(subscription).rejects.toThrow('Timed out waiting for Redis subscriber readiness') + } finally { + vi.useRealTimers() + } + }) + + it('gives a subscribe that joins another in-flight wait its own full budget', async () => { + vi.useFakeTimers() + try { + connection.status = 'connect' + const hub = getExecutionSignalHub() + const first = hub.subscribe('execution-first', vi.fn()) + const firstSettled = vi.fn() + void first.then(firstSettled, firstSettled) + // Late joiner: the first wait has almost spent its budget when this one begins. + await vi.advanceTimersByTimeAsync(readyBudgetMs() - 1000) + const second = hub.subscribe('execution-second', vi.fn()) + const secondSettled = vi.fn() + void second.then(secondSettled, secondSettled) + + await vi.advanceTimersByTimeAsync(1000) + await expect(first).rejects.toThrow('Timed out waiting for Redis subscriber readiness') + // The first deadline was its own; the second is still waiting on the shared signal. + expect(secondSettled).not.toHaveBeenCalled() + expect(connection.client?.listenerCount('ready')).toBe(2) + + connection.status = 'ready' + connection.client?.emit('ready') + await second + expect(mockSubscribe).toHaveBeenCalledWith( + 'execution:signal:execution-second', + 'execution:cancel' + ) + } finally { + vi.useRealTimers() + } + }) + + it('keeps its waiter accounting exact when one waiter times out before the signal settles', async () => { + vi.useFakeTimers() + try { + connection.status = 'connect' + const hub = getExecutionSignalHub() + // Waiter A will time out; waiter B, started later, is still waiting when it does. + const early = hub.subscribe('execution-early', vi.fn()) + const earlySettled = vi.fn() + void early.then(earlySettled, earlySettled) + await vi.advanceTimersByTimeAsync(readyBudgetMs() - 1000) + const late = hub.subscribe('execution-late', vi.fn()) + await vi.advanceTimersByTimeAsync(1000) + await expect(early).rejects.toThrow('Timed out waiting for Redis subscriber readiness') + + // The signal settles for B — and must not run A's cleanup a second time. + connection.status = 'ready' + connection.client?.emit('ready') + await late + expect(connection.client?.listenerCount('ready')).toBe(1) + expect(connection.client?.listenerCount('end')).toBe(0) + + // Connection drops again: a new subscribe must wait for a fresh ready, + // not reuse a readiness that has already passed. + connection.status = 'connect' + connection.client?.emit('close') + mockSubscribe.mockClear() + const again = hub.subscribe('execution-again', vi.fn()) + const againSettled = vi.fn() + void again.then(againSettled, againSettled) + await vi.advanceTimersByTimeAsync(readyBudgetMs() - 1) + expect(againSettled).not.toHaveBeenCalled() + expect(mockSubscribe).not.toHaveBeenCalled() + + // And when that lone waiter gives up, it must be the one that tears the + // signal down — which only holds if every earlier waiter left exactly once. + await vi.advanceTimersByTimeAsync(1) + await expect(again).rejects.toThrow('Timed out waiting for Redis subscriber readiness') + expect(connection.client?.listenerCount('ready')).toBe(1) + expect(connection.client?.listenerCount('end')).toBe(0) + expect(vi.getTimerCount()).toBe(0) + } finally { + vi.useRealTimers() + } + }) + + it('begins connecting when asked to connect ahead of a subscription', () => { + connection.status = 'connecting' + + connectExecutionSignalHub() + + // Constructing the hub is what dials; the client exists before any subscribe. + expect(connection.client).toBeDefined() + expect(mockSubscribe).not.toHaveBeenCalled() + }) + + it('does not throw when Redis is misconfigured, leaving that to the first subscriber', () => { + mockRedisUrl.error = new Error('Cache capability selected Redis but REDIS_URL is missing') + + expect(() => connectExecutionSignalHub()).not.toThrow() + expect(connection.client).toBeUndefined() + }) + it('uses a process-local signal hub when Redis is not configured', async () => { mockRedisUrl.value = undefined const handler = vi.fn() diff --git a/apps/sim/lib/execution/execution-signal.ts b/apps/sim/lib/execution/execution-signal.ts index c335f0123e9..65554f0f16c 100644 --- a/apps/sim/lib/execution/execution-signal.ts +++ b/apps/sim/lib/execution/execution-signal.ts @@ -3,10 +3,18 @@ import { toError } from '@sim/utils/errors' import { isRecordLike } from '@sim/utils/object' import Redis, { type RedisOptions } from 'ioredis' import { getConfiguredRedisUrl, getRedisConnectionDefaults } from '@/lib/core/config/redis' +import { coldConnectionBudgetMs } from '@/lib/core/config/redis-budget' const logger = createLogger('ExecutionSignalHub') const EXECUTION_SIGNAL_PREFIX = 'execution:signal:' -const SUBSCRIBER_TIMEOUT_MS = 5000 +/** + * Bounds the live `SUBSCRIBE` as well as the handshake commands — ioredis has + * one deadline for both — and an initial subscribe that rejects fails the run, + * so this stays at the tolerance a ready-but-slow server has always been + * given rather than being tightened to diagnose dead handshakes faster. + */ +const SUBSCRIBER_COMMAND_TIMEOUT_MS = 5_000 +const subscriberRetryDelayMs = (attempt: number): number => Math.min(attempt * 500, 5000) export const LEGACY_EXECUTION_CANCEL_CHANNEL = 'execution:cancel' export type ExecutionSignalReason = 'event' | 'cancelled' | 'reconnected' | 'unavailable' @@ -17,6 +25,17 @@ interface ChannelSubscription { acknowledged: boolean } +/** + * Settles on the subscriber's next `ready` or `end`. Shared by every waiter so + * the client carries a single listener pair; `waiters` counts them so the last + * one to leave can `detach` the listeners. + */ +interface ReadySignal { + promise: Promise + detach: () => void + waiters: number +} + export interface ExecutionSignalHub { subscribe(executionId: string, handler: ExecutionSignalHandler): Promise<() => void> } @@ -27,19 +46,36 @@ export function getExecutionSignalChannel(executionId: string): string { class RedisExecutionSignalHub implements ExecutionSignalHub { private readonly subscriber: Redis + /** + * How long any one subscribe waits for readiness: room for one dead attempt + * and then a healthy one, so ioredis's own reconnect can be what rescues a + * stalled connection instead of the wait expiring while the first attempt is + * still being diagnosed. Derived from the exact options the client is built + * with. It is paid against a Redis that is simply unreachable as well, where + * the connect deadline is what runs out and nothing is being diagnosed; that + * is the cost of the recovery, and it widens the window in which a short + * execution timeout can pre-empt the wait and report itself instead. + */ + private readonly readyTimeoutMs: number private readonly handlers = new Map>() private readonly subscriptions = new Map() - private connectionReady: Promise | undefined + private readySignal: ReadySignal | undefined private connectedOnce = false constructor(redisUrl: string) { const options = { ...getRedisConnectionDefaults(redisUrl), - commandTimeout: SUBSCRIBER_TIMEOUT_MS, + commandTimeout: SUBSCRIBER_COMMAND_TIMEOUT_MS, connectionName: 'execution-signal-hub', maxRetriesPerRequest: null, - retryStrategy: (attempt: number) => Math.min(attempt * 500, 5000), + retryStrategy: subscriberRetryDelayMs, } satisfies RedisOptions + this.readyTimeoutMs = coldConnectionBudgetMs({ + connectTimeoutMs: options.connectTimeout, + commandTimeoutMs: options.commandTimeout, + disconnectTimeoutMs: options.disconnectTimeout, + reconnectDelayMs: subscriberRetryDelayMs(1), + }) this.subscriber = new Redis(redisUrl, options) this.subscriber.on('message', (channel: string, message: string) => { if (channel === LEGACY_EXECUTION_CANCEL_CHANNEL) { @@ -131,37 +167,65 @@ class RedisExecutionSignalHub implements ExecutionSignalHub { await this.subscriber.subscribe(...channels) } + /** + * Each waiter runs its own deadline over the shared readiness signal. One + * shared timer — which is what the memoized promise had — hands a waiter that + * joins late only the remainder of the first waiter's budget, down to + * nothing. The last waiter to leave detaches the signal, so a timeout leaves + * nothing attached; a signal that settles clears itself, so a later waiter + * observes the connection afresh rather than a readiness that has passed. + */ private waitForConnectionReady(): Promise { - if (this.connectionReady) return this.connectionReady if (this.subscriber.status === 'end') { return Promise.reject(new Error('Redis subscriber connection ended')) } + const signal = (this.readySignal ??= this.createReadySignal()) + signal.waiters++ + let timer: NodeJS.Timeout | undefined + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout( + () => reject(new Error('Timed out waiting for Redis subscriber readiness')), + this.readyTimeoutMs + ) + }) + return Promise.race([signal.promise, deadline]).finally(() => { + clearTimeout(timer) + if (--signal.waiters === 0) { + signal.detach() + if (this.readySignal === signal) this.readySignal = undefined + } + }) + } - this.connectionReady = new Promise((resolve, reject) => { - const cleanup = () => { - clearTimeout(timeout) - this.subscriber.removeListener('ready', onReady) - this.subscriber.removeListener('end', onEnd) + /** + * The promise is deliberately marked handled: a waiter that gives up stops + * observing it, and an `end` that arrives after the last one has left must + * not surface as an unhandled rejection. + */ + private createReadySignal(): ReadySignal { + const signal: ReadySignal = { promise: Promise.resolve(), detach: () => undefined, waiters: 0 } + signal.promise = new Promise((resolve, reject) => { + const settle = () => { + signal.detach() + if (this.readySignal === signal) this.readySignal = undefined } const onReady = () => { - cleanup() + settle() resolve() } - const fail = (error: Error) => { - cleanup() - reject(error) + const onEnd = () => { + settle() + reject(new Error('Redis subscriber connection ended')) + } + signal.detach = () => { + this.subscriber.removeListener('ready', onReady) + this.subscriber.removeListener('end', onEnd) } - const onEnd = () => fail(new Error('Redis subscriber connection ended')) - const timeout = setTimeout( - () => fail(new Error('Timed out waiting for Redis subscriber readiness')), - SUBSCRIBER_TIMEOUT_MS - ) this.subscriber.once('ready', onReady) this.subscriber.once('end', onEnd) - }).finally(() => { - this.connectionReady = undefined }) - return this.connectionReady + signal.promise.catch(() => undefined) + return signal } private async handleReady(): Promise { @@ -261,6 +325,24 @@ export function getExecutionSignalHub(): ExecutionSignalHub { return executionSignalGlobal._executionSignalHub } +/** + * Begins the hub's subscriber connection ahead of a cancellation subscription, + * so that subscribe does not pay the handshake inside its own readiness + * budget. Constructing the hub is what connects — ioredis dials in its + * constructor — so there is nothing to await. Called at the execution entry + * point, the one path every execution shares, early enough to overlap the work + * ahead of the subscribe. Never throws: a misconfigured URL belongs to the + * first real subscriber, which reports it against the execution that needed + * signals, and a cold hub is only slower, not wrong. + */ +export function connectExecutionSignalHub(): void { + try { + getExecutionSignalHub() + } catch { + return + } +} + export function publishLocalExecutionSignal( executionId: string, reason: Extract diff --git a/apps/sim/lib/execution/payloads/file-secret-provenance.test.ts b/apps/sim/lib/execution/payloads/file-secret-provenance.test.ts new file mode 100644 index 00000000000..0afa418d05b --- /dev/null +++ b/apps/sim/lib/execution/payloads/file-secret-provenance.test.ts @@ -0,0 +1,124 @@ +/** @vitest-environment node */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { metadata, readWorkspaceFile } = vi.hoisted(() => ({ + metadata: vi.fn(), + readWorkspaceFile: vi.fn(), +})) + +vi.mock('@/lib/uploads/server/metadata', () => ({ getFileMetadataByKey: metadata })) +vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', () => ({ + readWorkspaceFileRecordByKey: { execute: readWorkspaceFile }, +})) + +import { resolveStoredFileProvenanceSource } from '@/lib/execution/payloads/file-secret-provenance' + +const context = { + principal: { kind: 'session', userId: 'reader', sessionId: 'session' } as const, + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', +} +const file = { + key: 'execution/workspace-1/workflow-1/execution-1/unique/archive.zip', + context: 'execution' as const, +} +const revision = new Date('2026-09-11T00:00:00.000Z') +const record = { + id: 'canonical-file', + key: file.key, + context: 'execution', + workspaceId: context.workspaceId, + userId: 'writer', + contentUpdatedAt: revision, +} + +describe('stored file provenance source', () => { + beforeEach(() => { + vi.clearAllMocks() + metadata.mockResolvedValue(record) + readWorkspaceFile.mockResolvedValue({ file: {} }) + }) + + it('uses the canonical execution file identity and revision', async () => { + expect(await resolveStoredFileProvenanceSource(file, context)).toEqual({ + identity: { + fileId: record.id, + key: record.key, + context: 'execution', + contentUpdatedAt: revision, + }, + ownerUserId: 'writer', + }) + expect(metadata).toHaveBeenCalledWith(file.key, undefined, { includeDeleted: true }) + }) + + it.each([ + { workspaceId: 'foreign-workspace' }, + { workflowId: 'foreign-workflow' }, + { executionId: 'foreign-execution' }, + ])('refuses an out-of-scope file before metadata lookup: %j', async (scope) => { + await expect(resolveStoredFileProvenanceSource(file, { ...context, ...scope })).rejects.toThrow( + 'File not found' + ) + expect(metadata).not.toHaveBeenCalled() + }) + + it('accepts a causally inherited file key in the same workflow', async () => { + await expect( + resolveStoredFileProvenanceSource(file, { + ...context, + executionId: 'resumed-execution', + fileKeys: [file.key], + }) + ).resolves.toMatchObject({ identity: { fileId: 'canonical-file' } }) + }) + + it('does not let a file key allowlist cross workspaces', async () => { + await expect( + resolveStoredFileProvenanceSource(file, { + ...context, + workspaceId: 'foreign-workspace', + fileKeys: [file.key], + }) + ).rejects.toThrow('File not found') + expect(metadata).not.toHaveBeenCalled() + }) + + it('rejects a forged context before metadata lookup', async () => { + await expect( + resolveStoredFileProvenanceSource({ ...file, context: 'workspace' }, context) + ).rejects.toThrow('File context does not match its storage key') + expect(metadata).not.toHaveBeenCalled() + }) + + it.each([ + { workspaceId: 'foreign-workspace' }, + { context: 'workspace' }, + { context: 'knowledge-base' }, + ])('rejects mismatched canonical metadata: %j', async (changes) => { + metadata.mockResolvedValue({ ...record, ...changes }) + await expect(resolveStoredFileProvenanceSource(file, context)).rejects.toThrow('File not found') + }) + + it('preserves a missing legacy record as absence', async () => { + metadata.mockResolvedValue(null) + await expect(resolveStoredFileProvenanceSource(file, context)).resolves.toBeUndefined() + }) + + it('does not turn metadata lookup failures into legacy absence', async () => { + metadata.mockRejectedValue(new Error('database unavailable')) + await expect(resolveStoredFileProvenanceSource(file, context)).rejects.toThrow( + 'database unavailable' + ) + }) + + it('requires the workspace file use case before resolving a workspace source', async () => { + const workspaceFile = { key: 'workspace/workspace-1/file.txt', context: 'workspace' as const } + readWorkspaceFile.mockRejectedValue(new Error('Workspace access denied')) + await expect(resolveStoredFileProvenanceSource(workspaceFile, context)).rejects.toThrow( + 'Workspace access denied' + ) + expect(metadata).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/execution/payloads/file-secret-provenance.ts b/apps/sim/lib/execution/payloads/file-secret-provenance.ts new file mode 100644 index 00000000000..1af18b999b3 --- /dev/null +++ b/apps/sim/lib/execution/payloads/file-secret-provenance.ts @@ -0,0 +1,61 @@ +import type { Principal } from '@sim/auth/principal' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { + assertUserFileContentAccess, + ExecutionFileAccessError, + type ExecutionMaterializationContext, +} from '@/lib/execution/payloads/materialization.server' +import type { WorkspaceFileSecretProvenanceIdentity } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' +import { getFileMetadataByKey } from '@/lib/uploads/server/metadata' +import { inferContextFromKey } from '@/lib/uploads/utils/file-utils' +import type { UserFile } from '@/executor/types' + +export interface StoredFileProvenanceSource { + identity: WorkspaceFileSecretProvenanceIdentity + ownerUserId: string +} + +/** + * Binds an authorized stored-file read to its canonical content revision. Execution callers pass + * the same trusted run capability used to read the bytes; a file object's id is never an identity. + * Files predating metadata registration retain the caller's existing absence policy. + */ +export async function resolveStoredFileProvenanceSource( + file: Pick, + context: ExecutionMaterializationContext & { principal: Principal; workspaceId: string } +): Promise { + if (!file.key) return undefined + try { + await assertUserFileContentAccess(file, context) + } catch (error) { + if (error instanceof ExecutionFileAccessError) { + throw new OrchestrationError('not_found', 'File not found') + } + throw error + } + const storageContext = inferContextFromKey(file.key) + if (storageContext !== 'workspace' && storageContext !== 'execution') return undefined + + const metadata = await getFileMetadataByKey(file.key, undefined, { includeDeleted: true }) + if (!metadata) return undefined + if ( + (metadata.context !== 'workspace' && + metadata.context !== 'mothership' && + metadata.context !== 'execution') || + metadata.workspaceId !== context.workspaceId || + (storageContext === 'execution' + ? metadata.context !== 'execution' + : metadata.context !== 'workspace' && metadata.context !== 'mothership') + ) { + throw new OrchestrationError('not_found', 'File not found') + } + return { + identity: { + fileId: metadata.id, + key: metadata.key, + context: metadata.context, + contentUpdatedAt: metadata.contentUpdatedAt, + }, + ownerUserId: metadata.userId, + } +} diff --git a/apps/sim/lib/execution/payloads/materialization.server.test.ts b/apps/sim/lib/execution/payloads/materialization.server.test.ts index cdcdcd225d3..c7208738e04 100644 --- a/apps/sim/lib/execution/payloads/materialization.server.test.ts +++ b/apps/sim/lib/execution/payloads/materialization.server.test.ts @@ -1,6 +1,7 @@ /** * @vitest-environment node */ +import { resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' const { mockDownloadServableFileFromStorage, mockReadWorkspaceFileByKey, mockVerifyFileAccess } = @@ -22,7 +23,10 @@ vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', readWorkspaceFileRecordByKey: { execute: mockReadWorkspaceFileByKey }, })) -import { readUserFileContent } from '@/lib/execution/payloads/materialization.server' +import { + readUserFileContent, + readUserFileContentWithContributors, +} from '@/lib/execution/payloads/materialization.server' import type { UserFile } from '@/executor/types' const PDF_SOURCE = Buffer.from('from reportlab.pdfgen import canvas') @@ -40,6 +44,7 @@ const generatedPdf: UserFile = { describe('readUserFileContent', () => { beforeEach(() => { vi.clearAllMocks() + resetDbChainMock() generatedPdf.size = PDF_SOURCE.length mockVerifyFileAccess.mockResolvedValue(true) mockReadWorkspaceFileByKey.mockResolvedValue({ file: { id: 'file-1' } }) @@ -49,6 +54,32 @@ describe('readUserFileContent', () => { }) }) + it('returns rendered contributor identities for the consuming boundary to classify', async () => { + const identity = { + fileId: 'image', + key: 'workspace/workspace-1/image.png', + context: 'workspace' as const, + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + } + const html = '' + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from(html), + contentType: 'text/html', + contributingFiles: [identity], + }) + + await expect( + readUserFileContentWithContributors( + { ...generatedPdf, name: 'page', type: 'text/x-sim-page' }, + { userId: 'user-1', encoding: 'text' } + ) + ).resolves.toEqual({ + content: html, + contributingFiles: [identity], + renderedContributingFiles: [identity], + }) + }) + it('returns the compiled artifact instead of the stored generation source', async () => { const content = await readUserFileContent(generatedPdf, { userId: 'user-1', diff --git a/apps/sim/lib/execution/payloads/materialization.server.ts b/apps/sim/lib/execution/payloads/materialization.server.ts index c5eafdedc6b..b919b3ca20b 100644 --- a/apps/sim/lib/execution/payloads/materialization.server.ts +++ b/apps/sim/lib/execution/payloads/materialization.server.ts @@ -63,6 +63,8 @@ export interface ReadUserFileContentOptions extends ExecutionMaterializationCont export interface ReadUserFileContentResult { content: string contributingFiles?: readonly WorkspaceFileSecretProvenanceIdentity[] + /** Subset transformed by the renderer; consumers apply their own admission policy. */ + renderedContributingFiles?: readonly WorkspaceFileSecretProvenanceIdentity[] } function getLogger(options: ExecutionMaterializationContext): Logger { @@ -215,10 +217,17 @@ function getExecutionKeyParts(key: string): } } +export class ExecutionFileAccessError extends Error { + constructor() { + super('File is not available in this execution.') + this.name = 'ExecutionFileAccessError' + } +} + function assertExecutionFileScope(key: string, options: ExecutionMaterializationContext): void { const parts = getExecutionKeyParts(key) if (!parts) { - throw new Error('File is not available in this execution.') + throw new ExecutionFileAccessError() } const allowedExecutionIds = new Set([ @@ -232,11 +241,11 @@ function assertExecutionFileScope(key: string, options: ExecutionMaterialization options.workflowId === parts.workflowId if (options.workspaceId && parts.workspaceId !== options.workspaceId) { - throw new Error('File is not available in this execution.') + throw new ExecutionFileAccessError() } if (options.workflowId && parts.workflowId !== options.workflowId) { - throw new Error('File is not available in this execution.') + throw new ExecutionFileAccessError() } if (allowedFileKeys.has(key)) { @@ -247,7 +256,7 @@ function assertExecutionFileScope(key: string, options: ExecutionMaterialization !options.executionId || (!allowedExecutionIds.has(parts.executionId) && !workflowScopeAllowed) ) { - throw new Error('File is not available in this execution.') + throw new ExecutionFileAccessError() } } @@ -344,7 +353,26 @@ export async function readUserFileContentWithContributors( throw new Error('Expected a file object with metadata.') } - await assertUserFileContentAccess(file, options) + let sourceIdentity: WorkspaceFileSecretProvenanceIdentity | undefined + const storageContext = file.key ? inferContextFromKey(file.key) : undefined + if ( + (storageContext === 'execution' || storageContext === 'workspace') && + options.principal && + options.workspaceId + ) { + const { resolveStoredFileProvenanceSource } = await import( + '@/lib/execution/payloads/file-secret-provenance' + ) + sourceIdentity = ( + await resolveStoredFileProvenanceSource(file, { + ...options, + principal: options.principal, + workspaceId: options.workspaceId, + }) + )?.identity + } else { + await assertUserFileContentAccess(file, options) + } const maxSourceBytes = options.maxSourceBytes ?? MAX_FUNCTION_FILE_BYTES if (Number.isFinite(file.size) && file.size > maxSourceBytes) { @@ -357,6 +385,7 @@ export async function readUserFileContentWithContributors( let buffer: Buffer | null = null let contributingFiles: readonly WorkspaceFileSecretProvenanceIdentity[] | undefined + let renderedContributingFiles: readonly WorkspaceFileSecretProvenanceIdentity[] | undefined const log = getLogger(options) const requestId = options.requestId ?? 'unknown' @@ -365,7 +394,10 @@ export async function readUserFileContentWithContributors( maxBytes: maxSourceBytes, }) buffer = servable.buffer - contributingFiles = servable.contributingFiles + renderedContributingFiles = servable.contributingFiles + contributingFiles = sourceIdentity + ? [sourceIdentity, ...(servable.contributingFiles ?? [])] + : servable.contributingFiles } catch (error) { if (isPayloadSizeLimitError(error)) { if (isGeneratedDocumentSourceType(file.type) && error.observedBytes !== undefined) { @@ -402,6 +434,7 @@ export async function readUserFileContentWithContributors( return { content: options.encoding === 'base64' ? bufferToBase64(selected) : selected.toString('utf8'), ...(contributingFiles && contributingFiles.length > 0 ? { contributingFiles } : {}), + ...(renderedContributingFiles?.length ? { renderedContributingFiles } : {}), } } diff --git a/apps/sim/lib/function-execution/application/execute-function.test.ts b/apps/sim/lib/function-execution/application/execute-function.test.ts index 156ee2d7542..4f4fa7542d0 100644 --- a/apps/sim/lib/function-execution/application/execute-function.test.ts +++ b/apps/sim/lib/function-execution/application/execute-function.test.ts @@ -26,6 +26,7 @@ vi.mock('@sim/platform-authz/workspace', () => ({ import { FUNCTION_EXECUTION_DELEGATION_AUDIENCE } from '@/lib/function-execution/application/authorization' import { executeFunction } from '@/lib/function-execution/application/execute-function' +import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' const principal: WorkflowExecutionDelegatedPrincipal = { kind: 'delegated', @@ -154,4 +155,28 @@ describe('executeFunction', () => { expect(mocks.loadWorkspace).not.toHaveBeenCalled() expect(mocks.executeRequest).not.toHaveBeenCalled() }) + + it('passes trusted registry state outside the parsed Function wire body', async () => { + const registry = new ResolvedSecretTraceRegistry([], { + userId: 'workspace-owner', + workspaceId: 'workspace-1', + }) + await executeFunction.execute({ + principal, + input: { + workspaceId: 'workspace-1', + body: { + code: 'return 1', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }, + headers: new Headers(), + resolvedSecretTraceRegistry: registry, + }, + }) + + expect(mocks.executeRequest.mock.calls[0][2].resolvedSecretTraceRegistry).toBe(registry) + expect(mocks.executeRequest.mock.calls[0][1]).not.toHaveProperty('resolvedSecretTraceRegistry') + }) }) diff --git a/apps/sim/lib/function-execution/application/execute-function.ts b/apps/sim/lib/function-execution/application/execute-function.ts index 66b73058abe..c5a8e2998c4 100644 --- a/apps/sim/lib/function-execution/application/execute-function.ts +++ b/apps/sim/lib/function-execution/application/execute-function.ts @@ -5,6 +5,7 @@ import { OrchestrationError } from '@/lib/core/orchestration/types' import { functionExecutionDelegationPolicy } from '@/lib/function-execution/application/authorization' import { functionExecutionOperations } from '@/lib/function-execution/application/operations' import { resolveActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' +import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' export interface ExecuteFunctionInput { workspaceId: string @@ -12,6 +13,8 @@ export interface ExecuteFunctionInput { headers: Headers signal?: AbortSignal sandboxProfile?: 'mothership' + /** Trusted in-process provenance state; never accepted from the Function request body. */ + resolvedSecretTraceRegistry?: ResolvedSecretTraceRegistry } /** @@ -57,6 +60,9 @@ export const executeFunction = defineAuthorizedWorkspaceUseCase({ { attributedUserId, principal, + ...(input.resolvedSecretTraceRegistry + ? { resolvedSecretTraceRegistry: input.resolvedSecretTraceRegistry } + : {}), ...(subject?.kind === 'sim_user' ? { fileAccessUserId: subject.userId } : {}), ...(input.sandboxProfile ? { sandboxProfile: input.sandboxProfile } : {}), } diff --git a/apps/sim/lib/function-execution/execute-request.test.ts b/apps/sim/lib/function-execution/execute-request.test.ts index 4b821c084cb..36ceb0a0e59 100644 --- a/apps/sim/lib/function-execution/execute-request.test.ts +++ b/apps/sim/lib/function-execution/execute-request.test.ts @@ -6,11 +6,14 @@ import { readFileSync } from 'node:fs' import { resolve } from 'node:path' import { createMockRequest, + dbChainMockFns, envFlagsMock, hybridAuthMockFns, + resetDbChainMock, resetEnvFlagsMock, workflowsUtilsMock, } from '@sim/testing' +import JSZip from 'jszip' import { NextRequest } from 'next/server' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' import { functionExecuteBodySchema } from '@/lib/api/contracts' @@ -42,6 +45,9 @@ const { mockUploadFile, mockValidateWorkspaceFileWriteTarget, mockWriteWorkspaceFileByPath, + mockUploadExecutionFile, + mockMountContributors, + mockRenderedMountContributors, } = vi.hoisted(() => ({ mockExecuteInSandbox: vi.fn(), mockExecuteInIsolatedVM: vi.fn(), @@ -60,6 +66,9 @@ const { mockUploadFile: vi.fn(), mockValidateWorkspaceFileWriteTarget: vi.fn(), mockWriteWorkspaceFileByPath: vi.fn(), + mockUploadExecutionFile: vi.fn(), + mockMountContributors: vi.fn(), + mockRenderedMountContributors: vi.fn(), })) vi.mock('@/lib/core/security/encryption', () => ({ @@ -146,6 +155,10 @@ vi.mock('@/lib/uploads', () => ({ }, })) +vi.mock('@/lib/uploads/contexts/execution/execution-file-manager', () => ({ + uploadExecutionFile: mockUploadExecutionFile, +})) + vi.mock('@/lib/workflows/utils', () => workflowsUtilsMock) /** @@ -162,6 +175,8 @@ vi.mock('@/lib/function-execution/sandbox-mounts', () => ({ }: { planned: Array<{ userFile: { name: string }; mountPath: string }> }) => ({ + contributingFiles: mockMountContributors(), + renderedContributingFiles: mockRenderedMountContributors(), sandboxFiles: planned.map(({ mountPath }) => ({ type: 'url' as const, path: mountPath, @@ -180,9 +195,14 @@ import { validateExternalUrl } from '@/lib/core/security/input-validation' import { clearLargeValueCacheForTests } from '@/lib/execution/payloads/cache' import { isLargeArrayManifest } from '@/lib/execution/payloads/large-array-manifest-metadata' import { isLargeValueRef } from '@/lib/execution/payloads/large-value-ref' +import * as fileMaterialization from '@/lib/execution/payloads/materialization.server' import { executeFunctionRequest } from '@/lib/function-execution/execute-request' +import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' -async function POST(request: NextRequest): Promise { +async function POST( + request: NextRequest, + resolvedSecretTraceRegistry?: ResolvedSecretTraceRegistry +): Promise { const auth = await hybridAuthMockFns.mockCheckInternalAuth(request) if (!auth.success || !auth.userId) { return Response.json({ error: auth.error || 'Unauthorized' }, { status: 401 }) @@ -204,6 +224,7 @@ async function POST(request: NextRequest): Promise { return executeFunctionRequest({ headers: request.headers, signal: request.signal }, parsed.data, { attributedUserId: auth.userId, + resolvedSecretTraceRegistry, principal: { kind: 'delegated', serviceId: 'executor', @@ -247,6 +268,18 @@ const MOUNT_REF = { describe('Function execution request', () => { beforeEach(() => { vi.clearAllMocks() + resetDbChainMock() + mockMountContributors.mockReturnValue(undefined) + mockRenderedMountContributors.mockReturnValue(undefined) + mockUploadExecutionFile.mockImplementation(async (context, buffer, name, type) => ({ + id: 'execution-file-1', + key: `execution/${context.workspaceId}/${context.workflowId}/${context.executionId}/file/${name}`, + context: 'execution', + name, + type, + size: buffer.length, + url: 'https://presigned.example/output', + })) envFlagsMock.isRemoteSandboxEnabled = false envFlagsMock.isMothershipSandboxEnabled = false @@ -1792,6 +1825,375 @@ describe('Function execution request', () => { expect(data.error).toContain('21 files') }) + it.each([ + { name: 'report.zip', secret: undefined, expectedStatus: 'exact' }, + { name: 'report.zip', secret: 'super-secret-value', expectedStatus: 'unknown' }, + { name: 'report.txt', secret: 'super-secret-value', expectedStatus: 'unknown' }, + ])( + 'preserves binary provenance for harvested $name with secret=$secret', + async ({ name, secret, expectedStatus }) => { + envFlagsMock.isRemoteSandboxEnabled = true + const zip = new JSZip() + zip.file('report.txt', secret ?? 'ordinary report') + const buffer = await zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' }) + expect(buffer.includes('super-secret-value')).toBe(false) + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: name, + path: `/tmp/sim/outputs/${name}`, + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + + const response = await POST( + createMockRequest('POST', { + code: secret ? 'token = {{MY_SECRET}}' : 'x = 1', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + ...(secret ? { envVars: { MY_SECRET: secret } } : {}), + }) + ) + + expect(response.status).toBe(200) + expect(mockUploadExecutionFile).toHaveBeenCalledWith( + expect.any(Object), + buffer, + name, + expect.any(String), + 'user-123', + expectedStatus === 'exact' ? { status: 'exact', entries: [] } : { status: 'unknown' } + ) + const data = await response.json() + expect(data.output.files[0]).not.toHaveProperty('secretProvenance') + } + ) + + it('keeps text-looking bytes opaque when their declared format is an archive', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + const buffer = Buffer.from('ASCII archive placeholder') + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: 'report.zip', + path: '/tmp/sim/outputs/report.zip', + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + const response = await POST( + createMockRequest('POST', { + code: 'token = {{MY_SECRET}}', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + envVars: { MY_SECRET: 'super-secret-value' }, + }) + ) + expect(response.status).toBe(200) + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'unknown' }) + }) + + it('refuses an unknown tracked execution mount before running code', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + const updatedAt = new Date('2026-01-01T00:00:00Z') + mockMountContributors.mockReturnValue([ + { + fileId: 'execution-file-1', + key: 'execution/workspace-1/workflow-1/execution-1/a/input.zip', + context: 'execution', + contentUpdatedAt: updatedAt, + }, + ]) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: updatedAt, + secretProvenanceVersion: 1, + provenanceContentUpdatedAt: updatedAt, + status: 'unknown', + entries: [], + }, + ]) + + const response = await POST( + createMockRequest('POST', { + code: 'x = 1', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }) + ) + expect(response.status).toBe(400) + expect((await response.json()).error).toContain('File secret provenance is unavailable') + expect(mockExecuteInSandbox).not.toHaveBeenCalled() + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + }) + + it('imports exact mount secrets into the trusted result registry and binary export classifier', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + const updatedAt = new Date('2026-01-01T00:00:00Z') + const registry = new ResolvedSecretTraceRegistry([], { + userId: 'user-123', + workspaceId: 'workspace-1', + }) + const completePending = registry.beginPendingActivation() + mockMountContributors.mockReturnValue([ + { + fileId: 'execution-file-1', + key: 'execution/workspace-1/workflow-1/execution-1/a/input.txt', + context: 'execution', + contentUpdatedAt: updatedAt, + }, + ]) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: updatedAt, + secretProvenanceVersion: 1, + provenanceContentUpdatedAt: updatedAt, + status: 'exact', + entries: [ + { + name: 'API_KEY', + encryptedValue: 'encrypted:mounted-secret', + sourceUserId: 'user-123', + sourceWorkspaceId: 'workspace-1', + }, + ], + }, + ]) + const zip = new JSZip() + zip.file('result.txt', 'mounted-secret') + const buffer = await zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' }) + mockExecuteInSandbox.mockResolvedValueOnce({ + result: 'mounted-secret', + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: 'result.zip', + path: '/tmp/sim/outputs/result.zip', + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + const response = await POST( + createMockRequest('POST', { + code: 'x = 1', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }), + registry + ) + completePending() + expect(response.status).toBe(200) + expect(registry.exportProvenance().entries).toEqual([ + expect.objectContaining({ encryptedValue: 'encrypted:mounted-secret' }), + ]) + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'unknown' }) + }) + + it.each([ + { input: 'contextVariables', archive: true, unredacted: false }, + { input: 'params', archive: true, unredacted: false }, + { input: 'contextVariables', archive: false, unredacted: false }, + { input: 'contextVariables', archive: true, unredacted: true }, + ] as const)( + 'classifies secret-bearing $input with archive=$archive and unredacted=$unredacted', + async ({ input, archive, unredacted }) => { + envFlagsMock.isRemoteSandboxEnabled = true + const plaintext = 'table-input-secret-value' + const registry = new ResolvedSecretTraceRegistry( + [ + { + name: 'API_KEY', + plaintext, + encryptedValue: plaintext, + scope: 'workspace', + ...(unredacted ? { unredacted: true as const } : {}), + }, + ], + { userId: 'user-123', workspaceId: 'workspace-1' } + ) + registry.recordResolvedAtInputPath('API_KEY', plaintext, [input, 'token']) + const zip = new JSZip() + zip.file('report.txt', plaintext) + const buffer = archive + ? await zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' }) + : Buffer.from(plaintext) + const name = archive ? 'report.zip' : 'report.txt' + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: name, + path: `/tmp/sim/outputs/${name}`, + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + + const response = await POST( + createMockRequest('POST', { + code: input === 'params' ? "x = params['token']" : 'x = token', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + [input]: { token: plaintext }, + }), + registry + ) + + expect(response.status).toBe(archive || unredacted ? 200 : 400) + if (archive) { + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual( + unredacted ? { status: 'exact', entries: [] } : { status: 'unknown' } + ) + } else { + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + } + } + ) + + it.each([ + { reason: 'source-provenance-incomplete', status: 200 }, + { reason: 'entry-decrypt-failed', status: 400 }, + ] as const)( + 'distinguishes historical absence from provenance faults: $reason', + async ({ reason, status }) => { + envFlagsMock.isRemoteSandboxEnabled = true + const registry = new ResolvedSecretTraceRegistry([], { + userId: 'user-123', + workspaceId: 'workspace-1', + }) + registry.markIncomplete(reason) + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + mockMountContributors.mockReturnValue([ + { + fileId: 'legacy-file', + key: 'execution/workspace-1/workflow-1/execution-1/a/input.txt', + context: 'execution', + contentUpdatedAt, + }, + ]) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: contentUpdatedAt, + secretProvenanceVersion: null, + provenanceContentUpdatedAt: null, + status: null, + entries: null, + }, + ]) + const buffer = Buffer.from('ordinary file') + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: 'report.zip', + path: '/tmp/sim/outputs/report.zip', + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + const response = await POST( + createMockRequest('POST', { + code: 'x = 1', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }), + registry + ) + expect(response.status).toBe(status) + if (status === 200) + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'unrecorded' }) + else expect(mockUploadExecutionFile).not.toHaveBeenCalled() + } + ) + + it('does not taint a secret-free mounted file with unrelated secrets from an earlier block', async () => { + envFlagsMock.isRemoteSandboxEnabled = true + const updatedAt = new Date('2026-01-01T00:00:00Z') + const scope = { userId: 'user-123', workspaceId: 'workspace-1' } + const registry = new ResolvedSecretTraceRegistry([], scope) + await registry.importProvenance( + { + version: 1, + complete: true, + scope, + entries: [{ name: 'OTHER_SECRET', encryptedValue: 'unrelated-secret-value' }], + }, + { trusted: true } + ) + mockMountContributors.mockReturnValue([ + { + fileId: 'execution-file-1', + key: 'execution/workspace-1/workflow-1/execution-1/a/input.txt', + context: 'execution', + contentUpdatedAt: updatedAt, + }, + ]) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: updatedAt, + secretProvenanceVersion: 1, + provenanceContentUpdatedAt: updatedAt, + status: 'exact', + entries: [], + }, + ]) + const buffer = Buffer.from('archive without secret inputs') + mockExecuteInSandbox.mockResolvedValueOnce({ + result: null, + stdout: '', + sandboxId: 'sbx', + collectedFiles: [ + { + relativePath: 'result.zip', + path: '/tmp/sim/outputs/result.zip', + contentBase64: buffer.toString('base64'), + byteLength: buffer.length, + }, + ], + }) + const response = await POST( + createMockRequest('POST', { + code: 'x = 1', + language: 'python', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }), + registry + ) + expect(response.status).toBe(200) + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'exact', entries: [] }) + }) + it('scans a harvested plaintext secret even under a binary file name', async () => { envFlagsMock.isRemoteSandboxEnabled = true mockExecuteInSandbox.mockResolvedValueOnce({ @@ -2479,6 +2881,112 @@ describe('Function execution request', () => { expect(data.success).toBe(true) expect(options?.brokers).toHaveProperty('sim.values.readArray') }) + + it.each([ + { status: 'exact', version: 1, secret: true, safe: false }, + { status: 'unknown', version: 1, secret: false, safe: false }, + { status: 'exact', version: 1, secret: false, safe: true }, + { status: null, version: null, secret: false, safe: true }, + ])( + 'applies rendered asset policy at Function admission while retaining legacy compatibility: %j', + async ({ status, version, secret, safe }) => { + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + const identity = { + fileId: 'image-1', + key: 'workspace/workspace-1/image.png', + context: 'workspace' as const, + contentUpdatedAt, + } + const materialized = { + content: '', + contributingFiles: [identity], + renderedContributingFiles: [identity], + } + const read = vi + .spyOn(fileMaterialization, 'readUserFileContentWithContributors') + .mockResolvedValue(materialized) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: contentUpdatedAt, + provenanceContentUpdatedAt: contentUpdatedAt, + secretProvenanceVersion: version, + status, + entries: secret + ? [{ name: 'TOKEN', encryptedValue: 'ciphertext', sourceUserId: 'user-1' }] + : [], + }, + ]) + mockExecuteInIsolatedVM.mockImplementationOnce(async (_input, options) => ({ + result: await options.brokers['sim.files.readText']({ file: MOUNT_REF.file }), + stdout: '', + })) + const request = { + code: 'return 1', + language: 'javascript', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + } + try { + const brokerResponse = await POST(createMockRequest('POST', request)) + const brokerBody = await brokerResponse.json() + expect(brokerBody.success).toBe(safe) + if (!safe) expect(JSON.stringify(brokerBody)).not.toContain(materialized.content) + + mockMountContributors.mockReturnValue([identity]) + mockRenderedMountContributors.mockReturnValue([identity]) + const mountResponse = await POST(createMockRequest('POST', request)) + expect(mountResponse.status).toBe(safe ? 200 : 400) + expect((await mountResponse.json()).success).toBe(safe) + } finally { + read.mockRestore() + } + } + ) + + it('refuses unknown execution provenance returned by the runtime file broker', async () => { + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + vi.spyOn(fileMaterialization, 'readUserFileContentWithContributors').mockResolvedValueOnce({ + content: 'private file content', + contributingFiles: [ + { + fileId: 'execution-file-1', + key: 'execution/workspace-1/workflow-1/execution-1/a/input.txt', + context: 'execution', + contentUpdatedAt, + }, + ], + }) + dbChainMockFns.limit.mockResolvedValue([ + { + fileContentUpdatedAt: contentUpdatedAt, + secretProvenanceVersion: 1, + provenanceContentUpdatedAt: contentUpdatedAt, + status: 'unknown', + entries: [], + }, + ]) + mockExecuteInIsolatedVM.mockImplementationOnce(async (_input, options) => ({ + result: await options.brokers['sim.files.readText']({ file: MOUNT_REF.file }), + stdout: '', + })) + + const response = await POST( + createMockRequest('POST', { + code: 'return 1', + language: 'javascript', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + }) + ) + + expect(response.status).toBe(500) + const data = await response.json() + expect(data.success).toBe(false) + expect(data.error).toContain('File secret provenance is unavailable') + expect(JSON.stringify(data)).not.toContain('private file content') + }) }) describe('Template Variable Resolution', () => { diff --git a/apps/sim/lib/function-execution/execute-request.ts b/apps/sim/lib/function-execution/execute-request.ts index 1392221fbd0..020b7b32f9d 100644 --- a/apps/sim/lib/function-execution/execute-request.ts +++ b/apps/sim/lib/function-execution/execute-request.ts @@ -55,7 +55,7 @@ import { MAX_INLINE_MATERIALIZATION_BYTES, } from '@/lib/execution/payloads/limits' import { - readUserFileContent, + readUserFileContentWithContributors, unavailableLargeValueError, } from '@/lib/execution/payloads/materialization.server' import { @@ -93,9 +93,13 @@ import { isExecutionResourceLimitError } from '@/lib/execution/resource-errors' import { planUserFileMounts, resolveUserFileMounts } from '@/lib/function-execution/sandbox-mounts' import { uploadExecutionFile } from '@/lib/uploads/contexts/execution/execution-file-manager' import { + createWorkspaceFileSecretProvenanceFromRegistry, EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE, + importWorkspaceFileSecretProvenanceForRuntime, + isOpaqueWorkspaceFileEgressSafe, mergeWorkspaceFileSecretProvenance, type WorkspaceFileSecretProvenance, + type WorkspaceFileSecretProvenanceIdentity, } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' import { deleteFiles } from '@/lib/uploads/core/storage-service' import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' @@ -117,7 +121,12 @@ import { scanResolvedSecretString, } from '@/executor/utils/resolved-secret-content-projection' import { isNonIdentifyingSecretLiteral } from '@/executor/utils/resolved-secret-match-policy' -import type { ResolvedSecretTraceProvenanceV1 } from '@/executor/utils/resolved-secret-trace-registry' +import type { + ResolvedSecretTraceProvenanceV1, + ResolvedSecretTraceRegistry, +} from '@/executor/utils/resolved-secret-trace-registry' + +const TEXT_OUTPUT_MIME_TYPES = new Set(Object.values(FORMAT_TO_CONTENT_TYPE)) const logger = createLogger('FunctionExecuteAPI') @@ -1014,6 +1023,74 @@ interface FunctionRouteExecutionContext { */ unredactedSecretNames: Set mountedFileSecretProvenanceScanner?: MountedFileSecretProvenanceScanner + runtimeFileSecretProvenanceScanner?: MountedFileSecretProvenanceScanner + runtimeFileSecretTraceRegistry?: ResolvedSecretTraceRegistry + runtimeInputProvenanceUnrecorded?: boolean + resolvedSecretTraceRegistry?: ResolvedSecretTraceRegistry +} + +/** Keeps bound file provenance in both ordinary Function results and exported artifact bytes. */ +async function importRuntimeFileContributors( + context: FunctionRouteExecutionContext, + identities: readonly WorkspaceFileSecretProvenanceIdentity[] | undefined, + renderedIdentities: readonly WorkspaceFileSecretProvenanceIdentity[] = [] +): Promise { + if (!identities?.length && renderedIdentities.length === 0) return + if (!context.workspaceId) throw new Error('File provenance requires a workspace') + /** Sim-rendered assets can encode literals before user code runs; raw files retain runtime lineage. */ + for (const identity of renderedIdentities) { + if (!(await isOpaqueWorkspaceFileEgressSafe(context.workspaceId, identity))) { + throw new Error('File secret provenance is unavailable for Function execution') + } + } + if (!context.runtimeInputProvenanceUnrecorded) { + context.runtimeFileSecretTraceRegistry ??= + context.resolvedSecretTraceRegistry?.forkForInputPaths([]) + } + for (const identity of identities ?? []) { + const imported = await importWorkspaceFileSecretProvenanceForRuntime({ + workspaceId: context.workspaceId, + identity, + registry: context.runtimeFileSecretTraceRegistry, + actorUserId: context.fileAccessUserId, + }) + if (!imported) throw new Error('File secret provenance is unavailable for Function execution') + } + if (context.runtimeFileSecretTraceRegistry && context.resolvedSecretTraceRegistry) { + context.resolvedSecretTraceRegistry.mergeToolCallRegistry( + context.runtimeFileSecretTraceRegistry + ) + } +} + +/** Includes only lineage carried by the values this Function receives, including deferred refs. */ +async function importRuntimeInputProvenance( + context: FunctionRouteExecutionContext, + inputs: { + code: string + params: Record + contextVariables: Record + } +): Promise { + const registry = context.resolvedSecretTraceRegistry + if (!registry) return + const valueProvenance = registry.exportCommittedProvenanceForValue(inputs) + if (!valueProvenance.complete && context.workspaceId) { + const decision = await createWorkspaceFileSecretProvenanceFromRegistry(registry, inputs, { + userId: context.attributedUserId, + workspaceId: context.workspaceId, + }) + if (decision.safe && decision.provenance.status === 'unrecorded') { + context.runtimeInputProvenanceUnrecorded = true + return + } + } + const inputRegistry = registry.forkForInputPaths(Object.keys(inputs).map((key) => [key])) + await inputRegistry.importProvenance(valueProvenance, { + trusted: true, + origin: 'function.runtimeInputs', + }) + context.runtimeFileSecretTraceRegistry = inputRegistry } type ResolvedSecretNamesMetadataType = @@ -1117,7 +1194,7 @@ function createFunctionRuntimeBrokers( const readFile = async (args: unknown, encoding: 'base64' | 'text', chunked = false) => { const fileArgs = getBrokerFileArgs(args) - return readUserFileContent(fileArgs.file, { + const materialized = await readUserFileContentWithContributors(fileArgs.file, { ...base, encoding, maxBytes: fileArgs.maxBytes, @@ -1125,6 +1202,12 @@ function createFunctionRuntimeBrokers( offset: chunked ? fileArgs.offset : undefined, length: chunked ? fileArgs.length : undefined, }) + await importRuntimeFileContributors( + context, + materialized.contributingFiles, + materialized.renderedContributingFiles + ) + return materialized.content } return { @@ -1256,7 +1339,10 @@ function countProtectedOutputSecretNames(context: FunctionRouteExecutionContext) */ function hasSecretMaterialInScope(context: FunctionRouteExecutionContext): boolean { if (countProtectedOutputSecretNames(context) > 0) return true - return context.mountedFileSecretProvenanceScanner?.hasSecrets ?? false + return Boolean( + context.mountedFileSecretProvenanceScanner?.hasSecrets || + context.runtimeFileSecretProvenanceScanner?.hasSecrets + ) } /** @@ -1274,15 +1360,34 @@ async function getOutputFileSecretProvenance( context: FunctionRouteExecutionContext, scope: { userId: string; workspaceId: string } ): Promise { + /** Runtime reads have settled before export; a broker cannot replace this with an older snapshot. */ + if (context.runtimeFileSecretTraceRegistry && !context.runtimeFileSecretProvenanceScanner) { + const provenance = context.runtimeFileSecretTraceRegistry.exportProvenance() + context.runtimeFileSecretProvenanceScanner = + await createMountedFileSecretProvenanceScanner(provenance) + if (!context.runtimeFileSecretProvenanceScanner && provenance.entries.length > 0) { + context.runtimeFileSecretProvenanceScanner = { + hasSecrets: true, + scan: () => ({ status: 'unknown' }), + } + } + } if (isBinary) { return hasSecretMaterialInScope(context) ? { status: 'unknown' } - : EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE - } - const mountedFileProvenance = context.mountedFileSecretProvenanceScanner?.scan(buffer) ?? { - status: 'exact' as const, - entries: [], + : context.runtimeInputProvenanceUnrecorded + ? { status: 'unrecorded' } + : EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE } + const mountedFileProvenance = mergeWorkspaceFileSecretProvenance( + context.mountedFileSecretProvenanceScanner?.scan(buffer) ?? + EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE, + context.runtimeFileSecretProvenanceScanner?.scan(buffer) ?? + EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE, + context.runtimeInputProvenanceUnrecorded + ? { status: 'unrecorded' } + : EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE + ) if (countProtectedOutputSecretNames(context) === 0) { return mountedFileProvenance } @@ -1531,12 +1636,11 @@ async function maybeExportSandboxFileToWorkspace(args: { const fileName = normalizeOutputWorkspaceFileName(outputPath) - const TEXT_MIMES = new Set(Object.values(FORMAT_TO_CONTENT_TYPE)) const resolvedMimeType = outputMimeType || FORMAT_TO_CONTENT_TYPE[resolveOutputFormat(fileName, outputFormat)] || 'application/octet-stream' - const isBinary = !TEXT_MIMES.has(resolvedMimeType) + const isBinary = !TEXT_OUTPUT_MIME_TYPES.has(resolvedMimeType) const outputBytes = Buffer.byteLength(exportedFileContent, isBinary ? 'base64' : 'utf-8') if (outputBytes > MAX_SANDBOX_OUTPUT_BYTES) { return exportFailure( @@ -1711,7 +1815,7 @@ async function maybeExportSandboxFilesToWorkspace(args: { file.mimeType || FORMAT_TO_CONTENT_TYPE[resolveOutputFormat(fileName, file.format)] || 'application/octet-stream' - const isBinary = !new Set(Object.values(FORMAT_TO_CONTENT_TYPE)).has(resolvedMimeType) + const isBinary = !TEXT_OUTPUT_MIME_TYPES.has(resolvedMimeType) const size = Buffer.byteLength(content, isBinary ? 'base64' : 'utf-8') totalOutputBytes += size if (totalOutputBytes > MAX_SANDBOX_OUTPUT_BYTES) { @@ -1926,16 +2030,6 @@ function collectedFileName(relativePath: string): string { return sanitizeFileName(relativePath.split('/').filter(Boolean).join('-')) || 'file' } -/** - * Persists files harvested from the sandbox output directory as platform file - * objects, so any downstream tool that accepts a file can consume them. - * - * Uploaded here, one at a time, rather than handed to the declarative - * file-output pipeline as bytes: that path would carry the whole export budget - * as base64 through `JSON.stringify`, a response buffer, and a re-parse, so - * several multiples of the payload would be live at once for a value that is a - * couple of hundred bytes per file once stored. - */ /** * Removes files already uploaded when a later one in the same harvest is refused. * @@ -1959,6 +2053,7 @@ async function discardUploadedExecutionFiles(files: readonly UserFile[]): Promis } } +/** Uploads harvested files sequentially, retaining their private provenance beside stored bytes. */ async function collectExecutionOutputFiles(args: { routeContext: FunctionRouteExecutionContext authUserId: string @@ -2001,38 +2096,41 @@ async function collectExecutionOutputFiles(args: { const name = collectedFileName(collected.relativePath) const mimeType = getMimeTypeFromExtension(getFileExtension(name)) - // Scanned unconditionally — never gated on whether the bytes look textual. - // Both a filename check and a UTF-8 round-trip were trivially defeated: name - // the file `.png`, or append one invalid byte, and a plaintext secret sailed - // past. A lossy UTF-8 decode preserves ASCII runs, so a literal secret is - // findable in any buffer, textual or not. - // - // What stays out of reach is a secret carried in transformed form — deflated - // inside a PDF, re-encoded — which no substring scan can see. That is an - // inherent limit of scanning, not a hole in the gate, and it is why these - // files are execution-scoped rather than durable workspace files. - { - const provenance = await getOutputFileSecretProvenance(buffer, false, routeContext, { - userId: args.authUserId, - workspaceId: resolvedWorkspaceId, - }) - // An execution-scoped file has nowhere to record a provenance envelope, so - // one carrying a resolved secret cannot ship under a lock the way a - // workspace file can — it is refused instead. - if (provenance.status !== 'exact' || provenance.entries.length > 0) { - await discardUploadedExecutionFiles(files) - return { - response: exportFailure( - `Sandbox output file "${name}" contains a resolved secret value and was not returned. Write the file without embedding secret values, or export it to a workspace file where its provenance can be recorded.`, - 400, - args.stdout, - args.executionTime, - args.cost - ), - } + /** Literal secrets must be refused regardless of the export's name or encoding. */ + const scannedProvenance = await getOutputFileSecretProvenance(buffer, false, routeContext, { + userId: args.authUserId, + workspaceId: resolvedWorkspaceId, + }) + if ( + scannedProvenance.status === 'unknown' || + (scannedProvenance.status === 'exact' && scannedProvenance.entries.length > 0) + ) { + await discardUploadedExecutionFiles(files) + return { + response: exportFailure( + `Sandbox output file "${name}" contains a resolved secret value and was not returned. Write the file without embedding secret values, or export it to a workspace file where its provenance can be recorded.`, + 400, + args.stdout, + args.executionTime, + args.cost + ), } } + /** + * A literal scan cannot vouch for encoded secrets in an archive or binary document. + * Persist that uncertainty so a later conversion cannot turn these bytes into a trusted + * workspace file. Both the format and bytes must be textual before a scan is sufficient. + */ + const isBinary = + !TEXT_OUTPUT_MIME_TYPES.has(mimeType) || !isUtf8(buffer) || buffer.includes(0) + const secretProvenance = isBinary + ? await getOutputFileSecretProvenance(buffer, true, routeContext, { + userId: args.authUserId, + workspaceId: resolvedWorkspaceId, + }) + : scannedProvenance + const userFile = await uploadExecutionFile( { workspaceId: resolvedWorkspaceId, @@ -2042,7 +2140,8 @@ async function collectExecutionOutputFiles(args: { buffer, name, mimeType, - args.authUserId + args.authUserId, + secretProvenance ) files.push(userFile) } @@ -2070,6 +2169,7 @@ export interface TrustedFunctionExecutionAuth { fileAccessUserId?: string principal: DelegatedPrincipal sandboxProfile?: 'mothership' + resolvedSecretTraceRegistry?: ResolvedSecretTraceRegistry } /** Executes the Function protocol after the application operation authorizes its principal. */ @@ -2284,6 +2384,7 @@ export async function executeFunctionRequest( unredactedSecretNames.filter((name) => Object.hasOwn(envVars, name)) ), mountedFileSecretProvenanceScanner, + resolvedSecretTraceRegistry: auth.resolvedSecretTraceRegistry, } const lang = isValidCodeLanguage(language) ? language : DEFAULT_CODE_LANGUAGE @@ -2377,6 +2478,11 @@ export async function executeFunctionRequest( for (const binding of compilation.bindings) { setRecordValue(contextVariables, binding.name, binding.value) } + await importRuntimeInputProvenance(routeContext, { + code: resolvedCode, + params: executionParams, + contextVariables, + }) if (lang === CodeLanguage.Shell && containsLargeValueRef(contextVariables)) { throw new Error( 'Large execution values require the JavaScript isolated-vm runtime. Select a nested field or read the value in a JavaScript function.' @@ -2479,6 +2585,11 @@ export async function executeFunctionRequest( logger, }, }) + await importRuntimeFileContributors( + routeContext, + resolvedMounts.contributingFiles, + resolvedMounts.renderedContributingFiles + ) } catch (error) { // Everything this can raise is about the files the caller named — a mount // it may not read, one over a size ceiling, a set over the aggregate. The @@ -3258,3 +3369,5 @@ export async function executeFunctionRequest( executionDeadlineController?.cleanup() } } + +import { isUtf8 } from 'node:buffer' diff --git a/apps/sim/lib/function-execution/sandbox-mounts.test.ts b/apps/sim/lib/function-execution/sandbox-mounts.test.ts index 2f49c989885..6557a35cb0d 100644 --- a/apps/sim/lib/function-execution/sandbox-mounts.test.ts +++ b/apps/sim/lib/function-execution/sandbox-mounts.test.ts @@ -14,11 +14,13 @@ const { mockGeneratePresignedDownloadUrl, mockDownloadServableFileFromStorage, mockReadWorkspaceFileRecordByKey, + mockGetFileMetadataByKey, } = vi.hoisted(() => ({ mockHasCloudStorage: vi.fn(), mockGeneratePresignedDownloadUrl: vi.fn(), mockDownloadServableFileFromStorage: vi.fn(), mockReadWorkspaceFileRecordByKey: vi.fn(), + mockGetFileMetadataByKey: vi.fn(), })) vi.mock('@/lib/uploads/core/storage-service', () => ({ @@ -34,6 +36,10 @@ vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', readWorkspaceFileRecordByKey: { execute: mockReadWorkspaceFileRecordByKey }, })) +vi.mock('@/lib/uploads/server/metadata', () => ({ + getFileMetadataByKey: mockGetFileMetadataByKey, +})) + import { MOUNT_URL_TTL_SECONDS, planUserFileMounts, @@ -137,6 +143,7 @@ describe('resolveUserFileMounts', () => { mockHasCloudStorage.mockReturnValue(true) mockGeneratePresignedDownloadUrl.mockResolvedValue('https://presigned.example/object') mockReadWorkspaceFileRecordByKey.mockResolvedValue({ file: { id: 'wf_1' } }) + mockGetFileMetadataByKey.mockResolvedValue(null) // Sized from the file being read: the aggregate budget counts bytes actually // buffered, so a fixed-size stub would never let the total ceiling trip. mockDownloadServableFileFromStorage.mockImplementation(async (file: UserFile) => ({ @@ -175,6 +182,88 @@ describe('resolveUserFileMounts', () => { ]) }) + it('carries canonical execution provenance through a URL mount without buffering bytes', async () => { + const file = executionFile({ id: 'untrusted-public-id' }) + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + mockGetFileMetadataByKey.mockResolvedValue({ + id: 'canonical-file-id', + key: file.key, + context: 'execution', + workspaceId: WORKSPACE_ID, + userId: 'user-1', + contentUpdatedAt, + }) + + const result = await resolveUserFileMounts({ + planned: planUserFileMounts([file]), + context: { + ...executionContext, + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + }, + }) + + expect(result.contributingFiles).toEqual([ + { + fileId: 'canonical-file-id', + key: file.key, + context: 'execution', + contentUpdatedAt, + }, + ]) + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it('preserves contributors introduced when an inline mount renders generated source', async () => { + const contributor = { + fileId: 'image-file', + key: 'workspace/ws-1/image.png', + context: 'workspace' as const, + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + } + mockHasCloudStorage.mockReturnValue(false) + mockDownloadServableFileFromStorage.mockResolvedValueOnce({ + buffer: Buffer.from('rendered'), + contributingFiles: [contributor], + }) + const result = await resolveUserFileMounts({ + planned: planUserFileMounts([executionFile()]), + context: executionContext, + }) + expect(result.contributingFiles).toEqual([contributor]) + expect(result.renderedContributingFiles).toEqual([contributor]) + }) + + it('retains both revisions when a file changes between two mount resolutions', async () => { + const oldFile = workspaceFile({ key: 'workspace/ws-1/old.pdf' }) + const newFile = workspaceFile({ key: 'workspace/ws-1/new.pdf' }) + const revisions = [new Date('2026-01-01T00:00:00Z'), new Date('2026-01-01T00:01:00Z')] + for (const [index, file] of [oldFile, newFile].entries()) { + mockGetFileMetadataByKey.mockResolvedValueOnce({ + id: 'canonical-file-id', + key: file.key, + context: 'workspace', + workspaceId: WORKSPACE_ID, + userId: 'user-1', + contentUpdatedAt: revisions[index], + }) + } + const result = await resolveUserFileMounts({ + planned: planUserFileMounts([oldFile, newFile]), + context: { + ...executionContext, + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + }, + }) + expect(result.contributingFiles).toEqual( + [oldFile, newFile].map((file, index) => ({ + fileId: 'canonical-file-id', + key: file.key, + context: 'workspace', + contentUpdatedAt: revisions[index], + })) + ) + }) + it('buffers bytes inline when there is no cloud storage to presign from', async () => { mockHasCloudStorage.mockReturnValue(false) diff --git a/apps/sim/lib/function-execution/sandbox-mounts.ts b/apps/sim/lib/function-execution/sandbox-mounts.ts index cbad1fa2859..8ba9069dcef 100644 --- a/apps/sim/lib/function-execution/sandbox-mounts.ts +++ b/apps/sim/lib/function-execution/sandbox-mounts.ts @@ -1,4 +1,5 @@ import { createLogger } from '@sim/logger' +import { resolveStoredFileProvenanceSource } from '@/lib/execution/payloads/file-secret-provenance' import { assertUserFileContentAccess, type ExecutionMaterializationContext, @@ -7,6 +8,7 @@ import { import { MAX_SANDBOX_URL_MOUNT_BYTES } from '@/lib/execution/remote-sandbox/output-limits' import { SANDBOX_INPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths' import type { SandboxFile } from '@/lib/execution/remote-sandbox/types' +import type { WorkspaceFileSecretProvenanceIdentity } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' import { buildStorageKeySegment } from '@/lib/uploads/core/storage-key' import { generatePresignedDownloadUrl, hasCloudStorage } from '@/lib/uploads/core/storage-service' import type { StorageContext } from '@/lib/uploads/shared/types' @@ -270,14 +272,39 @@ export function planUserFileMounts( export async function resolveUserFileMounts(args: { planned: readonly PlannedUserFileMount[] context: ExecutionMaterializationContext -}): Promise<{ sandboxFiles: SandboxFile[]; manifest: SandboxMountManifestEntry[] }> { +}): Promise<{ + sandboxFiles: SandboxFile[] + manifest: SandboxMountManifestEntry[] + contributingFiles?: readonly WorkspaceFileSecretProvenanceIdentity[] + renderedContributingFiles?: readonly WorkspaceFileSecretProvenanceIdentity[] +}> { const sandboxFiles: SandboxFile[] = [] const manifest: SandboxMountManifestEntry[] = [] const budget = createSandboxMountBudget() + const contributingFiles = new Map() + const renderedContributingFiles = new Map() + const addContributor = (identity: WorkspaceFileSecretProvenanceIdentity, rendered = false) => { + const revision = JSON.stringify([ + identity.fileId, + identity.key, + identity.context, + identity.contentUpdatedAt?.getTime(), + ]) + contributingFiles.set(revision, identity) + if (rendered) renderedContributingFiles.set(revision, identity) + } for (const { userFile, mountPath } of args.planned) { const storageContext = resolveTrustedFileContext(userFile.key, userFile.context) await assertUserFileContentAccess(userFile, args.context) + if (args.context.principal && args.context.workspaceId) { + const source = await resolveStoredFileProvenanceSource(userFile, { + ...args.context, + principal: args.context.principal, + workspaceId: args.context.workspaceId, + }) + if (source) addContributor(source.identity) + } await pushSandboxFileMount( sandboxFiles, @@ -291,12 +318,22 @@ export async function resolveUserFileMounts(args: { // Base64 regardless of content type: the payload is reproduced exactly // for any byte sequence, and picking utf8 for a mistyped binary would // substitute U+FFFD and hand the code a corrupted file. - const { content } = await readUserFileContentWithContributors(userFile, { + const { + content, + contributingFiles: contributors, + renderedContributingFiles, + } = await readUserFileContentWithContributors(userFile, { ...args.context, encoding: 'base64', maxBytes, maxSourceBytes: maxBytes, }) + for (const contributor of contributors ?? []) { + addContributor(contributor) + } + for (const contributor of renderedContributingFiles ?? []) { + addContributor(contributor, true) + } return { content, encoding: 'base64' as const, @@ -321,5 +358,12 @@ export async function resolveUserFileMounts(args: { urlBytes: budget.url, }) - return { sandboxFiles, manifest } + return { + sandboxFiles, + manifest, + ...(contributingFiles.size > 0 ? { contributingFiles: [...contributingFiles.values()] } : {}), + ...(renderedContributingFiles.size > 0 + ? { renderedContributingFiles: [...renderedContributingFiles.values()] } + : {}), + } } diff --git a/apps/sim/lib/integrations/icon-mapping.ts b/apps/sim/lib/integrations/icon-mapping.ts index a9318bcd494..bad2b73663e 100644 --- a/apps/sim/lib/integrations/icon-mapping.ts +++ b/apps/sim/lib/integrations/icon-mapping.ts @@ -301,7 +301,7 @@ export const blockTypeToIconMap: Record = { azure_data_explorer: AzureDataExplorerIcon, azure_devops: AzureIcon, bitbucket: BitbucketIcon, - box: BoxCompanyIcon, + box_v2: BoxCompanyIcon, brandfetch: BrandfetchIcon, brex: BrexIcon, brightdata: BrightDataIcon, @@ -337,10 +337,10 @@ export const blockTypeToIconMap: Record = { discord: DiscordIcon, docusign: DocuSignIcon, downdetector: DowndetectorIcon, - dropbox: DropboxIcon, + dropbox_v2: DropboxIcon, dropcontact: DropcontactIcon, dspy: DsPyIcon, - dub: DubIcon, + dub_v2: DubIcon, duckduckgo: DuckDuckGoIcon, dynamodb: DynamoDBIcon, dynatrace: DynatraceIcon, @@ -416,7 +416,7 @@ export const blockTypeToIconMap: Record = { jira_service_management: JiraServiceManagementIcon, jotform: JotformIcon, jsm: JiraServiceManagementIcon, - jupyter: JupyterIcon, + jupyter_v2: JupyterIcon, kalshi_v2: KalshiIcon, ketch: KetchIcon, knowledge: PackageSearchIcon, @@ -443,7 +443,7 @@ export const blockTypeToIconMap: Record = { mem0: Mem0Icon, memory: BrainIcon, microsoft_ad: AzureIcon, - microsoft_dataverse: MicrosoftDataverseIcon, + microsoft_dataverse_v2: MicrosoftDataverseIcon, microsoft_dynamics_365: MicrosoftDataverseIcon, microsoft_excel_v2: MicrosoftExcelIcon, microsoft_planner: MicrosoftPlannerIcon, @@ -486,7 +486,7 @@ export const blockTypeToIconMap: Record = { qdrant: QdrantIcon, quartr: QuartrIcon, quickbooks: QuickBooksIcon, - quiver: QuiverIcon, + quiver_v2: QuiverIcon, rabbitmq: RabbitmqIcon, railway: RailwayIcon, rb2b: RB2BIcon, @@ -512,8 +512,9 @@ export const blockTypeToIconMap: Record = { sentry: SentryIcon, serper: SerperIcon, servicenow: ServiceNowIcon, + servicenow_v2: ServiceNowIcon, ses: SESIcon, - sftp: SftpIcon, + sftp_v2: SftpIcon, sharepoint_v2: MicrosoftSharepointIcon, shopify: ShopifyIcon, sim_workspace_event: SimTriggerIcon, @@ -529,7 +530,7 @@ export const blockTypeToIconMap: Record = { sportmonks: SportmonksIcon, sqs: SQSIcon, square: SquareIcon, - ssh: SshIcon, + ssh_v2: SshIcon, ssm: SSMIcon, stagehand: StagehandIcon, stripe: StripeIcon, diff --git a/apps/sim/lib/internal/agiloft/execute-tool.test.ts b/apps/sim/lib/internal/agiloft/execute-tool.test.ts index d0c98cc55f8..79df94a6178 100644 --- a/apps/sim/lib/internal/agiloft/execute-tool.test.ts +++ b/apps/sim/lib/internal/agiloft/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const operationMocks = vi.hoisted(() => ({ executeAgiloftAsyncStatus: vi.fn(), @@ -28,7 +29,7 @@ const operationMocks = vi.hoisted(() => ({ vi.mock('@/lib/internal/agiloft/operations', () => operationMocks) import { AgiloftOperationError } from '@/lib/internal/agiloft/errors' -import { executeAgiloftTool } from '@/lib/internal/agiloft/execute-tool' +import { executeAgiloftTool as executeAgiloftToolOperation } from '@/lib/internal/agiloft/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' const CREDENTIALS = { @@ -145,6 +146,14 @@ const TOOL_CASES = [ ], ] as const +async function executeAgiloftTool( + request: Parameters[0] +): Promise { + const result = await executeAgiloftToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeAgiloftTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -164,6 +173,22 @@ describe('executeAgiloftTool', () => { ) }) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + operationMocks.executeAgiloftRetrieveAttachment.mockResolvedValueOnce(fileResult) + expect( + await executeAgiloftToolOperation( + createRequest({ + toolId: 'agiloft_retrieve_attachment', + input: { ...BASE, recordId: '1', fieldName: 'files', position: '0' }, + }) + ) + ).toBe(fileResult) + }) + it('uses the trusted delegation origin and forwards cancellation', async () => { const controller = new AbortController() const input = { ...BASE, data: '{"name":"Contract"}' } diff --git a/apps/sim/lib/internal/agiloft/execute-tool.ts b/apps/sim/lib/internal/agiloft/execute-tool.ts index e59c8ba22ba..5d5377815cd 100644 --- a/apps/sim/lib/internal/agiloft/execute-tool.ts +++ b/apps/sim/lib/internal/agiloft/execute-tool.ts @@ -42,9 +42,11 @@ import { executeAgiloftUpdateRecord, executeAgiloftUpsertRecord, } from '@/lib/internal/agiloft/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' function parseInput(contract: C, input: unknown) { @@ -69,7 +71,7 @@ async function executeOperation( contract: C, request: InternalToolOperationCall, operation: (input: ContractBody, context: AgiloftOperationContext) => Promise -): Promise { +): Promise { request.signal?.throwIfAborted() const parsed = parseInput(contract, request.input) if (!parsed.success) return parsed.response @@ -80,7 +82,7 @@ async function executeOperation( signal: request.signal, }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof AgiloftOperationError) { @@ -93,7 +95,9 @@ async function executeOperation( } } -export const executeAgiloftTool: InternalToolOperationHandler = async (request) => { +export const executeAgiloftTool: InternalToolOperationHandler = async ( + request +) => { switch (request.toolId) { case 'agiloft_async_status': return executeOperation(agiloftAsyncStatusContract, request, executeAgiloftAsyncStatus) diff --git a/apps/sim/lib/internal/agiloft/operations.test.ts b/apps/sim/lib/internal/agiloft/operations.test.ts index a9e373f3244..cd5709a1893 100644 --- a/apps/sim/lib/internal/agiloft/operations.test.ts +++ b/apps/sim/lib/internal/agiloft/operations.test.ts @@ -77,6 +77,17 @@ function createResponse( type ResponseTransform = (response: SecureFetchResponse) => Promise +const storedFile = { + id: 'stored-file', + name: 'stored.bin', + size: 5, + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + url: '/api/files/stored', + key: 'execution/workspace/workflow/run/stored.bin', + context: 'execution', +} as const + describe('Agiloft operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -163,17 +174,10 @@ describe('Agiloft operations', () => { { requestId: 'request-1', signal: controller.signal } ) - expect(result).toEqual({ - success: true, - output: { - file: { - name: 'evidence.txt', - mimeType: 'text/plain', - data: Buffer.from('hello').toString('base64'), - size: 5, - }, - }, - }) + expect(result.files).toEqual([ + { name: 'evidence.txt', mimeType: 'text/plain', buffer: Buffer.from('hello') }, + ]) + expect(result.present([storedFile])).toEqual({ success: true, output: { file: storedFile } }) expect(providerMocks.secureFetchWithPinnedIP).toHaveBeenCalledWith( expect.stringContaining('/ewws/EWRetrieve'), '203.0.113.10', diff --git a/apps/sim/lib/internal/agiloft/operations.ts b/apps/sim/lib/internal/agiloft/operations.ts index a346fd60ef8..ba25c691692 100644 --- a/apps/sim/lib/internal/agiloft/operations.ts +++ b/apps/sim/lib/internal/agiloft/operations.ts @@ -65,6 +65,10 @@ import { getLockHttpMethod, parseFieldList, } from '@/lib/internal/agiloft/urls' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { resolveEffectiveMimeType } from '@/lib/uploads/utils/file-utils' import type { AgiloftAsyncStatusResponse, @@ -893,7 +897,7 @@ export async function executeAgiloftAttachFile( export async function executeAgiloftRetrieveAttachment( input: AgiloftRetrieveBody, context: AgiloftOperationContext -): Promise { +): Promise { let resolvedIP: string try { resolvedIP = await resolveAgiloftInstance(input.instanceUrl, context.signal) @@ -930,15 +934,8 @@ export async function executeAgiloftRetrieveAttachment( error: `Agiloft error: ${buffer.toString('utf8').slice(0, 300)}`, }) } - return { - success: true, - output: { - file: { - name: fileName, - mimeType: resolveEffectiveMimeType(contentType, fileName), - data: buffer.toString('base64'), - size: buffer.length, - }, - }, - } + return createInternalToolFileResult( + { buffer, name: fileName, mimeType: resolveEffectiveMimeType(contentType, fileName) }, + (file) => ({ success: true, output: { file } }) + ) } diff --git a/apps/sim/lib/internal/cursor/execute-tool.test.ts b/apps/sim/lib/internal/cursor/execute-tool.test.ts index a290d6a7f06..e68a9e6bf1d 100644 --- a/apps/sim/lib/internal/cursor/execute-tool.test.ts +++ b/apps/sim/lib/internal/cursor/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ downloadCursorArtifact: vi.fn() })) @@ -15,7 +16,7 @@ vi.mock('@/lib/internal/cursor/operations', () => ({ })) import { CursorOperationError } from '@/lib/internal/cursor/errors' -import { executeCursorTool } from '@/lib/internal/cursor/execute-tool' +import { executeCursorTool as executeCursorToolOperation } from '@/lib/internal/cursor/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' function request(overrides: Partial = {}): InternalToolOperationCall { @@ -29,6 +30,14 @@ function request(overrides: Partial = {}): InternalTo } } +async function executeCursorTool( + request: Parameters[0] +): Promise { + const result = await executeCursorToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeCursorTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -45,13 +54,26 @@ describe('executeCursorTool', () => { const response = await executeCursorTool(request({ toolId, signal: controller.signal })) expect(response.status).toBe(200) - expect(mocks.downloadCursorArtifact).toHaveBeenCalledWith( + const expectedArgs: unknown[] = [ { apiKey: 'cursor-key', agentId: 'agent-1', path: '/src/index.ts' }, - { requestId: 'request-1', signal: controller.signal } - ) + { requestId: 'request-1', signal: controller.signal }, + ] + if (toolId.endsWith('_v2')) expectedArgs.push('v2') + expect(mocks.downloadCursorArtifact.mock.calls[0]).toEqual(expectedArgs) } ) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.downloadCursorArtifact.mockResolvedValueOnce(fileResult) + expect( + await executeCursorToolOperation(request({ toolId: 'cursor_download_artifact_v2' })) + ).toBe(fileResult) + }) + it('rejects invalid input before provider work', async () => { const response = await executeCursorTool(request({ input: { apiKey: '' } })) diff --git a/apps/sim/lib/internal/cursor/execute-tool.ts b/apps/sim/lib/internal/cursor/execute-tool.ts index 9436106fdec..eba25730a9d 100644 --- a/apps/sim/lib/internal/cursor/execute-tool.ts +++ b/apps/sim/lib/internal/cursor/execute-tool.ts @@ -4,7 +4,11 @@ import { cursorOperationErrorMessage, downloadCursorArtifact, } from '@/lib/internal/cursor/operations' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const inputSchema = z.object({ apiKey: z.string().min(1, 'API key is required'), @@ -12,7 +16,9 @@ const inputSchema = z.object({ path: z.string().min(1, 'Artifact path is required'), }) -export const executeCursorTool: InternalToolOperationHandler = async (request) => { +export const executeCursorTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() if ( request.toolId !== 'cursor_download_artifact' && @@ -29,12 +35,15 @@ export const executeCursorTool: InternalToolOperationHandler = async (request) = } try { - return Response.json( - await downloadCursorArtifact(parsed.data, { - requestId: request.requestId, - signal: request.signal, - }) - ) + const context = { + requestId: request.requestId, + signal: request.signal, + } + const result = + request.toolId === 'cursor_download_artifact_v2' + ? await downloadCursorArtifact(parsed.data, context, 'v2') + : await downloadCursorArtifact(parsed.data, context) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() const status = error instanceof CursorOperationError ? error.status : 500 diff --git a/apps/sim/lib/internal/cursor/operations.test.ts b/apps/sim/lib/internal/cursor/operations.test.ts index 1eff865fe46..9909d7a9bbd 100644 --- a/apps/sim/lib/internal/cursor/operations.test.ts +++ b/apps/sim/lib/internal/cursor/operations.test.ts @@ -15,6 +15,17 @@ vi.mock('@/lib/core/security/input-validation.server', () => ({ import { downloadCursorArtifact } from '@/lib/internal/cursor/operations' +const storedFile = { + id: 'stored-file', + name: 'stored.bin', + size: 5, + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + url: '/api/files/stored', + key: 'execution/workspace/workflow/run/stored.bin', + context: 'execution', +} as const + describe('downloadCursorArtifact', () => { beforeEach(() => { vi.clearAllMocks() @@ -34,7 +45,8 @@ describe('downloadCursorArtifact', () => { const result = await downloadCursorArtifact( { apiKey: 'cursor-key', agentId: 'agent-1', path: '/src/index.ts' }, - { requestId: 'request-1', signal: controller.signal } + { requestId: 'request-1', signal: controller.signal }, + 'v2' ) expect(fetchMock).toHaveBeenCalledOnce() @@ -47,11 +59,34 @@ describe('downloadCursorArtifact', () => { '203.0.113.1', { profile: 'contentFetch', signal: controller.signal } ) - expect(result.output.file).toEqual({ - name: 'index.ts', - mimeType: 'text/plain', - data: Buffer.from('artifact').toString('base64'), - size: 8, + expect(result.files).toEqual([ + { name: 'index.ts', mimeType: 'text/plain', buffer: Buffer.from('artifact') }, + ]) + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { file: storedFile }, + }) + }) + + it('preserves inline file data for the legacy tool', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue(Response.json({ url: 'https://download.example/artifact' })) + ) + const result = await downloadCursorArtifact( + { apiKey: 'cursor-key', agentId: 'agent-1', path: '/src/index.ts' }, + { requestId: 'request-1' } + ) + expect(result).toEqual({ + success: true, + output: { + file: { + name: 'index.ts', + mimeType: 'text/plain', + data: Buffer.from('artifact').toString('base64'), + size: 8, + }, + }, }) }) }) diff --git a/apps/sim/lib/internal/cursor/operations.ts b/apps/sim/lib/internal/cursor/operations.ts index 3365cd7969d..4ac9ba2cc81 100644 --- a/apps/sim/lib/internal/cursor/operations.ts +++ b/apps/sim/lib/internal/cursor/operations.ts @@ -9,6 +9,10 @@ import { readResponseTextWithLimit, } from '@/lib/core/utils/stream-limits' import { CursorOperationError } from '@/lib/internal/cursor/errors' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import type { DownloadArtifactParams } from '@/tools/cursor/types' const logger = createLogger('CursorOperations') @@ -25,13 +29,32 @@ export interface CursorOperationContext { signal?: AbortSignal } -export async function downloadCursorArtifact( +interface LegacyCursorArtifactResult { + success: boolean + output: { + file: { + name: string + mimeType: string + data: string + size: number + } + } +} + +export function downloadCursorArtifact( input: DownloadArtifactParams, context: CursorOperationContext -): Promise<{ - success: true - output: { file: { name: string; mimeType: string; data: string; size: number } } -}> { +): Promise +export function downloadCursorArtifact( + input: DownloadArtifactParams, + context: CursorOperationContext, + version: 'v2' +): Promise +export async function downloadCursorArtifact( + input: DownloadArtifactParams, + context: CursorOperationContext, + version: 'v1' | 'v2' = 'v1' +): Promise { context.signal?.throwIfAborted() const authHeader = `Basic ${Buffer.from(`${input.apiKey}:`).toString('base64')}` const artifactResponse = await fetch( @@ -86,15 +109,30 @@ export async function downloadCursorArtifact( const file = { name: input.path.split('/').pop() || 'artifact', mimeType: downloadResponse.headers.get('content-type') || 'application/octet-stream', - data: fileBuffer.toString('base64'), - size: fileBuffer.length, + buffer: fileBuffer, } logger.info(`[${context.requestId}] Cursor artifact downloaded`, { agentId: input.agentId, path: input.path, - size: file.size, + size: fileBuffer.length, }) - return { success: true, output: { file } } + if (version === 'v1') { + return { + success: true, + output: { + file: { + name: file.name, + mimeType: file.mimeType, + data: fileBuffer.toString('base64'), + size: fileBuffer.length, + }, + }, + } + } + return createInternalToolFileResult(file, (storedFile) => ({ + success: true, + output: { file: storedFile }, + })) } export function cursorOperationErrorMessage(error: unknown): string { diff --git a/apps/sim/lib/internal/discord/execute-tool.ts b/apps/sim/lib/internal/discord/execute-tool.ts index 023a0be5a79..a3bc4aa7fb6 100644 --- a/apps/sim/lib/internal/discord/execute-tool.ts +++ b/apps/sim/lib/internal/discord/execute-tool.ts @@ -5,7 +5,11 @@ import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' import { DiscordOperationError } from '@/lib/internal/discord/errors' import { executeDiscordSendMessage } from '@/lib/internal/discord/operations' import { discordSendMessageInputSchema } from '@/lib/internal/discord/schema' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const logger = createLogger('DiscordToolExecution') @@ -26,7 +30,9 @@ function inputSizeError(input: unknown): Response | null { : null } -export const executeDiscordTool: InternalToolOperationHandler = async (request) => { +export const executeDiscordTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() if (request.toolId !== 'discord_send_message') { return Response.json( @@ -54,7 +60,7 @@ export const executeDiscordTool: InternalToolOperationHandler = async (request) userId, }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof DiscordOperationError) { diff --git a/apps/sim/lib/internal/discord/operations.test.ts b/apps/sim/lib/internal/discord/operations.test.ts index 6dfac7443b6..2843a8d2746 100644 --- a/apps/sim/lib/internal/discord/operations.test.ts +++ b/apps/sim/lib/internal/discord/operations.test.ts @@ -2,6 +2,7 @@ * @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const fileMocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), @@ -96,25 +97,39 @@ describe('executeDiscordSendMessage', () => { return { id: 'message-1', content: 'hello' } }) - await expect( - executeDiscordSendMessage( - { - botToken: 'bot-token', - channelId: '123', - content: 'hello', - files: [{ key: 'workspace/file.txt', name: 'file.txt', size: 4 }], - }, - { - requestId: 'request-1', - signal: controller.signal, - userId: 'user-1', - } - ) - ).resolves.toMatchObject({ + const result = await executeDiscordSendMessage( + { + botToken: 'bot-token', + channelId: '123', + content: 'hello', + files: [{ key: 'workspace/file.txt', name: 'file.txt', size: 4 }], + }, + { + requestId: 'request-1', + signal: controller.signal, + userId: 'user-1', + } + ) + if (!isInternalToolFileResult(result)) throw new Error('Expected a file output') + expect(result.files).toEqual([ + { name: 'file.txt', mimeType: 'text/plain', buffer: Buffer.from('file') }, + ]) + const storedFile = { + id: 'stored', + name: 'file.txt', + size: 4, + type: 'text/plain', + mimeType: 'text/plain', + url: '/api/files/stored', + key: 'execution/file.txt', + context: 'execution' as const, + } + expect(result.present([storedFile])).toMatchObject({ success: true, output: { data: { id: 'message-1', content: 'hello' }, fileCount: 1, + files: [storedFile], message: 'hello', }, }) diff --git a/apps/sim/lib/internal/discord/operations.ts b/apps/sim/lib/internal/discord/operations.ts index 00a5d2211eb..a4571e27c66 100644 --- a/apps/sim/lib/internal/discord/operations.ts +++ b/apps/sim/lib/internal/discord/operations.ts @@ -6,6 +6,7 @@ import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { sendDiscordMessage } from '@/lib/internal/discord/client' import { DiscordOperationError } from '@/lib/internal/discord/errors' import type { DiscordSendMessageInput } from '@/lib/internal/discord/schema' +import { createInternalToolFilesResult } from '@/lib/internal/tool-operations/file-result' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' @@ -92,8 +93,7 @@ export async function executeDiscordSendMessage( return { name: file.name, mimeType, - data: downloaded.buffer.toString('base64'), - size: downloaded.buffer.length, + buffer: downloaded.buffer, } }) const data = await sendDiscordMessage( @@ -103,13 +103,13 @@ export async function executeDiscordSendMessage( 'multipart', context.signal ) - return { + return createInternalToolFilesResult(files, (storedFiles) => ({ success: true, output: { message: typeof data.content === 'string' ? data.content : undefined, data, fileCount: userFiles.length, - files, + files: storedFiles, }, - } + })) } diff --git a/apps/sim/lib/internal/file/execute-tool.test.ts b/apps/sim/lib/internal/file/execute-tool.test.ts index 82853d417ac..05534d68ab1 100644 --- a/apps/sim/lib/internal/file/execute-tool.test.ts +++ b/apps/sim/lib/internal/file/execute-tool.test.ts @@ -298,8 +298,11 @@ describe('executeFileTool', () => { }) it.each(PARSER_TOOL_IDS)('dispatches %s with trusted execution scope', async (toolId) => { + const headers = new Headers({ + 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1', + }) const response = await executeFileTool( - request(toolId, { filePath: 'https://example.com/report.txt', fileType: '' }) + request(toolId, { filePath: 'https://example.com/report.txt', fileType: '' }, { headers }) ) expect(response.status).toBe(200) @@ -311,6 +314,7 @@ describe('executeFileTool', () => { executionId: 'execution-1', attributedUserId: 'user-1', fileAccessUserId: 'user-1', + headers, }) ) expect(mocks.executeManage).not.toHaveBeenCalled() diff --git a/apps/sim/lib/internal/file/execute-tool.ts b/apps/sim/lib/internal/file/execute-tool.ts index 4ed4dd0a2d5..5641a165781 100644 --- a/apps/sim/lib/internal/file/execute-tool.ts +++ b/apps/sim/lib/internal/file/execute-tool.ts @@ -166,6 +166,7 @@ export const executeFileTool: InternalToolOperationHandler = async (request) => fileKeys: request.context.fileKeys, allowLargeValueWorkflowScope: request.context.allowLargeValueWorkflowScope, requestId: request.requestId, + headers: request.headers, signal: request.signal, }) } else { diff --git a/apps/sim/lib/internal/file/operations.provenance.test.ts b/apps/sim/lib/internal/file/operations.provenance.test.ts index 9e129d0f1bd..e82ebb64904 100644 --- a/apps/sim/lib/internal/file/operations.provenance.test.ts +++ b/apps/sim/lib/internal/file/operations.provenance.test.ts @@ -177,7 +177,10 @@ vi.mock('@/lib/core/security/encryption', () => ({ import { fileManageBodySchema } from '@/lib/api/contracts/tools/file' import type { DbTransaction } from '@/lib/db/types' -import { executeFileManageOperation } from '@/lib/internal/file/operations' +import { + executeFileManageOperation, + getFileContentProvenance, +} from '@/lib/internal/file/operations' import { importWorkspaceFileSecretProvenanceForModelView, isOpaqueWorkspaceFileEgressSafe, @@ -395,3 +398,102 @@ describe('appended file provenance', () => { } ) }) + +describe('execution-file content provenance', () => { + const identity = { + fileId: 'execution-file', + key: 'execution/workspace-1/workflow-1/execution-1/report.txt', + context: 'execution' as const, + contentUpdatedAt: CONTENT_UPDATED_AT, + } + const principal = createWorkspaceFileDelegatedPrincipal({ + serviceId: 'executor', + subjectUserId: 'user-1', + workspaceId: 'workspace-1', + delegationId: 'test-file-content', + }) + + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + }) + + it.each([ + { status: 'exact', version: 1, stale: false, enforced: false, complete: true }, + { status: 'exact', version: 1, stale: false, enforced: true, complete: true }, + { status: 'unrecorded', version: 1, stale: false, enforced: false, complete: true }, + { status: 'unrecorded', version: 1, stale: false, enforced: true, complete: false }, + { status: 'unknown', version: 1, stale: false, enforced: false, complete: false }, + { status: 'unknown', version: 1, stale: false, enforced: true, complete: false }, + { status: 'unknown', version: null, stale: false, enforced: false, complete: true }, + { status: 'unknown', version: null, stale: false, enforced: true, complete: true }, + { status: 'exact', version: 1, stale: true, enforced: false, complete: false }, + { status: 'exact', version: 1, stale: true, enforced: true, complete: false }, + ])( + 'reads $status version=$version stale=$stale with enforcement=$enforced', + async ({ status, version, stale, enforced, complete }) => { + mockEnforced.mockReturnValue(enforced) + queueTableRows(workspaceFiles, [ + { + ...joinedRow(status), + secretProvenanceVersion: version, + ...(stale ? { provenanceContentUpdatedAt: new Date(0) } : {}), + }, + ]) + + const provenance = await getFileContentProvenance(principal, 'workspace-1', [ + { identity, ownerUserId: 'user-1' }, + ]) + + expect(provenance).toMatchObject({ version: 1, complete, entries: [] }) + } + ) + + it('retains exact secret-bearing execution lineage for downstream text projections', async () => { + mockEnforced.mockReturnValue(true) + queueTableRows(workspaceFiles, [ + joinedRow('exact', [ + { + name: 'TOKEN', + encryptedValue: 'synthetic-ciphertext', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + ]), + ]) + + const provenance = await getFileContentProvenance(principal, 'workspace-1', [ + { identity, ownerUserId: 'user-1' }, + ]) + const registry = new ResolvedSecretTraceRegistry([], SCOPE) + expect(provenance.complete).toBe(true) + expect(await registry.importProvenance(provenance, { trusted: true })).toBe(true) + expect(projectResolvedSecretModelContent(`parsed: ${SECRET}`, registry)).toEqual({ + safe: true, + value: 'parsed: {{TOKEN}}', + }) + }) + + it.each([ + { sourceUserId: 'other-user', sourceWorkspaceId: 'workspace-1' }, + { sourceUserId: 'user-1', sourceWorkspaceId: 'other-workspace' }, + ])('anonymizes names from a different source scope: %j', async (sourceScope) => { + queueTableRows(workspaceFiles, [ + joinedRow('exact', [ + { name: 'PRIVATE_SOURCE_NAME', encryptedValue: 'synthetic-ciphertext', ...sourceScope }, + ]), + ]) + + const provenance = await getFileContentProvenance(principal, 'workspace-1', [ + { identity, ownerUserId: 'user-1' }, + ]) + + expect(provenance).toEqual({ + version: 1, + complete: true, + entries: [{ encryptedValue: 'synthetic-ciphertext' }], + scope: SCOPE, + }) + expect(JSON.stringify(provenance)).not.toContain('PRIVATE_SOURCE_NAME') + }) +}) diff --git a/apps/sim/lib/internal/file/operations.test.ts b/apps/sim/lib/internal/file/operations.test.ts index 2516a1873a7..3cffe984456 100644 --- a/apps/sim/lib/internal/file/operations.test.ts +++ b/apps/sim/lib/internal/file/operations.test.ts @@ -128,6 +128,8 @@ vi.mock('@/lib/uploads/contexts/workspace', () => ({ getWorkspaceFileByName: (...args: unknown[]) => mockGetWorkspaceFileByName(...args), getWorkspaceFile: (...args: unknown[]) => mockGetWorkspaceFile(...args), loadActiveWorkspaceContext: (...args: unknown[]) => mockLoadActiveWorkspaceContext(...args), + loadActiveWorkspaceFileContext: (...args: unknown[]) => + mockLoadActiveWorkspaceFileContext(...args), updateWorkspaceFileContent: (...args: unknown[]) => mockUpdateWorkspaceFileContent(...args), uploadWorkspaceFile: (...args: unknown[]) => mockUploadWorkspaceFile(...args), })) @@ -1068,13 +1070,30 @@ describe('file manage operations', () => { ? { status: 'exact', entries: [ - { name: 'TOKEN', encryptedValue: 'encrypted-token' }, - { name: 'ALPHA', encryptedValue: 'encrypted-alpha' }, + { + name: 'TOKEN', + encryptedValue: 'encrypted-token', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + { + name: 'ALPHA', + encryptedValue: 'encrypted-alpha', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, ], } : { status: 'exact', - entries: [{ name: 'TOKEN', encryptedValue: 'encrypted-token' }], + entries: [ + { + name: 'TOKEN', + encryptedValue: 'encrypted-token', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + ], } ) @@ -1115,6 +1134,215 @@ describe('file manage operations', () => { ) }) + describe('rendered file contributors', () => { + const contributor = { + fileId: 'image', + key: 'workspace/workspace-1/image.txt', + context: 'workspace' as const, + contentUpdatedAt: CONTENT_UPDATED_AT, + } + const secretEntries = [ + { + name: 'IMAGE_TOKEN', + encryptedValue: 'encrypted-image-token', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + ] + + beforeEach(() => { + mockResolveWorkspaceFileReference.mockResolvedValue(workspaceFile('document')) + mockGetFileMetadataByKey.mockResolvedValue({ + id: contributor.fileId, + key: contributor.key, + context: contributor.context, + workspaceId: 'workspace-1', + userId: 'user-1', + contentUpdatedAt: CONTENT_UPDATED_AT, + }) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('rendered image content'), + contentType: 'text/plain', + contributingFiles: [contributor], + }) + }) + + function renderedRequest(operation: 'write' | 'compress' | 'content') { + return createMockRequest( + 'POST', + { + operation, + workspaceId: 'workspace-1', + ...(operation === 'write' + ? { + fileName: 'copy.txt', + fileInput: { + id: 'document', + name: 'document.txt', + key: 'workspace/workspace-1/document.txt', + url: '/api/files/serve/document', + context: 'workspace', + size: 1, + type: 'text/plain', + }, + } + : { fileId: 'document' }), + }, + PRIVATE_REQUEST_HEADER + ) + } + + it.each(['write', 'compress', 'content'] as const)( + '%s keeps transformed secret-bearing assets unknown even when the source is exact-empty', + async (operation) => { + mockGetBoundWorkspaceFileSecretProvenance.mockImplementation( + async (_workspaceId: string, identity: { fileId: string }) => ({ + status: 'exact', + entries: identity.fileId === contributor.fileId ? secretEntries : [], + }) + ) + + mockDownloadServableFileFromStorage.mockResolvedValueOnce({ + buffer: Buffer.from(''), + contentType: 'text/html', + contributingFiles: [contributor], + }) + const response = await POST(renderedRequest(operation)) + + expect(response.status, await response.clone().text()).toBe(200) + if (operation === 'content') { + await expect(response.json()).resolves.toMatchObject({ + __resolvedSecretTraceProvenance: { + complete: false, + entries: [], + }, + }) + } else { + expect(mockUploadWorkspaceFile.mock.calls[0]?.[5]).toMatchObject({ + secretProvenance: { status: 'unknown' }, + }) + } + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledWith( + expect.anything(), + 'request-1', + expect.anything(), + expect.objectContaining({ + filePrincipal: expect.objectContaining({ subjectUserId: 'user-1' }), + signal: expect.any(AbortSignal), + }) + ) + expect(mockGetBoundWorkspaceFileSecretProvenance).toHaveBeenCalledWith( + 'workspace-1', + contributor + ) + } + ) + + it.each(['write', 'compress', 'content'] as const)( + '%s preserves unknown rendered-asset provenance', + async (operation) => { + mockGetBoundWorkspaceFileSecretProvenance.mockImplementation( + async (_workspaceId: string, identity: { fileId: string }) => + identity.fileId === contributor.fileId + ? { status: 'unknown' } + : { status: 'exact', entries: [] } + ) + + const response = await POST(renderedRequest(operation)) + + expect(response.status, await response.clone().text()).toBe(200) + if (operation === 'content') { + await expect(response.json()).resolves.toMatchObject({ + __resolvedSecretTraceProvenance: { complete: false, entries: [] }, + }) + } else { + expect(mockUploadWorkspaceFile.mock.calls[0]?.[5]).toMatchObject({ + secretProvenance: { status: 'unknown' }, + }) + } + } + ) + + it.each(['write', 'compress', 'content'] as const)( + '%s does not replace an older rendered revision with a safe revision of the same file', + async (operation) => { + const oldRevision = new Date(CONTENT_UPDATED_AT.getTime() - 1_000) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('both old and new image bytes'), + contentType: 'text/plain', + contributingFiles: [{ ...contributor, contentUpdatedAt: oldRevision }, contributor], + }) + mockGetBoundWorkspaceFileSecretProvenance.mockImplementation( + async (_workspaceId: string, identity: { contentUpdatedAt?: Date }) => + identity.contentUpdatedAt?.getTime() === oldRevision.getTime() + ? { status: 'unknown' } + : { status: 'exact', entries: [] } + ) + + const response = await POST(renderedRequest(operation)) + + expect(response.status, await response.clone().text()).toBe(200) + expect(mockGetBoundWorkspaceFileSecretProvenance).toHaveBeenCalledWith('workspace-1', { + ...contributor, + contentUpdatedAt: oldRevision, + }) + if (operation === 'content') { + await expect(response.json()).resolves.toMatchObject({ + __resolvedSecretTraceProvenance: { complete: false, entries: [] }, + }) + } else { + expect(mockUploadWorkspaceFile.mock.calls[0]?.[5]).toMatchObject({ + secretProvenance: { status: 'unknown' }, + }) + } + } + ) + + it.each(['write', 'compress'] as const)( + '%s retains the secret owner guard for rendered contributors', + async (operation) => { + mockGetFileMetadataByKey.mockResolvedValue({ + id: contributor.fileId, + key: contributor.key, + context: contributor.context, + workspaceId: 'workspace-1', + userId: 'other-user', + contentUpdatedAt: CONTENT_UPDATED_AT, + }) + mockGetBoundWorkspaceFileSecretProvenance.mockImplementation( + async (_workspaceId: string, identity: { fileId: string }) => ({ + status: 'exact', + entries: identity.fileId === contributor.fileId ? secretEntries : [], + }) + ) + + const response = await POST(renderedRequest(operation)) + + expect(response.status, await response.clone().text()).toBe(200) + expect(mockUploadWorkspaceFile.mock.calls[0]?.[5]).toMatchObject({ + secretProvenance: { status: 'unknown' }, + }) + } + ) + + it('refuses a rendered contributor whose canonical scope differs', async () => { + mockGetFileMetadataByKey.mockResolvedValue({ + id: contributor.fileId, + key: contributor.key, + context: contributor.context, + workspaceId: 'other-workspace', + userId: 'user-1', + contentUpdatedAt: CONTENT_UPDATED_AT, + }) + mockGetBoundWorkspaceFileSecretProvenance.mockResolvedValue({ status: 'exact', entries: [] }) + + const response = await POST(renderedRequest('write')) + + expect(response.status).toBe(404) + expect(mockUploadWorkspaceFile).not.toHaveBeenCalled() + }) + }) + it('pins resolved file-input provenance to the captured content revision', async () => { mockResolveWorkspaceFileReference.mockResolvedValue(workspaceFile('file-1')) mockGetBoundWorkspaceFileSecretProvenance.mockResolvedValue({ @@ -1870,7 +2098,14 @@ describe('file manage operations', () => { ) mockGetBoundWorkspaceFileSecretProvenance.mockResolvedValue({ status: 'exact', - entries: [{ name: 'TOKEN', encryptedValue: 'encrypted-token' }], + entries: [ + { + name: 'TOKEN', + encryptedValue: 'encrypted-token', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + ], }) const response = await POST( @@ -1885,7 +2120,7 @@ describe('file manage operations', () => { expect(body.__resolvedSecretTraceProvenance).toEqual({ version: 1, complete: true, - entries: [{ name: 'TOKEN', encryptedValue: 'encrypted-token' }], + entries: [{ encryptedValue: 'encrypted-token' }], }) }) diff --git a/apps/sim/lib/internal/file/operations.ts b/apps/sim/lib/internal/file/operations.ts index 9907e9103c0..5e84bdcac74 100644 --- a/apps/sim/lib/internal/file/operations.ts +++ b/apps/sim/lib/internal/file/operations.ts @@ -19,6 +19,7 @@ import { inspectPrivateSecretProvenanceRequest, isPrivateSecretProvenanceBundleV1, } from '@/lib/execution/model-input-provenance' +import { resolveStoredFileProvenanceSource } from '@/lib/execution/payloads/file-secret-provenance' import { assertUserFileContentAccess } from '@/lib/execution/payloads/materialization.server' import { PRIVATE_TOOL_METADATA_RESPONSE_HEADER, @@ -44,6 +45,8 @@ import type { WorkspaceFileRecord, } from '@/lib/uploads/contexts/workspace/workspace-file-manager' import { + getBoundWorkspaceFileSecretProvenance, + mayReadUnrecordedWorkspaceFile, mergeWorkspaceFileSecretProvenance, type WorkspaceFileSecretProvenance, type WorkspaceFileSecretProvenanceIdentity, @@ -198,11 +201,10 @@ const fileInputToUserFile = (fileInput: unknown) => { ? record.fileId.trim() : '' - // Objects with ids are resolved through workspace metadata. This fallback is for - // picker/upload values that only carry storage fields. - if (id) return null - const key = typeof record.key === 'string' ? record.key.trim() : '' + /** Execution ids are not workspace file ids; their storage key carries the run scope. */ + if (id && (!key || tryInferContextFromKey(key) !== 'execution')) return null + const path = typeof record.path === 'string' ? record.path.trim() : '' const url = typeof record.url === 'string' ? record.url.trim() : '' const fileUrl = @@ -217,7 +219,7 @@ const fileInputToUserFile = (fileInput: unknown) => { if (key && !context) return null return { - id: key || fileUrl, + id: id || key || fileUrl, name: typeof record.name === 'string' && record.name.trim() ? record.name.trim() : 'workspace-file', url: fileUrl ? ensureAbsoluteUrl(fileUrl) : '', @@ -284,6 +286,12 @@ const extractFileIdsFromInput = (fileInput: unknown): string[] => { if (typeof input === 'string') return normalizeFileIdList(input) if (input && typeof input === 'object') { const record = input as Record + if ( + typeof record.key === 'string' && + tryInferContextFromKey(record.key.trim()) === 'execution' + ) { + return [] + } if (typeof record.id === 'string') return normalizeFileIdList(record.id) if (typeof record.fileId === 'string') return normalizeFileIdList(record.fileId) } @@ -424,15 +432,23 @@ function sliceTextLines( interface ExtractedFileText { text: string truncated: boolean + contributingFiles?: readonly WorkspaceFileSecretProvenanceIdentity[] } const extractUserFileTextContent = async ( userFile: UserFile, - requestId: string + context: FileManageOperationContext ): Promise => { - const { buffer } = await downloadServableFileFromStorage(userFile, requestId, logger, { - maxBytes: MAX_GET_CONTENT_FILE_BYTES, - }) + const { buffer, contributingFiles } = await downloadServableFileFromStorage( + userFile, + context.requestId, + logger, + { + maxBytes: MAX_GET_CONTENT_FILE_BYTES, + filePrincipal: context.principal, + signal: context.signal, + } + ) const extension = getFileExtension(userFile.name) if (extension && isSupportedFileType(extension)) { @@ -442,7 +458,11 @@ const extractUserFileTextContent = async ( /** Scraped or placeholder output is a failure, not the file's content. */ throw new Error(result.metadata.warning ?? 'Parser returned degraded output') } - return { text: result.content ?? '', truncated: result.metadata?.truncated === true } + return { + text: result.content ?? '', + truncated: result.metadata?.truncated === true, + contributingFiles, + } } catch (error) { logger.warn('Falling back to raw text after parser failure', { name: userFile.name, @@ -452,16 +472,19 @@ const extractUserFileTextContent = async ( } if (isLikelyTextBuffer(buffer)) { - return { text: buffer.toString('utf-8'), truncated: false } + return { text: buffer.toString('utf-8'), truncated: false, contributingFiles } } return { text: `[Binary file: ${userFile.name} (${userFile.type || 'application/octet-stream'}, ${buffer.length} bytes). Cannot extract text content.]`, truncated: false, + contributingFiles, } } export interface FileContentProvenanceSource { + /** Rendering may encode these bytes; original secret literals cannot describe the transformed value. */ + opaque?: boolean identity?: WorkspaceFileSecretProvenanceIdentity ownerUserId?: string } @@ -471,10 +494,17 @@ interface FileContentSource extends FileContentProvenanceSource { } async function bindSelectedContentFile( - principal: Principal, - workspaceId: string, + context: FileManageOperationContext, file: UserFile ): Promise { + const { principal, workspaceId } = context + if (file.key && tryInferContextFromKey(file.key) === 'execution') { + const source = await resolveStoredFileProvenanceSource(file, { + ...context, + userId: context.fileAccessUserId, + }) + return { file, ...source } + } if (!file.key || file.context !== 'workspace') return { file } let metadata: Awaited> @@ -503,6 +533,67 @@ async function bindSelectedContentFile( } } +async function bindSelectedContentFiles( + context: FileManageOperationContext, + files: readonly UserFile[] +): Promise { + const sources: FileContentSource[] = [] + for (const file of files) { + context.signal?.throwIfAborted() + sources.push(await bindSelectedContentFile(context, file)) + } + return sources +} + +/** Preserves the renderer's consumed revision while checking each contributor's current scope. */ +async function bindRenderedContentSources( + context: FileManageOperationContext, + identities: readonly WorkspaceFileSecretProvenanceIdentity[] = [] +): Promise { + const sources: FileContentProvenanceSource[] = [] + for (const identity of identities) { + context.signal?.throwIfAborted() + const canonical = await resolveStoredFileProvenanceSource( + { + key: identity.key, + context: identity.context === 'mothership' ? 'workspace' : identity.context, + }, + { ...context, userId: context.fileAccessUserId } + ) + const matches = + canonical && + canonical.identity.fileId === identity.fileId && + canonical.identity.key === identity.key && + canonical.identity.context === identity.context + sources.push({ + identity, + opaque: true, + ...(matches ? { ownerUserId: canonical.ownerUserId } : {}), + }) + } + return sources +} + +/** Execution identities have already passed the same run capability that authorized their bytes. */ +async function readFileSourceSecretProvenance( + principal: Principal, + workspaceId: string, + identity: WorkspaceFileSecretProvenanceIdentity +): Promise { + if (identity.context === 'execution' || identity.context === 'mothership') { + return getBoundWorkspaceFileSecretProvenance(workspaceId, identity) + } + const { provenance } = await readWorkspaceFileSecretProvenance.execute({ + principal, + input: { + fileId: identity.fileId, + assertedWorkspaceId: workspaceId, + expectedContentUpdatedAt: identity.contentUpdatedAt, + }, + }) + return provenance +} + export async function getFileContentProvenance( principal: Principal, workspaceId: string, @@ -527,27 +618,25 @@ export async function getFileContentProvenance( accumulator.markIncomplete('file-source-unidentified') continue } - const { provenance } = await readWorkspaceFileSecretProvenance.execute({ - principal, - input: { - fileId: source.identity.fileId, - assertedWorkspaceId: workspaceId, - expectedContentUpdatedAt: source.identity.contentUpdatedAt, - }, - }) + const provenance = await readFileSourceSecretProvenance(principal, workspaceId, source.identity) signal?.throwIfAborted() - /** - * `unrecorded` is a more specific `unknown`, and this accumulator has not opted into the - * workspace file surface's policy, so it latches exactly as it did before. - */ - if (provenance.status !== 'exact') { + if (provenance.status === 'unrecorded' && mayReadUnrecordedWorkspaceFile(workspaceId)) continue + if (provenance.status !== 'exact' || (source.opaque && provenance.entries.length > 0)) { accumulator.markIncomplete('workspace-file-provenance-unknown') continue } accumulator.record({ version: 1, complete: true, - entries: [...provenance.entries], + entries: provenance.entries.map((entry) => ({ + encryptedValue: entry.encryptedValue, + ...(entry.name && + scope && + entry.sourceUserId === scope.userId && + entry.sourceWorkspaceId === scope.workspaceId + ? { name: entry.name } + : {}), + })), ...(scope ? { scope } : {}), }) } @@ -678,25 +767,27 @@ async function deriveWorkspaceFileSecretProvenance(options: { principal: Principal workspaceId: string targetOwnerUserId: string - sources: readonly FileContentSource[] + sources: readonly FileContentProvenanceSource[] }): Promise { - const provenances: WorkspaceFileSecretProvenance[] = [] + let combined: WorkspaceFileSecretProvenance = { status: 'exact', entries: [] } for (const source of options.sources) { if (!source.identity || !source.ownerUserId) return { status: 'unknown' } - const { provenance } = await readWorkspaceFileSecretProvenance.execute({ - principal: options.principal, - input: { fileId: source.identity.fileId, assertedWorkspaceId: options.workspaceId }, - }) + const provenance = await readFileSourceSecretProvenance( + options.principal, + options.workspaceId, + source.identity + ) if ( provenance.status === 'exact' && provenance.entries.length > 0 && - source.ownerUserId !== options.targetOwnerUserId + (source.opaque || source.ownerUserId !== options.targetOwnerUserId) ) { return { status: 'unknown' } } - provenances.push(provenance) + combined = mergeWorkspaceFileSecretProvenance(combined, provenance) + if (combined.status === 'unknown') return combined } - return mergeWorkspaceFileSecretProvenance(...provenances) + return combined } export function fileContentJsonResponse( @@ -1097,10 +1188,9 @@ export async function executeFileManageOperation( }, ] }) - const selectedSources = await Promise.all( - selectedInputFiles.map((file) => bindSelectedContentFile(principal, workspaceId, file)) - ) + const selectedSources = await bindSelectedContentFiles(context, selectedInputFiles) const sources = canonicalSources.concat(selectedSources) + const provenanceSources: FileContentProvenanceSource[] = [...sources] const contents: string[] = [] const lineRanges: FileContentLineRange[] = [] @@ -1119,7 +1209,14 @@ export async function executeFileManageOperation( }) } - const extracted = await extractUserFileTextContent(source.file, requestId) + const extracted = await extractUserFileTextContent(source.file, context) + if (includePrivateContentProvenance) { + const renderedSources = await bindRenderedContentSources( + context, + extracted.contributingFiles + ) + for (const renderedSource of renderedSources) provenanceSources.push(renderedSource) + } const { text: content, range } = sliceTextLines( extracted.text, body.offset, @@ -1144,7 +1241,7 @@ export async function executeFileManageOperation( logger.info('File content extracted', { count: contents.length }) const provenance = includePrivateContentProvenance - ? await getFileContentProvenance(principal, workspaceId, sources, signal) + ? await getFileContentProvenance(principal, workspaceId, provenanceSources, signal) : undefined return contentResponse( @@ -1186,8 +1283,8 @@ export async function executeFileManageOperation( * "safe" state — and a file the platform had locked as secret-derived * would be readable again under its new id. * - * A source with no workspace row resolves to `unknown` rather than empty, - * because nothing durable records what went into it. + * Workspace and execution files carry their canonical sidecars across the copy. + * An unidentified source cannot establish exact provenance. */ let inputProvenance: WorkspaceFileSecretProvenance | undefined if (fileInput !== undefined && fileInput !== null) { @@ -1220,12 +1317,7 @@ export async function executeFileManageOperation( const denied = await assertOperationFileAccess(sourceFile, context) if (denied) return denied - inputProvenance = await deriveWorkspaceFileSecretProvenance({ - principal, - workspaceId, - targetOwnerUserId: userId, - sources: [await bindSelectedContentFile(principal, workspaceId, sourceFile)], - }) + const source = await bindSelectedContentFile(context, sourceFile) const downloaded = await downloadServableFileFromStorage(sourceFile, requestId, logger, { maxBytes: MAX_WRITE_FILE_INPUT_BYTES, @@ -1235,6 +1327,15 @@ export async function executeFileManageOperation( // already-published artifact and throws when there is none. filePrincipal: principal, }) + inputProvenance = await deriveWorkspaceFileSecretProvenance({ + principal, + workspaceId, + targetOwnerUserId: userId, + sources: [ + source, + ...(await bindRenderedContentSources(context, downloaded.contributingFiles)), + ], + }) sourceEncoding = 'base64' sourceContent = downloaded.buffer.toString('base64') sourceName = fileName?.trim() || sourceFile.name @@ -1735,16 +1836,19 @@ export async function executeFileManageOperation( return [ { file: userFile, - identity: { fileId: file.id, key: file.key, context: 'workspace' }, + identity: { + fileId: file.id, + key: file.key, + context: 'workspace', + contentUpdatedAt: file.contentUpdatedAt ?? undefined, + }, ownerUserId: file.uploadedBy, }, ] }) - const selectedArchiveSources = await Promise.all( - selectedInputFiles.map((file) => bindSelectedContentFile(principal, workspaceId, file)) - ) + const selectedArchiveSources = await bindSelectedContentFiles(context, selectedInputFiles) const archiveSources = canonicalArchiveSources.concat(selectedArchiveSources) - const archiveProvenance = await deriveWorkspaceFileSecretProvenance({ + let archiveProvenance = await deriveWorkspaceFileSecretProvenance({ principal, workspaceId, targetOwnerUserId: userId, @@ -1775,9 +1879,26 @@ export async function executeFileManageOperation( // the archive must carry the servable bytes instead of the raw source text. // A still-compiling artifact throws, and the handler's catch turns that into // the shared 409 via `docNotReadyResponse`. - const { buffer } = await downloadServableFileFromStorage(userFile, requestId, logger, { - maxBytes: MAX_COMPRESS_FILE_BYTES, - }) + const { buffer, contributingFiles } = await downloadServableFileFromStorage( + userFile, + requestId, + logger, + { + maxBytes: MAX_COMPRESS_FILE_BYTES, + filePrincipal: principal, + signal, + } + ) + const renderedSources = await bindRenderedContentSources(context, contributingFiles) + archiveProvenance = mergeWorkspaceFileSecretProvenance( + archiveProvenance, + await deriveWorkspaceFileSecretProvenance({ + principal, + workspaceId, + targetOwnerUserId: userId, + sources: renderedSources, + }) + ) totalBytes += buffer.length if (totalBytes > MAX_COMPRESS_TOTAL_BYTES) { return Response.json( @@ -1905,14 +2026,17 @@ export async function executeFileManageOperation( return [ { file: userFile, - identity: { fileId: file.id, key: file.key, context: 'workspace' }, + identity: { + fileId: file.id, + key: file.key, + context: 'workspace', + contentUpdatedAt: file.contentUpdatedAt ?? undefined, + }, ownerUserId: file.uploadedBy, }, ] }) - const selectedArchiveSource = await Promise.all( - selectedInputFiles.map((file) => bindSelectedContentFile(principal, workspaceId, file)) - ) + const selectedArchiveSource = await bindSelectedContentFiles(context, selectedInputFiles) const archiveSource = canonicalArchiveSource.concat(selectedArchiveSource)[0] if (!archiveSource?.identity) { const denied = await assertOperationFileAccess(archive, context) diff --git a/apps/sim/lib/internal/file/parser.test.ts b/apps/sim/lib/internal/file/parser.test.ts index 9df0f56b89f..c3018aeef8c 100644 --- a/apps/sim/lib/internal/file/parser.test.ts +++ b/apps/sim/lib/internal/file/parser.test.ts @@ -37,6 +37,11 @@ const { mockUploadExecutionFile, mockUploadWorkspaceFile, mockReadWorkspaceFileNameByKey, + mockResolveProvenanceSource, + mockGetBoundProvenance, + mockGetFileContentProvenance, + storageConfig, + mockGetBlobContainerClient, } = vi.hoisted(() => { // eslint-disable-next-line @typescript-eslint/no-require-imports const actualPath = require('path') as typeof import('path') @@ -80,9 +85,40 @@ const { }) ), mockReadWorkspaceFileNameByKey: vi.fn(), + mockResolveProvenanceSource: vi.fn(), + mockGetBoundProvenance: vi.fn(), + mockGetFileContentProvenance: vi.fn(), + storageConfig: { + provider: 's3', + bucket: 'sim-execution-files', + containerName: 'execution-files', + }, + mockGetBlobContainerClient: vi.fn(), } }) +vi.mock('@/lib/execution/payloads/file-secret-provenance', () => ({ + resolveStoredFileProvenanceSource: mockResolveProvenanceSource, +})) + +vi.mock('@/lib/uploads/contexts/workspace/workspace-file-secret-provenance', () => ({ + getBoundWorkspaceFileSecretProvenance: mockGetBoundProvenance, +})) + +vi.mock('@/lib/internal/file/operations', () => ({ + getFileContentProvenance: mockGetFileContentProvenance, + fileContentJsonResponse: ( + body: Record, + includePrivate: boolean, + init?: ResponseInit, + provenance?: unknown + ) => + Response.json( + includePrivate ? { ...body, __resolvedSecretTraceProvenance: provenance } : body, + init + ), +})) + vi.mock('@/lib/execution/payloads/materialization.server', () => ({ assertUserFileContentAccess: async (file: { key: string }) => { if (!(await mockVerifyFileAccess(file.key))) throw new Error('File not found') @@ -95,6 +131,24 @@ vi.mock('@/lib/uploads', () => ({ StorageService: storageServiceMock, })) +vi.mock('@/lib/uploads/config', () => ({ + getStorageConfig: () => storageConfig, + S3_CONFIG: {}, + get USE_S3_STORAGE() { + return storageConfig.provider === 's3' + }, + get USE_BLOB_STORAGE() { + return storageConfig.provider === 'blob' + }, + get USE_GCS_STORAGE() { + return storageConfig.provider === 'gcs' + }, +})) + +vi.mock('@/lib/uploads/providers/blob/client', () => ({ + getBlobServiceClient: async () => ({ getContainerClient: mockGetBlobContainerClient }), +})) + vi.mock('@/lib/file-parsers', () => ({ isSupportedFileType: mockIsSupportedFileType, parseBuffer: mockParseBuffer, @@ -186,6 +240,7 @@ async function POST(request: NextRequest): Promise { executionId: parsed.data.executionId || 'execution-id', attributedUserId: 'test-user-id', fileAccessUserId: 'test-user-id', + headers: request.headers, signal: request.signal, }) } @@ -233,6 +288,8 @@ describe('file parser operation', () => { beforeEach(() => { vi.clearAllMocks() + storageConfig.provider = 's3' + mockGetBlobContainerClient.mockReset() setupFileApiMocks({ authenticated: true, }) @@ -254,6 +311,9 @@ describe('file parser operation', () => { }) mockUploadWorkspaceFile.mockClear() mockReadWorkspaceFileNameByKey.mockResolvedValue({ name: null }) + mockResolveProvenanceSource.mockResolvedValue(undefined) + mockGetBoundProvenance.mockResolvedValue({ status: 'exact', entries: [] }) + mockGetFileContentProvenance.mockResolvedValue({ version: 1, complete: true, entries: [] }) mockParseBuffer.mockResolvedValue({ content: 'parsed buffer content', metadata: { pageCount: 1 }, @@ -278,6 +338,321 @@ describe('file parser operation', () => { expect(data).toHaveProperty('error', 'No file path provided') }) + it('exports negotiated canonical execution-file lineage without changing public content', async () => { + const source = { + identity: { + fileId: 'canonical-file', + key: 'execution/workspace-id/workflow-id/execution-id/report.txt', + context: 'execution', + contentUpdatedAt: new Date('2026-09-10T00:00:00Z'), + }, + ownerUserId: 'test-user-id', + } + mockResolveProvenanceSource.mockResolvedValue(source) + const lineage = { + version: 1, + complete: true, + scope: { userId: 'test-user-id', workspaceId: 'workspace-id' }, + entries: [{ name: 'SECRET', encryptedValue: 'encrypted-value' }], + } + mockGetFileContentProvenance.mockResolvedValue(lineage) + + const response = await POST( + createMockRequest( + 'POST', + { filePath: source.identity.key }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + const body = await response.json() + + expect(body.output.content).toBe('parsed buffer content') + expect(body.__resolvedSecretTraceProvenance).toEqual(lineage) + expect(body).not.toHaveProperty('provenanceSource') + expect(body.output).not.toHaveProperty('provenanceSource') + expect(mockGetFileContentProvenance).toHaveBeenCalledWith( + expect.any(Object), + 'workspace-id', + [source], + expect.any(AbortSignal) + ) + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + }) + + it('keeps private canonical provenance out of unnegotiated parser responses', async () => { + mockResolveProvenanceSource.mockResolvedValue({ + identity: { fileId: 'canonical-file', key: 'workspace/report.txt', context: 'workspace' }, + ownerUserId: 'test-user-id', + }) + const response = await POST(createMockRequest('POST', { filePath: 'workspace/report.txt' })) + const body = await response.json() + + expect(body.output.content).toBe('parsed buffer content') + expect(body).not.toHaveProperty('__resolvedSecretTraceProvenance') + expect(body).not.toHaveProperty('provenanceSource') + expect(mockGetFileContentProvenance).not.toHaveBeenCalled() + }) + + it.each([ + { ownerUserId: 'test-user-id', expectedStatus: 'exact' }, + { ownerUserId: 'other-user', expectedStatus: 'unknown' }, + ])('preserves safe copy provenance for $ownerUserId', async ({ ownerUserId, expectedStatus }) => { + const source = { + identity: { fileId: 'canonical-file', key: 'workspace/report.txt', context: 'workspace' }, + ownerUserId, + } + const entries = [{ name: 'SECRET', encryptedValue: 'encrypted-value' }] + mockResolveProvenanceSource.mockResolvedValue(source) + mockGetBoundProvenance.mockResolvedValue({ status: 'exact', entries }) + + await POST(createMockRequest('POST', { filePath: source.identity.key })) + + expect(mockGetBoundProvenance).toHaveBeenCalledWith('workspace-id', source.identity) + expect(mockUploadExecutionFile).toHaveBeenCalledWith( + expect.any(Object), + expect.any(Buffer), + 'report.txt', + 'text/plain', + 'test-user-id', + expectedStatus === 'exact' ? { status: 'exact', entries } : { status: 'unknown' } + ) + }) + + it('keeps tracked unknown sources in the private response instead of treating them as legacy', async () => { + const source = { + identity: { fileId: 'canonical-file', key: 'workspace/report.txt', context: 'workspace' }, + ownerUserId: 'test-user-id', + } + mockResolveProvenanceSource.mockResolvedValue(source) + mockGetBoundProvenance.mockResolvedValue({ status: 'unknown' }) + mockGetFileContentProvenance.mockResolvedValue({ version: 1, complete: false, entries: [] }) + + const response = await POST( + createMockRequest( + 'POST', + { filePath: source.identity.key }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + + expect((await response.json()).__resolvedSecretTraceProvenance.complete).toBe(false) + expect(mockGetFileContentProvenance).toHaveBeenCalledWith( + expect.any(Object), + 'workspace-id', + [source], + expect.any(AbortSignal) + ) + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'unknown' }) + }) + + it('preserves missing historical metadata as absence on copied files', async () => { + const response = await POST( + createMockRequest( + 'POST', + { filePath: 'workspace/legacy.txt' }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + + expect((await response.json()).success).toBe(true) + expect(mockGetBoundProvenance).not.toHaveBeenCalled() + expect(mockUploadExecutionFile.mock.calls[0][5]).toEqual({ status: 'unrecorded' }) + expect(mockGetFileContentProvenance).toHaveBeenCalledWith( + expect.any(Object), + 'workspace-id', + [], + expect.any(AbortSignal) + ) + }) + + it('does not return content when canonical provenance resolution rejects the file scope', async () => { + mockResolveProvenanceSource.mockRejectedValue(new Error('File not found')) + const response = await POST(createMockRequest('POST', { filePath: 'workspace/other.txt' })) + + expect((await response.json()).success).toBe(false) + expect(storageServiceMockFns.mockDownloadFile).not.toHaveBeenCalled() + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + }) + + it('reads owned presigned URLs through canonical authorized storage', async () => { + const key = 'execution/workspace-id/workflow-id/execution-id/report.txt' + const source = { + identity: { fileId: 'canonical-file', key, context: 'execution' }, + ownerUserId: 'test-user-id', + } + mockResolveProvenanceSource.mockResolvedValue(source) + const response = await POST( + createMockRequest( + 'POST', + { filePath: `https://sim-execution-files.s3.us-east-1.amazonaws.com/${key}?signature=old` }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + + expect((await response.json()).success).toBe(true) + expect(mockResolveProvenanceSource).toHaveBeenCalledWith( + { key, context: 'execution' }, + expect.objectContaining({ workspaceId: 'workspace-id', executionId: 'execution-id' }) + ) + expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({ + key, + context: 'execution', + maxBytes: 100 * 1024 * 1024, + }) + expect(inputValidationMockFns.mockSecureFetchWithPinnedIP).not.toHaveBeenCalled() + }) + + it.each([ + 'https://exampleaccount.blob.core.windows.net/execution-files', + 'https://exampleaccount.blob.core.usgovcloudapi.net/execution-files', + 'https://storage.example.test/account/execution-files', + ])( + 'recognizes the configured Azure container endpoint without a separate account name: %s', + async (containerUrl) => { + storageConfig.provider = 'blob' + mockGetBlobContainerClient.mockReturnValue({ url: containerUrl }) + const key = 'execution/workspace-id/workflow-id/execution-id/report.txt' + const source = { + identity: { fileId: 'canonical-file', key, context: 'execution' }, + ownerUserId: 'test-user-id', + } + mockResolveProvenanceSource.mockResolvedValue(source) + mockGetBoundProvenance.mockResolvedValue({ status: 'unknown' }) + + const response = await POST( + createMockRequest('POST', { + filePath: `${containerUrl}/${key}?sig=placeholder`, + }) + ) + + expect((await response.json()).success).toBe(true) + expect(mockGetBlobContainerClient).toHaveBeenCalledWith('execution-files') + expect(mockResolveProvenanceSource).toHaveBeenCalledWith( + { key, context: 'execution' }, + expect.objectContaining({ workspaceId: 'workspace-id' }) + ) + expect(mockGetBoundProvenance).toHaveBeenCalledWith('workspace-id', source.identity) + expect(mockUploadExecutionFile).not.toHaveBeenCalled() + expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({ + key, + context: 'execution', + maxBytes: 100 * 1024 * 1024, + }) + expect(inputValidationMockFns.mockSecureFetchWithPinnedIP).not.toHaveBeenCalled() + } + ) + + it.each([ + 'https://exampleaccount.blob.core.windows.net.attacker.test/execution-files', + 'https://exampleaccount.blob.core.windows.net/execution-files-other', + ])( + 'does not attribute another Azure origin or container to owned storage: %s', + async (containerUrl) => { + storageConfig.provider = 'blob' + mockGetBlobContainerClient.mockReturnValue({ + url: 'https://exampleaccount.blob.core.windows.net/execution-files', + }) + inputValidationMockFns.mockSecureFetchWithPinnedIP.mockResolvedValue( + new Response('external content', { headers: { 'content-type': 'text/plain' } }) + ) + await POST(createMockRequest('POST', { filePath: `${containerUrl}/report.txt` })) + + expect(mockResolveProvenanceSource).not.toHaveBeenCalled() + expect(storageServiceMockFns.mockDownloadFile).not.toHaveBeenCalled() + } + ) + + it('does not attribute an external hostname prefix to canonical storage provenance', async () => { + inputValidationMockFns.mockSecureFetchWithPinnedIP.mockResolvedValue( + new Response('external content', { headers: { 'content-type': 'text/plain' } }) + ) + await POST( + createMockRequest('POST', { + filePath: + 'https://sim-execution-files.s3.us-east-1.amazonaws.com.attacker.test/execution/workspace-id/workflow-id/execution-id/report.txt', + }) + ) + + expect(mockResolveProvenanceSource).not.toHaveBeenCalled() + expect(storageServiceMockFns.mockDownloadFile).not.toHaveBeenCalled() + }) + + it('retains only returned contributors when the multi-file output budget stops parsing', async () => { + const first = { + identity: { fileId: 'first', key: 'workspace/first.txt', context: 'workspace' }, + ownerUserId: 'test-user-id', + } + const second = { + identity: { fileId: 'second', key: 'workspace/second.txt', context: 'workspace' }, + ownerUserId: 'test-user-id', + } + mockResolveProvenanceSource.mockResolvedValueOnce(first).mockResolvedValueOnce(second) + mockParseBuffer + .mockResolvedValueOnce({ content: 'a'.repeat(3 * 1024 * 1024) }) + .mockResolvedValueOnce({ content: 'b'.repeat(3 * 1024 * 1024) }) + const response = await POST( + createMockRequest( + 'POST', + { filePath: ['workspace/first.txt', 'workspace/second.txt', 'workspace/third.txt'] }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + const body = await response.json() + + expect(body.success).toBe(true) + expect(body.results).toHaveLength(1) + expect(body.error).toContain('too large') + expect(mockResolveProvenanceSource).toHaveBeenCalledTimes(2) + expect(mockGetFileContentProvenance).toHaveBeenCalledWith( + expect.any(Object), + 'workspace-id', + [first], + expect.any(AbortSignal) + ) + }) + + it.each([{ filePath: 'workspace/failed.txt' }, { filePath: ['workspace/failed.txt'] }])( + 'keeps failed parser provenance out of the public payload for %j', + async ({ filePath }) => { + mockResolveProvenanceSource.mockResolvedValue({ + identity: { fileId: 'private-source', key: 'workspace/failed.txt', context: 'workspace' }, + ownerUserId: 'private-owner', + }) + mockGetBoundProvenance.mockResolvedValue({ + status: 'exact', + entries: [{ name: 'SECRET', encryptedValue: 'private-ciphertext' }], + }) + mockParseBuffer.mockResolvedValue({ + content: 'discarded parser output', + metadata: { degraded: true, warning: 'Unable to parse format' }, + }) + + const response = await POST( + createMockRequest( + 'POST', + { filePath }, + { 'x-sim-request-private-tool-metadata': 'resolved-secret-provenance-v1' } + ) + ) + const body = await response.json() + const serialized = JSON.stringify(body) + + expect(Array.isArray(filePath) ? body.results[0].success : body.success).toBe(false) + for (const privateValue of [ + 'provenanceSource', + 'private-source', + 'private-owner', + 'private-ciphertext', + 'discarded parser output', + ]) { + expect(serialized).not.toContain(privateValue) + } + for (const call of mockGetFileContentProvenance.mock.calls) { + expect(call[2]).toEqual([]) + } + } + ) + it('should accept and process a local file', async () => { setupFileApiMocks({ cloudEnabled: false, @@ -458,7 +833,8 @@ describe('file parser operation', () => { parsedBuffer, 'report.pdf', 'application/pdf', - 'test-user-id' + 'test-user-id', + { status: 'unrecorded' } ) }) diff --git a/apps/sim/lib/internal/file/parser.ts b/apps/sim/lib/internal/file/parser.ts index 3318736429b..19f24e02ef7 100644 --- a/apps/sim/lib/internal/file/parser.ts +++ b/apps/sim/lib/internal/file/parser.ts @@ -7,6 +7,7 @@ import type { Principal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { generateShortId } from '@sim/utils/id' +import { omit } from '@sim/utils/object' import binaryExtensionsList from 'binary-extensions' import type { ContractBody } from '@/lib/api/contracts' import type { fileParseContract } from '@/lib/api/contracts/storage-transfer' @@ -16,18 +17,32 @@ import { isPayloadSizeLimitError, readNodeStreamToBufferWithLimit, } from '@/lib/core/utils/stream-limits' +import { resolveStoredFileProvenanceSource } from '@/lib/execution/payloads/file-secret-provenance' import { assertUserFileContentAccess, type ExecutionMaterializationContext, } from '@/lib/execution/payloads/materialization.server' +import { + RESOLVED_SECRET_PROVENANCE_METADATA_V1, + requestsPrivateToolMetadata, +} from '@/lib/execution/private-tool-metadata' import { isSupportedFileType, parseBuffer } from '@/lib/file-parsers' import { isFileParserError } from '@/lib/file-parsers/errors' +import { + type FileContentProvenanceSource, + fileContentJsonResponse, + getFileContentProvenance, +} from '@/lib/internal/file/operations' import { isUsingCloudStorage, StorageService } from '@/lib/uploads' import { uploadExecutionFile } from '@/lib/uploads/contexts/execution' import { ExternalUrlValidationError, fetchExternalUrlToWorkspace, } from '@/lib/uploads/contexts/workspace' +import { + getBoundWorkspaceFileSecretProvenance, + type WorkspaceFileSecretProvenance, +} from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' import { UPLOAD_DIR_SERVER } from '@/lib/uploads/core/setup.server' import { isWorkspaceScopedContext } from '@/lib/uploads/shared/types' import { @@ -74,6 +89,7 @@ export interface FileParserOperationContext { fileKeys?: string[] allowLargeValueWorkflowScope?: boolean requestId?: string + headers?: Headers signal?: AbortSignal } @@ -91,6 +107,8 @@ interface ParseResult { originalName?: string // Original filename from database (for workspace files) viewerUrl?: string | null // Viewer URL for the file if available userFile?: UserFile // UserFile object for the raw file + /** Canonical lineage used only when presenting the private tool response. */ + provenanceSource?: FileContentProvenanceSource metadata?: { fileType: string size: number @@ -103,6 +121,32 @@ function getContentBytes(content: unknown): number { return typeof content === 'string' ? Buffer.byteLength(content, 'utf8') : 0 } +/** Keeps stored source lineage on byte-for-byte copies, including legacy absence. */ +async function resolveParserFileProvenance( + file: Pick, + access: FileReadAccessContext, + targetOwnerUserId: string +): Promise<{ + source?: FileContentProvenanceSource + copyProvenance: WorkspaceFileSecretProvenance +}> { + const source = await resolveStoredFileProvenanceSource(file, access) + if (!source) return { copyProvenance: { status: 'unrecorded' } } + const provenance = await getBoundWorkspaceFileSecretProvenance( + access.workspaceId, + source.identity + ) + return { + source, + copyProvenance: + provenance.status === 'exact' && + provenance.entries.length > 0 && + source.ownerUserId !== targetOwnerUserId + ? { status: 'unknown' } + : provenance, + } +} + export async function executeFileParserOperation( input: FileParserOperationInput, context: FileParserOperationContext @@ -122,6 +166,24 @@ export async function executeFileParserOperation( return Response.json({ success: false, error: 'Execution access denied' }, { status: 403 }) } const { attributedUserId, workspaceId } = context + const sources: FileContentProvenanceSource[] = [] + const includePrivateProvenance = Boolean( + context.headers && + requestsPrivateToolMetadata(context.headers, RESOLVED_SECRET_PROVENANCE_METADATA_V1) + ) + const contentResponse = async (body: Record, init?: ResponseInit) => + fileContentJsonResponse( + body, + includePrivateProvenance, + init, + includePrivateProvenance + ? await getFileContentProvenance(context.principal, workspaceId, sources, context.signal) + : undefined + ) + const partialResponse = async (results: unknown[]) => { + const response = parsedOutputTooLargeResponse(results) + return contentResponse(await response.json(), { status: response.status }) + } const fileReadAccess: FileReadAccessContext = { principal: context.principal, workspaceId, @@ -168,7 +230,7 @@ export async function executeFileParserOperation( const remainingOutputBytes = MAX_MULTI_FILE_PARSE_OUTPUT_BYTES - totalOutputBytes if (remainingOutputBytes <= 0) { - return parsedOutputTooLargeResponse(results) + return await partialResponse(results) } const result = await parseFileSingle( @@ -191,8 +253,9 @@ export async function executeFileParserOperation( if (result.success) { totalOutputBytes += getContentBytes(result.content) if (totalOutputBytes > MAX_MULTI_FILE_PARSE_OUTPUT_BYTES) { - return parsedOutputTooLargeResponse(results) + return await partialResponse(results) } + if (result.provenanceSource) sources.push(result.provenanceSource) const displayName = result.originalName || extractCleanFilename(result.filePath) || 'unknown' @@ -213,13 +276,13 @@ export async function executeFileParserOperation( } if (result.error?.startsWith('Parsed file output is too large')) { - return parsedOutputTooLargeResponse(results) + return await partialResponse(results) } - results.push(result) + results.push(omit(result, ['provenanceSource'])) } - return Response.json({ + return await contentResponse({ success: true, results, }) @@ -242,8 +305,9 @@ export async function executeFileParserOperation( } if (result.success) { + if (result.provenanceSource) sources.push(result.provenanceSource) const displayName = result.originalName || extractCleanFilename(result.filePath) || 'unknown' - return Response.json({ + return await contentResponse({ success: true, output: { content: result.content, @@ -258,7 +322,7 @@ export async function executeFileParserOperation( }) } - return Response.json(result) + return Response.json(omit(result, ['provenanceSource'])) } catch (error) { logger.error('Error in file parse API:', error) return Response.json( @@ -337,6 +401,7 @@ async function parseFileSingle( fileType, workspaceId, attributedUserId, + fileReadAccess, executionContext, headers, signal, @@ -498,15 +563,15 @@ function validateFilePath(filePath: string): { isValid: boolean; error?: string * so keying a cache by filename returns stale bytes. `fetchExternalUrlToWorkspace` * delegates to `uploadWorkspaceFile`, which suffix-disambiguates collisions on save. * - * Workspace save is skipped when the URL already points at our execution-files - * bucket (re-uploading our own bytes is wasteful and would generate `image (1).png` - * style aliases for files we already own). + * URLs for our execution-files storage resolve through the authorized canonical + * read path, keeping stored provenance bound to the same bytes the parser reads. */ async function handleExternalUrl( url: string, fileType: string, workspaceId: string, userId: string, + fileReadAccess: FileReadAccessContext, executionContext?: ExecutionContext, headers?: Record, signal?: AbortSignal, @@ -516,36 +581,81 @@ async function handleExternalUrl( try { logger.info('Fetching external URL:', url) - const { getStorageConfig, USE_S3_STORAGE, USE_BLOB_STORAGE, USE_GCS_STORAGE } = await import( - '@/lib/uploads/config' - ) + const { getStorageConfig, S3_CONFIG, USE_S3_STORAGE, USE_BLOB_STORAGE, USE_GCS_STORAGE } = + await import('@/lib/uploads/config') const executionConfig = getStorageConfig('execution') - let isExecutionFile = false + let executionFileKey: string | undefined try { const parsedUrl = new URL(url) if (USE_S3_STORAGE && executionConfig.bucket) { - const bucketInHost = parsedUrl.hostname.startsWith(executionConfig.bucket) - const bucketInPath = parsedUrl.pathname.startsWith(`/${executionConfig.bucket}/`) - isExecutionFile = bucketInHost || bucketInPath + const endpointHost = S3_CONFIG.endpoint ? new URL(S3_CONFIG.endpoint).host : undefined + const bucketHostPrefix = `${executionConfig.bucket}.` + const storageHost = parsedUrl.host.startsWith(bucketHostPrefix) + ? parsedUrl.host.slice(bucketHostPrefix.length) + : parsedUrl.host + const matchesStorageHost = endpointHost + ? storageHost === endpointHost + : /^s3(?:[.-][a-z0-9-]+)?\.amazonaws\.com$/.test(storageHost) + const bucketInHost = matchesStorageHost && parsedUrl.host.startsWith(bucketHostPrefix) + const bucketInPath = + matchesStorageHost && parsedUrl.pathname.startsWith(`/${executionConfig.bucket}/`) + if (bucketInHost || bucketInPath) { + executionFileKey = decodeURIComponent( + bucketInHost + ? parsedUrl.pathname.slice(1) + : parsedUrl.pathname.slice(executionConfig.bucket.length + 2) + ) + } } else if (USE_BLOB_STORAGE && executionConfig.containerName) { - isExecutionFile = url.includes(`/${executionConfig.containerName}/`) + const { getBlobServiceClient } = await import('@/lib/uploads/providers/blob/client') + const client = await getBlobServiceClient() + const containerUrl = new URL(client.getContainerClient(executionConfig.containerName).url) + const prefix = `${containerUrl.pathname.replace(/\/$/, '')}/` + if (parsedUrl.origin === containerUrl.origin && parsedUrl.pathname.startsWith(prefix)) { + executionFileKey = decodeURIComponent(parsedUrl.pathname.slice(prefix.length)) + } } else if (USE_GCS_STORAGE && executionConfig.bucket) { - const bucketInHost = parsedUrl.hostname.startsWith(`${executionConfig.bucket}.`) - const bucketInPath = parsedUrl.pathname.startsWith(`/${executionConfig.bucket}/`) - isExecutionFile = bucketInHost || bucketInPath + const bucketInHost = + parsedUrl.hostname === `${executionConfig.bucket}.storage.googleapis.com` + const bucketInPath = + parsedUrl.hostname === 'storage.googleapis.com' && + parsedUrl.pathname.startsWith(`/${executionConfig.bucket}/`) + if (bucketInHost || bucketInPath) { + executionFileKey = decodeURIComponent( + bucketInHost + ? parsedUrl.pathname.slice(1) + : parsedUrl.pathname.slice(executionConfig.bucket.length + 2) + ) + } } } catch (error) { logger.warn('Failed to parse URL for execution file check:', error) - isExecutionFile = false + executionFileKey = undefined + } + + /** Read owned storage through its authorized, canonical bytes and provenance together. */ + if (executionFileKey) { + return handleCloudFile( + executionFileKey, + fileType, + userId, + fileReadAccess, + fileReadAccess.principal, + workspaceId, + executionContext, + signal, + maxDownloadBytes, + maxParsedOutputBytes + ) } const { filename, buffer, mimeType } = await fetchExternalUrlToWorkspace({ url, userId, workspaceId: workspaceId || undefined, - saveToWorkspace: Boolean(workspaceId) && !isExecutionFile, + saveToWorkspace: Boolean(workspaceId), headers, signal, maxDownloadBytes, @@ -558,7 +668,9 @@ async function handleExternalUrl( let userFile: UserFile | undefined if (executionContext) { try { - userFile = await uploadExecutionFile(executionContext, buffer, filename, mimeType, userId) + userFile = await uploadExecutionFile(executionContext, buffer, filename, mimeType, userId, { + status: 'unrecorded', + }) logger.info(`Stored file in execution storage: ${filename}`, { key: userFile.key }) } catch (uploadError) { logger.warn('Failed to store file in execution storage:', uploadError) @@ -672,6 +784,12 @@ async function handleCloudFile( } } + const sourceProvenance = await resolveParserFileProvenance( + { key: cloudKey, context }, + fileReadAccess, + attributedUserId + ) + let originalFilename: string | undefined // Not filtered to `context = 'workspace'`: a chat attachment carries the same key // prefix and has an `originalName` worth recovering too, and without it the parse @@ -745,7 +863,8 @@ async function handleCloudFile( fileBuffer, filename, mimeType, - attributedUserId + attributedUserId, + sourceProvenance.copyProvenance ) logger.info(`Copied file to execution storage: ${filename}`, { key: userFile.key }) } catch (uploadError) { @@ -803,6 +922,9 @@ async function handleCloudFile( if (userFile) { parseResult.userFile = userFile } + if (parseResult.success && sourceProvenance.source) { + parseResult.provenanceSource = sourceProvenance.source + } signal?.throwIfAborted() @@ -873,6 +995,12 @@ async function handleLocalFile( } } + const sourceProvenance = await resolveParserFileProvenance( + { key: storageKey, context }, + fileReadAccess, + attributedUserId + ) + const fullPath = path.join(UPLOAD_DIR_SERVER, storageKey) logger.info('Processing local file:', fullPath) @@ -916,7 +1044,8 @@ async function handleLocalFile( fileBuffer, filename, mimeType, - attributedUserId + attributedUserId, + sourceProvenance.copyProvenance ) logger.info(`Stored local file in execution storage: ${filename}`, { key: userFile.key }) } catch (uploadError) { @@ -930,6 +1059,7 @@ async function handleLocalFile( content, filePath, userFile, + provenanceSource: sourceProvenance.source, metadata: { fileType: mimeType, size: fileBuffer.length, diff --git a/apps/sim/lib/internal/function/execute.test.ts b/apps/sim/lib/internal/function/execute.test.ts index a5ed37edef3..676a24ecd0b 100644 --- a/apps/sim/lib/internal/function/execute.test.ts +++ b/apps/sim/lib/internal/function/execute.test.ts @@ -18,6 +18,7 @@ vi.mock('@/lib/function-execution/application/execute-function', () => ({ import { FUNCTION_EXECUTION_DELEGATION_AUDIENCE } from '@/lib/function-execution/application/authorization' import { executeFunctionTool } from '@/lib/internal/function/execute' +import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' describe('executeFunctionTool', () => { beforeEach(() => { @@ -59,6 +60,10 @@ describe('executeFunctionTool', () => { executionId: 'execution-1', userId: 'workspace-owner', executorDelegationOrigin: origin, + resolvedSecretTraceRegistry: new ResolvedSecretTraceRegistry([], { + userId: 'workspace-owner', + workspaceId: 'workspace-1', + }), } const headers = new Headers() @@ -92,6 +97,7 @@ describe('executeFunctionTool', () => { userId: undefined, }), headers, + resolvedSecretTraceRegistry: context.resolvedSecretTraceRegistry, }), }) }) diff --git a/apps/sim/lib/internal/function/execute.ts b/apps/sim/lib/internal/function/execute.ts index 4d68ba712a4..291dddd4fa1 100644 --- a/apps/sim/lib/internal/function/execute.ts +++ b/apps/sim/lib/internal/function/execute.ts @@ -71,6 +71,9 @@ export async function executeFunctionTool(input: ExecuteFunctionToolInput): Prom workspaceId: context.workspaceId, body: trustedBody, headers, + ...(context.resolvedSecretTraceRegistry + ? { resolvedSecretTraceRegistry: context.resolvedSecretTraceRegistry } + : {}), ...(signal ? { signal } : {}), ...(sandboxProfile ? { sandboxProfile } : {}), }, diff --git a/apps/sim/lib/internal/google-drive/execute-tool.test.ts b/apps/sim/lib/internal/google-drive/execute-tool.test.ts index 2081e5db43f..5fcc8255c29 100644 --- a/apps/sim/lib/internal/google-drive/execute-tool.test.ts +++ b/apps/sim/lib/internal/google-drive/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ download: vi.fn(), @@ -20,7 +21,7 @@ vi.mock('@/lib/internal/google-drive/operations', () => ({ import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { GoogleDriveOperationError } from '@/lib/internal/google-drive/errors' -import { executeGoogleDriveTool } from '@/lib/internal/google-drive/execute-tool' +import { executeGoogleDriveTool as executeGoogleDriveToolOperation } from '@/lib/internal/google-drive/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' const INPUTS = { @@ -64,6 +65,14 @@ function request( } } +async function executeGoogleDriveTool( + request: Parameters[0] +): Promise { + const result = await executeGoogleDriveToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeGoogleDriveTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -88,6 +97,15 @@ describe('executeGoogleDriveTool', () => { } ) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.download.mockResolvedValueOnce(fileResult) + expect(await executeGoogleDriveToolOperation(request('google_drive_download'))).toBe(fileResult) + }) + it('preserves validation and provider error envelopes', async () => { const invalid = await executeGoogleDriveTool( request('google_drive_export', { input: { accessToken: 'token' } }) diff --git a/apps/sim/lib/internal/google-drive/execute-tool.ts b/apps/sim/lib/internal/google-drive/execute-tool.ts index d96ac227fdd..3d76c819c7c 100644 --- a/apps/sim/lib/internal/google-drive/execute-tool.ts +++ b/apps/sim/lib/internal/google-drive/execute-tool.ts @@ -20,9 +20,11 @@ import { executeGoogleDriveUpload, type GoogleDriveOperationContext, } from '@/lib/internal/google-drive/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' @@ -92,7 +94,9 @@ function unexpectedResponse(request: InternalToolOperationCall, error: unknown): return Response.json({ success: false, error: message }, { status }) } -export const executeGoogleDriveTool: InternalToolOperationHandler = async (request) => { +export const executeGoogleDriveTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() let serialized: string try { @@ -118,7 +122,9 @@ export const executeGoogleDriveTool: InternalToolOperationHandler = async (reque userId: request.context.userId, }) request.signal?.throwIfAborted() - return result instanceof Response ? result : Response.json(result) + return result instanceof Response || isInternalToolFileResult(result) + ? result + : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof GoogleDriveOperationError) { diff --git a/apps/sim/lib/internal/google-drive/operations.test.ts b/apps/sim/lib/internal/google-drive/operations.test.ts index bd3046b17cb..aaab39e66fb 100644 --- a/apps/sim/lib/internal/google-drive/operations.test.ts +++ b/apps/sim/lib/internal/google-drive/operations.test.ts @@ -49,6 +49,17 @@ const context = { userId: 'user-1', } +const storedFile = { + id: 'stored-file', + name: 'stored.bin', + size: 5, + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + url: '/api/files/stored', + key: 'execution/workspace/workflow/run/stored.bin', + context: 'execution', +} as const + describe('Google Drive operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -82,11 +93,12 @@ describe('Google Drive operations', () => { maxResponseBytes: MAX_FILE_SIZE, signal: context.signal, }) - expect(result.output.file).toEqual({ - name: 'report.pdf', - mimeType: 'application/pdf', - data: 'AAAAAA==', - size: 4, + expect(result.files).toEqual([ + { name: 'report.pdf', mimeType: 'application/pdf', buffer: Buffer.alloc(4) }, + ]) + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { file: storedFile }, }) }) @@ -112,7 +124,9 @@ describe('Google Drive operations', () => { label: 'revisionsUrl', signal: context.signal, }) - expect(result.output.metadata.revisions).toEqual([{ id: 'rev-1' }]) + expect(result.present([storedFile])).toMatchObject({ + output: { metadata: { revisions: [{ id: 'rev-1' }] } }, + }) }) it('preserves the export byte limit and exact error', async () => { diff --git a/apps/sim/lib/internal/google-drive/operations.ts b/apps/sim/lib/internal/google-drive/operations.ts index c7d02d82f26..e07fc6b552c 100644 --- a/apps/sim/lib/internal/google-drive/operations.ts +++ b/apps/sim/lib/internal/google-drive/operations.ts @@ -17,6 +17,7 @@ import type { GoogleDriveMoveInput, GoogleDriveUploadInput, } from '@/lib/internal/google-drive/input' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' import type { GoogleDriveFile, GoogleDriveRevision } from '@/tools/google_drive/types' import { @@ -204,18 +205,14 @@ export async function executeGoogleDriveDownload( } context.signal?.throwIfAborted() - return { - success: true, - output: { - file: { - name: input.fileName || metadata.name || 'download', - mimeType: finalMimeType, - data: fileBuffer.toString('base64'), - size: fileBuffer.length, - }, - metadata, + return createInternalToolFileResult( + { + buffer: fileBuffer, + name: input.fileName || metadata.name || 'download', + mimeType: finalMimeType, }, - } + (file) => ({ success: true, output: { file, metadata } }) + ) } export async function executeGoogleDriveExport( @@ -279,18 +276,14 @@ export async function executeGoogleDriveExport( ) } const fileBuffer = Buffer.from(arrayBuffer) - return { - success: true, - output: { - file: { - name: input.fileName || metadata.name || 'export', - mimeType: input.mimeType, - data: fileBuffer.toString('base64'), - size: fileBuffer.length, - }, - exportedMimeType: input.mimeType, + return createInternalToolFileResult( + { + buffer: fileBuffer, + name: input.fileName || metadata.name || 'export', + mimeType: input.mimeType, }, - } + (file) => ({ success: true, output: { file, exportedMimeType: input.mimeType } }) + ) } export async function executeGoogleDriveMove( diff --git a/apps/sim/lib/internal/google-vault/execute-tool.test.ts b/apps/sim/lib/internal/google-vault/execute-tool.test.ts index cebe945359a..c6f86679f64 100644 --- a/apps/sim/lib/internal/google-vault/execute-tool.test.ts +++ b/apps/sim/lib/internal/google-vault/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ downloadGoogleVaultExportFile: vi.fn() })) @@ -10,15 +11,45 @@ vi.mock('@/lib/internal/google-vault/operations', () => ({ downloadGoogleVaultExportFile: mocks.downloadGoogleVaultExportFile, })) -import { executeGoogleVaultTool } from '@/lib/internal/google-vault/execute-tool' +import { executeGoogleVaultTool as executeGoogleVaultToolOperation } from '@/lib/internal/google-vault/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' +async function executeGoogleVaultTool( + request: Parameters[0] +): Promise { + const result = await executeGoogleVaultToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeGoogleVaultTool', () => { beforeEach(() => { vi.clearAllMocks() mocks.downloadGoogleVaultExportFile.mockResolvedValue({ success: true, output: {} }) }) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.downloadGoogleVaultExportFile.mockResolvedValueOnce(fileResult) + expect( + await executeGoogleVaultToolOperation({ + toolId: 'google_vault_download_export_file', + input: { + accessToken: 'token', + matterId: 'matter-1', + bucketName: 'bucket', + objectName: 'file.zip', + }, + headers: new Headers(), + context: createExecutionContext(), + requestId: 'request-1', + }) + ).toBe(fileResult) + }) + it('dispatches typed input and cancellation without HTTP metadata', async () => { const controller = new AbortController() const request: InternalToolOperationCall = { diff --git a/apps/sim/lib/internal/google-vault/execute-tool.ts b/apps/sim/lib/internal/google-vault/execute-tool.ts index c18a7c55327..665b423b162 100644 --- a/apps/sim/lib/internal/google-vault/execute-tool.ts +++ b/apps/sim/lib/internal/google-vault/execute-tool.ts @@ -3,7 +3,11 @@ import { z } from 'zod' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { GoogleVaultOperationError } from '@/lib/internal/google-vault/errors' import { downloadGoogleVaultExportFile } from '@/lib/internal/google-vault/operations' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const inputSchema = z.object({ accessToken: z.string().min(1, 'Access token is required'), @@ -13,7 +17,9 @@ const inputSchema = z.object({ fileName: z.string().optional(), }) -export const executeGoogleVaultTool: InternalToolOperationHandler = async (request) => { +export const executeGoogleVaultTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (request.toolId !== 'google_vault_download_export_file') { return Response.json( @@ -26,9 +32,8 @@ export const executeGoogleVaultTool: InternalToolOperationHandler = async (reque return Response.json({ success: false, error: 'Invalid request data' }, { status: 400 }) } try { - return Response.json( - await downloadGoogleVaultExportFile(parsed.data, { signal: request.signal }) - ) + const result = await downloadGoogleVaultExportFile(parsed.data, { signal: request.signal }) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() const status = isPayloadSizeLimitError(error) diff --git a/apps/sim/lib/internal/google-vault/operations.test.ts b/apps/sim/lib/internal/google-vault/operations.test.ts index fb7d34b9045..03daa3b8da5 100644 --- a/apps/sim/lib/internal/google-vault/operations.test.ts +++ b/apps/sim/lib/internal/google-vault/operations.test.ts @@ -16,6 +16,17 @@ vi.mock('@/lib/core/security/input-validation.server', () => ({ import { downloadGoogleVaultExportFile } from '@/lib/internal/google-vault/operations' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' +const storedFile = { + id: 'stored-file', + name: 'stored.bin', + size: 5, + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + url: '/api/files/stored', + key: 'execution/workspace/workflow/run/stored.bin', + context: 'execution', +} as const + describe('downloadGoogleVaultExportFile', () => { beforeEach(() => { vi.clearAllMocks() @@ -53,11 +64,12 @@ describe('downloadGoogleVaultExportFile', () => { signal: controller.signal, } ) - expect(result.output.file).toEqual({ - name: 'vault export.zip', - mimeType: 'application/zip', - data: 'AQID', - size: 3, + expect(result.files).toEqual([ + { name: 'vault export.zip', mimeType: 'application/zip', buffer: Buffer.from([1, 2, 3]) }, + ]) + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { file: storedFile }, }) }) }) diff --git a/apps/sim/lib/internal/google-vault/operations.ts b/apps/sim/lib/internal/google-vault/operations.ts index 87683ae4b2b..045a55f4096 100644 --- a/apps/sim/lib/internal/google-vault/operations.ts +++ b/apps/sim/lib/internal/google-vault/operations.ts @@ -8,6 +8,7 @@ import { readResponseToBufferWithLimit, } from '@/lib/core/utils/stream-limits' import { GoogleVaultOperationError } from '@/lib/internal/google-vault/errors' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import type { GoogleVaultDownloadExportFileParams } from '@/tools/google_vault/types' import { enhanceGoogleVaultError } from '@/tools/google_vault/utils' @@ -84,10 +85,8 @@ export async function downloadGoogleVaultExportFile( input.fileName, input.objectName ) - return { + return createInternalToolFileResult({ buffer, name, mimeType }, (file) => ({ success: true, - output: { - file: { name, mimeType, data: buffer.toString('base64'), size: buffer.length }, - }, - } + output: { file }, + })) } diff --git a/apps/sim/lib/internal/jupyter/client.test.ts b/apps/sim/lib/internal/jupyter/client.test.ts index 9f2256bf54e..d6e46c42e43 100644 --- a/apps/sim/lib/internal/jupyter/client.test.ts +++ b/apps/sim/lib/internal/jupyter/client.test.ts @@ -14,7 +14,11 @@ vi.mock('@/lib/core/security/input-validation.server', () => ({ secureFetchWithPinnedIP: securityMocks.secureFetchWithPinnedIP, })) -import { InvalidJupyterTargetError, requestJupyterApi } from '@/lib/internal/jupyter/client' +import { + InvalidJupyterTargetError, + requestJupyterApi, + requestJupyterFile, +} from '@/lib/internal/jupyter/client' describe('Jupyter client', () => { beforeEach(() => { @@ -109,4 +113,51 @@ describe('Jupyter client', () => { ).rejects.toMatchObject({ name: 'AbortError' }) expect(securityMocks.validateUrlWithDNS).not.toHaveBeenCalled() }) + + it('downloads raw bytes with token auth, the server base path, and a separate 100 MiB cap', async () => { + const controller = new AbortController() + await requestJupyterFile( + { + serverUrl: 'https://jupyter.example.com/user/alice/', + token: 'secret-token', + path: 'datasets/report #1.xlsx', + }, + controller.signal + ) + + const url = + 'https://jupyter.example.com/user/alice/files/datasets/report%20%231.xlsx?download=1' + expect(securityMocks.validateUrlWithDNS).toHaveBeenCalledWith( + url, + 'serverUrl', + 'selfHostedService' + ) + expect(securityMocks.secureFetchWithPinnedIP).toHaveBeenCalledWith(url, '192.0.2.10', { + method: 'GET', + headers: { Authorization: 'token secret-token' }, + body: undefined, + profile: 'selfHostedService', + maxRedirects: 0, + maxResponseBytes: 100 * 1024 * 1024, + signal: controller.signal, + }) + }) + + it.each(['../secret', '%2e%2e/secret', 'data/../secret'])( + 'rejects raw download traversal before DNS: %s', + async (path) => { + await expect( + requestJupyterFile({ serverUrl: 'jupyter.example.com', token: 'token', path }) + ).rejects.toMatchObject({ name: 'UnsafeJupyterPathError' }) + expect(securityMocks.validateUrlWithDNS).not.toHaveBeenCalled() + } + ) + + it('rejects a raw file target blocked by DNS policy', async () => { + securityMocks.validateUrlWithDNS.mockResolvedValue({ isValid: false, error: 'blocked' }) + await expect( + requestJupyterFile({ serverUrl: 'jupyter.example.com', token: 'token', path: 'data.csv' }) + ).rejects.toBeInstanceOf(InvalidJupyterTargetError) + expect(securityMocks.secureFetchWithPinnedIP).not.toHaveBeenCalled() + }) }) diff --git a/apps/sim/lib/internal/jupyter/client.ts b/apps/sim/lib/internal/jupyter/client.ts index 3e1b3acc3cb..06348db6b29 100644 --- a/apps/sim/lib/internal/jupyter/client.ts +++ b/apps/sim/lib/internal/jupyter/client.ts @@ -7,9 +7,11 @@ import { } from '@/lib/core/security/input-validation.server' import { buildJupyterAuthHeaders, + encodeJupyterPath, InvalidJupyterServerUrlError, normalizeJupyterServerUrl, } from '@/lib/internal/jupyter/protocol' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' export class InvalidJupyterTargetError extends Error { constructor(message: string) { @@ -30,6 +32,30 @@ export interface JupyterApiRequest { export async function requestJupyterApi( input: JupyterApiRequest, signal?: AbortSignal +): Promise { + return requestJupyter(input, `api/${input.path}`, MAX_JSON_API_RESPONSE_BYTES, signal) +} + +/** Downloads raw file bytes through Jupyter's authenticated `/files/` handler. */ +export async function requestJupyterFile( + input: Pick, + signal?: AbortSignal +): Promise { + signal?.throwIfAborted() + const path = encodeJupyterPath(input.path) + return requestJupyter( + { serverUrl: input.serverUrl, token: input.token, path: input.path, method: 'GET' }, + `files/${path}?download=1`, + MAX_BUFFERED_TRANSFER_BYTES, + signal + ) +} + +async function requestJupyter( + input: JupyterApiRequest, + route: string, + maxResponseBytes: number, + signal?: AbortSignal ): Promise { signal?.throwIfAborted() let base: string @@ -41,7 +67,7 @@ export async function requestJupyterApi( } throw error } - const url = `${base}/api/${input.path}` + const url = `${base}/${route}` const urlValidation = await validateUrlWithDNS(url, 'serverUrl', 'selfHostedService') signal?.throwIfAborted() @@ -59,7 +85,7 @@ export async function requestJupyterApi( body: hasBody ? JSON.stringify(input.body) : undefined, profile: 'selfHostedService', maxRedirects: 0, - maxResponseBytes: MAX_JSON_API_RESPONSE_BYTES, + maxResponseBytes, signal, }) } diff --git a/apps/sim/lib/internal/jupyter/execute-tool.test.ts b/apps/sim/lib/internal/jupyter/execute-tool.test.ts index e956ae7e1d5..69464995721 100644 --- a/apps/sim/lib/internal/jupyter/execute-tool.test.ts +++ b/apps/sim/lib/internal/jupyter/execute-tool.test.ts @@ -7,11 +7,14 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' const operationMocks = vi.hoisted(() => ({ executeJupyterProxy: vi.fn(), executeJupyterUpload: vi.fn(), + executeJupyterGetContent: vi.fn(), })) vi.mock('@/lib/internal/jupyter/operations', () => operationMocks) +import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { executeJupyterTool, JUPYTER_PROXY_TOOL_IDS } from '@/lib/internal/jupyter/execute-tool' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' const PROXY_BODY = { @@ -38,6 +41,12 @@ function createRequest( } } +async function executeResponse(request: InternalToolOperationCall): Promise { + const response = await executeJupyterTool(request) + if (!(response instanceof Response)) throw new Error('Expected a JSON response') + return response +} + describe('executeJupyterTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -50,7 +59,7 @@ describe('executeJupyterTool', () => { }) it.each(JUPYTER_PROXY_TOOL_IDS)('recognizes proxy tool ID %s', async (toolId) => { - const response = await executeJupyterTool(createRequest({ toolId })) + const response = await executeResponse(createRequest({ toolId })) expect(response.status).toBe(200) expect(operationMocks.executeJupyterProxy).toHaveBeenCalledWith(PROXY_BODY, { @@ -60,7 +69,7 @@ describe('executeJupyterTool', () => { }) it('validates the canonical proxy contract before provider work', async () => { - const response = await executeJupyterTool(createRequest({ input: { ...PROXY_BODY, path: '' } })) + const response = await executeResponse(createRequest({ input: { ...PROXY_BODY, path: '' } })) expect(response.status).toBe(400) await expect(response.json()).resolves.toMatchObject({ @@ -79,7 +88,7 @@ describe('executeJupyterTool', () => { fileName: 'hello.txt', } - const response = await executeJupyterTool( + const response = await executeResponse( createRequest({ toolId: 'jupyter_upload_file', input, @@ -96,7 +105,7 @@ describe('executeJupyterTool', () => { }) it('fails upload closed without a trusted execution user', async () => { - const response = await executeJupyterTool( + const response = await executeResponse( createRequest({ toolId: 'jupyter_upload_file', context: { @@ -122,11 +131,50 @@ describe('executeJupyterTool', () => { }) it('returns a deterministic error for unsupported IDs', async () => { - const response = await executeJupyterTool(createRequest({ toolId: 'jupyter_unknown' })) + const response = await executeResponse(createRequest({ toolId: 'jupyter_unknown' })) expect(response.status).toBe(500) await expect(response.json()).resolves.toEqual({ error: 'Unsupported Jupyter tool: jupyter_unknown', }) }) + + it('preserves the typed v2 file result for central storage', async () => { + const result = createInternalToolFileResult( + { buffer: Buffer.from('hello'), name: 'notes.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + const input = { ...PROXY_BODY, path: 'notes.txt' } + const controller = new AbortController() + operationMocks.executeJupyterGetContent.mockResolvedValue(result) + expect( + await executeJupyterTool( + createRequest({ toolId: 'jupyter_get_content_v2', input, signal: controller.signal }) + ) + ).toBe(result) + expect(operationMocks.executeJupyterGetContent).toHaveBeenCalledWith(input, { + requestId: 'request-1', + signal: controller.signal, + }) + expect(operationMocks.executeJupyterProxy).not.toHaveBeenCalled() + }) + + it('rejects non-GET v2 reads before provider work', async () => { + const response = await executeResponse( + createRequest({ toolId: 'jupyter_get_content_v2', input: { ...PROXY_BODY, method: 'POST' } }) + ) + expect(response.status).toBe(400) + expect(operationMocks.executeJupyterGetContent).not.toHaveBeenCalled() + }) + + it('projects a download byte limit failure as 413', async () => { + operationMocks.executeJupyterGetContent.mockRejectedValue( + new PayloadSizeLimitError({ label: 'Jupyter file download', maxBytes: 100 * 1024 * 1024 }) + ) + const response = await executeResponse(createRequest({ toolId: 'jupyter_get_content_v2' })) + expect(response.status).toBe(413) + await expect(response.json()).resolves.toMatchObject({ + error: expect.stringContaining('Jupyter file download exceeds maximum size'), + }) + }) }) diff --git a/apps/sim/lib/internal/jupyter/execute-tool.ts b/apps/sim/lib/internal/jupyter/execute-tool.ts index 26ba77d704b..d732610cb30 100644 --- a/apps/sim/lib/internal/jupyter/execute-tool.ts +++ b/apps/sim/lib/internal/jupyter/execute-tool.ts @@ -4,9 +4,19 @@ import type { AnyApiRouteContract } from '@/lib/api/contracts' import { jupyterUploadContract } from '@/lib/api/contracts/storage-transfer' import { jupyterProxyContract } from '@/lib/api/contracts/tools/jupyter' import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' -import { executeJupyterProxy, executeJupyterUpload } from '@/lib/internal/jupyter/operations' +import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { InvalidJupyterTargetError } from '@/lib/internal/jupyter/client' +import { + executeJupyterGetContent, + executeJupyterProxy, + executeJupyterUpload, +} from '@/lib/internal/jupyter/operations' +import { UnsafeJupyterPathError } from '@/lib/internal/jupyter/protocol' import { parseInternalToolInput } from '@/lib/internal/tool-operations/parse-input' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const proxyLogger = createLogger('JupyterProxyAPI') const uploadLogger = createLogger('JupyterUploadAPI') @@ -53,15 +63,31 @@ function unexpectedErrorResponse( } /** Executes every Jupyter tool without routing through the application's HTTP listener. */ -export const executeJupyterTool: InternalToolOperationHandler = async ({ - toolId, - input, - context, - requestId, - signal, -}) => { +export const executeJupyterTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async ({ toolId, input, context, requestId, signal }) => { signal?.throwIfAborted() + if (toolId === 'jupyter_get_content_v2') { + const parsed = parseJupyterBody(jupyterProxyContract, input) + if (!parsed.success) return parsed.response + if (parsed.data.method !== 'GET') { + return Response.json({ error: 'Get Content requires a GET request' }, { status: 400 }) + } + try { + return await executeJupyterGetContent(parsed.data, { requestId, signal }) + } catch (error) { + signal?.throwIfAborted() + if (error instanceof UnsafeJupyterPathError || error instanceof InvalidJupyterTargetError) { + return Response.json({ error: error.message }, { status: 400 }) + } + if (isPayloadSizeLimitError(error)) { + return Response.json({ error: error.message }, { status: 413 }) + } + return unexpectedErrorResponse('proxy', requestId, error, signal) + } + } + if (JUPYTER_PROXY_TOOL_ID_SET.has(toolId)) { const parsed = parseJupyterBody(jupyterProxyContract, input) if (!parsed.success) return parsed.response diff --git a/apps/sim/lib/internal/jupyter/get-content.test.ts b/apps/sim/lib/internal/jupyter/get-content.test.ts new file mode 100644 index 00000000000..6f73d7fd3b0 --- /dev/null +++ b/apps/sim/lib/internal/jupyter/get-content.test.ts @@ -0,0 +1,179 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const clientMocks = vi.hoisted(() => ({ + InvalidJupyterTargetError: class extends Error {}, + requestJupyterApi: vi.fn(), + requestJupyterFile: vi.fn(), +})) + +vi.mock('@/lib/internal/jupyter/client', () => clientMocks) +vi.mock('@/lib/internal/jupyter/file-input', () => ({ resolveJupyterUploadFile: vi.fn() })) + +import { executeJupyterGetContent } from '@/lib/internal/jupyter/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' + +const INPUT = { + serverUrl: 'https://jupyter.example.com/user/alice', + token: 'token', + path: 'data/report #1.xlsx', +} +const CONTEXT = { requestId: 'request-1' } +const MIME_TYPE = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' + +describe('Jupyter Get Content v2', () => { + beforeEach(() => vi.clearAllMocks()) + + it('downloads a 12 MiB workbook through the raw endpoint and presents only its stored file', async () => { + const controller = new AbortController() + const buffer = Buffer.alloc(12 * 1024 * 1024, 42) + clientMocks.requestJupyterApi.mockResolvedValue( + Response.json({ name: 'report #1.xlsx', type: 'file', size: buffer.length, content: null }) + ) + clientMocks.requestJupyterFile.mockResolvedValue( + new Response(buffer, { headers: { 'content-type': MIME_TYPE } }) + ) + + const result = await executeJupyterGetContent(INPUT, { ...CONTEXT, signal: controller.signal }) + expect(isInternalToolFileResult(result)).toBe(true) + if (!isInternalToolFileResult(result)) throw new Error('Expected a file result') + expect(result.files).toHaveLength(1) + expect(result.files[0]?.buffer.length).toBe(buffer.length) + expect(result.files[0]?.buffer.equals(buffer)).toBe(true) + expect(result.files[0]?.name).toBe('report #1.xlsx') + expect(result.files[0]?.mimeType).toBe(MIME_TYPE) + const file = { + id: 'file-1', + name: 'report #1.xlsx', + type: MIME_TYPE, + mimeType: MIME_TYPE, + size: buffer.length, + key: 'execution/file-1', + url: '/api/files/serve/execution/file-1', + } + expect(result.present([file])).toEqual({ success: true, output: { file } }) + expect(clientMocks.requestJupyterApi).toHaveBeenCalledExactlyOnceWith( + { ...INPUT, method: 'GET', path: 'contents/data/report%20%231.xlsx?content=0' }, + controller.signal + ) + expect(clientMocks.requestJupyterFile).toHaveBeenCalledExactlyOnceWith(INPUT, controller.signal) + }) + + it('returns text files as stored files without an inline text alias', async () => { + clientMocks.requestJupyterApi.mockResolvedValue( + Response.json({ type: 'file', name: 'notes.txt', size: 5 }) + ) + clientMocks.requestJupyterFile.mockResolvedValue( + new Response('hello', { headers: { 'content-type': 'text/plain; charset=UTF-8' } }) + ) + const result = await executeJupyterGetContent({ ...INPUT, path: 'notes.txt' }, CONTEXT) + if (!isInternalToolFileResult(result)) throw new Error('Expected a file result') + expect(result.files[0]?.buffer.toString('utf8')).toBe('hello') + expect(result.files[0]?.mimeType).toBe('text/plain') + }) + + it.each([ + { type: 'notebook', content: { cells: [{ cell_type: 'code', source: ['1 + 1'] }] } }, + { type: 'directory', content: [{ type: 'file', name: 'data.csv', path: 'docs/data.csv' }] }, + ])('preserves structured $type output without a raw file request', async ({ type, content }) => { + clientMocks.requestJupyterApi + .mockResolvedValueOnce(Response.json({ name: 'docs', path: 'docs', type, content: null })) + .mockResolvedValueOnce( + Response.json({ name: 'docs', path: 'docs', type, content, format: 'json', mimetype: null }) + ) + const response = await executeJupyterGetContent({ ...INPUT, path: 'docs' }, CONTEXT) + if (!(response instanceof Response)) throw new Error('Expected structured content') + await expect(response.json()).resolves.toEqual({ + success: true, + output: { + name: 'docs', + path: 'docs', + mimetype: null, + text: JSON.stringify(content), + file: null, + }, + }) + expect(clientMocks.requestJupyterApi).toHaveBeenNthCalledWith( + 2, + { ...INPUT, method: 'GET', path: `contents/docs?content=1&type=${type}` }, + undefined + ) + expect(clientMocks.requestJupyterFile).not.toHaveBeenCalled() + }) + + it('rejects oversized metadata before reading file bytes', async () => { + clientMocks.requestJupyterApi.mockResolvedValue( + Response.json({ type: 'file', size: MAX_BUFFERED_TRANSFER_BYTES + 1 }) + ) + const response = await executeJupyterGetContent(INPUT, CONTEXT) + if (!(response instanceof Response)) throw new Error('Expected a size error') + expect(response.status).toBe(413) + expect(clientMocks.requestJupyterFile).not.toHaveBeenCalled() + }) + + it('enforces the raw byte cap when metadata size is missing or wrong', async () => { + const cancel = vi.fn() + clientMocks.requestJupyterApi.mockResolvedValue(Response.json({ type: 'file', size: 1 })) + clientMocks.requestJupyterFile.mockResolvedValue( + new Response(new ReadableStream({ cancel }), { + headers: { 'content-length': String(MAX_BUFFERED_TRANSFER_BYTES + 1) }, + }) + ) + await expect(executeJupyterGetContent(INPUT, CONTEXT)).rejects.toMatchObject({ + name: 'PayloadSizeLimitError', + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + }) + expect(cancel).toHaveBeenCalledOnce() + }) + + it('keeps notebook JSON bounded at 10 MiB', async () => { + clientMocks.requestJupyterApi + .mockResolvedValueOnce(Response.json({ type: 'notebook' })) + .mockResolvedValueOnce( + new Response('', { headers: { 'content-length': String(10 * 1024 * 1024 + 1) } }) + ) + await expect(executeJupyterGetContent(INPUT, CONTEXT)).rejects.toMatchObject({ + name: 'PayloadSizeLimitError', + maxBytes: 10 * 1024 * 1024, + }) + expect(clientMocks.requestJupyterFile).not.toHaveBeenCalled() + }) + + it.each(['metadata', 'download'])( + 'preserves an upstream %s failure without returning a file', + async (stage) => { + const errorResponse = new Response('not found', { status: 404 }) + clientMocks.requestJupyterApi.mockResolvedValue( + stage === 'metadata' ? errorResponse : Response.json({ type: 'file' }) + ) + clientMocks.requestJupyterFile.mockResolvedValue(errorResponse) + const response = await executeJupyterGetContent(INPUT, CONTEXT) + if (!(response instanceof Response)) throw new Error('Expected an upstream error') + expect(response.status).toBe(404) + await expect(response.json()).resolves.toEqual({ error: 'Jupyter API error: 404 not found' }) + } + ) + + it('does not fetch raw bytes when cancellation arrives after metadata', async () => { + const controller = new AbortController() + clientMocks.requestJupyterApi.mockImplementation(async () => { + controller.abort(new DOMException('cancelled', 'AbortError')) + return Response.json({ type: 'file' }) + }) + await expect( + executeJupyterGetContent(INPUT, { ...CONTEXT, signal: controller.signal }) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(clientMocks.requestJupyterFile).not.toHaveBeenCalled() + }) + + it('rejects malformed metadata without requesting raw bytes', async () => { + clientMocks.requestJupyterApi.mockResolvedValue(Response.json({ type: 'unexpected' })) + const response = await executeJupyterGetContent(INPUT, CONTEXT) + if (!(response instanceof Response)) throw new Error('Expected an invalid model error') + expect(response.status).toBe(502) + expect(clientMocks.requestJupyterFile).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/internal/jupyter/operations.test.ts b/apps/sim/lib/internal/jupyter/operations.test.ts index b8a8d33ca7b..a705d1f6f38 100644 --- a/apps/sim/lib/internal/jupyter/operations.test.ts +++ b/apps/sim/lib/internal/jupyter/operations.test.ts @@ -8,6 +8,7 @@ const clientMocks = vi.hoisted(() => { return { InvalidJupyterTargetError, requestJupyterApi: vi.fn(), + requestJupyterFile: vi.fn(), } }) const fileInputMocks = vi.hoisted(() => ({ diff --git a/apps/sim/lib/internal/jupyter/operations.ts b/apps/sim/lib/internal/jupyter/operations.ts index a42129cd911..3beeef70bb3 100644 --- a/apps/sim/lib/internal/jupyter/operations.ts +++ b/apps/sim/lib/internal/jupyter/operations.ts @@ -1,7 +1,18 @@ import { createLogger } from '@sim/logger' import type { JupyterUploadBody } from '@/lib/api/contracts/storage-transfer' import type { JupyterProxyBody } from '@/lib/api/contracts/tools/jupyter' -import { InvalidJupyterTargetError, requestJupyterApi } from '@/lib/internal/jupyter/client' +import { MAX_JSON_API_RESPONSE_BYTES } from '@/lib/core/security/input-validation.server' +import { + DEFAULT_MAX_ERROR_BODY_BYTES, + readResponseJsonWithLimit, + readResponseTextWithLimit, + readResponseToBufferWithLimit, +} from '@/lib/core/utils/stream-limits' +import { + InvalidJupyterTargetError, + requestJupyterApi, + requestJupyterFile, +} from '@/lib/internal/jupyter/client' import { resolveJupyterUploadFile } from '@/lib/internal/jupyter/file-input' import { assertSafeJupyterProxyPath, @@ -9,6 +20,10 @@ import { parseJupyterContentModel, UnsafeJupyterPathError, } from '@/lib/internal/jupyter/protocol' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { InternalToolOperationResult } from '@/lib/internal/tool-operations/types' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' +import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' const uploadLogger = createLogger('JupyterUploadAPI') @@ -25,6 +40,107 @@ function validationErrorResponse(error: UnsafeJupyterPathError | InvalidJupyterT return Response.json({ success: false, error: error.message }, { status: 400 }) } +/** Stores files before JSON presentation while preserving structured notebook and directory reads. */ +export async function executeJupyterGetContent( + input: Pick, + context: JupyterOperationContext +): Promise { + const { signal } = context + signal?.throwIfAborted() + const path = encodeJupyterPath(input.path) + const auth = { serverUrl: input.serverUrl, token: input.token } + const metadataResponse = await requestJupyterApi( + { ...auth, method: 'GET', path: `contents/${path}?content=0` }, + signal + ) + if (!metadataResponse.ok) return jupyterReadErrorResponse(metadataResponse, signal) + const metadata = parseJupyterContentModel( + await readResponseJsonWithLimit(metadataResponse, { + maxBytes: MAX_JSON_API_RESPONSE_BYTES, + label: 'Jupyter content metadata', + signal, + }) + ) + signal?.throwIfAborted() + if (!metadata?.type) { + return Response.json({ error: 'Jupyter returned an invalid content model' }, { status: 502 }) + } + + if (metadata.type === 'file') { + if (metadata.size !== undefined && metadata.size > MAX_BUFFERED_TRANSFER_BYTES) { + return Response.json( + { error: 'Jupyter file exceeds the 100 MB download limit' }, + { status: 413 } + ) + } + const response = await requestJupyterFile(input, signal) + if (!response.ok) return jupyterReadErrorResponse(response, signal) + const buffer = await readResponseToBufferWithLimit(response, { + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + label: 'Jupyter file download', + signal, + }) + signal?.throwIfAborted() + const name = metadata.name || input.path.split('/').filter(Boolean).at(-1) || 'file' + const mimeType = + response.headers.get('content-type')?.split(';')[0]?.trim() || + metadata.mimetype || + getMimeTypeFromExtension(getFileExtension(name)) + return createInternalToolFileResult({ buffer, name, mimeType }, (file) => ({ + success: true, + output: { file }, + })) + } + + const response = await requestJupyterApi( + { ...auth, method: 'GET', path: `contents/${path}?content=1&type=${metadata.type}` }, + signal + ) + if (!response.ok) return jupyterReadErrorResponse(response, signal) + const data = parseJupyterContentModel( + await readResponseJsonWithLimit(response, { + maxBytes: MAX_JSON_API_RESPONSE_BYTES, + label: 'Jupyter structured content', + signal, + }) + ) + signal?.throwIfAborted() + if (data?.type !== metadata.type) { + return Response.json({ error: 'Jupyter returned an invalid content model' }, { status: 502 }) + } + const text = + data.format === 'json' || typeof data.content === 'object' + ? JSON.stringify(data.content) + : typeof data.content === 'string' + ? data.content + : null + return Response.json({ + success: true, + output: { + name: data.name ?? metadata.name ?? '', + path: data.path ?? input.path, + mimetype: data.mimetype ?? null, + text, + file: null, + }, + }) +} + +async function jupyterReadErrorResponse( + response: Awaited>, + signal?: AbortSignal +): Promise { + const errorText = await readResponseTextWithLimit(response, { + maxBytes: DEFAULT_MAX_ERROR_BODY_BYTES, + label: 'Jupyter read error', + signal, + }) + return Response.json( + { error: `Jupyter API error: ${response.status} ${errorText}` }, + { status: response.status } + ) +} + /** Executes the shared Jupyter proxy contract and mirrors the upstream response verbatim. */ export async function executeJupyterProxy( input: JupyterProxyBody, diff --git a/apps/sim/lib/internal/microsoft-teams/execute-tool.test.ts b/apps/sim/lib/internal/microsoft-teams/execute-tool.test.ts index da09c978423..873618b9d72 100644 --- a/apps/sim/lib/internal/microsoft-teams/execute-tool.test.ts +++ b/apps/sim/lib/internal/microsoft-teams/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ deleteMicrosoftTeamsChatMessage: vi.fn(), @@ -16,9 +17,17 @@ vi.mock('@/lib/internal/microsoft-teams/operations', () => ({ writeMicrosoftTeamsChatMessage: mocks.writeMicrosoftTeamsChatMessage, })) -import { executeMicrosoftTeamsTool } from '@/lib/internal/microsoft-teams/execute-tool' +import { executeMicrosoftTeamsTool as executeMicrosoftTeamsToolOperation } from '@/lib/internal/microsoft-teams/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' +async function executeMicrosoftTeamsTool( + request: Parameters[0] +): Promise { + const result = await executeMicrosoftTeamsToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeMicrosoftTeamsTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -27,6 +36,23 @@ describe('executeMicrosoftTeamsTool', () => { mocks.writeMicrosoftTeamsChatMessage.mockResolvedValue({ success: true, output: {} }) }) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.writeMicrosoftTeamsChatMessage.mockResolvedValueOnce(fileResult) + expect( + await executeMicrosoftTeamsToolOperation({ + toolId: 'microsoft_teams_write_chat', + input: { accessToken: 'token', chatId: 'chat-1', content: 'hello', files: null }, + headers: new Headers(), + context: createExecutionContext(), + requestId: 'request-1', + }) + ).toBe(fileResult) + }) + it('dispatches typed input with cancellation', async () => { const controller = new AbortController() const input = { accessToken: 'token', chatId: 'chat-1', messageId: 'message-1' } diff --git a/apps/sim/lib/internal/microsoft-teams/execute-tool.ts b/apps/sim/lib/internal/microsoft-teams/execute-tool.ts index 0c371d540b9..442be760b82 100644 --- a/apps/sim/lib/internal/microsoft-teams/execute-tool.ts +++ b/apps/sim/lib/internal/microsoft-teams/execute-tool.ts @@ -12,7 +12,11 @@ import { microsoftTeamsWriteChannelInputSchema, microsoftTeamsWriteChatInputSchema, } from '@/lib/internal/microsoft-teams/schema' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const deleteInputSchema = z.object({ accessToken: z.string().min(1, 'Access token is required'), @@ -36,7 +40,9 @@ function inputSizeError(input: unknown): Response | null { ) } -export const executeMicrosoftTeamsTool: InternalToolOperationHandler = async (request) => { +export const executeMicrosoftTeamsTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() const sizeError = inputSizeError(request.input) if (sizeError) return sizeError @@ -50,12 +56,14 @@ export const executeMicrosoftTeamsTool: InternalToolOperationHandler = async (re case 'microsoft_teams_write_chat': { const parsed = microsoftTeamsWriteChatInputSchema.safeParse(request.input) if (!parsed.success) return validationErrorResponse(parsed.error) - return Response.json(await writeMicrosoftTeamsChatMessage(parsed.data, context)) + const result = await writeMicrosoftTeamsChatMessage(parsed.data, context) + return isInternalToolFileResult(result) ? result : Response.json(result) } case 'microsoft_teams_write_channel': { const parsed = microsoftTeamsWriteChannelInputSchema.safeParse(request.input) if (!parsed.success) return validationErrorResponse(parsed.error) - return Response.json(await writeMicrosoftTeamsChannelMessage(parsed.data, context)) + const result = await writeMicrosoftTeamsChannelMessage(parsed.data, context) + return isInternalToolFileResult(result) ? result : Response.json(result) } case 'microsoft_teams_delete_chat_message': { const parsed = deleteInputSchema.safeParse(request.input) diff --git a/apps/sim/lib/internal/microsoft-teams/operations.test.ts b/apps/sim/lib/internal/microsoft-teams/operations.test.ts index 356c68571f7..927769c2429 100644 --- a/apps/sim/lib/internal/microsoft-teams/operations.test.ts +++ b/apps/sim/lib/internal/microsoft-teams/operations.test.ts @@ -2,6 +2,7 @@ * @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), @@ -54,6 +55,7 @@ describe('deleteMicrosoftTeamsChatMessage', () => { expect(mocks.fetch.mock.calls[1][1]).toEqual( expect.objectContaining({ method: 'POST', signal: controller.signal }) ) + if (isInternalToolFileResult(result)) throw new Error('Expected a JSON result') expect(result.output).toEqual({ deleted: true, messageId: 'message-1', @@ -83,6 +85,7 @@ describe('deleteMicrosoftTeamsChatMessage', () => { expect(mocks.fetch.mock.calls[0][1]).toEqual( expect.objectContaining({ method: 'POST', signal: controller.signal }) ) + if (isInternalToolFileResult(result)) throw new Error('Expected a JSON result') expect(result.output).toEqual({ updatedContent: true, metadata: { @@ -95,6 +98,50 @@ describe('deleteMicrosoftTeamsChatMessage', () => { }) }) + it('keeps multiple valid attachments out of the message response body', async () => { + const buffer = Buffer.alloc(4 * 1024 * 1024, 1) + const sourceFiles = Array.from({ length: 3 }, (_, index) => ({ + id: `source-${index}`, + key: `workspace/file-${index}.txt`, + name: `file-${index}.txt`, + size: buffer.length, + type: 'text/plain', + })) + mocks.processFilesToUserFiles.mockReturnValue(sourceFiles) + mocks.downloadServableFileFromStorage.mockResolvedValue({ buffer, contentType: 'text/plain' }) + mocks.fetch.mockReset() + for (const file of sourceFiles) { + mocks.fetch + .mockResolvedValueOnce(Response.json({ id: file.id })) + .mockResolvedValueOnce( + Response.json({ id: file.id, webDavUrl: `https://teams.example/${file.name}` }) + ) + } + mocks.fetch.mockResolvedValueOnce(Response.json({ id: 'message-1', chatId: 'chat-1' })) + + const result = await writeMicrosoftTeamsChatMessage( + { accessToken: 'token', chatId: 'chat-1', content: 'hello', files: sourceFiles }, + { requestId: 'request-1', userId: 'user-1' } + ) + + if (!isInternalToolFileResult(result)) throw new Error('Expected a file output') + expect(result.files).toHaveLength(3) + for (const file of result.files) expect(file.buffer).toBe(buffer) + expect(mocks.downloadServableFileFromStorage).toHaveBeenCalledTimes(3) + expect(mocks.fetch).toHaveBeenCalledTimes(7) + const storedFiles = sourceFiles.map((file) => ({ + ...file, + mimeType: file.type, + url: `/api/files/${file.id}`, + context: 'execution' as const, + })) + const presented = result.present(storedFiles) + expect(presented).toMatchObject({ + output: { files: storedFiles, metadata: { attachmentCount: 3 } }, + }) + expect(Buffer.byteLength(JSON.stringify(presented))).toBeLessThan(10 * 1024) + }) + it('resolves mentions in-process while preserving the enhanced output envelope', async () => { mocks.fetch.mockReset() mocks.fetch @@ -120,6 +167,7 @@ describe('deleteMicrosoftTeamsChatMessage', () => { body: { contentType: 'html', content: 'Ada hello' }, mentions: [{ id: 0, mentionText: 'Ada' }], }) + if (isInternalToolFileResult(result)) throw new Error('Expected a JSON result') expect(result.output).toMatchObject({ updatedContent: true, metadata: { chatId: 'chat-1', attachmentCount: 0 }, diff --git a/apps/sim/lib/internal/microsoft-teams/operations.ts b/apps/sim/lib/internal/microsoft-teams/operations.ts index 4de7a0cd846..7cca5977186 100644 --- a/apps/sim/lib/internal/microsoft-teams/operations.ts +++ b/apps/sim/lib/internal/microsoft-teams/operations.ts @@ -11,6 +11,10 @@ import type { MicrosoftTeamsWriteChannelInput, MicrosoftTeamsWriteChatInput, } from '@/lib/internal/microsoft-teams/schema' +import { + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' @@ -35,13 +39,6 @@ export interface MicrosoftTeamsOperationContext { userId?: string } -interface TeamsFileOutput { - name: string - mimeType: string - data: string - size: number -} - interface TeamsAttachmentRef { id: string contentType: 'reference' @@ -95,13 +92,13 @@ async function uploadFilesForMessage( rawFiles: NonNullable, client: MicrosoftTeamsClient, context: MicrosoftTeamsOperationContext -): Promise<{ attachments: TeamsAttachmentRef[]; files: TeamsFileOutput[] }> { +): Promise<{ attachments: TeamsAttachmentRef[]; files: InternalToolFile[] }> { if (rawFiles.length === 0) return { attachments: [], files: [] } if (!context.userId) throw new MicrosoftTeamsOperationError('Authentication required', 401) const requestId = context.requestId || 'microsoft-teams-operation' const userFiles = processFilesToUserFiles(rawFiles, requestId, logger) const attachments: TeamsAttachmentRef[] = [] - const files: TeamsFileOutput[] = [] + const files: InternalToolFile[] = [] let totalBytes = 0 for (const file of userFiles) { @@ -141,8 +138,7 @@ async function uploadFilesForMessage( files.push({ name: file.name, mimeType: contentType, - data: buffer.toString('base64'), - size: buffer.length, + buffer, }) let uploaded: MicrosoftTeamsGraphObject @@ -353,17 +349,20 @@ async function sendMessage(args: { 'Failed to send Teams message', args.context.signal ) - return { - success: true as const, - output: { - updatedContent: true, - metadata: { - ...args.enhancedMetadata(data), - attachmentCount: uploaded.attachments.length, - }, - files: uploaded.files, + const output = { + updatedContent: true, + metadata: { + ...args.enhancedMetadata(data), + attachmentCount: uploaded.attachments.length, }, } + if (uploaded.files.length === 0) { + return { success: true as const, output: { ...output, files: [] } } + } + return createInternalToolFilesResult(uploaded.files, (files) => ({ + success: true, + output: { ...output, files }, + })) } export async function writeMicrosoftTeamsChatMessage( diff --git a/apps/sim/lib/internal/microsoft-word/execute-tool.test.ts b/apps/sim/lib/internal/microsoft-word/execute-tool.test.ts index bee8eae27e3..f1ac631927f 100644 --- a/apps/sim/lib/internal/microsoft-word/execute-tool.test.ts +++ b/apps/sim/lib/internal/microsoft-word/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const operationMocks = vi.hoisted(() => ({ executeMicrosoftWordAppend: vi.fn(), @@ -18,7 +19,7 @@ vi.mock('@/lib/internal/microsoft-word/operations', () => operationMocks) import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' import { GraphRequestError } from '@/lib/internal/microsoft-word/client' -import { executeMicrosoftWordTool } from '@/lib/internal/microsoft-word/execute-tool' +import { executeMicrosoftWordTool as executeMicrosoftWordToolOperation } from '@/lib/internal/microsoft-word/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' const READ_INPUT = { accessToken: 'token', documentId: 'document-1' } @@ -70,6 +71,14 @@ const TOOL_CASES = [ ], ] as const +async function executeMicrosoftWordTool( + request: Parameters[0] +): Promise { + const result = await executeMicrosoftWordToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeMicrosoftWordTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -96,6 +105,22 @@ describe('executeMicrosoftWordTool', () => { }) }) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + operationMocks.executeMicrosoftWordExportPdf.mockResolvedValueOnce(fileResult) + expect( + await executeMicrosoftWordToolOperation( + createRequest({ + toolId: 'microsoft_word_export_pdf', + input: { accessToken: 'token', documentId: 'document-1' }, + }) + ) + ).toBe(fileResult) + }) + it('returns validation errors before provider work', async () => { const response = await executeMicrosoftWordTool( createRequest({ input: { accessToken: '', documentId: '' } }) diff --git a/apps/sim/lib/internal/microsoft-word/execute-tool.ts b/apps/sim/lib/internal/microsoft-word/execute-tool.ts index 9ea1b0ea7cf..b4948fc0c9c 100644 --- a/apps/sim/lib/internal/microsoft-word/execute-tool.ts +++ b/apps/sim/lib/internal/microsoft-word/execute-tool.ts @@ -24,7 +24,11 @@ import { microsoftWordReplaceTextInputSchema, microsoftWordUpdateInputSchema, } from '@/lib/internal/microsoft-word/schema' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const logger = createLogger('MicrosoftWordToolExecution') @@ -34,7 +38,7 @@ async function executeOperation( execute: (input: z.output, context: MicrosoftWordOperationContext) => Promise, context: MicrosoftWordOperationContext, toolId: string -): Promise { +): Promise { context.signal?.throwIfAborted() let serializedInput: string try { @@ -61,7 +65,7 @@ async function executeOperation( try { const result = await execute(parsed.data, context) context.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { context.signal?.throwIfAborted() const message = getErrorMessage(error, 'Unknown error occurred') @@ -81,7 +85,9 @@ async function executeOperation( } } -export const executeMicrosoftWordTool: InternalToolOperationHandler = async (request) => { +export const executeMicrosoftWordTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { const { input, requestId, signal, toolId } = request const context: MicrosoftWordOperationContext = { requestId, signal } diff --git a/apps/sim/lib/internal/microsoft-word/operations.test.ts b/apps/sim/lib/internal/microsoft-word/operations.test.ts index 7fcc9bb4170..64add99b95d 100644 --- a/apps/sim/lib/internal/microsoft-word/operations.test.ts +++ b/apps/sim/lib/internal/microsoft-word/operations.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext, inputValidationMock, inputValidationMockFns } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { executeMicrosoftWordExportPdf } from '@/lib/internal/microsoft-word/operations' vi.mock('@/lib/core/security/input-validation.server', () => inputValidationMock) @@ -93,7 +94,7 @@ beforeEach(() => { }) }) -function executeTool(toolId: string, input: unknown): Promise { +async function executeTool(toolId: string, input: unknown): Promise { const request: InternalToolOperationCall = { toolId, input, @@ -101,13 +102,44 @@ function executeTool(toolId: string, input: unknown): Promise { context: createExecutionContext({ workflowId: 'workflow-1' }), requestId: 'request-1', } - return executeMicrosoftWordTool(request) + const result = await executeMicrosoftWordTool(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result } function executeAppend(input: typeof baseBody): Promise { return executeTool('microsoft_word_append', input) } +describe('Microsoft Word PDF file output', () => { + it('keeps large PDFs in process for the executor to store', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024, 1) + mockSecureFetchWithPinnedIP + .mockResolvedValueOnce(itemResponse('version-1')) + .mockResolvedValueOnce(new Response(buffer)) + const result = await executeMicrosoftWordExportPdf( + { accessToken: 'token-123', documentId: 'doc-abc' }, + { requestId: 'request-1' } + ) + expect(result.files).toHaveLength(1) + expect(result.files[0]?.name).toBe('notes.pdf') + expect(result.files[0]?.mimeType).toBe('application/pdf') + expect(result.files[0]?.buffer.length).toBe(buffer.length) + expect(result.files[0]?.buffer.equals(buffer)).toBe(true) + const file = { + id: 'stored', + name: 'notes.pdf', + size: buffer.length, + type: 'application/pdf', + mimeType: 'application/pdf', + url: '/api/files/stored', + key: 'execution/notes.pdf', + context: 'execution' as const, + } + expect(result.present([file])).toEqual({ success: true, output: { file } }) + }) +}) + describe('Microsoft Word direct input validation', () => { it('rejects a whitespace-only document name before provider work', async () => { const response = await executeTool('microsoft_word_create', { diff --git a/apps/sim/lib/internal/microsoft-word/operations.ts b/apps/sim/lib/internal/microsoft-word/operations.ts index 4a3d990bf45..82999611f04 100644 --- a/apps/sim/lib/internal/microsoft-word/operations.ts +++ b/apps/sim/lib/internal/microsoft-word/operations.ts @@ -17,6 +17,7 @@ import type { MicrosoftWordReplaceTextInput, MicrosoftWordUpdateInput, } from '@/lib/internal/microsoft-word/schema' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { appendParagraphsToDocx, buildDocxFromContent, @@ -322,15 +323,8 @@ export async function executeMicrosoftWordExportPdf( name, size: pdfBuffer.length, }) - return { - success: true as const, - output: { - file: { - name, - mimeType: PDF_MIME_TYPE, - data: pdfBuffer.toString('base64'), - size: pdfBuffer.length, - }, - }, - } + return createInternalToolFileResult( + { buffer: pdfBuffer, name, mimeType: PDF_MIME_TYPE }, + (file) => ({ success: true, output: { file } }) + ) } diff --git a/apps/sim/lib/internal/onedrive/execute-tool.test.ts b/apps/sim/lib/internal/onedrive/execute-tool.test.ts index ae324b007b6..4644ef711aa 100644 --- a/apps/sim/lib/internal/onedrive/execute-tool.test.ts +++ b/apps/sim/lib/internal/onedrive/execute-tool.test.ts @@ -1,8 +1,10 @@ /** * @vitest-environment node */ + import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ downloadOneDriveFile: vi.fn(), @@ -17,10 +19,15 @@ vi.mock('@/lib/internal/onedrive/operations', () => ({ import { executeOneDriveTool } from '@/lib/internal/onedrive/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' +const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.pdf', mimeType: 'application/pdf' }, + (file) => ({ success: true, output: { file } }) +) + describe('executeOneDriveTool', () => { beforeEach(() => { vi.clearAllMocks() - mocks.downloadOneDriveFile.mockResolvedValue({ success: true, output: {} }) + mocks.downloadOneDriveFile.mockResolvedValue(fileResult) mocks.uploadOneDriveFile.mockResolvedValue({ success: true, output: {} }) }) @@ -35,7 +42,7 @@ describe('executeOneDriveTool', () => { signal: controller.signal, } - expect((await executeOneDriveTool(request)).status).toBe(200) + expect(await executeOneDriveTool(request)).toBe(fileResult) expect(mocks.downloadOneDriveFile).toHaveBeenCalledWith( { accessToken: 'token', fileId: 'file-1', fileName: undefined }, { signal: controller.signal } diff --git a/apps/sim/lib/internal/onedrive/execute-tool.ts b/apps/sim/lib/internal/onedrive/execute-tool.ts index a4791fd31d5..12bc5ade032 100644 --- a/apps/sim/lib/internal/onedrive/execute-tool.ts +++ b/apps/sim/lib/internal/onedrive/execute-tool.ts @@ -6,7 +6,10 @@ import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { OneDriveOperationError } from '@/lib/internal/onedrive/errors' import { downloadOneDriveFile, uploadOneDriveFile } from '@/lib/internal/onedrive/operations' import { oneDriveUploadInputSchema } from '@/lib/internal/onedrive/schema' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const downloadInputSchema = z.object({ accessToken: z.string().min(1, 'Access token is required'), @@ -30,7 +33,9 @@ function inputSizeError(input: unknown): Response | null { ) } -export const executeOneDriveTool: InternalToolOperationHandler = async (request) => { +export const executeOneDriveTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() const sizeError = inputSizeError(request.input) if (sizeError) return sizeError @@ -39,11 +44,9 @@ export const executeOneDriveTool: InternalToolOperationHandler = async (request) case 'onedrive_download': { const parsed = downloadInputSchema.safeParse(request.input) if (!parsed.success) return validationErrorResponse(parsed.error) - return Response.json( - await downloadOneDriveFile( - { ...parsed.data, fileName: parsed.data.fileName ?? undefined }, - { signal: request.signal } - ) + return await downloadOneDriveFile( + { ...parsed.data, fileName: parsed.data.fileName ?? undefined }, + { signal: request.signal } ) } case 'onedrive_upload': { diff --git a/apps/sim/lib/internal/onedrive/operations.test.ts b/apps/sim/lib/internal/onedrive/operations.test.ts index 975cd038361..db46c90a502 100644 --- a/apps/sim/lib/internal/onedrive/operations.test.ts +++ b/apps/sim/lib/internal/onedrive/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), @@ -63,12 +68,40 @@ describe('downloadOneDriveFile', () => { expect(mocks.secureFetchWithPinnedIP.mock.calls[1][2]).toEqual( expect.objectContaining({ signal: controller.signal }) ) - expect(result.output.file).toEqual({ + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { + name: 'report.pdf', + mimeType: 'application/pdf', + buffer: Buffer.from([1, 2, 3]), + }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', name: 'report.pdf', + type: 'application/pdf', mimeType: 'application/pdf', - data: 'AQID', size: 3, - }) + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ success: true, output: { file: storedFile } }) + }) + + it('preserves downloads above the JSON response limit as bytes', async () => { + const buffer = Buffer.alloc(11 * 1024 * 1024, 1) + mocks.secureFetchWithPinnedIP.mockReset() + mocks.secureFetchWithPinnedIP + .mockResolvedValueOnce( + Response.json({ name: 'large.xlsx', file: { mimeType: 'application/vnd.ms-excel' } }) + ) + .mockResolvedValueOnce(new Response(buffer)) + + const result = await downloadOneDriveFile({ accessToken: 'token', fileId: 'large-file' }, {}) + + expect(result.files[0]?.buffer.equals(buffer)).toBe(true) + expect(result.files[0]).not.toHaveProperty('data') }) it('uploads plain content without an HTTP route hop and preserves text-file behavior', async () => { diff --git a/apps/sim/lib/internal/onedrive/operations.ts b/apps/sim/lib/internal/onedrive/operations.ts index c28f6002ab6..f1d5db4bf19 100644 --- a/apps/sim/lib/internal/onedrive/operations.ts +++ b/apps/sim/lib/internal/onedrive/operations.ts @@ -17,6 +17,10 @@ import { } from '@/lib/core/utils/stream-limits' import { OneDriveOperationError } from '@/lib/internal/onedrive/errors' import type { OneDriveUploadInput } from '@/lib/internal/onedrive/schema' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' import { getExtensionFromMimeType, @@ -25,7 +29,7 @@ import { import { downloadServableFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' import { assertToolFileAccess } from '@/app/api/files/authorization' -import type { OneDriveDownloadResponse, OneDriveToolParams } from '@/tools/onedrive/types' +import type { OneDriveToolParams } from '@/tools/onedrive/types' import { normalizeExcelValues } from '@/tools/onedrive/utils' const MAX_GRAPH_JSON_BYTES = 2 * 1024 * 1024 @@ -452,7 +456,7 @@ async function graphError(response: SecureFetchResponse, fallback: string, signa export async function downloadOneDriveFile( input: OneDriveDownloadInput, context: OneDriveOperationContext -): Promise { +): Promise { context.signal?.throwIfAborted() const fileId = encodeURIComponent(input.fileId) const metadataResponse = await fetchGraph( @@ -498,15 +502,12 @@ export async function downloadOneDriveFile( label: 'OneDrive file download', signal: context.signal, }) - return { - success: true, - output: { - file: { - name: input.fileName || metadata.name || 'download', - mimeType: metadata.file?.mimeType || 'application/octet-stream', - data: buffer.toString('base64'), - size: buffer.length, - }, + return createInternalToolFileResult( + { + buffer, + name: input.fileName || metadata.name || 'download', + mimeType: metadata.file?.mimeType || 'application/octet-stream', }, - } + (file) => ({ success: true, output: { file } }) + ) } diff --git a/apps/sim/lib/internal/onepassword/execute-tool.ts b/apps/sim/lib/internal/onepassword/execute-tool.ts index 47aec72789c..4b8144709a1 100644 --- a/apps/sim/lib/internal/onepassword/execute-tool.ts +++ b/apps/sim/lib/internal/onepassword/execute-tool.ts @@ -27,10 +27,12 @@ import { executeOnePasswordUpdateItem, type OnePasswordOperationContext, } from '@/lib/internal/onepassword/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { parseInternalToolInput } from '@/lib/internal/tool-operations/parse-input' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' const logger = createLogger('OnePasswordToolExecution') @@ -40,7 +42,7 @@ async function executeOperation( request: InternalToolOperationCall, operation: (input: ContractBody, context: OnePasswordOperationContext) => Promise, failureMessage: string -): Promise { +): Promise { request.signal?.throwIfAborted() const parsed = parseInternalToolInput(contract, request.input) if (!parsed.success) return parsed.response @@ -48,7 +50,7 @@ async function executeOperation( try { const result = await operation(parsed.data, { signal: request.signal }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof OnePasswordOperationError) { @@ -64,7 +66,9 @@ async function executeOperation( } } -export const executeOnePasswordTool: InternalToolOperationHandler = async (request) => { +export const executeOnePasswordTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { switch (request.toolId) { case 'onepassword_list_vaults': return executeOperation( diff --git a/apps/sim/lib/internal/onepassword/operations.test.ts b/apps/sim/lib/internal/onepassword/operations.test.ts index 76d7518f514..aafdf3f4dc9 100644 --- a/apps/sim/lib/internal/onepassword/operations.test.ts +++ b/apps/sim/lib/internal/onepassword/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const clientMocks = vi.hoisted(() => ({ connectItemToSdkItem: vi.fn(), @@ -201,20 +206,65 @@ describe('1Password operations', () => { { signal: controller.signal } ) - expect(result).toEqual({ - file: { + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'secret.txt', mimeType: 'text/plain', - data: Buffer.from('hello').toString('base64'), - size: 5, + buffer: Buffer.from('hello'), }, - }) + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'secret.txt', + type: 'text/plain', + mimeType: 'text/plain', + size: 5, + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ file: storedFile }) expect(clientMocks.connectRequest.mock.calls[1]?.[0]).toMatchObject({ maxResponseBytes: 5, signal: controller.signal, }) }) + it('returns SDK attachment bytes for storage with the actual byte length', async () => { + clientMocks.createOnePasswordClient.mockResolvedValue({ + items: { + get: vi.fn().mockResolvedValue({ id: 'item-1' }), + files: { read: vi.fn().mockResolvedValue(new Uint8Array([1, 2, 3])) }, + }, + }) + clientMocks.findItemFileAttributes.mockReturnValue({ + id: 'file-1', + name: 'secret.bin', + size: 3, + }) + + const result = await executeOnePasswordGetItemFile( + { ...SERVICE_CREDENTIALS, vaultId: 'vault-1', itemId: 'item-1', fileId: 'file-1' }, + {} + ) + + expect(result.files).toEqual([ + { name: 'secret.bin', mimeType: 'application/octet-stream', buffer: Buffer.from([1, 2, 3]) }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'secret.bin', + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + size: 3, + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ file: storedFile }) + }) + it('preserves the private secret value and rejects Connect mode', async () => { const resolve = vi.fn().mockResolvedValue('resolved-secret') clientMocks.createOnePasswordClient.mockResolvedValue({ secrets: { resolve } }) diff --git a/apps/sim/lib/internal/onepassword/operations.ts b/apps/sim/lib/internal/onepassword/operations.ts index 27721610f4b..d33b3ba1cb0 100644 --- a/apps/sim/lib/internal/onepassword/operations.ts +++ b/apps/sim/lib/internal/onepassword/operations.ts @@ -32,6 +32,10 @@ import { applyOnePasswordPatch, type JsonPatchOperation, } from '@/lib/internal/onepassword/json-patch' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' export interface OnePasswordOperationContext { @@ -373,9 +377,7 @@ export async function executeOnePasswordResolveSecret( export async function executeOnePasswordGetItemFile( input: GetItemFileInput, context: OnePasswordOperationContext -): Promise<{ - file: { name: string; mimeType: string; data: string; size: number } -}> { +): Promise { const credentials = resolveCredentials(input) if (credentials.mode === 'service_account') { const client = await createOnePasswordClient(credentials.serviceAccountToken, context.signal) @@ -390,14 +392,10 @@ export async function executeOnePasswordGetItemFile( ) assertKnownSizeWithinLimit(content.byteLength, MAX_FILE_SIZE, '1Password item file') const buffer = Buffer.from(content.buffer, content.byteOffset, content.byteLength) - return { - file: { - name: attributes.name, - mimeType: 'application/octet-stream', - data: buffer.toString('base64'), - size: attributes.size, - }, - } + return createInternalToolFileResult( + { buffer, name: attributes.name, mimeType: 'application/octet-stream' }, + (file) => ({ file }) + ) } const metadataResponse = await connectRequest({ @@ -434,12 +432,12 @@ export async function executeOnePasswordGetItemFile( } const buffer = Buffer.from(await contentResponse.arrayBuffer()) context.signal?.throwIfAborted() - return { - file: { + return createInternalToolFileResult( + { + buffer, name: typeof metadata.name === 'string' ? metadata.name : 'attachment', mimeType: contentResponse.headers.get('content-type') || 'application/octet-stream', - data: buffer.toString('base64'), - size: typeof metadata.size === 'number' ? metadata.size : buffer.length, }, - } + (file) => ({ file }) + ) } diff --git a/apps/sim/lib/internal/outlook/client.test.ts b/apps/sim/lib/internal/outlook/client.test.ts index 1954a636768..c085bca9b04 100644 --- a/apps/sim/lib/internal/outlook/client.test.ts +++ b/apps/sim/lib/internal/outlook/client.test.ts @@ -2,7 +2,7 @@ * @vitest-environment node */ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { DEFAULT_MAX_ERROR_BODY_BYTES, PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { OutlookClient } from '@/lib/internal/outlook/client' import { OutlookOperationError } from '@/lib/internal/outlook/errors' @@ -86,6 +86,94 @@ describe('OutlookClient', () => { ) }) + it('reads raw attachments larger than 10 MiB with OAuth and cancellation', async () => { + const bytes = Buffer.alloc(12 * 1024 * 1024, 4) + fetchMock.mockResolvedValue( + new Response(bytes, { headers: { 'content-type': 'application/pdf' } }) + ) + const controller = new AbortController() + + const result = await new OutlookClient('access-token').buffer( + '/me/messages/message-1/attachments/file-1/$value', + 100 * 1024 * 1024, + 'Failed to download attachment', + controller.signal + ) + + expect(result.buffer.equals(bytes)).toBe(true) + expect(result.contentType).toBe('application/pdf') + expect(fetchMock).toHaveBeenCalledWith( + 'https://graph.microsoft.com/v1.0/me/messages/message-1/attachments/file-1/$value', + { + method: 'GET', + headers: { Authorization: 'Bearer access-token' }, + signal: controller.signal, + } + ) + }) + + it('accepts a zero-byte attachment body', async () => { + fetchMock.mockResolvedValue(new Response(new Uint8Array(0))) + + const result = await new OutlookClient('access-token').buffer( + '/attachment/$value', + 1024, + 'Failed' + ) + + expect(result.buffer.byteLength).toBe(0) + }) + + it.each(['declared', 'actual'])('enforces the %s raw-body limit', async (sizeSource) => { + fetchMock.mockResolvedValue( + new Response(new Uint8Array(1025), { + headers: { 'content-length': sizeSource === 'declared' ? '1025' : '1' }, + }) + ) + + await expect( + new OutlookClient('access-token').buffer('/attachment/$value', 1024, 'Failed') + ).rejects.toBeInstanceOf(PayloadSizeLimitError) + }) + + it('preserves raw-download Graph error messages and status', async () => { + fetchMock.mockResolvedValue( + Response.json({ error: { message: 'Access denied' } }, { status: 403 }) + ) + + await expect( + new OutlookClient('access-token').buffer('/attachment/$value', 1024, 'Failed to download') + ).rejects.toEqual(new OutlookOperationError('Access denied', 403)) + }) + + it('bounds raw-download error bodies while preserving provider status', async () => { + fetchMock.mockResolvedValue( + new Response('bad gateway', { + status: 502, + headers: { 'content-length': String(DEFAULT_MAX_ERROR_BODY_BYTES + 1) }, + }) + ) + + await expect( + new OutlookClient('access-token').buffer('/attachment/$value', 1024, 'Failed to download') + ).rejects.toEqual(new OutlookOperationError('Failed to download', 502)) + }) + + it('rejects cancelled raw downloads before fetch', async () => { + const controller = new AbortController() + controller.abort(new DOMException('cancelled', 'AbortError')) + + await expect( + new OutlookClient('access-token').buffer( + '/attachment/$value', + 1024, + 'Failed', + controller.signal + ) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(fetchMock).not.toHaveBeenCalled() + }) + it('stops before provider work when already cancelled', async () => { const controller = new AbortController() controller.abort(new DOMException('cancelled', 'AbortError')) diff --git a/apps/sim/lib/internal/outlook/client.ts b/apps/sim/lib/internal/outlook/client.ts index 7c4e70035bb..2b7a2ac7e2b 100644 --- a/apps/sim/lib/internal/outlook/client.ts +++ b/apps/sim/lib/internal/outlook/client.ts @@ -1,5 +1,9 @@ import { getErrorMessage } from '@sim/utils/errors' -import { readResponseTextWithLimit } from '@/lib/core/utils/stream-limits' +import { + DEFAULT_MAX_ERROR_BODY_BYTES, + readResponseTextWithLimit, + readResponseToBufferWithLimit, +} from '@/lib/core/utils/stream-limits' import { OutlookOperationError } from '@/lib/internal/outlook/errors' const MICROSOFT_GRAPH_BASE_URL = 'https://graph.microsoft.com/v1.0' @@ -98,4 +102,40 @@ export class OutlookClient { await response.body?.cancel() signal?.throwIfAborted() } + + async buffer( + path: string, + maxBytes: number, + fallbackError: string, + signal?: AbortSignal + ): Promise<{ buffer: Buffer; contentType: string | null }> { + signal?.throwIfAborted() + const response = await fetch(this.url(path), { + method: 'GET', + headers: { Authorization: `Bearer ${this.accessToken}` }, + signal, + }) + if (!response.ok) { + let data: OutlookJsonObject = {} + try { + data = parseJson( + await readResponseTextWithLimit(response, { + maxBytes: DEFAULT_MAX_ERROR_BODY_BYTES, + label: 'Microsoft Graph error response', + signal, + }) + ) + } catch { + signal?.throwIfAborted() + } + throw new OutlookOperationError(graphErrorMessage(data, fallbackError), response.status) + } + const buffer = await readResponseToBufferWithLimit(response, { + maxBytes, + label: 'Outlook attachment', + signal, + }) + signal?.throwIfAborted() + return { buffer, contentType: response.headers.get('content-type') } + } } diff --git a/apps/sim/lib/internal/outlook/execute-tool.test.ts b/apps/sim/lib/internal/outlook/execute-tool.test.ts index 63a032b41b4..cdeb57e9030 100644 --- a/apps/sim/lib/internal/outlook/execute-tool.test.ts +++ b/apps/sim/lib/internal/outlook/execute-tool.test.ts @@ -8,6 +8,7 @@ const operationMocks = vi.hoisted(() => ({ executeOutlookCopy: vi.fn(), executeOutlookDelete: vi.fn(), executeOutlookDraft: vi.fn(), + executeOutlookGetAttachment: vi.fn(), executeOutlookMarkRead: vi.fn(), executeOutlookMarkUnread: vi.fn(), executeOutlookMove: vi.fn(), @@ -18,9 +19,16 @@ vi.mock('@/lib/internal/outlook/operations', () => operationMocks) import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' import { OutlookOperationError } from '@/lib/internal/outlook/errors' -import { executeOutlookTool } from '@/lib/internal/outlook/execute-tool' +import { executeOutlookTool as executeOutlookToolOperation } from '@/lib/internal/outlook/execute-tool' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' +async function executeOutlookTool(request: InternalToolOperationCall): Promise { + const result = await executeOutlookToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + const MESSAGE_BODY = { accessToken: 'access-token', messageId: 'message-1' } const COPY_MOVE_BODY = { ...MESSAGE_BODY, destinationId: 'folder-1' } const MAIL_BODY = { @@ -51,6 +59,12 @@ const TOOL_CASES = [ ['outlook_copy', COPY_MOVE_BODY, operationMocks.executeOutlookCopy, 'provider'], ['outlook_delete', MESSAGE_BODY, operationMocks.executeOutlookDelete, 'provider'], ['outlook_draft', MAIL_BODY, operationMocks.executeOutlookDraft, 'mail'], + [ + 'outlook_get_attachment', + { ...MESSAGE_BODY, attachmentId: 'attachment-1' }, + operationMocks.executeOutlookGetAttachment, + 'provider', + ], ['outlook_mark_read', MESSAGE_BODY, operationMocks.executeOutlookMarkRead, 'provider'], ['outlook_mark_unread', MESSAGE_BODY, operationMocks.executeOutlookMarkUnread, 'provider'], ['outlook_move', COPY_MOVE_BODY, operationMocks.executeOutlookMove, 'provider'], @@ -86,6 +100,55 @@ describe('executeOutlookTool', () => { } ) + it('passes large attachment bytes to the central presenter without serializing them', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024) + const fileResult = createInternalToolFileResult( + { buffer, name: 'report.xlsx', mimeType: 'application/octet-stream' }, + (file) => ({ success: true, output: { attachments: [file] } }) + ) + operationMocks.executeOutlookGetAttachment.mockResolvedValue(fileResult) + + const result = await executeOutlookToolOperation( + createRequest({ + toolId: 'outlook_get_attachment', + input: { ...MESSAGE_BODY, attachmentId: 'attachment-1' }, + }) + ) + + expect(result).toBe(fileResult) + }) + + it('rejects invalid attachment IDs before provider work', async () => { + const response = await executeOutlookTool( + createRequest({ + toolId: 'outlook_get_attachment', + input: { ...MESSAGE_BODY, attachmentId: ' ' }, + }) + ) + + expect(response.status).toBe(400) + expect(operationMocks.executeOutlookGetAttachment).not.toHaveBeenCalled() + }) + + it('preserves attachment size errors as 413 responses', async () => { + operationMocks.executeOutlookGetAttachment.mockRejectedValue( + new OutlookOperationError('Outlook attachment exceeds the size limit', 413) + ) + + const response = await executeOutlookTool( + createRequest({ + toolId: 'outlook_get_attachment', + input: { ...MESSAGE_BODY, attachmentId: 'attachment-1' }, + }) + ) + + expect(response.status).toBe(413) + await expect(response.json()).resolves.toEqual({ + success: false, + error: 'Outlook attachment exceeds the size limit', + }) + }) + it('returns the canonical validation envelope before provider work', async () => { const response = await executeOutlookTool( createRequest({ input: { accessToken: '', messageId: '' } }) diff --git a/apps/sim/lib/internal/outlook/execute-tool.ts b/apps/sim/lib/internal/outlook/execute-tool.ts index b209433362c..6c219f184e4 100644 --- a/apps/sim/lib/internal/outlook/execute-tool.ts +++ b/apps/sim/lib/internal/outlook/execute-tool.ts @@ -11,34 +11,48 @@ import { } from '@/lib/api/contracts/tools/microsoft' import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' import { OutlookOperationError } from '@/lib/internal/outlook/errors' +import { outlookGetAttachmentInputSchema } from '@/lib/internal/outlook/get-attachment-input' import { executeOutlookCopy, executeOutlookDelete, executeOutlookDraft, + executeOutlookGetAttachment, executeOutlookMarkRead, executeOutlookMarkUnread, executeOutlookMove, executeOutlookSend, type OutlookMailOperationContext, } from '@/lib/internal/outlook/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import { parseInternalOperationInput } from '@/lib/internal/tool-operations/parse-contract-input' import { parseInternalToolInput } from '@/lib/internal/tool-operations/parse-input' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' async function executeOperation( contract: C, input: unknown, execute: (input: ContractBody) => Promise, signal?: AbortSignal -): Promise { +): Promise { signal?.throwIfAborted() const parsed = parseInternalToolInput(contract, input, { maxInputBytes: DEFAULT_MAX_JSON_BODY_BYTES, }) if (!parsed.success) return parsed.response + return executeAndPresent(() => execute(parsed.data), signal) +} + +async function executeAndPresent( + execute: () => Promise, + signal?: AbortSignal +): Promise { try { - const result = await execute(parsed.data) + const result = await execute() signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { signal?.throwIfAborted() if (error instanceof OutlookOperationError) { @@ -51,14 +65,24 @@ async function executeOperation( } } -export const executeOutlookTool: InternalToolOperationHandler = async (request) => { +export const executeOutlookTool: InternalToolOperationHandler = async ( + request +) => { const { input, context, requestId, signal, toolId } = request + signal?.throwIfAborted() const mailContext: OutlookMailOperationContext = { requestId, signal, userId: context.userId, } switch (toolId) { + case 'outlook_get_attachment': { + const parsed = parseInternalOperationInput({ body: outlookGetAttachmentInputSchema }, input, { + maxInputBytes: DEFAULT_MAX_JSON_BODY_BYTES, + }) + if (!parsed.success) return parsed.response + return executeAndPresent(() => executeOutlookGetAttachment(parsed.data.body, signal), signal) + } case 'outlook_copy': return executeOperation( outlookCopyContract, diff --git a/apps/sim/lib/internal/outlook/get-attachment-input.ts b/apps/sim/lib/internal/outlook/get-attachment-input.ts new file mode 100644 index 00000000000..8e423a4a6d7 --- /dev/null +++ b/apps/sim/lib/internal/outlook/get-attachment-input.ts @@ -0,0 +1,10 @@ +import { z } from 'zod' +import { accessTokenSchema, messageIdSchema } from '@/lib/api/contracts/tools/microsoft' + +export const outlookGetAttachmentInputSchema = z.object({ + accessToken: accessTokenSchema, + messageId: messageIdSchema.trim().min(1, 'Message ID is required'), + attachmentId: z.string().trim().min(1, 'Attachment ID is required'), +}) + +export type OutlookGetAttachmentInput = z.infer diff --git a/apps/sim/lib/internal/outlook/operations.test.ts b/apps/sim/lib/internal/outlook/operations.test.ts index 2587dc1acca..e5d39b3fbc8 100644 --- a/apps/sim/lib/internal/outlook/operations.test.ts +++ b/apps/sim/lib/internal/outlook/operations.test.ts @@ -7,6 +7,7 @@ const mocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), downloadServableFilesWithinBudget: vi.fn(), empty: vi.fn(), + buffer: vi.fn(), json: vi.fn(), processFilesToUserFiles: vi.fn(), })) @@ -17,6 +18,10 @@ vi.mock('@/lib/internal/outlook/client', () => ({ return mocks.json(...args) } + buffer(...args: unknown[]) { + return mocks.buffer(...args) + } + empty(...args: unknown[]) { return mocks.empty(...args) } @@ -38,11 +43,17 @@ import { executeOutlookCopy, executeOutlookDelete, executeOutlookDraft, + executeOutlookGetAttachment, executeOutlookMarkRead, executeOutlookMarkUnread, executeOutlookMove, executeOutlookSend, } from '@/lib/internal/outlook/operations' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' const MAIL_INPUT = { accessToken: 'access-token', @@ -81,6 +92,7 @@ describe('Outlook operations', () => { { buffer: Buffer.from('report'), contentType: 'application/pdf' }, ]) mocks.empty.mockResolvedValue(undefined) + mocks.buffer.mockResolvedValue({ buffer: Buffer.alloc(0), contentType: null }) mocks.json.mockResolvedValue({}) mocks.processFilesToUserFiles.mockReturnValue([]) }) @@ -333,3 +345,182 @@ describe('Outlook operations', () => { expect(mocks.assertToolFileAccess).not.toHaveBeenCalled() }) }) + +const ATTACHMENT_INPUT = { + accessToken: 'access-token', + messageId: ' message/1 ', + attachmentId: ' attachment/1 ', +} + +const ATTACHMENT_METADATA = { + '@odata.type': '#microsoft.graph.fileAttachment', + id: 'attachment/1', + name: 'report.xlsx', + contentType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', + size: 12 * 1024 * 1024, + isInline: false, + lastModifiedDateTime: '2026-09-11T10:00:00Z', +} + +describe('Outlook attachment downloads', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.json.mockResolvedValue(ATTACHMENT_METADATA) + mocks.buffer.mockResolvedValue({ buffer: Buffer.alloc(0), contentType: null }) + }) + + it('fetches metadata separately and presents a large file as a stored reference', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024, 1) + const controller = new AbortController() + mocks.buffer.mockResolvedValue({ buffer, contentType: 'application/octet-stream' }) + + const result = await executeOutlookGetAttachment(ATTACHMENT_INPUT, controller.signal) + + expect(mocks.json).toHaveBeenCalledWith( + '/me/messages/message%2F1/attachments/attachment%2F1?$select=id,name,contentType,size,isInline,lastModifiedDateTime', + { method: 'GET' }, + 'Failed to retrieve attachment', + controller.signal + ) + expect(mocks.buffer).toHaveBeenCalledWith( + '/me/messages/message%2F1/attachments/attachment%2F1/$value', + MAX_BUFFERED_TRANSFER_BYTES, + 'Failed to download attachment', + controller.signal + ) + if (!isInternalToolFileResult(result)) throw new Error('Expected a file result') + expect(result.files).toHaveLength(1) + expect(result.files[0]?.buffer).toBe(buffer) + expect(result.files[0]?.name).toBe(ATTACHMENT_METADATA.name) + expect(result.files[0]?.mimeType).toBe(ATTACHMENT_METADATA.contentType) + const stored: StoredToolFile = { + id: 'stored-1', + key: 'execution/stored-1', + name: ATTACHMENT_METADATA.name, + size: buffer.byteLength, + type: ATTACHMENT_METADATA.contentType, + mimeType: ATTACHMENT_METADATA.contentType, + url: '/api/files/serve/stored-1', + } + const body = result.present([stored]) + expect(body).toEqual({ + success: true, + output: { + message: 'Successfully retrieved attachment "report.xlsx".', + results: { + id: 'attachment/1', + name: ATTACHMENT_METADATA.name, + contentType: ATTACHMENT_METADATA.contentType, + size: buffer.byteLength, + isInline: false, + attachmentType: '#microsoft.graph.fileAttachment', + lastModifiedDateTime: ATTACHMENT_METADATA.lastModifiedDateTime, + }, + attachments: [stored], + }, + }) + expect(Buffer.byteLength(JSON.stringify(body))).toBeLessThan(2000) + expect(JSON.stringify(body)).not.toContain('contentBytes') + }) + + it('preserves zero-byte file attachments', async () => { + mocks.json.mockResolvedValue({ ...ATTACHMENT_METADATA, size: 0 }) + + const result = await executeOutlookGetAttachment(ATTACHMENT_INPUT) + + if (!isInternalToolFileResult(result)) throw new Error('Expected a file result') + expect(result.files[0]?.buffer.byteLength).toBe(0) + expect(mocks.buffer).toHaveBeenCalledOnce() + }) + + it.each(['#microsoft.graph.itemAttachment', '#microsoft.graph.referenceAttachment'])( + 'preserves %s metadata without trying to download raw content', + async (attachmentType) => { + mocks.json.mockResolvedValue({ ...ATTACHMENT_METADATA, '@odata.type': attachmentType }) + + const result = await executeOutlookGetAttachment(ATTACHMENT_INPUT) + + expect(result).toMatchObject({ + success: true, + output: { + results: { attachmentType, name: 'report.xlsx' }, + attachments: [], + }, + }) + expect(mocks.buffer).not.toHaveBeenCalled() + } + ) + + it('rejects metadata above 100 MiB before fetching file bytes', async () => { + mocks.json.mockResolvedValue({ + ...ATTACHMENT_METADATA, + size: MAX_BUFFERED_TRANSFER_BYTES + 1, + }) + + await expect(executeOutlookGetAttachment(ATTACHMENT_INPUT)).rejects.toMatchObject({ + status: 413, + }) + expect(mocks.buffer).not.toHaveBeenCalled() + }) + + it('accepts the exact metadata size limit and bounds the raw body independently', async () => { + mocks.json.mockResolvedValue({ ...ATTACHMENT_METADATA, size: MAX_BUFFERED_TRANSFER_BYTES }) + mocks.buffer.mockRejectedValue( + new PayloadSizeLimitError({ + label: 'Outlook attachment', + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + observedBytes: MAX_BUFFERED_TRANSFER_BYTES + 1, + }) + ) + + await expect(executeOutlookGetAttachment(ATTACHMENT_INPUT)).rejects.toMatchObject({ + status: 413, + }) + expect(mocks.buffer).toHaveBeenCalledOnce() + }) + + it.each(['metadata', 'download'])('preserves Graph %s errors and HTTP status', async (phase) => { + const error = new OutlookOperationError('Attachment not found', 404) + if (phase === 'metadata') mocks.json.mockRejectedValueOnce(error) + else mocks.buffer.mockRejectedValueOnce(error) + + await expect(executeOutlookGetAttachment(ATTACHMENT_INPUT)).rejects.toBe(error) + if (phase === 'metadata') expect(mocks.buffer).not.toHaveBeenCalled() + }) + + it('uses response MIME and a fallback filename when metadata omits them', async () => { + mocks.json.mockResolvedValue({ '@odata.type': '#microsoft.graph.fileAttachment' }) + mocks.buffer.mockResolvedValue({ buffer: Buffer.from('text'), contentType: 'text/plain' }) + + const result = await executeOutlookGetAttachment(ATTACHMENT_INPUT) + + if (!isInternalToolFileResult(result)) throw new Error('Expected a file result') + expect(result.files[0]?.name).toBe('attachment') + expect(result.files[0]?.mimeType).toBe('text/plain') + }) + + it('stops between metadata and raw downloads when cancelled', async () => { + const controller = new AbortController() + mocks.json.mockImplementationOnce(async () => { + controller.abort(new DOMException('cancelled', 'AbortError')) + return ATTACHMENT_METADATA + }) + + await expect( + executeOutlookGetAttachment(ATTACHMENT_INPUT, controller.signal) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(mocks.buffer).not.toHaveBeenCalled() + }) + + it('does not produce a file result if cancellation occurs during download', async () => { + const controller = new AbortController() + mocks.buffer.mockImplementationOnce(async () => { + controller.abort(new DOMException('cancelled', 'AbortError')) + return { buffer: Buffer.alloc(0), contentType: null } + }) + + await expect( + executeOutlookGetAttachment(ATTACHMENT_INPUT, controller.signal) + ).rejects.toMatchObject({ name: 'AbortError' }) + }) +}) diff --git a/apps/sim/lib/internal/outlook/operations.ts b/apps/sim/lib/internal/outlook/operations.ts index 1f6594c1f0a..972e1c4cb33 100644 --- a/apps/sim/lib/internal/outlook/operations.ts +++ b/apps/sim/lib/internal/outlook/operations.ts @@ -9,17 +9,22 @@ import type { OutlookMoveBody, OutlookSendBody, } from '@/lib/api/contracts/tools/microsoft' -import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { assertKnownSizeWithinLimit, isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { OutlookClient, type OutlookJsonObject } from '@/lib/internal/outlook/client' import { OutlookOperationError } from '@/lib/internal/outlook/errors' +import type { OutlookGetAttachmentInput } from '@/lib/internal/outlook/get-attachment-input' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { downloadServableFilesWithinBudget } from '@/lib/uploads/utils/file-utils.server' import { assertToolFileAccess } from '@/app/api/files/authorization' +import type { CleanedOutlookAttachmentMetadata } from '@/tools/outlook/types' const logger = createLogger('OutlookOperations') const OUTLOOK_SEND_ATTACHMENT_MAX_BYTES = 3 * 1024 * 1024 const OUTLOOK_DRAFT_ATTACHMENT_MAX_BYTES = 4 * 1024 * 1024 +const OUTLOOK_ATTACHMENT_METADATA_FIELDS = 'id,name,contentType,size,isInline,lastModifiedDateTime' interface OutlookMailOperationContext { requestId: string @@ -148,6 +153,62 @@ async function buildMessage( return message } +export async function executeOutlookGetAttachment( + input: OutlookGetAttachmentInput, + signal?: AbortSignal +) { + signal?.throwIfAborted() + const client = new OutlookClient(input.accessToken) + const path = `/me/messages/${encodeURIComponent(input.messageId.trim())}/attachments/${encodeURIComponent(input.attachmentId.trim())}` + try { + const data = await client.json( + `${path}?$select=${OUTLOOK_ATTACHMENT_METADATA_FIELDS}`, + { method: 'GET' }, + 'Failed to retrieve attachment', + signal + ) + signal?.throwIfAborted() + const results: CleanedOutlookAttachmentMetadata = { + id: optionalString(data, 'id') ?? input.attachmentId.trim(), + name: optionalString(data, 'name') ?? null, + contentType: optionalString(data, 'contentType') ?? null, + size: typeof data.size === 'number' ? data.size : null, + isInline: optionalBoolean(data, 'isInline') ?? null, + attachmentType: optionalString(data, '@odata.type') ?? null, + lastModifiedDateTime: optionalString(data, 'lastModifiedDateTime') ?? null, + } + const output = { + message: `Successfully retrieved attachment "${results.name ?? ''}".`, + results, + } + if (results.attachmentType !== '#microsoft.graph.fileAttachment') { + return { success: true, output: { ...output, attachments: [] } } + } + if (results.size !== null && results.size !== undefined) { + assertKnownSizeWithinLimit(results.size, MAX_BUFFERED_TRANSFER_BYTES, 'Outlook attachment') + } + const { buffer, contentType } = await client.buffer( + `${path}/$value`, + MAX_BUFFERED_TRANSFER_BYTES, + 'Failed to download attachment', + signal + ) + signal?.throwIfAborted() + return createInternalToolFileResult( + { + buffer, + name: results.name || 'attachment', + mimeType: results.contentType || contentType || 'application/octet-stream', + }, + (file) => ({ success: true, output: { ...output, attachments: [file] } }) + ) + } catch (error) { + signal?.throwIfAborted() + if (isPayloadSizeLimitError(error)) throw new OutlookOperationError(error.message, 413) + throw error + } +} + export async function executeOutlookCopy(input: OutlookCopyBody, signal?: AbortSignal) { const client = new OutlookClient(input.accessToken) const data = await client.json( diff --git a/apps/sim/lib/internal/pipedrive/execute-tool.ts b/apps/sim/lib/internal/pipedrive/execute-tool.ts index 78394ababff..ba5baf9db86 100644 --- a/apps/sim/lib/internal/pipedrive/execute-tool.ts +++ b/apps/sim/lib/internal/pipedrive/execute-tool.ts @@ -5,7 +5,11 @@ import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' import { PipedriveOperationError } from '@/lib/internal/pipedrive/errors' import { executePipedriveGetFiles } from '@/lib/internal/pipedrive/operations' import { pipedriveGetFilesInputSchema } from '@/lib/internal/pipedrive/schema' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' const logger = createLogger('PipedriveToolExecution') @@ -26,7 +30,9 @@ function inputSizeError(input: unknown): Response | null { : null } -export const executePipedriveTool: InternalToolOperationHandler = async (request) => { +export const executePipedriveTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (request.toolId !== 'pipedrive_get_files') { return Response.json( @@ -52,7 +58,7 @@ export const executePipedriveTool: InternalToolOperationHandler = async (request signal: request.signal, }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof PipedriveOperationError) { diff --git a/apps/sim/lib/internal/pipedrive/operations.test.ts b/apps/sim/lib/internal/pipedrive/operations.test.ts new file mode 100644 index 00000000000..5b686ca2863 --- /dev/null +++ b/apps/sim/lib/internal/pipedrive/operations.test.ts @@ -0,0 +1,84 @@ +/** + * @vitest-environment node + */ +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + downloadPipedriveFile: vi.fn(), + listPipedriveFiles: vi.fn(), +})) + +vi.mock('@/lib/internal/pipedrive/client', () => mocks) + +import { executePipedriveGetFiles } from '@/lib/internal/pipedrive/operations' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' + +describe('executePipedriveGetFiles', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.listPipedriveFiles.mockResolvedValue({ + files: [{ id: 1, name: 'report.pdf', url: 'https://files.example/report.pdf' }], + hasMore: true, + nextStart: 1, + }) + }) + + it('keeps large downloads out of JSON while preserving file-list pagination', async () => { + const buffer = Buffer.alloc(11 * 1024 * 1024, 1) + mocks.downloadPipedriveFile.mockResolvedValue({ buffer, contentType: 'application/pdf' }) + const controller = new AbortController() + const input = { accessToken: 'token', downloadFiles: true } + const result = await executePipedriveGetFiles(input, { + requestId: 'request-1', + signal: controller.signal, + }) + + assert(isInternalToolFileResult(result)) + expect(result.files).toHaveLength(1) + expect(result.files[0]?.buffer).toBe(buffer) + expect(result.files[0]?.name).toBe('report.pdf') + expect(result.files[0]?.mimeType).toBe('application/pdf') + expect(mocks.downloadPipedriveFile).toHaveBeenCalledWith( + 'https://files.example/report.pdf', + input, + MAX_BUFFERED_TRANSFER_BYTES, + controller.signal + ) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'report.pdf', + type: 'application/pdf', + mimeType: 'application/pdf', + size: buffer.length, + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ + success: true, + output: { + files: [{ id: 1, name: 'report.pdf', url: 'https://files.example/report.pdf' }], + downloadedFiles: [storedFile], + total_items: 1, + has_more: true, + next_start: 1, + success: true, + }, + }) + }) + + it('returns metadata without file persistence when downloads are disabled', async () => { + const result = await executePipedriveGetFiles( + { accessToken: 'token', downloadFiles: false }, + { requestId: 'request-1' } + ) + + expect(isInternalToolFileResult(result)).toBe(false) + expect(result).toMatchObject({ success: true, output: { has_more: true, next_start: 1 } }) + expect(mocks.downloadPipedriveFile).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/internal/pipedrive/operations.ts b/apps/sim/lib/internal/pipedrive/operations.ts index 0418852eeca..068def9a162 100644 --- a/apps/sim/lib/internal/pipedrive/operations.ts +++ b/apps/sim/lib/internal/pipedrive/operations.ts @@ -1,6 +1,10 @@ import { createLogger } from '@sim/logger' import { downloadPipedriveFile, listPipedriveFiles } from '@/lib/internal/pipedrive/client' import type { PipedriveGetFilesInput } from '@/lib/internal/pipedrive/schema' +import { + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' @@ -17,12 +21,7 @@ export async function executePipedriveGetFiles( ) { context.signal?.throwIfAborted() const page = await listPipedriveFiles(input, context.signal) - const downloadedFiles: Array<{ - data: string - mimeType: string - name: string - size: number - }> = [] + const downloadedFiles: InternalToolFile[] = [] let downloadedBytes = 0 if (input.downloadFiles) { @@ -43,8 +42,7 @@ export async function executePipedriveGetFiles( downloadedFiles.push({ name, mimeType: downloaded.contentType || getMimeTypeFromExtension(extension), - data: downloaded.buffer.toString('base64'), - size: downloaded.buffer.length, + buffer: downloaded.buffer, }) } catch (error) { context.signal?.throwIfAborted() @@ -56,15 +54,16 @@ export async function executePipedriveGetFiles( } } context.signal?.throwIfAborted() - return { + const output = { + files: page.files, + total_items: page.files.length, + has_more: page.hasMore, + next_start: page.nextStart, success: true, - output: { - files: page.files, - downloadedFiles: downloadedFiles.length > 0 ? downloadedFiles : undefined, - total_items: page.files.length, - has_more: page.hasMore, - next_start: page.nextStart, - success: true, - }, } + if (downloadedFiles.length === 0) return { success: true, output } + return createInternalToolFilesResult(downloadedFiles, (files) => ({ + success: true, + output: { ...output, downloadedFiles: files }, + })) } diff --git a/apps/sim/lib/internal/quiver/execute-tool.test.ts b/apps/sim/lib/internal/quiver/execute-tool.test.ts index d89b4394be8..83c4e3bb4f8 100644 --- a/apps/sim/lib/internal/quiver/execute-tool.test.ts +++ b/apps/sim/lib/internal/quiver/execute-tool.test.ts @@ -4,6 +4,7 @@ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' import { DEFAULT_MAX_JSON_BODY_BYTES } from '@/lib/api/server/validation' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ executeImage: vi.fn(), @@ -16,7 +17,7 @@ vi.mock('@/lib/internal/quiver/operations', () => ({ })) import { QuiverOperationError } from '@/lib/internal/quiver/errors' -import { executeQuiverTool } from '@/lib/internal/quiver/execute-tool' +import { executeQuiverTool as executeQuiverToolOperation } from '@/lib/internal/quiver/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' function request(overrides: Partial = {}) { @@ -30,6 +31,14 @@ function request(overrides: Partial = {}) { } as InternalToolOperationCall } +async function executeQuiverTool( + request: Parameters[0] +): Promise { + const result = await executeQuiverToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeQuiverTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -47,6 +56,18 @@ describe('executeQuiverTool', () => { mocks.executeImage.mockResolvedValue(result) }) + it('forwards binary file results without serializing them', async () => { + const result = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ file }) + ) + mocks.executeText.mockResolvedValueOnce(result) + expect(await executeQuiverToolOperation(request({ toolId: 'quiver_text_to_svg_v2' }))).toBe( + result + ) + expect(mocks.executeText.mock.calls[0]?.[2]).toBe('v2') + }) + it.each([ ['quiver_text_to_svg', mocks.executeText], ['quiver_image_to_svg', mocks.executeImage], @@ -65,6 +86,28 @@ describe('executeQuiverTool', () => { ) }) + it.each([ + ['quiver_text_to_svg_v2', mocks.executeText], + ['quiver_image_to_svg_v2', mocks.executeImage], + ])('selects the stored file projection for %s', async (toolId, execute) => { + const input = + toolId === 'quiver_image_to_svg_v2' + ? { apiKey: 'secret', model: 'arrow-preview', image: 'https://example.com/image.png' } + : { apiKey: 'secret', model: 'arrow-preview', prompt: 'A compass' } + const result = createInternalToolFileResult( + { buffer: Buffer.from(''), name: 'file.svg', mimeType: 'image/svg+xml' }, + (file) => ({ success: true, output: { file, files: [file] } }) + ) + execute.mockResolvedValueOnce(result) + + expect(await executeQuiverToolOperation(request({ toolId, input }))).toBe(result) + expect(execute).toHaveBeenCalledWith( + expect.objectContaining({ apiKey: 'secret', model: 'arrow-preview' }), + expect.objectContaining({ userId: 'user-1', requestId: 'request-1' }), + 'v2' + ) + }) + it('authenticates before parsing input', async () => { const response = await executeQuiverTool( request({ input: null, context: createExecutionContext({ workflowId: 'workflow-1' }) }) diff --git a/apps/sim/lib/internal/quiver/execute-tool.ts b/apps/sim/lib/internal/quiver/execute-tool.ts index f4e4663e4d1..545023779c0 100644 --- a/apps/sim/lib/internal/quiver/execute-tool.ts +++ b/apps/sim/lib/internal/quiver/execute-tool.ts @@ -14,9 +14,11 @@ import { quiverImageToSvgInputSchema, quiverTextToSvgInputSchema, } from '@/lib/internal/quiver/schema' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' const logger = createLogger('QuiverToolExecution') @@ -44,7 +46,7 @@ async function executeOperation( request: InternalToolOperationCall, schema: z.ZodType, execute: (input: Input, context: QuiverOperationContext) => Promise -): Promise { +): Promise { request.signal?.throwIfAborted() const sizeError = validateInputSize(request.input) if (sizeError) return sizeError @@ -70,7 +72,7 @@ async function executeOperation( userId, }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof QuiverOperationError) { @@ -89,7 +91,9 @@ async function executeOperation( } } -export const executeQuiverTool: InternalToolOperationHandler = async (request) => { +export const executeQuiverTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() if (!request.context.userId) { return Response.json({ success: false, error: 'Unauthorized' }, { status: 401 }) @@ -99,6 +103,14 @@ export const executeQuiverTool: InternalToolOperationHandler = async (request) = return executeOperation(request, quiverTextToSvgInputSchema, executeQuiverTextToSvg) case 'quiver_image_to_svg': return executeOperation(request, quiverImageToSvgInputSchema, executeQuiverImageToSvg) + case 'quiver_text_to_svg_v2': + return executeOperation(request, quiverTextToSvgInputSchema, (input, context) => + executeQuiverTextToSvg(input, context, 'v2') + ) + case 'quiver_image_to_svg_v2': + return executeOperation(request, quiverImageToSvgInputSchema, (input, context) => + executeQuiverImageToSvg(input, context, 'v2') + ) default: return Response.json( { success: false, error: `Unsupported Quiver tool: ${request.toolId}` }, diff --git a/apps/sim/lib/internal/quiver/operations.test.ts b/apps/sim/lib/internal/quiver/operations.test.ts index 65fc8133fda..ca4637f6d05 100644 --- a/apps/sim/lib/internal/quiver/operations.test.ts +++ b/apps/sim/lib/internal/quiver/operations.test.ts @@ -48,6 +48,19 @@ const context = { userId: 'user-1', } +function storedSvg(name: string) { + return { + id: name, + name, + size: 14, + type: 'image/svg+xml', + mimeType: 'image/svg+xml', + url: `/api/files/${name}`, + key: `execution/${name}`, + context: 'execution' as const, + } +} + describe('Quiver operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -78,7 +91,8 @@ describe('Quiver operations', () => { n: 2, temperature: 0.5, }, - { ...context, signal: controller.signal } + { ...context, signal: controller.signal }, + 'v2' ) expect(mocks.assertToolFileAccess).toHaveBeenCalledTimes(2) @@ -110,13 +124,17 @@ describe('Quiver operations', () => { }, controller.signal ) - expect(result.output).toMatchObject({ - file: { name: 'generated-1.svg', mimeType: 'image/svg+xml' }, + expect(result.files).toHaveLength(2) + const storedFiles = [storedSvg('generated-1.svg'), storedSvg('generated-2.svg')] + const presented = result.present(storedFiles) as { output: { files: unknown[] } } + expect(presented.output.files).toBe(storedFiles) + expect(presented.output).toMatchObject({ files: [{ name: 'generated-1.svg' }, { name: 'generated-2.svg' }], - svgContent: 'one', id: 'generation-1', usage: { totalTokens: 9, inputTokens: 4, outputTokens: 5 }, }) + expect(Object.keys(presented.output).sort()).toEqual(['files', 'id', 'usage']) + expect(presented.output).not.toHaveProperty('svgContent') }) it('preserves image URL inputs without reading local files', async () => { @@ -128,7 +146,8 @@ describe('Quiver operations', () => { auto_crop: false, target_size: 512, }, - context + context, + 'v2' ) expect(mocks.assertToolFileAccess).not.toHaveBeenCalled() @@ -143,11 +162,39 @@ describe('Quiver operations', () => { }, undefined ) - expect(result.output.file.name).toBe('vectorized.svg') - expect(result.output.files).toHaveLength(1) - expect(result.output.svgContent).toBe('one') + expect(result.files[0]?.name).toBe('vectorized.svg') + expect(result.files).toHaveLength(1) + const file = storedSvg('vectorized.svg') + const presented = result.present([file]) + expect(presented).toMatchObject({ + output: { files: [file] }, + }) + expect(presented).not.toHaveProperty('output.file') + expect(presented).not.toHaveProperty('output.svgContent') }) + it.each([0, 12 * 1024 * 1024])( + 'returns only stored file references for %i bytes of SVG content', + async (size) => { + const svg = `${'x'.repeat(size)}` + mocks.requestQuiverSvg.mockResolvedValue({ data: [{ svg }] }) + const result = await executeQuiverTextToSvg( + { apiKey: 'secret', model: 'arrow-preview', prompt: 'A map' }, + context, + 'v2' + ) + expect(result.files).toHaveLength(1) + expect(result.files[0]?.buffer.length).toBe(Buffer.byteLength(svg)) + const file = storedSvg('generated.svg') + const presented = result.present([file]) + expect(presented).toEqual({ + success: true, + output: { files: [file], id: null, usage: null }, + }) + expect(JSON.stringify(presented).length).toBeLessThan(1024) + } + ) + it('authorizes stored image inputs and sends their bytes', async () => { await executeQuiverImageToSvg( { apiKey: 'secret', model: 'arrow-preview', image: rawFile }, @@ -174,6 +221,46 @@ describe('Quiver operations', () => { ) }) + it('preserves v1 inline file data and every generated SVG', async () => { + const result = await executeQuiverTextToSvg( + { apiKey: 'secret', model: 'arrow-preview', prompt: 'A compass', n: 2 }, + context + ) + const files = ['one', 'two'].map((name, index) => ({ + name: `generated-${index + 1}.svg`, + mimeType: 'image/svg+xml', + data: Buffer.from(`${name}`).toString('base64'), + size: Buffer.byteLength(`${name}`), + })) + + expect(result).toEqual({ + success: true, + output: { + file: files[0], + files, + svgContent: 'one', + id: 'generation-1', + usage: { totalTokens: 9, inputTokens: 4, outputTokens: 5 }, + }, + }) + }) + + it('preserves v1 vectorization first-file projection and inline markup', async () => { + const result = await executeQuiverImageToSvg( + { apiKey: 'secret', model: 'arrow-preview', image: 'https://images.example.com/source.png' }, + context + ) + expect(result.output.files).toHaveLength(1) + expect(result.output.file).toEqual({ + name: 'vectorized.svg', + mimeType: 'image/svg+xml', + data: Buffer.from('one').toString('base64'), + size: Buffer.byteLength('one'), + }) + expect(result.output.file).toBe(result.output.files[0]) + expect(result.output.svgContent).toBe('one') + }) + it('fails closed on incomplete private model-input provenance', async () => { const headers = new Headers({ [PRIVATE_MODEL_INPUT_PROVENANCE_HEADER]: RESOLVED_SECRET_PROVENANCE_METADATA_V1, diff --git a/apps/sim/lib/internal/quiver/operations.ts b/apps/sim/lib/internal/quiver/operations.ts index 809830b7bb1..33828b10e3f 100644 --- a/apps/sim/lib/internal/quiver/operations.ts +++ b/apps/sim/lib/internal/quiver/operations.ts @@ -4,6 +4,10 @@ import { validateOpaqueModelInputProvenance } from '@/lib/execution/model-input- import { requestQuiverSvg } from '@/lib/internal/quiver/client' import { QuiverOperationError } from '@/lib/internal/quiver/errors' import type { QuiverImageToSvgInput, QuiverTextToSvgInput } from '@/lib/internal/quiver/schema' +import { + createInternalToolFilesResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { isModelSafeWorkspaceFileKey, MODEL_UNSAFE_WORKSPACE_FILE_ERROR_MESSAGE, @@ -13,6 +17,7 @@ import type { RawFileInput } from '@/lib/uploads/utils/file-schemas' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { assertToolFileAccess } from '@/app/api/files/authorization' +import type { QuiverSvgResponse } from '@/tools/quiver/types' const logger = createLogger('QuiverOperations') @@ -23,32 +28,10 @@ export interface QuiverOperationContext { userId: string } -interface QuiverFile { - name: string - mimeType: 'image/svg+xml' - data: string - size: number -} - -interface QuiverUsage { - totalTokens: number - inputTokens: number - outputTokens: number -} - -export interface QuiverSvgOutput { - success: true - output: { - file: QuiverFile - files: QuiverFile[] - svgContent: string - id: string | null - usage: QuiverUsage | null - } -} - type ApiImage = { url: string } | { base64: string } +export type QuiverSvgOutput = QuiverSvgResponse & { success: true } + function fail(message: string, status: number, body?: Record): never { throw new QuiverOperationError(message, status, body) } @@ -155,8 +138,9 @@ async function resolveImage( function projectResult( result: unknown, fileName: (index: number, total: number) => string, + version: 'v1' | 'v2', firstOnly = false -): QuiverSvgOutput { +): QuiverSvgOutput | InternalToolFileResult { const root = record(result) const data = root.data if (!Array.isArray(data) || data.length === 0) { @@ -171,8 +155,7 @@ function projectResult( return { name: fileName(index, projectedData.length), mimeType: 'image/svg+xml' as const, - data: buffer.toString('base64'), - size: buffer.length, + buffer, } }) const usage = isRecordLike(root.usage) @@ -183,22 +166,49 @@ function projectResult( } : null - return { + if (version === 'v1') { + const inlineFiles = files.map(({ buffer, name, mimeType }) => ({ + name, + mimeType, + data: buffer.toString('base64'), + size: buffer.length, + })) + return { + success: true, + output: { + file: inlineFiles[0], + files: inlineFiles, + svgContent: record(data[0]).svg as string, + id: typeof root.id === 'string' ? root.id : null, + usage, + }, + } + } + + return createInternalToolFilesResult(files, (storedFiles) => ({ success: true, output: { - file: files[0], - files, - svgContent: record(data[0]).svg as string, + files: storedFiles, id: typeof root.id === 'string' ? root.id : null, usage, }, - } + })) } -export async function executeQuiverTextToSvg( +export function executeQuiverTextToSvg( input: QuiverTextToSvgInput, context: QuiverOperationContext -): Promise { +): Promise +export function executeQuiverTextToSvg( + input: QuiverTextToSvgInput, + context: QuiverOperationContext, + version: 'v2' +): Promise +export async function executeQuiverTextToSvg( + input: QuiverTextToSvgInput, + context: QuiverOperationContext, + version: 'v1' | 'v2' = 'v1' +): Promise { context.signal?.throwIfAborted() validateProvenance(input, context) const references: ApiImage[] = [] @@ -223,15 +233,27 @@ export async function executeQuiverTextToSvg( const result = await requestQuiverSvg('generations', input.apiKey, body, context.signal) context.signal?.throwIfAborted() - return projectResult(result, (index, total) => - total > 1 ? `generated-${index + 1}.svg` : 'generated.svg' + return projectResult( + result, + (index, total) => (total > 1 ? `generated-${index + 1}.svg` : 'generated.svg'), + version ) } -export async function executeQuiverImageToSvg( +export function executeQuiverImageToSvg( input: QuiverImageToSvgInput, context: QuiverOperationContext -): Promise { +): Promise +export function executeQuiverImageToSvg( + input: QuiverImageToSvgInput, + context: QuiverOperationContext, + version: 'v2' +): Promise +export async function executeQuiverImageToSvg( + input: QuiverImageToSvgInput, + context: QuiverOperationContext, + version: 'v1' | 'v2' = 'v1' +): Promise { context.signal?.throwIfAborted() validateProvenance(input, context) const image = await resolveImage(input.image, context) @@ -245,5 +267,5 @@ export async function executeQuiverImageToSvg( const result = await requestQuiverSvg('vectorizations', input.apiKey, body, context.signal) context.signal?.throwIfAborted() - return projectResult(result, () => 'vectorized.svg', true) + return projectResult(result, () => 'vectorized.svg', version, true) } diff --git a/apps/sim/lib/internal/sftp/execute-tool.test.ts b/apps/sim/lib/internal/sftp/execute-tool.test.ts index 901baba7b09..bf75f3c6194 100644 --- a/apps/sim/lib/internal/sftp/execute-tool.test.ts +++ b/apps/sim/lib/internal/sftp/execute-tool.test.ts @@ -2,6 +2,7 @@ * @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ executeDelete: vi.fn(), @@ -19,9 +20,9 @@ vi.mock('@/lib/internal/sftp/operations', () => ({ executeSftpUpload: mocks.executeUpload, })) -import { executeSftpTool } from '@/lib/internal/sftp/execute-tool' +import { executeSftpTool as executeSftpToolOperation } from '@/lib/internal/sftp/execute-tool' import { sftpDeleteTool } from '@/tools/sftp/delete' -import { sftpDownloadTool } from '@/tools/sftp/download' +import { sftpDownloadTool, sftpDownloadV2Tool } from '@/tools/sftp/download' import { sftpListTool } from '@/tools/sftp/list' import { sftpMkdirTool } from '@/tools/sftp/mkdir' import { sftpUploadTool } from '@/tools/sftp/upload' @@ -34,6 +35,14 @@ const baseInput = { remotePath: '/files', } +async function executeSftpTool( + request: Parameters[0] +): Promise { + const result = await executeSftpToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('SFTP tool execution', () => { beforeEach(() => { vi.clearAllMocks() @@ -42,9 +51,29 @@ describe('SFTP tool execution', () => { } }) + it('forwards binary file results without serializing them', async () => { + const result = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ file }) + ) + mocks.executeDownload.mockResolvedValueOnce(result) + expect( + await executeSftpToolOperation({ + toolId: 'sftp_download_v2', + input: baseInput, + headers: new Headers(), + context: { userId: 'user-1' }, + requestId: 'request-1', + }) + ).toBe(result) + expect(mocks.executeDownload.mock.calls[0][0]).toEqual(baseInput) + expect(mocks.executeDownload.mock.calls[0][2]).toBe('v2') + }) + it.each([ ['sftp_delete', mocks.executeDelete], ['sftp_download', mocks.executeDownload], + ['sftp_download_v2', mocks.executeDownload], ['sftp_list', mocks.executeList], ['sftp_mkdir', mocks.executeMkdir], ['sftp_upload', mocks.executeUpload], @@ -81,6 +110,7 @@ describe('SFTP tool execution', () => { for (const tool of [ sftpDeleteTool, sftpDownloadTool, + sftpDownloadV2Tool, sftpListTool, sftpMkdirTool, sftpUploadTool, diff --git a/apps/sim/lib/internal/sftp/execute-tool.ts b/apps/sim/lib/internal/sftp/execute-tool.ts index 70d5b1d947b..79ed60322dd 100644 --- a/apps/sim/lib/internal/sftp/execute-tool.ts +++ b/apps/sim/lib/internal/sftp/execute-tool.ts @@ -14,6 +14,7 @@ import { import { sftpDeleteInputSchema, sftpDownloadInputSchema, + sftpDownloadV2InputSchema, sftpListInputSchema, sftpMkdirInputSchema, sftpUploadInputSchema, @@ -21,6 +22,7 @@ import { import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' const logger = createLogger('SftpToolExecution') @@ -28,8 +30,11 @@ const logger = createLogger('SftpToolExecution') async function executeParsed( request: InternalToolOperationCall, schema: S, - execute: (input: z.output, context: SftpOperationContext) => Promise -): Promise { + execute: ( + input: z.output, + context: SftpOperationContext + ) => Promise +): Promise { const parsed = schema.safeParse(request.input) if (!parsed.success) { return Response.json( @@ -51,7 +56,9 @@ async function executeParsed( }) } -export const executeSftpTool: InternalToolOperationHandler = async (request) => { +export const executeSftpTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() let serializedInput: string try { @@ -74,6 +81,10 @@ export const executeSftpTool: InternalToolOperationHandler = async (request) => return executeParsed(request, sftpDeleteInputSchema, executeSftpDelete) case 'sftp_download': return executeParsed(request, sftpDownloadInputSchema, executeSftpDownload) + case 'sftp_download_v2': + return executeParsed(request, sftpDownloadV2InputSchema, (input, context) => + executeSftpDownload(input, context, 'v2') + ) case 'sftp_list': return executeParsed(request, sftpListInputSchema, executeSftpList) case 'sftp_mkdir': diff --git a/apps/sim/lib/internal/sftp/operations.test.ts b/apps/sim/lib/internal/sftp/operations.test.ts index eaac26e5b46..3f07be5a173 100644 --- a/apps/sim/lib/internal/sftp/operations.test.ts +++ b/apps/sim/lib/internal/sftp/operations.test.ts @@ -66,6 +66,17 @@ const connectionInput = { } const context = { userId: 'user-1', requestId: 'request-1' } +const storedFile = { + id: 'stored-file', + name: 'file.txt', + size: 5, + type: 'text/plain', + mimeType: 'text/plain', + url: '/api/files/stored', + key: 'execution/file.txt', + context: 'execution', +} as const + describe('SFTP operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -128,11 +139,74 @@ describe('SFTP operations', () => { context ) + if (!(response instanceof Response)) throw new Error('Expected a JSON response') expect(response.status).toBe(413) expect(mocks.readFile).not.toHaveBeenCalled() expect(mocks.clientEnd).toHaveBeenCalledOnce() }) + it.each([5, 12 * 1024 * 1024])( + 'returns %i bytes through a stored file without inline content in v2', + async (size) => { + const buffer = Buffer.alloc(size, 1) + const sftp = { + stat: vi.fn((_path, callback) => callback(null, { size: buffer.length })), + } as unknown as SFTPWrapper + mocks.getSftp.mockResolvedValue(sftp) + mocks.readFile.mockResolvedValue(buffer) + const result = await executeSftpDownload( + { ...connectionInput, remotePath: '/file.txt' }, + context, + 'v2' + ) + if (result instanceof Response) throw new Error('Expected a file output') + expect(result.files[0]?.buffer).toBe(buffer) + expect(result.files[0]?.name).toBe('file.txt') + const file = { ...storedFile, size: buffer.length } + const presented = result.present([file]) + expect(presented).toEqual({ file }) + expect(JSON.stringify(presented)).not.toContain('"content"') + expect(JSON.stringify(presented)).not.toContain('"encoding"') + expect(mocks.clientEnd).toHaveBeenCalledOnce() + expect(mocks.readFile).toHaveBeenCalledWith( + sftp, + '/file.txt', + 50 * 1024 * 1024, + 'SFTP download', + undefined + ) + } + ) + + it.each(['base64', 'utf-8'] as const)( + 'preserves the complete v1 %s response', + async (encoding) => { + const buffer = Buffer.from('hello') + mocks.getSftp.mockResolvedValue({ + stat: vi.fn((_path, callback) => callback(null, { size: buffer.length })), + }) + mocks.readFile.mockResolvedValue(buffer) + const result = await executeSftpDownload( + { ...connectionInput, remotePath: '/file.txt', encoding }, + context + ) + expect(await result.json()).toEqual({ + success: true, + fileName: 'file.txt', + file: { + name: 'file.txt', + mimeType: 'text/plain', + data: buffer.toString('base64'), + size: 5, + }, + content: buffer.toString(encoding), + size: 5, + encoding, + message: 'Successfully downloaded file.txt', + }) + } + ) + it('authorizes every referenced Sim file before reading or uploading it', async () => { const denied = Response.json({ success: false, error: 'File not found' }, { status: 404 }) const file = { key: 'workspace/file', name: 'private.txt', size: 4 } diff --git a/apps/sim/lib/internal/sftp/operations.ts b/apps/sim/lib/internal/sftp/operations.ts index dc82c211ba4..9949f73f755 100644 --- a/apps/sim/lib/internal/sftp/operations.ts +++ b/apps/sim/lib/internal/sftp/operations.ts @@ -23,6 +23,8 @@ import type { SftpMkdirInput, SftpUploadInput, } from '@/lib/internal/sftp/schema' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { InternalToolOperationResult } from '@/lib/internal/tool-operations/types' import { getFileExtension, getMimeTypeFromExtension, @@ -335,10 +337,20 @@ export async function executeSftpList( } } -export async function executeSftpDownload( +export function executeSftpDownload( input: SftpDownloadInput, context: SftpOperationContext -): Promise { +): Promise +export function executeSftpDownload( + input: Omit, + context: SftpOperationContext, + version: 'v2' +): Promise +export async function executeSftpDownload( + input: Omit & { encoding?: SftpDownloadInput['encoding'] }, + context: SftpOperationContext, + version: 'v1' | 'v2' = 'v1' +): Promise { if (!isPathSafe(input.remotePath)) return unsafePathResponse() try { return await withSftp(input, context, async (sftp) => { @@ -370,20 +382,25 @@ export async function executeSftpDownload( const fileName = path.basename(remotePath) const extension = getFileExtension(fileName) const mimeType = getMimeTypeFromExtension(extension) - return Response.json({ - success: true, - fileName, - file: { - name: fileName, - mimeType, - data: buffer.toString('base64'), + if (version === 'v1') { + return Response.json({ + success: true, + fileName, + file: { + name: fileName, + mimeType, + data: buffer.toString('base64'), + size: buffer.length, + }, + content: buffer.toString(input.encoding === 'base64' ? 'base64' : 'utf-8'), size: buffer.length, - }, - content: buffer.toString(input.encoding === 'base64' ? 'base64' : 'utf-8'), - size: buffer.length, - encoding: input.encoding, - message: `Successfully downloaded ${fileName}`, - }) + encoding: input.encoding, + message: `Successfully downloaded ${fileName}`, + }) + } + return createInternalToolFileResult({ buffer, name: fileName, mimeType }, (file) => ({ + file, + })) }) } catch (error) { context.signal?.throwIfAborted() diff --git a/apps/sim/lib/internal/sftp/schema.ts b/apps/sim/lib/internal/sftp/schema.ts index 6a3ab54e14d..d07447ad5be 100644 --- a/apps/sim/lib/internal/sftp/schema.ts +++ b/apps/sim/lib/internal/sftp/schema.ts @@ -52,6 +52,13 @@ export const sftpDownloadInputSchema = requireCredentials( }) ) +export const sftpDownloadV2InputSchema = requireCredentials( + z.object({ + ...connectionFields, + remotePath: z.string().min(1, 'Remote path is required'), + }) +) + export const sftpUploadInputSchema = requireCredentials( z.object({ ...connectionFields, @@ -68,4 +75,5 @@ export type SftpListInput = z.output export type SftpDeleteInput = z.output export type SftpMkdirInput = z.output export type SftpDownloadInput = z.output +export type SftpDownloadV2Input = z.output export type SftpUploadInput = z.output diff --git a/apps/sim/lib/internal/sharepoint/execute-tool.test.ts b/apps/sim/lib/internal/sharepoint/execute-tool.test.ts index d268dc14ce3..dae8f0061ce 100644 --- a/apps/sim/lib/internal/sharepoint/execute-tool.test.ts +++ b/apps/sim/lib/internal/sharepoint/execute-tool.test.ts @@ -2,6 +2,7 @@ * @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ download: vi.fn(), @@ -13,7 +14,7 @@ vi.mock('@/lib/internal/sharepoint/operations', () => ({ executeSharePointUploadFile: mocks.upload, })) -import { executeSharePointTool } from '@/lib/internal/sharepoint/execute-tool' +import { executeSharePointTool as executeSharePointToolOperation } from '@/lib/internal/sharepoint/execute-tool' import { downloadFileTool } from '@/tools/sharepoint/download_file' import { uploadFileTool } from '@/tools/sharepoint/upload_file' @@ -24,6 +25,14 @@ const context = { executionId: 'execution-1', } +async function executeSharePointTool( + request: Parameters[0] +): Promise { + const result = await executeSharePointToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeSharePointTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -53,6 +62,23 @@ describe('executeSharePointTool', () => { ) }) + it('forwards file bytes without serializing the file result', async () => { + const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.download.mockResolvedValueOnce(fileResult) + expect( + await executeSharePointToolOperation({ + toolId: 'sharepoint_download_file', + input: { accessToken: 'token', driveId: 'drive', itemId: 'item' }, + headers: new Headers(), + context, + requestId: 'request-1', + }) + ).toBe(fileResult) + }) + it('requires trusted execution identity before parsing tool input', async () => { const response = await executeSharePointTool({ toolId: 'sharepoint_download_file', diff --git a/apps/sim/lib/internal/sharepoint/execute-tool.ts b/apps/sim/lib/internal/sharepoint/execute-tool.ts index 8d331672e52..c25c37b97aa 100644 --- a/apps/sim/lib/internal/sharepoint/execute-tool.ts +++ b/apps/sim/lib/internal/sharepoint/execute-tool.ts @@ -14,13 +14,17 @@ import { import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' async function executeParsed( request: InternalToolOperationCall, schema: S, - execute: (input: z.output, context: SharePointOperationContext) => Promise -): Promise { + execute: ( + input: z.output, + context: SharePointOperationContext + ) => Promise +): Promise { let serializedInput: string try { serializedInput = JSON.stringify(request.input) ?? '' @@ -57,7 +61,9 @@ async function executeParsed( }) } -export const executeSharePointTool: InternalToolOperationHandler = async (request) => { +export const executeSharePointTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (!request.context.userId) { return Response.json({ success: false, error: 'Authentication required' }, { status: 401 }) diff --git a/apps/sim/lib/internal/sharepoint/operations.test.ts b/apps/sim/lib/internal/sharepoint/operations.test.ts index 863cb6a0f6a..42b03434e1f 100644 --- a/apps/sim/lib/internal/sharepoint/operations.test.ts +++ b/apps/sim/lib/internal/sharepoint/operations.test.ts @@ -63,6 +63,17 @@ const userFile = { type: 'application/pdf', } +const storedFile = { + id: 'stored-file', + name: 'stored.bin', + size: 5, + type: 'application/octet-stream', + mimeType: 'application/octet-stream', + url: '/api/files/stored', + key: 'execution/workspace/workflow/run/stored.bin', + context: 'execution', +} as const + describe('SharePoint operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -147,13 +158,14 @@ describe('SharePoint operations', () => { expect(mocks.uploadGraph).not.toHaveBeenCalled() }) - it('preserves the inline download output contract and cancellation signal', async () => { + it('keeps a large download in process until a stored file can be presented', async () => { const controller = new AbortController() mocks.getMetadata.mockResolvedValue({ name: 'source.txt', file: { mimeType: 'text/plain' }, }) - mocks.downloadGraph.mockResolvedValue(Buffer.from('hello')) + const buffer = Buffer.alloc(12 * 1024 * 1024, 1) + mocks.downloadGraph.mockResolvedValue(buffer) const response = await executeSharePointDownloadFile( { accessToken: 'token', driveId: 'drive', itemId: 'item', fileName: 'renamed.txt' }, @@ -161,16 +173,11 @@ describe('SharePoint operations', () => { ) expect(mocks.clientConstructed).toHaveBeenCalledWith('token', controller.signal) - expect(await response.json()).toEqual({ - success: true, - output: { - file: { - name: 'renamed.txt', - mimeType: 'text/plain', - data: Buffer.from('hello').toString('base64'), - size: 5, - }, - }, - }) + if (response instanceof Response) throw new Error('Expected a file output') + expect(response.files).toHaveLength(1) + expect(response.files[0]?.name).toBe('renamed.txt') + expect(response.files[0]?.mimeType).toBe('text/plain') + expect(response.files[0]?.buffer).toBe(buffer) + expect(response.present([storedFile])).toEqual({ success: true, output: { file: storedFile } }) }) }) diff --git a/apps/sim/lib/internal/sharepoint/operations.ts b/apps/sim/lib/internal/sharepoint/operations.ts index 392185a2ee9..3782a8d689b 100644 --- a/apps/sim/lib/internal/sharepoint/operations.ts +++ b/apps/sim/lib/internal/sharepoint/operations.ts @@ -10,6 +10,8 @@ import type { SharePointDownloadFileInput, SharePointUploadFileInput, } from '@/lib/internal/sharepoint/schema' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { InternalToolOperationResult } from '@/lib/internal/tool-operations/types' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { downloadServableFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { docNotReadyResponse } from '@/lib/uploads/utils/servable-file-response' @@ -47,7 +49,7 @@ function uploadedFile(data: SharePointUploadedItem): SharePointUploadedItem { export async function executeSharePointDownloadFile( input: SharePointDownloadFileInput, context: SharePointOperationContext -): Promise { +): Promise { context.signal?.throwIfAborted() try { const client = new SharePointClient(input.accessToken, context.signal) @@ -61,17 +63,10 @@ export async function executeSharePointDownloadFile( const mimeType = metadata.file?.mimeType || 'application/octet-stream' const buffer = await client.download(input.driveId, input.itemId) context.signal?.throwIfAborted() - return Response.json({ - success: true, - output: { - file: { - name: input.fileName || metadata.name || 'download', - mimeType, - data: buffer.toString('base64'), - size: buffer.length, - }, - }, - }) + return createInternalToolFileResult( + { buffer, name: input.fileName || metadata.name || 'download', mimeType }, + (file) => ({ success: true, output: { file } }) + ) } catch (error) { context.signal?.throwIfAborted() if (error instanceof SharePointGraphError) { diff --git a/apps/sim/lib/internal/slack/execute-tool.ts b/apps/sim/lib/internal/slack/execute-tool.ts index 81905967129..f4f4cc2b131 100644 --- a/apps/sim/lib/internal/slack/execute-tool.ts +++ b/apps/sim/lib/internal/slack/execute-tool.ts @@ -28,10 +28,12 @@ import { executeSlackGetChannelHistoryOperation } from '@/lib/internal/slack/ope import { executeSlackGetThreadRepliesOperation } from '@/lib/internal/slack/operations/get-thread-replies' import { executeSlackListConversationsOperation } from '@/lib/internal/slack/operations/list-conversations' import { executeToolOperationImplementation } from '@/lib/internal/tool-operations/execute' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { parseInternalToolInput } from '@/lib/internal/tool-operations/parse-input' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' import { docNotReadyResponse } from '@/lib/uploads/utils/servable-file-response' @@ -41,14 +43,14 @@ async function executeOperation( contract: C, request: InternalToolOperationCall, execute: (input: ContractBody) => Promise -): Promise { +): Promise { request.signal?.throwIfAborted() const parsed = parseInternalToolInput(contract, request.input) if (!parsed.success) return parsed.response try { const result = await execute(parsed.data) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof SlackOperationError) { @@ -71,7 +73,9 @@ async function executeOperation( } } -export const executeSlackTool: InternalToolOperationHandler = async (request) => { +export const executeSlackTool: InternalToolOperationHandler = async ( + request +) => { const context: SlackOperationContext = { requestId: request.requestId, signal: request.signal, diff --git a/apps/sim/lib/internal/slack/operations.test.ts b/apps/sim/lib/internal/slack/operations.test.ts index 8f269c2f2ae..b4b76baf6be 100644 --- a/apps/sim/lib/internal/slack/operations.test.ts +++ b/apps/sim/lib/internal/slack/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +import { afterEach, assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ resolveFiles: vi.fn(), @@ -195,11 +200,23 @@ describe('Slack operations', () => { }), 'uploadUrl' ) - expect(result.output).toMatchObject({ - channel: 'C1', - fileCount: 1, - ts: '10', - files: [{ name: 'hello.txt', size: 5 }], + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'hello.txt', mimeType: 'text/plain', buffer: Buffer.from('hello') }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'hello.txt', + type: 'text/plain', + mimeType: 'text/plain', + size: 5, + context: 'execution', + } + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { channel: 'C1', fileCount: 1, ts: '10', files: [storedFile] }, }) }) @@ -234,12 +251,21 @@ describe('Slack operations', () => { signal: controller.signal, } ) - expect(result.output.file).toEqual({ + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'report.pdf', mimeType: 'application/pdf', buffer: Buffer.from('pdf') }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', name: 'report.pdf', + type: 'application/pdf', mimeType: 'application/pdf', - data: Buffer.from('pdf').toString('base64'), size: 3, - }) + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ success: true, output: { file: storedFile } }) }) it.each([ diff --git a/apps/sim/lib/internal/slack/operations.ts b/apps/sim/lib/internal/slack/operations.ts index a6c139d9ccf..c2d2512330b 100644 --- a/apps/sim/lib/internal/slack/operations.ts +++ b/apps/sim/lib/internal/slack/operations.ts @@ -25,8 +25,12 @@ import { } from '@/lib/internal/slack/client' import { SlackOperationError } from '@/lib/internal/slack/errors' import { forEachSlackAttachmentFile } from '@/lib/internal/slack/file-input' +import { + createInternalToolFileResult, + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' -import type { ToolFileData } from '@/tools/types' const logger = createLogger('SlackOperations') @@ -294,9 +298,9 @@ async function uploadSlackFiles( input: SlackSendMessageBody, channel: string, context: SlackOperationContext -): Promise<{ fileIds: string[]; files: ToolFileData[]; message?: unknown }> { +): Promise<{ fileIds: string[]; files: InternalToolFile[]; message?: unknown }> { const fileIds: string[] = [] - const files: ToolFileData[] = [] + const files: InternalToolFile[] = [] await forEachSlackAttachmentFile( input.files ?? [], @@ -344,8 +348,7 @@ async function uploadSlackFiles( files.push({ name: file.name, mimeType: file.contentType || file.type || 'application/octet-stream', - data: file.buffer.toString('base64'), - size: file.buffer.length, + buffer: file.buffer, }) } ) @@ -413,16 +416,17 @@ export async function executeSlackSendMessage( return { success: true as const, output: sentMessageOutput(data, input.text) } } - return { - success: true as const, + const { message, fileIds, files } = uploaded + return createInternalToolFilesResult(files, (storedFiles) => ({ + success: true, output: { - message: uploaded.message, - ts: record(uploaded.message).ts, + message, + ts: record(message).ts, channel, - fileCount: uploaded.fileIds.length, - files: uploaded.files, + fileCount: fileIds.length, + files: storedFiles, }, - } + })) } export async function executeSlackDownload(input: SlackDownloadBody, signal?: AbortSignal) { @@ -458,10 +462,8 @@ export async function executeSlackDownload(input: SlackDownloadBody, signal?: Ab if (!response.ok) failure(400, 'Failed to download file content') const buffer = Buffer.from(await response.arrayBuffer()) signal?.throwIfAborted() - return { - success: true as const, - output: { - file: { name, mimeType, data: buffer.toString('base64'), size: buffer.length }, - }, - } + return createInternalToolFileResult({ buffer, name, mimeType }, (file) => ({ + success: true, + output: { file }, + })) } diff --git a/apps/sim/lib/internal/ssh/execute-tool.test.ts b/apps/sim/lib/internal/ssh/execute-tool.test.ts index 7fe223caf1b..22149e60e25 100644 --- a/apps/sim/lib/internal/ssh/execute-tool.test.ts +++ b/apps/sim/lib/internal/ssh/execute-tool.test.ts @@ -3,6 +3,7 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const operationMocks = vi.hoisted(() => ({ executeSshCheckCommandExists: vi.fn(), @@ -24,7 +25,7 @@ vi.mock('@/lib/internal/ssh/operations', () => operationMocks) import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { SshOperationError } from '@/lib/internal/ssh/errors' -import { executeSshTool } from '@/lib/internal/ssh/execute-tool' +import { executeSshTool as executeSshToolOperation } from '@/lib/internal/ssh/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' const CONNECTION = { @@ -40,6 +41,7 @@ const TOOL_IDS = [ 'ssh_create_directory', 'ssh_delete_file', 'ssh_download_file', + 'ssh_download_file_v2', 'ssh_execute_command', 'ssh_execute_script', 'ssh_get_system_info', @@ -67,6 +69,14 @@ function createRequest( } } +async function executeSshTool( + request: Parameters[0] +): Promise { + const result = await executeSshToolOperation(request) + if (!(result instanceof Response)) throw new Error('Expected a JSON response') + return result +} + describe('executeSshTool', () => { beforeEach(() => { vi.clearAllMocks() @@ -75,6 +85,23 @@ describe('executeSshTool', () => { } }) + it('forwards binary file results without serializing them', async () => { + const result = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.txt', mimeType: 'text/plain' }, + (file) => ({ file }) + ) + operationMocks.executeSshDownloadFile.mockResolvedValueOnce(result) + expect( + await executeSshToolOperation( + createRequest({ + toolId: 'ssh_download_file_v2', + input: { ...CONNECTION, remotePath: '/file.txt' }, + }) + ) + ).toBe(result) + expect(operationMocks.executeSshDownloadFile.mock.calls[0][2]).toBe('v2') + }) + it('validates typed operation input and dispatches without reading a serialized body', async () => { const controller = new AbortController() const input = { ...CONNECTION, command: 'pwd' } diff --git a/apps/sim/lib/internal/ssh/execute-tool.ts b/apps/sim/lib/internal/ssh/execute-tool.ts index 58010742226..b1261db2d27 100644 --- a/apps/sim/lib/internal/ssh/execute-tool.ts +++ b/apps/sim/lib/internal/ssh/execute-tool.ts @@ -34,9 +34,11 @@ import { executeSshWriteFileContent, type SshOperationContext, } from '@/lib/internal/ssh/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { InternalToolOperationCall, InternalToolOperationHandler, + InternalToolOperationResult, } from '@/lib/internal/tool-operations/types' const logger = createLogger('SshToolExecution') @@ -46,7 +48,7 @@ async function executeOperation( request: InternalToolOperationCall, operation: (input: ContractBody, context: SshOperationContext) => Promise, failureMessage: string -): Promise { +): Promise { request.signal?.throwIfAborted() if (!contract.body) throw new Error(`SSH contract ${contract.path} has no operation input`) const parsed = contract.body.safeParse(request.input) @@ -60,7 +62,7 @@ async function executeOperation( try { const result = await operation(parsed.data as ContractBody, { signal: request.signal }) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof SshOperationError) { @@ -79,7 +81,9 @@ async function executeOperation( } } -export const executeSshTool: InternalToolOperationHandler = async (request) => { +export const executeSshTool: InternalToolOperationHandler = async ( + request +) => { switch (request.toolId) { case 'ssh_check_command_exists': return executeOperation( @@ -116,6 +120,13 @@ export const executeSshTool: InternalToolOperationHandler = async (request) => { executeSshDownloadFile, 'SSH file download failed' ) + case 'ssh_download_file_v2': + return executeOperation( + sshDownloadFileContract, + request, + (input, context) => executeSshDownloadFile(input, context, 'v2'), + 'SSH file download failed' + ) case 'ssh_execute_command': return executeOperation( sshExecuteCommandContract, diff --git a/apps/sim/lib/internal/ssh/operations.test.ts b/apps/sim/lib/internal/ssh/operations.test.ts index 5927e232ed4..99b394924b1 100644 --- a/apps/sim/lib/internal/ssh/operations.test.ts +++ b/apps/sim/lib/internal/ssh/operations.test.ts @@ -1,10 +1,12 @@ +import { Readable } from 'node:stream' /** * @vitest-environment node */ import { beforeEach, describe, expect, it, vi } from 'vitest' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ - client: { destroy: vi.fn(), end: vi.fn() }, + client: { destroy: vi.fn(), end: vi.fn(), sftp: vi.fn() }, createSSHConnection: vi.fn(), executeSSHCommand: vi.fn(), })) @@ -19,7 +21,7 @@ vi.mock('@/lib/internal/ssh/client', () => ({ sanitizePath: (value: string) => value.trim(), })) -import { executeSshExecuteCommand } from '@/lib/internal/ssh/operations' +import { executeSshDownloadFile, executeSshExecuteCommand } from '@/lib/internal/ssh/operations' const INPUT = { host: 'ssh.example.com', @@ -30,6 +32,17 @@ const INPUT = { workingDirectory: "/srv/app's", } +const storedFile = { + id: 'stored-file', + name: 'file.txt', + size: 5, + type: 'text/plain', + mimeType: 'text/plain', + url: '/api/files/stored', + key: 'execution/file.txt', + context: 'execution', +} as const + describe('SSH operations', () => { beforeEach(() => { vi.clearAllMocks() @@ -58,6 +71,65 @@ describe('SSH operations', () => { expect(mocks.client.end).toHaveBeenCalledOnce() }) + it.each([5, 12 * 1024 * 1024])( + 'downloads %i bytes through a stored file without inline content in v2', + async (size) => { + const buffer = Buffer.alloc(size, 65) + const sftp = { + stat: vi.fn((_path, callback) => callback(null, { size })), + createReadStream: vi.fn(() => Readable.from([buffer])), + } + mocks.client.sftp.mockImplementation((callback) => callback(null, sftp)) + const result = await executeSshDownloadFile( + { + host: INPUT.host, + port: INPUT.port, + username: INPUT.username, + password: INPUT.password, + remotePath: '/file.txt', + }, + {}, + 'v2' + ) + if (!isInternalToolFileResult(result)) throw new Error('Expected a file output') + expect(result.files[0]?.buffer.length).toBe(size) + expect(result.files[0]?.name).toBe('file.txt') + const file = { ...storedFile, size } + const presented = result.present([file]) + expect(presented).toEqual({ + file, + remotePath: '/file.txt', + }) + expect(JSON.stringify(presented)).not.toContain('"content"') + expect(sftp.createReadStream).toHaveBeenCalledOnce() + expect(mocks.client.end).toHaveBeenCalledOnce() + } + ) + + it('preserves the complete legacy v1 download response', async () => { + const buffer = Buffer.from('hello') + const sftp = { + stat: vi.fn((_path, callback) => callback(null, { size: buffer.length })), + createReadStream: vi.fn(() => Readable.from([buffer])), + } + mocks.client.sftp.mockImplementation((callback) => callback(null, sftp)) + const result = await executeSshDownloadFile({ ...INPUT, remotePath: '/file.txt' }, {}) + expect(result).toEqual({ + downloaded: true, + file: { + name: 'file.txt', + mimeType: 'text/plain', + data: buffer.toString('base64'), + size: 5, + }, + content: buffer.toString('base64'), + fileName: 'file.txt', + remotePath: '/file.txt', + size: 5, + message: 'File downloaded successfully from /file.txt', + }) + }) + it('destroys and closes the client when cancellation wins during provider work', async () => { const controller = new AbortController() mocks.executeSSHCommand.mockImplementationOnce(async () => { diff --git a/apps/sim/lib/internal/ssh/operations.ts b/apps/sim/lib/internal/ssh/operations.ts index 0072c49f4aa..7970d848c99 100644 --- a/apps/sim/lib/internal/ssh/operations.ts +++ b/apps/sim/lib/internal/ssh/operations.ts @@ -33,6 +33,7 @@ import { sanitizePath, } from '@/lib/internal/ssh/client' import { SshOperationError } from '@/lib/internal/ssh/errors' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' export interface SshOperationContext { @@ -281,7 +282,8 @@ export async function executeSshDeleteFile( export async function executeSshDownloadFile( input: DownloadFileInput, - context: SshOperationContext + context: SshOperationContext, + version: 'v1' | 'v2' = 'v1' ): Promise { return withClient(input, context, async (client) => { const sftp = await getSftp(client, context.signal) @@ -307,21 +309,34 @@ export async function executeSshDownloadFile( context.signal ) const fileName = path.basename(remotePath) - const base64Content = content.toString('base64') - return { - downloaded: true, - file: { + if (version === 'v1') { + const base64Content = content.toString('base64') + return { + downloaded: true, + file: { + name: fileName, + mimeType: getMimeTypeFromExtension(getFileExtension(fileName)), + data: base64Content, + size: content.length, + }, + content: base64Content, + fileName, + remotePath, + size: content.length, + message: `File downloaded successfully from ${remotePath}`, + } + } + return createInternalToolFileResult( + { + buffer: content, name: fileName, mimeType: getMimeTypeFromExtension(getFileExtension(fileName)), - data: base64Content, - size: content.length, }, - content: base64Content, - fileName, - remotePath, - size: content.length, - message: `File downloaded successfully from ${remotePath}`, - } + (file) => ({ + file, + remotePath, + }) + ) }) } diff --git a/apps/sim/lib/internal/telegram/execute-tool.test.ts b/apps/sim/lib/internal/telegram/execute-tool.test.ts index 8dd8f96435b..a71d51157f0 100644 --- a/apps/sim/lib/internal/telegram/execute-tool.test.ts +++ b/apps/sim/lib/internal/telegram/execute-tool.test.ts @@ -1,8 +1,10 @@ /** * @vitest-environment node */ + import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ sendTelegramDocument: vi.fn() })) @@ -13,10 +15,15 @@ vi.mock('@/lib/internal/telegram/operations', () => ({ import { executeTelegramTool } from '@/lib/internal/telegram/execute-tool' import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' +const fileResult = createInternalToolFileResult( + { buffer: Buffer.from('file'), name: 'file.pdf', mimeType: 'application/pdf' }, + (file) => ({ success: true, output: { file } }) +) + describe('executeTelegramTool', () => { beforeEach(() => { vi.clearAllMocks() - mocks.sendTelegramDocument.mockResolvedValue({ success: true, output: {} }) + mocks.sendTelegramDocument.mockResolvedValue(fileResult) }) it('uses trusted user context for protected files', async () => { @@ -35,7 +42,7 @@ describe('executeTelegramTool', () => { signal: controller.signal, } - expect((await executeTelegramTool(request)).status).toBe(200) + expect(await executeTelegramTool(request)).toBe(fileResult) expect(mocks.sendTelegramDocument).toHaveBeenCalledWith(input, { userId: 'user-1', requestId: 'request-1', diff --git a/apps/sim/lib/internal/telegram/execute-tool.ts b/apps/sim/lib/internal/telegram/execute-tool.ts index ae7f4d27200..eb0f24ba471 100644 --- a/apps/sim/lib/internal/telegram/execute-tool.ts +++ b/apps/sim/lib/internal/telegram/execute-tool.ts @@ -2,7 +2,10 @@ import { getErrorMessage } from '@sim/utils/errors' import { z } from 'zod' import { TelegramOperationError } from '@/lib/internal/telegram/errors' import { sendTelegramDocument } from '@/lib/internal/telegram/operations' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas' import { docNotReadyResponse } from '@/lib/uploads/utils/servable-file-response' @@ -13,7 +16,9 @@ const inputSchema = z.object({ caption: z.string().optional().nullable(), }) -export const executeTelegramTool: InternalToolOperationHandler = async (request) => { +export const executeTelegramTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (request.toolId !== 'telegram_send_document') { return Response.json( @@ -30,13 +35,11 @@ export const executeTelegramTool: InternalToolOperationHandler = async (request) return Response.json({ success: false, error: 'Invalid request data' }, { status: 400 }) } try { - return Response.json( - await sendTelegramDocument(parsed.data, { - userId, - requestId: request.requestId, - signal: request.signal, - }) - ) + return await sendTelegramDocument(parsed.data, { + userId, + requestId: request.requestId, + signal: request.signal, + }) } catch (error) { request.signal?.throwIfAborted() const notReady = docNotReadyResponse(error) diff --git a/apps/sim/lib/internal/telegram/operations.test.ts b/apps/sim/lib/internal/telegram/operations.test.ts index 862acbaae0a..1ca7855cac9 100644 --- a/apps/sim/lib/internal/telegram/operations.test.ts +++ b/apps/sim/lib/internal/telegram/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), @@ -53,9 +58,28 @@ describe('sendTelegramDocument', () => { expect(mocks.fetch.mock.calls[0][1]).toEqual( expect.objectContaining({ signal: controller.signal }) ) - expect(result.output.files?.[0]).toEqual( - expect.objectContaining({ name: 'file.pdf', data: 'AQID', size: 3 }) - ) + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'file.pdf', mimeType: 'application/pdf', buffer: Buffer.from([1, 2, 3]) }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'file.pdf', + type: 'application/pdf', + mimeType: 'application/pdf', + size: 3, + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ + success: true, + output: { + message: 'Document sent successfully', + data: { message_id: 1 }, + files: [storedFile], + }, + }) }) it('fails closed before materialization when file access is denied', async () => { diff --git a/apps/sim/lib/internal/telegram/operations.ts b/apps/sim/lib/internal/telegram/operations.ts index aa18bdaf50d..b321ac86ee3 100644 --- a/apps/sim/lib/internal/telegram/operations.ts +++ b/apps/sim/lib/internal/telegram/operations.ts @@ -2,6 +2,10 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { isPayloadSizeLimitError, readResponseJsonWithLimit } from '@/lib/core/utils/stream-limits' import { TelegramOperationError } from '@/lib/internal/telegram/errors' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import type { RawFileInput } from '@/lib/uploads/utils/file-schemas' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { downloadServableFileFromStorage } from '@/lib/uploads/utils/file-utils.server' @@ -35,7 +39,7 @@ interface TelegramApiResponse { export async function sendTelegramDocument( input: TelegramSendDocumentInput, context: TelegramOperationContext -): Promise { +): Promise { context.signal?.throwIfAborted() if (!input.files?.length) { throw new TelegramOperationError( @@ -118,19 +122,12 @@ export async function sendTelegramDocument( ) } - return { + return createInternalToolFileResult({ buffer, name: userFile.name, mimeType }, (file) => ({ success: true, output: { message: 'Document sent successfully', data: data.result, - files: [ - { - name: userFile.name, - mimeType, - data: buffer.toString('base64'), - size: buffer.length, - }, - ], + files: [file], }, - } + })) } diff --git a/apps/sim/lib/internal/tool-operations/file-result.server.test.ts b/apps/sim/lib/internal/tool-operations/file-result.server.test.ts new file mode 100644 index 00000000000..5a5dcc9c636 --- /dev/null +++ b/apps/sim/lib/internal/tool-operations/file-result.server.test.ts @@ -0,0 +1,527 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { + PayloadSizeLimitError, + readResponseToBufferWithLimit, +} from '@/lib/core/utils/stream-limits' +import { + createInternalToolFileResult, + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' +import { MAX_TOOL_RESPONSE_BODY_BYTES } from '@/lib/internal/tool-operations/response-limits' +import type { InternalToolOperationContext } from '@/lib/internal/tool-operations/types' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' +import type { UserFile } from '@/executor/types' + +const mocks = vi.hoisted(() => ({ + uploadExecution: vi.fn(), + uploadCopilot: vi.fn(), + deleteFile: vi.fn(), + deleteMetadata: vi.fn(), +})) + +vi.mock('@/lib/uploads/contexts/execution', () => ({ + uploadExecutionFile: mocks.uploadExecution, +})) + +vi.mock('@/lib/uploads/contexts/copilot', () => ({ + uploadCopilotFile: mocks.uploadCopilot, +})) + +vi.mock('@/lib/uploads/core/storage-service', () => ({ deleteFile: mocks.deleteFile })) +vi.mock('@/lib/uploads/server/metadata', () => ({ deleteFileMetadata: mocks.deleteMetadata })) + +import { + presentInternalToolOperationResult, + storeInternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result.server' + +const runContext: InternalToolOperationContext = { + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + executionId: 'execution-1', + userId: 'user-1', +} + +const copilotContext: InternalToolOperationContext = { + workspaceId: 'workspace-1', + workflowId: '', + userId: 'user-1', + copilotToolExecution: true, +} + +function file(buffer = Buffer.from('workbook')): InternalToolFile { + return { + buffer, + name: 'workbook.xlsx', + mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', + } +} + +function storedFile( + buffer: Buffer, + name: string, + type: string, + context: 'execution' | 'copilot', + index = 1 +): UserFile { + return { + id: `file-${index}`, + key: `${context}/file-${index}/${name}`, + url: `https://storage.example/file-${index}`, + name, + size: buffer.length, + type, + context, + } +} + +describe('presentInternalToolOperationResult', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.uploadExecution.mockReset() + mocks.uploadCopilot.mockReset() + mocks.deleteFile.mockReset() + mocks.deleteMetadata.mockReset() + mocks.uploadExecution.mockImplementation( + async (_scope: unknown, buffer: Buffer, name: string, type: string) => + storedFile(buffer, name, type, 'execution', mocks.uploadExecution.mock.calls.length) + ) + mocks.uploadCopilot.mockImplementation( + async (input: { buffer: Buffer; fileName: string; contentType: string }) => + storedFile(input.buffer, input.fileName, input.contentType, 'copilot') + ) + mocks.deleteFile.mockResolvedValue(undefined) + mocks.deleteMetadata.mockResolvedValue(true) + }) + + it('passes ordinary responses through without reading or changing them', async () => { + const original = Response.json({ error: 'Provider failed' }, { status: 403 }) + const controller = new AbortController() + controller.abort() + + const result = await presentInternalToolOperationResult( + original, + { workflowId: '' }, + controller.signal + ) + + expect(result).toBe(original) + expect(original.bodyUsed).toBe(false) + expect(mocks.uploadExecution).not.toHaveBeenCalled() + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('persists a 12 MiB workbook before serializing its descriptor and adjacent metadata', async () => { + const input = file(Buffer.alloc(12 * 1024 * 1024, 1)) + const result = createInternalToolFileResult( + input, + (stored) => ({ success: true, output: { file: stored, metadata: { sourceId: 'item-1' } } }), + { status: 201, headers: { 'x-provider-version': 'v1' } } + ) + + const response = await presentInternalToolOperationResult(result, runContext) + const body = await response.text() + + expect(body.length).toBeLessThan(1024) + expect(response.status).toBe(201) + expect(response.headers.get('x-provider-version')).toBe('v1') + expect(response.headers.get('content-type')).toBe('application/json') + expect(JSON.parse(body)).toMatchObject({ + success: true, + output: { + file: { + name: input.name, + size: input.buffer.length, + type: input.mimeType, + }, + metadata: { sourceId: 'item-1' }, + }, + }) + const [scope, buffer, name, mimeType, userId] = mocks.uploadExecution.mock.calls[0]! + expect(buffer).toBe(input.buffer) + expect({ scope, name, mimeType, userId }).toEqual({ + scope: { workspaceId: 'workspace-1', workflowId: 'workflow-1', executionId: 'execution-1' }, + name: input.name, + mimeType: input.mimeType, + userId: 'user-1', + }) + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('stores non-run files under the authenticated Copilot user', async () => { + const input = file() + const response = await presentInternalToolOperationResult( + createInternalToolFileResult(input, (stored) => ({ file: stored, fileUrl: stored.url })), + copilotContext + ) + + expect(await response.json()).toMatchObject({ + file: { context: 'copilot' }, + fileUrl: 'https://storage.example/file-1', + }) + expect(mocks.uploadCopilot).toHaveBeenCalledWith({ + buffer: input.buffer, + fileName: input.name, + contentType: input.mimeType, + userId: 'user-1', + }) + expect(mocks.uploadExecution).not.toHaveBeenCalled() + }) + + it('replaces binary representation headers before the JSON transport size check', async () => { + const input = file(Buffer.alloc(12 * 1024 * 1024)) + const headers = new Headers({ + 'content-length': String(input.buffer.length), + 'content-encoding': 'gzip', + 'content-type': input.mimeType, + 'x-provider-version': 'v1', + }) + const response = await presentInternalToolOperationResult( + createInternalToolFileResult(input, (stored) => ({ file: stored }), { + status: 201, + statusText: 'Created', + headers, + }), + runContext + ) + + const body = await readResponseToBufferWithLimit(response, { + maxBytes: MAX_TOOL_RESPONSE_BODY_BYTES, + label: 'Tool response body', + }) + + expect(body.length).toBeLessThan(1024) + expect(JSON.parse(body.toString('utf8'))).toMatchObject({ + file: { size: input.buffer.length, context: 'execution' }, + }) + expect(response.status).toBe(201) + expect(response.statusText).toBe('Created') + expect(response.headers.get('content-type')).toBe('application/json') + expect(response.headers.get('content-length')).toBeNull() + expect(response.headers.get('content-encoding')).toBeNull() + expect(response.headers.get('x-provider-version')).toBe('v1') + expect(headers.get('content-length')).toBe(String(input.buffer.length)) + expect(mocks.uploadExecution).toHaveBeenCalledTimes(1) + expect(mocks.deleteFile).not.toHaveBeenCalled() + }) + + it('allows actorless execution artifacts without requiring a human subject', async () => { + const result = createInternalToolFileResult(file(), (stored) => ({ file: stored })) + await presentInternalToolOperationResult(result, { + ...runContext, + userId: undefined, + executorDelegationOrigin: { + workflowId: 'workflow-1', + executionId: 'execution-1', + principal: { + kind: 'system', + serviceId: 'schedule', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + }, + }, + }) + + expect(mocks.uploadExecution.mock.calls[0]?.[4]).toBeUndefined() + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it("does not make an actorless principal's compatibility owner a Copilot user", async () => { + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(file(), (stored) => ({ file: stored })), + { + ...copilotContext, + userId: 'billing-owner', + executorDelegationOrigin: { + workflowId: 'workflow-1', + principal: { + kind: 'system', + serviceId: 'schedule', + workspaceId: 'workspace-1', + workflowId: 'workflow-1', + }, + }, + } + ) + ).rejects.toThrow('human subject') + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('uses a real principal subject and refuses a conflicting claimed owner', async () => { + const result = createInternalToolFileResult(file(), (stored) => ({ file: stored })) + const context: InternalToolOperationContext = { + ...copilotContext, + userId: undefined, + executorDelegationOrigin: { + workflowId: 'workflow-1', + principal: { kind: 'session', userId: 'actual-user', sessionId: 'session-1' }, + }, + } + await presentInternalToolOperationResult(result, context) + expect(mocks.uploadCopilot).toHaveBeenCalledWith( + expect.objectContaining({ userId: 'actual-user' }) + ) + + await expect( + presentInternalToolOperationResult(result, { ...context, userId: 'other-user' }) + ).rejects.toThrow('does not match') + expect(mocks.uploadCopilot).toHaveBeenCalledTimes(1) + }) + + it.each([ + { ...copilotContext, userId: undefined }, + { ...runContext, workspaceId: undefined }, + { ...runContext, workflowId: '' }, + ])('rejects missing storage authority before uploading: %j', async (context) => { + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(file(), (stored) => ({ file: stored })), + context + ) + ).rejects.toThrow() + expect(mocks.uploadExecution).not.toHaveBeenCalled() + expect(mocks.uploadCopilot).not.toHaveBeenCalled() + }) + + it('accepts exactly 100 MiB and rejects larger files before uploading', async () => { + const atLimit = file(Buffer.alloc(MAX_BUFFERED_TRANSFER_BYTES)) + await presentInternalToolOperationResult( + createInternalToolFileResult(atLimit, (stored) => ({ file: stored })), + runContext + ) + expect(mocks.uploadExecution).toHaveBeenCalledTimes(1) + + const oversized = file(Buffer.alloc(MAX_BUFFERED_TRANSFER_BYTES + 1)) + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(oversized, (stored) => ({ file: stored })), + runContext + ) + ).rejects.toBeInstanceOf(PayloadSizeLimitError) + expect(mocks.uploadExecution).toHaveBeenCalledTimes(1) + }) + + it('checks the aggregate buffer budget before storing the first file', async () => { + const files = [file(Buffer.alloc(60 * 1024 * 1024)), file(Buffer.alloc(41 * 1024 * 1024))] + await expect( + presentInternalToolOperationResult( + createInternalToolFilesResult(files, (stored) => ({ files: stored })), + runContext + ) + ).rejects.toMatchObject({ + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + observedBytes: 101 * 1024 * 1024, + }) + expect(mocks.uploadExecution).not.toHaveBeenCalled() + }) + + it('validates every file before any upload', async () => { + await expect( + presentInternalToolOperationResult( + createInternalToolFilesResult([file(), { ...file(), name: ' ' }], (stored) => ({ + files: stored, + })), + runContext + ) + ).rejects.toThrow('filename') + expect(mocks.uploadExecution).not.toHaveBeenCalled() + }) + + it('persists distinct files sequentially and reuses repeated file references', async () => { + const first = file() + const second = { ...file(), name: 'second.xlsx' } + let firstFinished = false + mocks.uploadExecution + .mockImplementationOnce(async () => { + await Promise.resolve() + firstFinished = true + return storedFile(first.buffer, first.name, first.mimeType, 'execution') + }) + .mockImplementationOnce(async () => { + expect(firstFinished).toBe(true) + return storedFile(second.buffer, second.name, second.mimeType, 'execution', 2) + }) + const result = createInternalToolFilesResult([first, second, first], (stored) => { + expect(stored[0]).toBe(stored[2]) + return { files: stored, echoedFile: stored[0] } + }) + + const response = await presentInternalToolOperationResult(result, runContext) + expect((await response.json()).files).toHaveLength(3) + expect(mocks.uploadExecution).toHaveBeenCalledTimes(2) + }) + + it('preserves image sniffing before returning an already stored descriptor', async () => { + const input = { buffer: Buffer.from(''), name: 'image.png', mimeType: 'image/png' } + const response = await presentInternalToolOperationResult( + createInternalToolFileResult(input, (stored) => ({ file: stored })), + runContext + ) + + expect(mocks.uploadExecution).toHaveBeenCalledWith( + expect.anything(), + input.buffer, + 'image.bin', + 'application/octet-stream', + 'user-1' + ) + expect(await response.json()).toMatchObject({ + file: { + name: 'image.bin', + type: 'application/octet-stream', + }, + }) + }) + + it('does not upload after cancellation', async () => { + const controller = new AbortController() + const error = new Error('cancelled') + controller.abort(error) + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(file(), (stored) => ({ file: stored })), + runContext, + controller.signal + ) + ).rejects.toBe(error) + expect(mocks.uploadExecution).not.toHaveBeenCalled() + }) + + it('rolls back an upload that completed after cancellation, without the aborted signal', async () => { + const controller = new AbortController() + const error = new Error('cancelled during upload') + mocks.uploadExecution.mockImplementationOnce(async () => { + controller.abort(error) + return storedFile(Buffer.from('file'), 'file.txt', 'text/plain', 'execution') + }) + + await expect( + presentInternalToolOperationResult( + createInternalToolFilesResult([file(), file()], (stored) => ({ files: stored })), + runContext, + controller.signal + ) + ).rejects.toBe(error) + expect(mocks.uploadExecution).toHaveBeenCalledTimes(1) + expect(mocks.deleteFile).toHaveBeenCalledWith({ + key: 'execution/file-1/file.txt', + context: 'execution', + }) + expect(mocks.deleteMetadata).toHaveBeenCalledWith('execution/file-1/file.txt') + }) + + it('rolls back preceding uploads when a later upload fails', async () => { + const error = new Error('Storage unavailable') + mocks.uploadExecution + .mockResolvedValueOnce( + storedFile(Buffer.from('file'), 'first.txt', 'text/plain', 'execution') + ) + .mockRejectedValueOnce(error) + await expect( + presentInternalToolOperationResult( + createInternalToolFilesResult([file(), file()], (stored) => ({ files: stored })), + runContext + ) + ).rejects.toBe(error) + expect(mocks.deleteFile).toHaveBeenCalledTimes(1) + expect(mocks.deleteMetadata).toHaveBeenCalledWith('execution/file-1/first.txt') + }) + + it.each([ + () => { + throw new Error('Presentation failed') + }, + () => ({ unsupported: 1n }), + () => undefined, + ])('rolls back Copilot files if presentation or serialization fails', async (present) => { + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(file(), present), + copilotContext + ) + ).rejects.toThrow() + expect(mocks.deleteFile).toHaveBeenCalledWith({ + key: 'copilot/file-1/workbook.xlsx', + context: 'copilot', + }) + expect(mocks.deleteMetadata).toHaveBeenCalledWith('copilot/file-1/workbook.xlsx') + }) + + it('rolls back when adjacent JSON exceeds the unchanged transport limit', async () => { + await expect( + presentInternalToolOperationResult( + createInternalToolFileResult(file(), (stored) => ({ + file: stored, + text: 'x'.repeat(MAX_TOOL_RESPONSE_BODY_BYTES), + })), + runContext + ) + ).rejects.toMatchObject({ maxBytes: MAX_TOOL_RESPONSE_BODY_BYTES }) + expect(mocks.deleteFile).toHaveBeenCalledTimes(1) + expect(mocks.deleteMetadata).toHaveBeenCalledTimes(1) + }) + + it('finalizes large binary outputs as stored file descriptors', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024) + const finalize = vi.fn((body: unknown) => body) + const output = await storeInternalToolFileResult( + createInternalToolFileResult(file(buffer), (stored) => ({ + success: true, + output: { file: stored }, + })), + copilotContext, + finalize + ) + + expect(output).toBe(finalize.mock.calls[0]?.[0]) + expect(output).toMatchObject({ + success: true, + output: { file: { context: 'copilot', size: buffer.length } }, + }) + expect(output).not.toHaveProperty('output.file.data') + expect(JSON.stringify(output).length).toBeLessThan(1024) + expect(mocks.uploadCopilot).toHaveBeenCalledTimes(1) + expect(mocks.uploadCopilot.mock.calls[0]?.[0].buffer).toBe(buffer) + expect(mocks.deleteFile).not.toHaveBeenCalled() + }) + + it('rolls back storage if the external result finalizer rejects its output', async () => { + const error = new Error('Invalid tool response') + await expect( + storeInternalToolFileResult( + createInternalToolFileResult(file(), (stored) => ({ file: stored })), + copilotContext, + () => { + throw error + } + ) + ).rejects.toBe(error) + + expect(mocks.deleteFile).toHaveBeenCalledWith({ + key: 'copilot/file-1/workbook.xlsx', + context: 'copilot', + }) + expect(mocks.deleteMetadata).toHaveBeenCalledTimes(1) + }) + + it('attempts remaining cleanup after a deletion failure and preserves the original error', async () => { + const error = new Error('Presentation failed') + mocks.deleteFile.mockRejectedValueOnce(new Error('Cleanup failed')) + await expect( + presentInternalToolOperationResult( + createInternalToolFilesResult([file(), { ...file(), name: 'second.xlsx' }], () => { + throw error + }), + runContext + ) + ).rejects.toBe(error) + expect(mocks.deleteFile).toHaveBeenCalledTimes(2) + expect(mocks.deleteMetadata).toHaveBeenCalledTimes(1) + expect(mocks.deleteMetadata).toHaveBeenCalledWith('execution/file-2/second.xlsx') + }) +}) diff --git a/apps/sim/lib/internal/tool-operations/file-result.server.ts b/apps/sim/lib/internal/tool-operations/file-result.server.ts new file mode 100644 index 00000000000..ab1b6e516d9 --- /dev/null +++ b/apps/sim/lib/internal/tool-operations/file-result.server.ts @@ -0,0 +1,185 @@ +import { PrincipalSubjectUserRequiredError, resolvePrincipalSubject } from '@sim/auth/principal' +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' +import { omit } from '@sim/utils/object' +import { assertKnownSizeWithinLimit } from '@/lib/core/utils/stream-limits' +import type { + InternalToolFile, + InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' +import { MAX_TOOL_RESPONSE_BODY_BYTES } from '@/lib/internal/tool-operations/response-limits' +import type { + InternalToolOperationContext, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' +import { uploadCopilotFile } from '@/lib/uploads/contexts/copilot' +import { uploadExecutionFile } from '@/lib/uploads/contexts/execution' +import type { ExecutionContext } from '@/lib/uploads/contexts/execution/utils' +import { deleteFile } from '@/lib/uploads/core/storage-service' +import { deleteFileMetadata } from '@/lib/uploads/server/metadata' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' +import { resolveStoredFileMetadata } from '@/lib/uploads/utils/stored-file-metadata' +import type { UserFile } from '@/executor/types' + +const logger = createLogger('InternalToolFileResult') + +type FileStorageScope = + | { kind: 'execution'; context: ExecutionContext; userId?: string } + | { kind: 'copilot'; userId: string } + +interface CreatedFile { + key: string + context: FileStorageScope['kind'] +} + +function resolveCopilotUserId(context: InternalToolOperationContext): string { + const origin = context.executorDelegationOrigin + const principal = origin?.principal + const subject = principal ? resolvePrincipalSubject(principal) : null + if (principal && subject?.kind !== 'sim_user') { + throw new PrincipalSubjectUserRequiredError(principal.kind) + } + const userId = + subject?.kind === 'sim_user' ? subject.userId : (origin?.subjectUserId ?? context.userId) + if (!userId?.trim()) throw new Error('Authentication required') + if ( + (origin?.subjectUserId !== undefined && origin.subjectUserId !== userId) || + (context.userId !== undefined && context.userId !== userId) + ) { + throw new Error('Tool file owner does not match the authenticated subject') + } + return userId +} + +function resolveFileStorageScope(context: InternalToolOperationContext): FileStorageScope { + if (context.executionId) { + if (!context.workspaceId?.trim() || !context.workflowId.trim() || !context.executionId.trim()) { + throw new Error('Execution file output requires a complete trusted execution scope') + } + return { + kind: 'execution', + context: { + workspaceId: context.workspaceId, + workflowId: context.workflowId, + executionId: context.executionId, + }, + userId: context.userId, + } + } + return { kind: 'copilot', userId: resolveCopilotUserId(context) } +} + +function validateFiles(result: InternalToolFileResult): readonly InternalToolFile[] { + const files = [...new Set(result.files)] + let totalBytes = 0 + for (const file of files) { + if ( + !Buffer.isBuffer(file.buffer) || + typeof file.name !== 'string' || + !file.name.trim() || + typeof file.mimeType !== 'string' || + !file.mimeType.trim() + ) { + throw new Error('Tool file output requires a buffer, filename, and MIME type') + } + assertKnownSizeWithinLimit(file.buffer.length, MAX_BUFFERED_TRANSFER_BYTES, 'Tool output file') + totalBytes += file.buffer.length + assertKnownSizeWithinLimit(totalBytes, MAX_BUFFERED_TRANSFER_BYTES, 'Tool output files') + } + return files +} + +/** Rollback must finish even when the operation's signal is already aborted. */ +async function rollbackCreatedFiles(files: readonly CreatedFile[]): Promise { + for (const file of files) { + try { + await deleteFile(file) + await deleteFileMetadata(file.key) + } catch (error) { + logger.error('Failed to roll back an unpublished tool output file', { + key: file.key, + context: file.context, + error: getErrorMessage(error), + }) + } + } +} + +/** Stores file results and rolls back their objects if final presentation fails. */ +export async function storeInternalToolFileResult( + result: InternalToolFileResult, + context: InternalToolOperationContext, + finalize: (body: unknown) => T, + signal?: AbortSignal +): Promise { + signal?.throwIfAborted() + const files = validateFiles(result) + const scope = resolveFileStorageScope(context) + const createdFiles: CreatedFile[] = [] + const storedFiles = new Map() + + try { + for (const file of files) { + signal?.throwIfAborted() + const metadata = resolveStoredFileMetadata(file.name, file.mimeType, file.buffer) + const storedFile = + scope.kind === 'execution' + ? await uploadExecutionFile( + scope.context, + file.buffer, + metadata.fileName, + metadata.mimeType, + scope.userId + ) + : await uploadCopilotFile({ + buffer: file.buffer, + fileName: metadata.fileName, + contentType: metadata.mimeType, + userId: scope.userId, + }) + createdFiles.push({ key: storedFile.key, context: scope.kind }) + storedFiles.set(file, 'mimeType' in storedFile ? omit(storedFile, ['mimeType']) : storedFile) + signal?.throwIfAborted() + } + const presentedFiles = result.files.map((file) => { + const storedFile = storedFiles.get(file) + if (!storedFile) throw new Error('Tool output file was not stored') + return storedFile + }) + const output = await finalize(result.present(presentedFiles)) + signal?.throwIfAborted() + return output + } catch (error) { + await rollbackCreatedFiles(createdFiles) + signal?.throwIfAborted() + throw error + } +} + +/** Stores trusted in-process file results before their small JSON envelope crosses transport. */ +export async function presentInternalToolOperationResult( + result: InternalToolOperationResult, + context: InternalToolOperationContext, + signal?: AbortSignal +): Promise { + if (result instanceof Response) return result + return storeInternalToolFileResult( + result, + context, + (presented) => { + const body = JSON.stringify(presented) + if (body === undefined) throw new TypeError('Tool file result must be JSON serializable') + assertKnownSizeWithinLimit( + Buffer.byteLength(body, 'utf8'), + MAX_TOOL_RESPONSE_BODY_BYTES, + 'Tool response body' + ) + const headers = new Headers(result.init?.headers) + headers.delete('content-length') + headers.delete('content-encoding') + headers.set('content-type', 'application/json') + return new Response(body, { ...result.init, headers }) + }, + signal + ) +} diff --git a/apps/sim/lib/internal/tool-operations/file-result.ts b/apps/sim/lib/internal/tool-operations/file-result.ts new file mode 100644 index 00000000000..42bb5167891 --- /dev/null +++ b/apps/sim/lib/internal/tool-operations/file-result.ts @@ -0,0 +1,45 @@ +import type { UserFile } from '@/executor/types' + +/** Binary output kept in process until the executor persists it. */ +export interface InternalToolFile { + buffer: Buffer + name: string + mimeType: string +} + +/** The presenter receives stored descriptors, never inline file bytes. */ +export interface InternalToolFileResult { + kind: 'file-output' + files: readonly InternalToolFile[] + present: (files: readonly UserFile[]) => unknown + init?: ResponseInit +} + +export function createInternalToolFilesResult( + files: readonly InternalToolFile[], + present: InternalToolFileResult['present'], + init?: ResponseInit +): InternalToolFileResult { + return { kind: 'file-output', files, present, ...(init ? { init } : {}) } +} + +export function createInternalToolFileResult( + file: InternalToolFile, + present: (file: UserFile) => unknown, + init?: ResponseInit +): InternalToolFileResult { + return createInternalToolFilesResult([file], (files) => present(files[0]!), init) +} + +export function isInternalToolFileResult(value: unknown): value is InternalToolFileResult { + return ( + typeof value === 'object' && + value !== null && + 'kind' in value && + value.kind === 'file-output' && + 'files' in value && + Array.isArray(value.files) && + 'present' in value && + typeof value.present === 'function' + ) +} diff --git a/apps/sim/lib/internal/tool-operations/registry.server.ts b/apps/sim/lib/internal/tool-operations/registry.server.ts index bc8fcf7b7f0..ea568ff50dd 100644 --- a/apps/sim/lib/internal/tool-operations/registry.server.ts +++ b/apps/sim/lib/internal/tool-operations/registry.server.ts @@ -1,7 +1,12 @@ -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { isMcpTool } from '@/executor/constants' -type InternalToolOperationHandlerLoader = () => Promise +type InternalToolOperationHandlerLoader = () => Promise< + InternalToolOperationHandler +> const STS_TOOL_IDS = [ 'sts_assume_role', @@ -456,6 +461,7 @@ const JUPYTER_TOOL_IDS = [ 'jupyter_delete_content', 'jupyter_delete_session', 'jupyter_get_content', + 'jupyter_get_content_v2', 'jupyter_interrupt_kernel', 'jupyter_list_contents', 'jupyter_list_kernels', @@ -730,6 +736,7 @@ const OUTLOOK_TOOL_IDS = [ 'outlook_copy', 'outlook_delete', 'outlook_draft', + 'outlook_get_attachment', 'outlook_mark_read', 'outlook_mark_unread', 'outlook_move', @@ -742,6 +749,7 @@ const SSH_TOOL_IDS = [ 'ssh_create_directory', 'ssh_delete_file', 'ssh_download_file', + 'ssh_download_file_v2', 'ssh_execute_command', 'ssh_execute_script', 'ssh_get_system_info', @@ -1012,6 +1020,7 @@ const CURSOR_TOOL_IDS = ['cursor_download_artifact', 'cursor_download_artifact_v const SFTP_TOOL_IDS = [ 'sftp_delete', 'sftp_download', + 'sftp_download_v2', 'sftp_list', 'sftp_mkdir', 'sftp_upload', @@ -1069,7 +1078,12 @@ const PERSONA_TOOL_IDS = ['persona_import_accounts'] as const const SHAREPOINT_TOOL_IDS = ['sharepoint_download_file', 'sharepoint_upload_file'] as const -const QUIVER_TOOL_IDS = ['quiver_text_to_svg', 'quiver_image_to_svg'] as const +const QUIVER_TOOL_IDS = [ + 'quiver_text_to_svg', + 'quiver_image_to_svg', + 'quiver_text_to_svg_v2', + 'quiver_image_to_svg_v2', +] as const const TELEGRAM_TOOL_IDS = ['telegram_send_document'] as const @@ -1787,7 +1801,7 @@ export function getRegisteredInternalToolOperationIds(): string[] { export async function getInternalToolOperationHandler( toolId: string -): Promise { +): Promise | null> { const loader = handlerLoaders.get(toolId) if (loader) return loader() if (isMcpTool(toolId)) { diff --git a/apps/sim/lib/internal/tool-operations/response-limits.ts b/apps/sim/lib/internal/tool-operations/response-limits.ts new file mode 100644 index 00000000000..fb8828b4ef1 --- /dev/null +++ b/apps/sim/lib/internal/tool-operations/response-limits.ts @@ -0,0 +1,2 @@ +/** Maximum inline tool response size; binary file outputs use stored descriptors. */ +export const MAX_TOOL_RESPONSE_BODY_BYTES = 10 * 1024 * 1024 diff --git a/apps/sim/lib/internal/tool-operations/types.ts b/apps/sim/lib/internal/tool-operations/types.ts index 2e1f52d3584..803c299741e 100644 --- a/apps/sim/lib/internal/tool-operations/types.ts +++ b/apps/sim/lib/internal/tool-operations/types.ts @@ -1,4 +1,5 @@ import type { BillingAttributionSnapshot } from '@/lib/billing/core/billing-attribution' +import type { InternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import type { ExecutorDelegationOrigin } from '@/executor/types' import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' import type { ToolResponse } from '@/tools/types' @@ -42,4 +43,8 @@ export interface InternalToolOperationCall { signal?: AbortSignal } -export type InternalToolOperationHandler = (request: InternalToolOperationCall) => Promise +export type InternalToolOperationResult = Response | InternalToolFileResult + +export type InternalToolOperationHandler = ( + request: InternalToolOperationCall +) => Promise diff --git a/apps/sim/lib/internal/twilio-voice/execute-tool.ts b/apps/sim/lib/internal/twilio-voice/execute-tool.ts index 8b9a555b9f4..840eedfea9a 100644 --- a/apps/sim/lib/internal/twilio-voice/execute-tool.ts +++ b/apps/sim/lib/internal/twilio-voice/execute-tool.ts @@ -1,7 +1,11 @@ import { getErrorMessage } from '@sim/utils/errors' import { z } from 'zod' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { TwilioVoiceOperationError } from '@/lib/internal/twilio-voice/errors' import { getTwilioRecording } from '@/lib/internal/twilio-voice/operations' @@ -11,7 +15,9 @@ const inputSchema = z.object({ recordingSid: z.string().min(1, 'Recording SID is required'), }) -export const executeTwilioVoiceTool: InternalToolOperationHandler = async (request) => { +export const executeTwilioVoiceTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (request.toolId !== 'twilio_voice_get_recording') { return Response.json( @@ -24,12 +30,11 @@ export const executeTwilioVoiceTool: InternalToolOperationHandler = async (reque return Response.json({ success: false, error: 'Invalid request data' }, { status: 400 }) } try { - return Response.json( - await getTwilioRecording(parsed.data, { - requestId: request.requestId, - signal: request.signal, - }) - ) + const result = await getTwilioRecording(parsed.data, { + requestId: request.requestId, + signal: request.signal, + }) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() const status = isPayloadSizeLimitError(error) diff --git a/apps/sim/lib/internal/twilio-voice/operations.test.ts b/apps/sim/lib/internal/twilio-voice/operations.test.ts index dd952e46dbb..8420edeb5cc 100644 --- a/apps/sim/lib/internal/twilio-voice/operations.test.ts +++ b/apps/sim/lib/internal/twilio-voice/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ secureFetchWithPinnedIP: vi.fn(), @@ -56,12 +61,23 @@ describe('getTwilioRecording', () => { signal: controller.signal, }) ) - expect(result.output).toEqual( - expect.objectContaining({ - duration: 42, - transcriptionText: 'hello', - file: expect.objectContaining({ name: 'RE123.mp3', data: 'AQID', size: 3 }), - }) - ) + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'RE123.mp3', mimeType: 'audio/mpeg', buffer: Buffer.from([1, 2, 3]) }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'RE123.mp3', + type: 'audio/mpeg', + mimeType: 'audio/mpeg', + size: 3, + context: 'execution', + } + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { duration: 42, transcriptionText: 'hello', file: storedFile }, + }) }) }) diff --git a/apps/sim/lib/internal/twilio-voice/operations.ts b/apps/sim/lib/internal/twilio-voice/operations.ts index 7acc77c44e8..bc551fbe692 100644 --- a/apps/sim/lib/internal/twilio-voice/operations.ts +++ b/apps/sim/lib/internal/twilio-voice/operations.ts @@ -8,10 +8,14 @@ import { readResponseJsonWithLimit, readResponseToBufferWithLimit, } from '@/lib/core/utils/stream-limits' +import { + createInternalToolFileResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' import { TwilioVoiceOperationError } from '@/lib/internal/twilio-voice/errors' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { getExtensionFromMimeType } from '@/lib/uploads/utils/file-utils' -import type { TwilioGetRecordingOutput, TwilioGetRecordingParams } from '@/tools/twilio_voice/types' +import type { TwilioGetRecordingParams } from '@/tools/twilio_voice/types' const logger = createLogger('TwilioGetRecordingOperation') const MAX_TWILIO_JSON_BYTES = 2 * 1024 * 1024 @@ -67,7 +71,7 @@ async function fetchPinned( export async function getTwilioRecording( input: TwilioGetRecordingParams, context: TwilioVoiceOperationContext -): Promise { +) { context.signal?.throwIfAborted() if (!input.accountSid.startsWith('AC')) { throw new TwilioVoiceOperationError( @@ -131,7 +135,7 @@ export async function getTwilioRecording( logger.warn('Failed to fetch Twilio transcription', { requestId: context.requestId, error }) } - let file: TwilioGetRecordingOutput['output']['file'] + let file: InternalToolFile | undefined if (mediaUrl) { try { const response = await fetchPinned( @@ -151,8 +155,7 @@ export async function getTwilioRecording( file = { name: `${data.sid || input.recordingSid}.${getExtensionFromMimeType(mimeType) || 'dat'}`, mimeType, - data: buffer.toString('base64'), - size: buffer.length, + buffer, } } } catch (error) { @@ -164,25 +167,26 @@ export async function getTwilioRecording( } } - return { + const output = { success: true, - output: { - success: true, - recordingSid: data.sid, - callSid: data.call_sid, - duration: data.duration ? Number.parseInt(data.duration, 10) : undefined, - status: data.status, - channels: data.channels, - source: data.source, - mediaUrl, - file, - price: data.price, - priceUnit: data.price_unit, - uri: data.uri, - transcriptionText: transcription?.transcription_text, - transcriptionStatus: transcription?.status, - transcriptionPrice: transcription?.price, - transcriptionPriceUnit: transcription?.price_unit, - }, + recordingSid: data.sid, + callSid: data.call_sid, + duration: data.duration ? Number.parseInt(data.duration, 10) : undefined, + status: data.status, + channels: data.channels, + source: data.source, + mediaUrl, + price: data.price, + priceUnit: data.price_unit, + uri: data.uri, + transcriptionText: transcription?.transcription_text, + transcriptionStatus: transcription?.status, + transcriptionPrice: transcription?.price, + transcriptionPriceUnit: transcription?.price_unit, } + if (!file) return { success: true, output } + return createInternalToolFileResult(file, (storedFile) => ({ + success: true, + output: { ...output, file: storedFile }, + })) } diff --git a/apps/sim/lib/internal/vanta/execute-tool.ts b/apps/sim/lib/internal/vanta/execute-tool.ts index b510d7453ae..26181300c4b 100644 --- a/apps/sim/lib/internal/vanta/execute-tool.ts +++ b/apps/sim/lib/internal/vanta/execute-tool.ts @@ -1,5 +1,9 @@ import { getErrorMessage } from '@sim/utils/errors' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { VantaOperationError } from '@/lib/internal/vanta/errors' import { vantaDownloadDocumentFileInputSchema, @@ -13,7 +17,9 @@ import { import { vantaQueryBodySchema } from '@/lib/internal/vanta/schema' /** Executes the Vanta tool family without a same-origin HTTP hop. */ -export const executeVantaTool: InternalToolOperationHandler = async (request) => { +export const executeVantaTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() const schema = request.toolId === 'vanta_upload_document_file' @@ -47,7 +53,7 @@ export const executeVantaTool: InternalToolOperationHandler = async (request) => context ) request.signal?.throwIfAborted() - return Response.json(result) + return isInternalToolFileResult(result) ? result : Response.json(result) } const query = vantaQueryBodySchema.parse(parsed.data) diff --git a/apps/sim/lib/internal/vanta/operations.test.ts b/apps/sim/lib/internal/vanta/operations.test.ts index 732f6561972..df66cd149cc 100644 --- a/apps/sim/lib/internal/vanta/operations.test.ts +++ b/apps/sim/lib/internal/vanta/operations.test.ts @@ -1,7 +1,12 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' const mocks = vi.hoisted(() => ({ fetchAuth: vi.fn(), @@ -98,19 +103,23 @@ describe('Vanta operations', () => { ) expect(mocks.fetchAuth.mock.calls[0]?.[2]).toEqual({ signal: context.signal }) - expect(result).toEqual({ + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'report final.pdf', mimeType: 'application/pdf', buffer: Buffer.from('hello') }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'report final.pdf', + type: 'application/pdf', + mimeType: 'application/pdf', + size: 5, + context: 'execution', + } + expect(result.present([storedFile])).toEqual({ success: true, - output: { - file: { - name: 'report final.pdf', - mimeType: 'application/pdf', - data: Buffer.from('hello').toString('base64'), - size: 5, - }, - name: 'report final.pdf', - mimeType: 'application/pdf', - size: 5, - }, + output: { file: storedFile, name: 'report final.pdf', mimeType: 'application/pdf', size: 5 }, }) }) diff --git a/apps/sim/lib/internal/vanta/operations.ts b/apps/sim/lib/internal/vanta/operations.ts index ced9bece5cd..c649d9b53af 100644 --- a/apps/sim/lib/internal/vanta/operations.ts +++ b/apps/sim/lib/internal/vanta/operations.ts @@ -4,6 +4,10 @@ import { readResponseJsonWithLimit, readResponseToBufferWithLimit, } from '@/lib/core/utils/stream-limits' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { fetchVantaWithAuth, getVantaBaseUrl, @@ -511,7 +515,7 @@ export async function executeVantaUploadDocumentFile( export async function executeVantaDownloadDocumentFile( input: VantaDownloadDocumentFileInput, context: VantaFileOperationContext -): Promise> { +): Promise { context.signal?.throwIfAborted() const mediaUrl = buildVantaUrl( getVantaBaseUrl(input.region), @@ -565,13 +569,8 @@ export async function executeVantaDownloadDocumentFile( const name = fileNameFromContentDisposition(response.headers.get('content-disposition')) || `vanta-document-file-${input.uploadedFileId}` - return { + return createInternalToolFileResult({ buffer, name, mimeType }, (file) => ({ success: true, - output: { - file: { name, mimeType, data: buffer.toString('base64'), size: buffer.length }, - name, - mimeType, - size: buffer.length, - }, - } + output: { file, name, mimeType, size: file.size }, + })) } diff --git a/apps/sim/lib/internal/zoho-desk/execute-tool.test.ts b/apps/sim/lib/internal/zoho-desk/execute-tool.test.ts index 907d649af75..126da373975 100644 --- a/apps/sim/lib/internal/zoho-desk/execute-tool.test.ts +++ b/apps/sim/lib/internal/zoho-desk/execute-tool.test.ts @@ -3,12 +3,14 @@ */ import { createExecutionContext } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { createInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' const mocks = vi.hoisted(() => ({ getZohoDeskAttachment: vi.fn() })) vi.mock('@/lib/internal/zoho-desk/operations', () => ({ getZohoDeskAttachment: mocks.getZohoDeskAttachment, - MAX_ZOHO_DESK_ATTACHMENT_BYTES: 7 * 1024 * 1024, })) import type { InternalToolOperationCall } from '@/lib/internal/tool-operations/types' @@ -33,28 +35,46 @@ function request(overrides: Partial = {}): InternalTo describe('executeZohoDeskTool', () => { beforeEach(() => { vi.clearAllMocks() - mocks.getZohoDeskAttachment.mockResolvedValue({ - success: true, - output: { file: { name: 'file.pdf', mimeType: 'application/pdf', data: 'YQ==' } }, - }) }) it('dispatches the typed operation with cancellation', async () => { const controller = new AbortController() - const response = await executeZohoDeskTool(request({ signal: controller.signal })) + const result = createInternalToolFileResult( + { buffer: Buffer.alloc(12 * 1024 * 1024), name: 'file.pdf', mimeType: 'application/pdf' }, + (file) => ({ success: true, output: { file } }) + ) + mocks.getZohoDeskAttachment.mockResolvedValue(result) - expect(response.status).toBe(200) + expect(await executeZohoDeskTool(request({ signal: controller.signal }))).toBe(result) expect(mocks.getZohoDeskAttachment).toHaveBeenCalledWith( expect.objectContaining({ orgId: 'org-1' }), { signal: controller.signal } ) }) + it('projects the buffered file limit as 413', async () => { + mocks.getZohoDeskAttachment.mockRejectedValue( + new PayloadSizeLimitError({ + label: 'Zoho Desk attachment', + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + observedBytes: MAX_BUFFERED_TRANSFER_BYTES + 1, + }) + ) + const response = await executeZohoDeskTool(request()) + if (!(response instanceof Response)) throw new Error('Expected an error response') + expect(response.status).toBe(413) + await expect(response.json()).resolves.toEqual({ + success: false, + error: 'Attachment exceeds the 100 MB download limit', + }) + }) + it('preserves operation status', async () => { mocks.getZohoDeskAttachment.mockRejectedValue( new ZohoDeskOperationError('Invalid attachment href', 400) ) const response = await executeZohoDeskTool(request()) + if (!(response instanceof Response)) throw new Error('Expected an error response') expect(response.status).toBe(400) }) }) diff --git a/apps/sim/lib/internal/zoho-desk/execute-tool.ts b/apps/sim/lib/internal/zoho-desk/execute-tool.ts index 0af245bbe06..1a99f24ba21 100644 --- a/apps/sim/lib/internal/zoho-desk/execute-tool.ts +++ b/apps/sim/lib/internal/zoho-desk/execute-tool.ts @@ -1,12 +1,13 @@ import { getErrorMessage } from '@sim/utils/errors' import { z } from 'zod' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { ZohoDeskOperationError } from '@/lib/internal/zoho-desk/errors' -import { - getZohoDeskAttachment, - MAX_ZOHO_DESK_ATTACHMENT_BYTES, -} from '@/lib/internal/zoho-desk/operations' +import { getZohoDeskAttachment } from '@/lib/internal/zoho-desk/operations' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' const inputSchema = z.object({ accessToken: z.string().min(1), @@ -16,7 +17,9 @@ const inputSchema = z.object({ fileName: z.string().optional(), }) -export const executeZohoDeskTool: InternalToolOperationHandler = async (request) => { +export const executeZohoDeskTool: InternalToolOperationHandler< + InternalToolOperationResult +> = async (request) => { request.signal?.throwIfAborted() if (request.toolId !== 'zoho_desk_get_attachment') { return Response.json( @@ -29,18 +32,14 @@ export const executeZohoDeskTool: InternalToolOperationHandler = async (request) return Response.json({ success: false, error: 'Invalid request data' }, { status: 400 }) } try { - return Response.json( - await getZohoDeskAttachment(parsed.data, { - signal: request.signal, - }) - ) + return await getZohoDeskAttachment(parsed.data, { signal: request.signal }) } catch (error) { request.signal?.throwIfAborted() if (isPayloadSizeLimitError(error)) { return Response.json( { success: false, - error: `Attachment exceeds the ${Math.floor(MAX_ZOHO_DESK_ATTACHMENT_BYTES / (1024 * 1024))} MB download limit`, + error: `Attachment exceeds the ${Math.floor(MAX_BUFFERED_TRANSFER_BYTES / (1024 * 1024))} MB download limit`, }, { status: 413 } ) diff --git a/apps/sim/lib/internal/zoho-desk/operations.test.ts b/apps/sim/lib/internal/zoho-desk/operations.test.ts new file mode 100644 index 00000000000..5283ee5c1e6 --- /dev/null +++ b/apps/sim/lib/internal/zoho-desk/operations.test.ts @@ -0,0 +1,146 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' + +const mocks = vi.hoisted(() => ({ secureFetchWithValidation: vi.fn() })) + +vi.mock('@/lib/core/security/input-validation.server', () => ({ + secureFetchWithValidation: mocks.secureFetchWithValidation, +})) + +import { getZohoDeskAttachment } from '@/lib/internal/zoho-desk/operations' + +const input = { + accessToken: 'token', + orgId: 'org-1', + href: '/api/v1/tickets/1/attachments/2/content', + apiDomain: 'https://desk.zoho.eu', +} + +describe('getZohoDeskAttachment', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('returns a 12 MiB attachment for central storage while preserving download guards', async () => { + const buffer = Buffer.alloc(12 * 1024 * 1024, 42) + const controller = new AbortController() + mocks.secureFetchWithValidation.mockResolvedValue( + new Response(new Uint8Array(buffer), { + headers: { + 'content-type': 'application/pdf', + 'content-disposition': "attachment; filename*=UTF-8''ticket%20attachment.pdf", + }, + }) + ) + + const result = await getZohoDeskAttachment(input, { signal: controller.signal }) + + expect(mocks.secureFetchWithValidation).toHaveBeenCalledWith( + 'https://desk.zoho.eu/api/v1/tickets/1/attachments/2/content', + { + profile: 'contentFetch', + method: 'GET', + headers: { + Authorization: 'Zoho-oauthtoken token', + orgId: 'org-1', + 'Content-Type': 'application/json', + }, + timeout: 30_000, + maxResponseBytes: MAX_BUFFERED_TRANSFER_BYTES, + stripAuthOnRedirect: true, + signal: controller.signal, + } + ) + expect(result.files).toHaveLength(1) + expect(result.files[0]?.name).toBe('ticket attachment.pdf') + expect(result.files[0]?.mimeType).toBe('application/pdf') + expect(result.files[0]?.buffer.equals(buffer)).toBe(true) + const file = { + id: 'file-1', + name: 'ticket attachment.pdf', + key: 'execution/workspace/workflow/run/file.pdf', + url: '/api/files/file-1', + type: 'application/pdf', + mimeType: 'application/pdf', + size: buffer.length, + context: 'execution', + } + const presented = result.present([file]) + expect(presented).toEqual({ success: true, output: { file } }) + expect(presented).not.toHaveProperty('output.file.data') + expect(JSON.stringify(presented).length).toBeLessThan(1024) + }) + + it('rejects a declared attachment size above the buffered transfer limit', async () => { + mocks.secureFetchWithValidation.mockResolvedValue( + new Response(new Uint8Array(), { + headers: { 'content-length': String(MAX_BUFFERED_TRANSFER_BYTES + 1) }, + }) + ) + + await expect(getZohoDeskAttachment(input, {})).rejects.toMatchObject({ + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + observedBytes: MAX_BUFFERED_TRANSFER_BYTES + 1, + }) + }) + + it('rejects an oversized stream even without a content-length header', async () => { + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(MAX_BUFFERED_TRANSFER_BYTES + 1)) + controller.close() + }, + }) + mocks.secureFetchWithValidation.mockResolvedValue(new Response(body)) + + await expect(getZohoDeskAttachment(input, {})).rejects.toMatchObject({ + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + observedBytes: MAX_BUFFERED_TRANSFER_BYTES + 1, + }) + }) + + it('retains filename overrides and the binary MIME fallback for empty files', async () => { + mocks.secureFetchWithValidation.mockResolvedValue(new Response(new Uint8Array())) + + const result = await getZohoDeskAttachment({ ...input, fileName: ' empty.bin ' }, {}) + + expect(result.files[0]?.name).toBe('empty.bin') + expect(result.files[0]?.mimeType).toBe('application/octet-stream') + expect(result.files[0]?.buffer.length).toBe(0) + }) + + it.each(['https://desk.zoho.com.attacker.example/attachment', 'http://desk.zoho.com/attachment'])( + 'rejects untrusted attachment URL %s before sending credentials', + async (href) => { + await expect(getZohoDeskAttachment({ ...input, href }, {})).rejects.toMatchObject({ + status: 400, + }) + expect(mocks.secureFetchWithValidation).not.toHaveBeenCalled() + } + ) + + it.each([ + [403, 403], + [500, 502], + [204, 502], + ])('preserves provider HTTP %i as operation status %i', async (status, expectedStatus) => { + mocks.secureFetchWithValidation.mockResolvedValue(new Response(null, { status })) + + await expect(getZohoDeskAttachment(input, {})).rejects.toMatchObject({ + status: expectedStatus, + }) + }) + + it('does not start a download after cancellation', async () => { + const controller = new AbortController() + controller.abort(new Error('Execution cancelled')) + + await expect(getZohoDeskAttachment(input, { signal: controller.signal })).rejects.toThrow( + 'Execution cancelled' + ) + expect(mocks.secureFetchWithValidation).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/internal/zoho-desk/operations.ts b/apps/sim/lib/internal/zoho-desk/operations.ts index 8cb3f45a30e..c225cb87ea0 100644 --- a/apps/sim/lib/internal/zoho-desk/operations.ts +++ b/apps/sim/lib/internal/zoho-desk/operations.ts @@ -1,5 +1,11 @@ import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server' +import { readResponseToBufferWithLimit } from '@/lib/core/utils/stream-limits' +import { + createInternalToolFileResult, + type InternalToolFileResult, +} from '@/lib/internal/tool-operations/file-result' import { ZohoDeskOperationError } from '@/lib/internal/zoho-desk/errors' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { isZohoHost } from '@/tools/zoho_desk/host-allowlist' import type { ZohoDeskGetAttachmentParams } from '@/tools/zoho_desk/types' import { @@ -9,8 +15,6 @@ import { resolveZohoAttachmentUrl, } from '@/tools/zoho_desk/utils' -export const MAX_ZOHO_DESK_ATTACHMENT_BYTES = 7 * 1024 * 1024 - export interface ZohoDeskOperationContext { signal?: AbortSignal } @@ -18,10 +22,7 @@ export interface ZohoDeskOperationContext { export async function getZohoDeskAttachment( input: ZohoDeskGetAttachmentParams, context: ZohoDeskOperationContext -): Promise<{ - success: true - output: { file: { data: string; mimeType: string; name: string } } -}> { +): Promise { context.signal?.throwIfAborted() let downloadUrl: URL try { @@ -41,7 +42,7 @@ export async function getZohoDeskAttachment( method: 'GET', headers: buildZohoDeskHeaders({ accessToken: input.accessToken, orgId: input.orgId }), timeout: 30_000, - maxResponseBytes: MAX_ZOHO_DESK_ATTACHMENT_BYTES, + maxResponseBytes: MAX_BUFFERED_TRANSFER_BYTES, stripAuthOnRedirect: true, signal: context.signal, }) @@ -57,20 +58,22 @@ export async function getZohoDeskAttachment( 502 ) } - const buffer = Buffer.from(await response.arrayBuffer()) + const buffer = await readResponseToBufferWithLimit(response, { + maxBytes: MAX_BUFFERED_TRANSFER_BYTES, + label: 'Zoho Desk attachment', + signal: context.signal, + }) context.signal?.throwIfAborted() - return { - success: true, - output: { - file: { - data: buffer.toString('base64'), - mimeType: response.headers.get('content-type') || 'application/octet-stream', - name: deriveAttachmentName( - input.fileName, - response.headers.get('content-disposition'), - downloadUrl.pathname - ), - }, + return createInternalToolFileResult( + { + buffer, + mimeType: response.headers.get('content-type') || 'application/octet-stream', + name: deriveAttachmentName( + input.fileName, + response.headers.get('content-disposition'), + downloadUrl.pathname + ), }, - } + (file) => ({ success: true, output: { file } }) + ) } diff --git a/apps/sim/lib/internal/zoom/execute-tool.ts b/apps/sim/lib/internal/zoom/execute-tool.ts index 803acb22ac3..49e56cc9a72 100644 --- a/apps/sim/lib/internal/zoom/execute-tool.ts +++ b/apps/sim/lib/internal/zoom/execute-tool.ts @@ -1,6 +1,10 @@ import { getErrorMessage } from '@sim/utils/errors' import { z } from 'zod' -import type { InternalToolOperationHandler } from '@/lib/internal/tool-operations/types' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' +import type { + InternalToolOperationHandler, + InternalToolOperationResult, +} from '@/lib/internal/tool-operations/types' import { ZoomOperationError } from '@/lib/internal/zoom/errors' import { getZoomMeetingRecordings } from '@/lib/internal/zoom/operations' @@ -12,7 +16,9 @@ const inputSchema = z.object({ downloadFiles: z.boolean().default(false), }) -export const executeZoomTool: InternalToolOperationHandler = async (request) => { +export const executeZoomTool: InternalToolOperationHandler = async ( + request +) => { request.signal?.throwIfAborted() if (request.toolId !== 'zoom_get_meeting_recordings') { return Response.json( @@ -25,12 +31,11 @@ export const executeZoomTool: InternalToolOperationHandler = async (request) => return Response.json({ success: false, error: 'Invalid request data' }, { status: 400 }) } try { - return Response.json( - await getZoomMeetingRecordings(parsed.data, { - requestId: request.requestId, - signal: request.signal, - }) - ) + const result = await getZoomMeetingRecordings(parsed.data, { + requestId: request.requestId, + signal: request.signal, + }) + return isInternalToolFileResult(result) ? result : Response.json(result) } catch (error) { request.signal?.throwIfAborted() if (error instanceof ZoomOperationError) { diff --git a/apps/sim/lib/internal/zoom/operations.test.ts b/apps/sim/lib/internal/zoom/operations.test.ts index d21429d74d8..f56d9a69a94 100644 --- a/apps/sim/lib/internal/zoom/operations.test.ts +++ b/apps/sim/lib/internal/zoom/operations.test.ts @@ -1,7 +1,7 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' const mocks = vi.hoisted(() => ({ @@ -16,6 +16,10 @@ vi.mock('@/lib/core/security/input-validation.server', () => ({ vi.mock('@/lib/uploads/shared/types', () => ({ MAX_BUFFERED_TRANSFER_BYTES: 5 })) +import { + isInternalToolFileResult, + type StoredToolFile, +} from '@/lib/internal/tool-operations/file-result' import { getZoomMeetingRecordings } from '@/lib/internal/zoom/operations' describe('getZoomMeetingRecordings', () => { @@ -24,6 +28,44 @@ describe('getZoomMeetingRecordings', () => { mocks.validateUrlWithDNS.mockResolvedValue({ isValid: true, resolvedIP: '203.0.113.1' }) }) + it('returns stored recording references with the original recording metadata', async () => { + mocks.secureFetchWithPinnedIP + .mockResolvedValueOnce( + Response.json({ + id: 'meeting-1', + recording_files: [{ id: 'one', download_url: 'https://files.example/one' }], + }) + ) + .mockResolvedValueOnce(new Response('one', { headers: { 'content-type': 'video/mp4' } })) + + const result = await getZoomMeetingRecordings( + { accessToken: 'token', meetingId: 'meeting-1', downloadFiles: true }, + { requestId: 'request-1' } + ) + + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'zoom-recording-one.mp4', mimeType: 'video/mp4', buffer: Buffer.from('one') }, + ]) + const storedFile: StoredToolFile = { + id: 'stored-file-1', + key: 'execution/stored-file-1', + url: '/api/files/serve/stored-file-1', + name: 'zoom-recording-one.mp4', + type: 'video/mp4', + mimeType: 'video/mp4', + size: 3, + context: 'execution', + } + expect(result.present([storedFile])).toMatchObject({ + success: true, + output: { + recording: { id: 'meeting-1', recording_files: [{ id: 'one' }] }, + files: [storedFile], + }, + }) + }) + it('downloads sequentially and rejects cumulative recording bytes', async () => { mocks.secureFetchWithPinnedIP .mockResolvedValueOnce( diff --git a/apps/sim/lib/internal/zoom/operations.ts b/apps/sim/lib/internal/zoom/operations.ts index 1860b06e667..72be9339db7 100644 --- a/apps/sim/lib/internal/zoom/operations.ts +++ b/apps/sim/lib/internal/zoom/operations.ts @@ -4,6 +4,10 @@ import { validateUrlWithDNS, } from '@/lib/core/security/input-validation.server' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { + createInternalToolFilesResult, + type InternalToolFile, +} from '@/lib/internal/tool-operations/file-result' import { ZoomOperationError } from '@/lib/internal/zoom/errors' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' import { getExtensionFromMimeType } from '@/lib/uploads/utils/file-utils' @@ -52,13 +56,7 @@ export interface ZoomOperationContext { export async function getZoomMeetingRecordings( input: ZoomGetMeetingRecordingsParams, context: ZoomOperationContext -): Promise<{ - success: true - output: { - recording: ZoomRecordingsResponse & { recording_files: ZoomRecordingFile[] } - files?: Array<{ name: string; mimeType: string; data: string; size: number }> - } -}> { +) { context.signal?.throwIfAborted() const query = new URLSearchParams() if (input.includeFolderItems != null) { @@ -87,7 +85,7 @@ export async function getZoomMeetingRecordings( throw new ZoomOperationError(errorData.message || `Zoom API error: ${response.status}`, 400) } const data = (await response.json()) as ZoomRecordingsResponse - const files: Array<{ name: string; mimeType: string; data: string; size: number }> = [] + const files: InternalToolFile[] = [] let bufferedBytes = 0 if (input.downloadFiles && Array.isArray(data.recording_files)) { @@ -134,8 +132,7 @@ export async function getZoomMeetingRecordings( files.push({ name: `zoom-recording-${file.id || file.recording_start || Date.now()}.${extension}`, mimeType, - data: buffer.toString('base64'), - size: buffer.length, + buffer, }) } catch (error) { context.signal?.throwIfAborted() @@ -153,36 +150,35 @@ export async function getZoomMeetingRecordings( } } - return { - success: true, - output: { - recording: { - uuid: data.uuid, - id: data.id, - account_id: data.account_id, - host_id: data.host_id, - topic: data.topic, - type: data.type, - start_time: data.start_time, - duration: data.duration, - total_size: data.total_size, - recording_count: data.recording_count, - share_url: data.share_url, - recording_files: (data.recording_files || []).map((file) => ({ - id: file.id, - meeting_id: file.meeting_id, - recording_start: file.recording_start, - recording_end: file.recording_end, - file_type: file.file_type, - file_extension: file.file_extension, - file_size: file.file_size, - play_url: file.play_url, - download_url: file.download_url, - status: file.status, - recording_type: file.recording_type, - })), - }, - files: files.length > 0 ? files : undefined, - }, + const recording = { + uuid: data.uuid, + id: data.id, + account_id: data.account_id, + host_id: data.host_id, + topic: data.topic, + type: data.type, + start_time: data.start_time, + duration: data.duration, + total_size: data.total_size, + recording_count: data.recording_count, + share_url: data.share_url, + recording_files: (data.recording_files || []).map((file) => ({ + id: file.id, + meeting_id: file.meeting_id, + recording_start: file.recording_start, + recording_end: file.recording_end, + file_type: file.file_type, + file_extension: file.file_extension, + file_size: file.file_size, + play_url: file.play_url, + download_url: file.download_url, + status: file.status, + recording_type: file.recording_type, + })), } + if (files.length === 0) return { success: true, output: { recording } } + return createInternalToolFilesResult(files, (storedFiles) => ({ + success: true, + output: { recording, files: storedFiles }, + })) } diff --git a/apps/sim/lib/knowledge/__integration__/application-acl.integration.ts b/apps/sim/lib/knowledge/__integration__/application-acl.integration.ts index 48d30b587e3..209f2142ebb 100644 --- a/apps/sim/lib/knowledge/__integration__/application-acl.integration.ts +++ b/apps/sim/lib/knowledge/__integration__/application-acl.integration.ts @@ -62,7 +62,10 @@ import { seedKnowledgeMemberFixture, } from '@/lib/knowledge/__integration__/seed-source-access-fixture' import { confluencePageAcl } from '@/lib/knowledge/access/confluence-permissions' -import { knowledgeAccessCondition } from '@/lib/knowledge/access/predicate' +import { + knowledgeAccessCondition, + knowledgeMetadataCandidateAccessCondition, +} from '@/lib/knowledge/access/predicate' import { createKnowledgeAccessProvider } from '@/lib/knowledge/access/scope' import { listKnowledgeChunks } from '@/lib/knowledge/application/chunks' import { readKnowledgeDocument } from '@/lib/knowledge/application/documents' @@ -299,9 +302,16 @@ describe('indexed source content through real application access', () => { return result.results.map((row) => row.documentId) } - it.each(['workspace', 'admin', 'members'] as const)( - 'allows a remaining workspace ACL only in workspace mode, not during a %s transition', - async (accessMode) => { + it.each([ + ['workspace', true], + ['workspace', false], + ['admin', true], + ['admin', false], + ['members', true], + ['members', false], + ] as const)( + 'requires settled workspace mode for a remaining workspace ACL (%s, rewrite pending=%s)', + async (accessMode, accessRewritePending) => { const [savedConnector] = await db .select({ accessMode: knowledgeConnector.accessMode, @@ -324,18 +334,25 @@ describe('indexed source content through real application access', () => { .where(eq(document.id, documentId)) await db .update(knowledgeConnector) - .set({ accessMode, accessRewritePending: true }) + .set({ accessMode, accessRewritePending }) .where(eq(knowledgeConnector.id, connectorId)) - const visible = await db - .select({ id: document.id }) - .from(document) - .where( - and( - eq(document.id, documentId), - knowledgeAccessCondition({ kind: 'workspace', tokens: ['pub', 'ws'] }) + for (const accessCondition of [ + knowledgeMetadataCandidateAccessCondition, + knowledgeAccessCondition, + ]) { + const visible = await db + .select({ id: document.id }) + .from(document) + .where( + and( + eq(document.id, documentId), + accessCondition({ kind: 'workspace', tokens: ['pub', 'ws'] }) + ) ) + expect(visible.map((row) => row.id)).toEqual( + accessMode === 'workspace' && !accessRewritePending ? [documentId] : [] ) - expect(visible.map((row) => row.id)).toEqual(accessMode === 'workspace' ? [documentId] : []) + } } finally { await db .update(knowledgeConnector) diff --git a/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts b/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts new file mode 100644 index 00000000000..6cbe10aeed6 --- /dev/null +++ b/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts @@ -0,0 +1,491 @@ +/** Real execution-file storage, ZIP extraction, durable provenance, table import, and KB indexing. */ +import { mkdtempSync } from 'node:fs' +import { rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import path from 'node:path' +import type { DelegatedPrincipal } from '@sim/auth/principal' +import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' +import { + document, + documentSecretProvenance, + knowledgeBase, + organization, + outboxEvent, + user, + userTableRowSecretProvenance, + userTableRows, + workspace, + workspaceFileColumns, + workspaceFiles, +} from '@sim/db/schema' +import { generateId } from '@sim/utils/id' +import { eq, inArray, sql } from 'drizzle-orm' +import JSZip from 'jszip' +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest' + +const fixtureStorage = vi.hoisted(() => ({ root: '' })) +vi.mock('@/lib/uploads/core/setup.server', () => ({ + get UPLOAD_DIR_SERVER() { + return fixtureStorage.root + }, +})) +vi.mock('@/lib/embeddings', async () => ({ + ...(await import('@/lib/embeddings/client')), + assertKnowledgeEmbeddingCapacity: async () => {}, + embedKnowledge: async (texts: string[]) => ({ + embeddings: texts.map(() => [1, ...Array(1535).fill(0)]), + totalTokens: texts.length, + billableTokens: 0, + isBYOK: true, + modelName: 'text-embedding-3-small', + pricingId: 'text-embedding-3-small', + }), +})) + +import { fileManageDecompressBodySchema } from '@/lib/api/contracts/tools/file' +import { processOutboxEventById } from '@/lib/core/outbox/service' +import { encryptSecret } from '@/lib/core/security/encryption' +import { isUserFile } from '@/lib/core/utils/user-file' +import { executeFileManageOperation } from '@/lib/internal/file/operations' +import { + createKnowledgeAclFixtureIds, + seedKnowledgeAclFixture, +} from '@/lib/knowledge/__integration__/seed-source-access-fixture' +import { addWorkspaceFilesToKnowledgeBase } from '@/lib/knowledge/application/add-workspace-files' +import { listKnowledgeChunks } from '@/lib/knowledge/application/chunks' +import { searchKnowledge } from '@/lib/knowledge/application/search' +import { KNOWLEDGE_DOCUMENT_PROCESSING_OUTBOX_EVENT } from '@/lib/knowledge/documents/processing-outbox-event' +import { knowledgeDocumentProcessingOutboxHandlers } from '@/lib/knowledge/documents/processing-outbox-handler' +import { createSingleDocument } from '@/lib/knowledge/documents/service' +import { loadKnowledgeDocumentSecretRegistry } from '@/lib/knowledge/secret-provenance' +import { createTableFromWorkspaceFile } from '@/lib/table/application/workspace-file-imports' +import { uploadExecutionFile } from '@/lib/uploads/contexts/execution/execution-file-manager' +import { + deleteWorkspaceFile, + getWorkspaceFile, +} from '@/lib/uploads/contexts/workspace/workspace-file-manager' +import { + filterModelSafeWorkspaceFileAttachments, + getBoundWorkspaceFileSecretProvenance, + isModelSafeWorkspaceFileKey, + isOpaqueWorkspaceFileEgressSafe, + type WorkspaceFileSecretProvenance, +} from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' +import { deleteFile, downloadFile } from '@/lib/uploads/core/storage-service' +import { createWorkspaceFileDelegatedPrincipal } from '@/lib/workspace-files/application/delegated-principal' +import type { UserFile } from '@/executor/types' + +const fixtures: ReturnType[] = [] +const trackedEventIds: string[] = [] +const REPORT_TEXT = + 'Orion archive import retains verified source bytes through every durable surface.' +const REPORT_CSV = `name,description\nOrion,${REPORT_TEXT}\n` +const FIXTURE_SECRET = 'fixture-resolved-secret-not-a-live-key' + +async function seed() { + const ids = createKnowledgeAclFixtureIds() + fixtures.push(ids) + await seedKnowledgeAclFixture(ids) + return { ...ids, workflowId: generateId(), executionId: generateId() } +} + +type Fixture = Awaited> + +function sessionPrincipal(ids: Fixture) { + return { kind: 'session', userId: ids.aliceId, sessionId: 'fixture-session' } as const +} + +function tablePrincipal(ids: Fixture): DelegatedPrincipal { + const issuedAt = new Date() + return { + kind: 'delegated', + serviceId: 'copilot', + subjectUserId: ids.aliceId, + workspaceId: ids.workspaceId, + delegationId: generateId(), + audience: 'sim:tables', + issuedAt, + expiresAt: new Date(issuedAt.getTime() + 5 * 60_000), + } +} + +async function uploadArchive( + ids: Fixture, + provenance?: WorkspaceFileSecretProvenance, + content = REPORT_CSV +) { + const zip = new JSZip() + zip.file('report.csv', content) + return uploadExecutionFile( + ids, + await zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' }), + 'report.zip', + 'application/zip', + ids.aliceId, + provenance + ) +} + +async function decompress(ids: Fixture, archive: UserFile, executionId = ids.executionId) { + return executeFileManageOperation( + fileManageDecompressBodySchema.parse({ + operation: 'decompress', + workspaceId: ids.workspaceId, + fileInput: archive, + }), + { + principal: createWorkspaceFileDelegatedPrincipal({ + serviceId: 'executor', + subjectUserId: ids.aliceId, + workspaceId: ids.workspaceId, + delegationId: generateId(), + executionId, + }), + workspaceId: ids.workspaceId, + attributedUserId: ids.aliceId, + fileAccessUserId: ids.aliceId, + workflowId: ids.workflowId, + executionId, + headers: new Headers(), + requestId: generateId(), + } + ) +} + +async function extract(ids: Fixture, archive: UserFile) { + const response = await decompress(ids, archive) + const body = await response.json() + expect(response.status, JSON.stringify(body)).toBe(200) + expect(body.success).toBe(true) + const candidates: unknown = body.data?.files + if (!Array.isArray(candidates) || !candidates.every(isUserFile)) { + throw new Error('Archive extraction returned invalid file metadata') + } + expect(candidates).toHaveLength(1) + const child = candidates[0] + const record = await getWorkspaceFile(ids.workspaceId, child.id) + if (!record) throw new Error('Extracted file has no canonical workspace record') + const identity = { + fileId: record.id, + key: record.key, + context: 'workspace' as const, + contentUpdatedAt: record.contentUpdatedAt ?? undefined, + } + return { child, record, identity, publicMetadata: JSON.stringify({ archive, body }) } +} + +async function assertBlockedConsumers(ids: Fixture, source: Awaited>) { + expect(await isOpaqueWorkspaceFileEgressSafe(ids.workspaceId, source.identity)).toBe(false) + const imported = await addWorkspaceFilesToKnowledgeBase.execute({ + principal: sessionPrincipal(ids), + input: { knowledgeBaseId: ids.knowledgeBaseId, fileReferences: [source.child.id] }, + }) + expect(imported).toMatchObject({ added: [], failed: [source.child.id] }) + await expect( + createTableFromWorkspaceFile.execute({ + principal: tablePrincipal(ids), + input: { workspaceId: ids.workspaceId, fileReference: source.child.id }, + }) + ).rejects.toThrow('cannot be verified as free of resolved secrets') +} + +beforeAll(() => { + fixtureStorage.root = mkdtempSync(path.join(tmpdir(), 'sim-execution-archive-provenance-')) +}) +afterAll(async () => { + if (trackedEventIds.length) { + await db.delete(outboxEvent).where(inArray(outboxEvent.id, trackedEventIds)) + } + for (const ids of fixtures) { + await db.delete(knowledgeBase).where(eq(knowledgeBase.id, ids.knowledgeBaseId)) + await db.delete(workspace).where(eq(workspace.id, ids.workspaceId)) + await db.delete(organization).where(eq(organization.id, ids.organizationId)) + await db.delete(user).where(inArray(user.id, [ids.aliceId, ids.bobId])) + } + await rm(fixtureStorage.root, { recursive: true, force: true }) + await db.$client.end() +}) + +describe('execution archive durable provenance', () => { + it('carries exact-empty lineage through extraction, table rows, and delayed KB indexing/search', async () => { + const ids = await seed() + const archive = await uploadArchive(ids, { status: 'exact', entries: [] }) + const [storedArchive] = await db + .select({ + secretProvenanceVersion: workspaceFiles.secretProvenanceVersion, + context: workspaceFiles.context, + }) + .from(workspaceFiles) + .where(eq(workspaceFiles.key, archive.key)) + expect(storedArchive.secretProvenanceVersion).toBe(1) + expect(storedArchive.context).toBe('execution') + const source = await extract(ids, archive) + expect(await getBoundWorkspaceFileSecretProvenance(ids.workspaceId, source.identity)).toEqual({ + status: 'exact', + entries: [], + }) + expect(await isOpaqueWorkspaceFileEgressSafe(ids.workspaceId, source.identity)).toBe(true) + expect((await downloadFile({ key: source.child.key, context: 'workspace' })).toString()).toBe( + REPORT_CSV + ) + + const table = await createTableFromWorkspaceFile.execute({ + principal: tablePrincipal(ids), + input: { workspaceId: ids.workspaceId, fileReference: source.child.id }, + }) + expect(table.kind).toBe('inline') + if (table.kind !== 'inline') throw new Error('Small CSV did not use the inline import path') + expect(table.insertedCount).toBe(1) + const rows = await db + .select({ + data: userTableRows.data, + updatedAt: userTableRows.updatedAt, + version: userTableRows.secretProvenanceVersion, + contentUpdatedAt: userTableRowSecretProvenance.contentUpdatedAt, + status: userTableRowSecretProvenance.status, + entries: userTableRowSecretProvenance.entries, + }) + .from(userTableRows) + .leftJoin( + userTableRowSecretProvenance, + eq(userTableRowSecretProvenance.rowId, userTableRows.id) + ) + .where(eq(userTableRows.tableId, table.table.id)) + expect(rows).toHaveLength(1) + const nameColumn = table.table.schema.columns.find((column) => column.name === 'name') + const descriptionColumn = table.table.schema.columns.find( + (column) => column.name === 'description' + ) + if (!nameColumn?.id || !descriptionColumn?.id) { + throw new Error('Imported table lost its canonical source columns') + } + expect(rows[0]).toMatchObject({ + data: { [nameColumn.id]: 'Orion', [descriptionColumn.id]: REPORT_TEXT }, + version: 1, + status: 'exact', + entries: [], + }) + expect(rows[0].contentUpdatedAt).toEqual(rows[0].updatedAt) + + const imported = await addWorkspaceFilesToKnowledgeBase.execute({ + principal: sessionPrincipal(ids), + input: { knowledgeBaseId: ids.knowledgeBaseId, fileReferences: [source.child.id] }, + }) + expect(imported.failed).toEqual([]) + expect(imported.added).toHaveLength(1) + const documentId = imported.added[0].documentId + const [admitted] = await db.select().from(document).where(eq(document.id, documentId)) + expect(admitted.secretProvenanceVersion).toBe(1) + expect(admitted.storageKey).toMatch(/^kb\//) + const events = await db + .select() + .from(outboxEvent) + .where(sql`${outboxEvent.payload}::jsonb ->> 'documentId' = ${documentId}`) + trackedEventIds.push(...events.map((event) => event.id)) + const dispatch = events.find( + (event) => event.eventType === KNOWLEDGE_DOCUMENT_PROCESSING_OUTBOX_EVENT + ) + if (!dispatch) throw new Error('Knowledge import did not atomically admit processing') + await deleteWorkspaceFile(ids.workspaceId, source.child.id) + await deleteFile({ key: source.child.key, context: 'workspace' }) + await deleteFile({ key: archive.key, context: 'execution' }) + await processOutboxEventById(dispatch.id, knowledgeDocumentProcessingOutboxHandlers) + const [indexed] = await db.select().from(document).where(eq(document.id, documentId)) + expect(indexed.processingStatus, indexed.processingError ?? undefined).toBe('completed') + const chunks = await listKnowledgeChunks.execute({ + principal: sessionPrincipal(ids), + input: { knowledgeBaseId: ids.knowledgeBaseId, documentId }, + }) + expect(chunks.chunks.map((chunk) => chunk.content).join('\n')).toContain(REPORT_TEXT) + const search = await searchKnowledge.execute({ + principal: sessionPrincipal(ids), + input: { + workspaceId: ids.workspaceId, + knowledgeBaseIds: [ids.knowledgeBaseId], + query: 'Orion', + searchMode: 'hybrid', + topK: 10, + }, + }) + expect(search.results.map((entry) => entry.documentId)).toContain(documentId) + }) + + it('keeps an explicitly unknown execution source unavailable to model, KB, and table consumers', async () => { + const ids = await seed() + const archive = await uploadArchive(ids, { status: 'unknown' }) + const source = await extract(ids, archive) + expect(await getBoundWorkspaceFileSecretProvenance(ids.workspaceId, source.identity)).toEqual({ + status: 'unknown', + }) + await assertBlockedConsumers(ids, source) + }) + + it('does not infer safe extracted bytes from a secret-bearing archive or expose private metadata', async () => { + const ids = await seed() + const { encrypted } = await encryptSecret(FIXTURE_SECRET) + const archive = await uploadArchive( + ids, + { + status: 'exact', + entries: [ + { + name: 'FIXTURE_SECRET', + encryptedValue: encrypted, + sourceUserId: ids.aliceId, + sourceWorkspaceId: ids.workspaceId, + }, + ], + }, + `name,description\nOrion,${FIXTURE_SECRET}\n` + ) + const source = await extract(ids, archive) + expect(source.publicMetadata).not.toContain(FIXTURE_SECRET) + expect(source.publicMetadata).not.toContain(encrypted) + expect(source.publicMetadata).not.toContain('encryptedValue') + expect(await getBoundWorkspaceFileSecretProvenance(ids.workspaceId, source.identity)).toEqual({ + status: 'unknown', + }) + await assertBlockedConsumers(ids, source) + }) + + it('preserves compatibility for execution files created before provenance stamping', async () => { + const ids = await seed() + const archive = await uploadArchive(ids) + const [storedArchive] = await db + .select({ + secretProvenanceVersion: workspaceFiles.secretProvenanceVersion, + context: workspaceFiles.context, + }) + .from(workspaceFiles) + .where(eq(workspaceFiles.key, archive.key)) + expect(storedArchive.secretProvenanceVersion).toBeNull() + const source = await extract(ids, archive) + expect(await isOpaqueWorkspaceFileEgressSafe(ids.workspaceId, source.identity)).toBe(true) + const imported = await createTableFromWorkspaceFile.execute({ + principal: tablePrincipal(ids), + input: { workspaceId: ids.workspaceId, fileReference: source.child.id }, + }) + expect(imported.kind).toBe('inline') + }) + + it.each([false, true])( + 'refuses tracked unknown execution attachments with historical metadata (archivedOnly=%s)', + async (archivedOnly) => { + const ids = await seed() + const file = await uploadExecutionFile( + ids, + Buffer.from(REPORT_CSV), + 'report.csv', + 'text/csv', + ids.aliceId, + { status: 'unknown' } + ) + if (archivedOnly) { + await db + .update(workspaceFiles) + .set({ deletedAt: new Date() }) + .where(eq(workspaceFiles.key, file.key)) + } else { + await db.insert(withInsertColumns(workspaceFiles, workspaceFileColumns)).values({ + id: generateId(), + key: file.key, + userId: ids.aliceId, + workspaceId: ids.workspaceId, + context: 'execution', + originalName: 'historical-report.csv', + contentType: file.type, + sizeBytes: file.size, + deletedAt: new Date(), + contentUpdatedAt: new Date(Date.now() + 60_000), + secretProvenanceVersion: null, + }) + } + + expect( + await filterModelSafeWorkspaceFileAttachments([file], { workspaceId: ids.workspaceId }) + ).toEqual([]) + expect(await isModelSafeWorkspaceFileKey(file.key, { workspaceId: ids.workspaceId })).toBe( + false + ) + } + ) + + it.each([ + { status: 'exact', deleted: false }, + { status: 'unknown', deleted: false }, + { status: 'exact', deleted: true }, + { status: 'unknown', deleted: true }, + ] as const)( + 'binds $status execution bytes into KB admission despite URL-only classification (deleted=$deleted)', + async ({ status, deleted }) => { + const ids = await seed() + const file = await uploadExecutionFile( + ids, + Buffer.from(REPORT_CSV), + 'report.csv', + 'text/csv', + ids.aliceId, + status === 'exact' ? { status, entries: [] } : { status } + ) + if (deleted) { + await db + .update(workspaceFiles) + .set({ deletedAt: new Date() }) + .where(eq(workspaceFiles.key, file.key)) + } + const admitted = await createSingleDocument( + { + filename: file.name, + fileUrl: `/api/files/serve/${encodeURIComponent(file.key)}?context=workspace`, + fileSize: file.size, + mimeType: file.type, + }, + ids.knowledgeBaseId, + generateId(), + ids.aliceId, + undefined, + { + filename: { status: 'exact', entries: [] }, + content: { status: 'exact', entries: [] }, + tags: [], + } + ) + const [stored] = await db + .select({ + version: document.secretProvenanceVersion, + status: documentSecretProvenance.status, + }) + .from(document) + .leftJoin(documentSecretProvenance, eq(documentSecretProvenance.documentId, document.id)) + .where(eq(document.id, admitted.id)) + expect(stored).toEqual({ version: 1, status }) + const registry = loadKnowledgeDocumentSecretRegistry(admitted.id, { + userId: ids.aliceId, + workspaceId: ids.workspaceId, + }) + if (status === 'exact') { + await expect(registry).resolves.toMatchObject({ + tracked: true, + provenance: { status: 'exact', entries: [] }, + }) + } else { + await expect(registry).rejects.toThrow( + 'Knowledge document secret provenance is unavailable' + ) + } + } + ) + + it('refuses another execution before extracting any workspace files', async () => { + const ids = await seed() + const archive = await uploadArchive(ids, { status: 'exact', entries: [] }) + const response = await decompress(ids, archive, generateId()) + expect(response.status).toBe(404) + const files = await db + .select({ context: workspaceFiles.context }) + .from(workspaceFiles) + .where(eq(workspaceFiles.workspaceId, ids.workspaceId)) + expect(files).toEqual([{ context: 'execution' }]) + }) +}) diff --git a/apps/sim/lib/knowledge/__integration__/organization-mcp-search.integration.ts b/apps/sim/lib/knowledge/__integration__/organization-mcp-search.integration.ts index 8f8374ebb9f..0b7060df4c6 100644 --- a/apps/sim/lib/knowledge/__integration__/organization-mcp-search.integration.ts +++ b/apps/sim/lib/knowledge/__integration__/organization-mcp-search.integration.ts @@ -12,6 +12,7 @@ import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/ import { CallToolResultSchema } from '@modelcontextprotocol/sdk/types.js' import type { Principal } from '@sim/auth/principal' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { apiKey, document, @@ -25,6 +26,7 @@ import { oauthClient, oauthConsent, organization, + organizationColumns, organizationSearchIntegration, rateLimitBucket, user, @@ -252,7 +254,7 @@ describe('organization Search MCP with real ingestion and current access', () => updatedAt: new Date(), })) ) - await db.insert(organization).values({ + await db.insert(withInsertColumns(organization, organizationColumns)).values({ id: otherOrganizationId, name: 'Other organization MCP fixture', slug: otherOrganizationId, diff --git a/apps/sim/lib/knowledge/__integration__/search-latency.integration.ts b/apps/sim/lib/knowledge/__integration__/search-latency.integration.ts new file mode 100644 index 00000000000..5646bd707d1 --- /dev/null +++ b/apps/sim/lib/knowledge/__integration__/search-latency.integration.ts @@ -0,0 +1,558 @@ +/** Real Assistant tool, application authorization, PostgreSQL/pgvector, and result processing. */ +import { readFileSync, statSync, writeFileSync } from 'node:fs' +import { db } from '@sim/db' +import { + copilotChats, + credential, + credentialGroup, + document, + knowledgeBase, + knowledgeConnector, + member, + organization, + user, + workspace, +} from '@sim/db/schema' +import { createLogger, Logger } from '@sim/logger' +import { generateId } from '@sim/utils/id' +import { and, eq, inArray, sql } from 'drizzle-orm' +import { afterAll, beforeAll, describe, expect, it, type MockInstance, vi } from 'vitest' +import { z } from 'zod' +import { searchWorkspaceServerTool } from '@/lib/copilot/tools/server/knowledge/workspace-search' +import { seedSearchReaderFixture } from '@/lib/knowledge/__integration__/seed-search-reader-fixture' +import { + createKnowledgeAclFixtureIds, + seedKnowledgeAclFixture, +} from '@/lib/knowledge/__integration__/seed-source-access-fixture' +import { searchScopedKnowledge } from '@/lib/knowledge/application/workspace-search' +import type { WorkspaceSearchFilters } from '@/lib/knowledge/search/filters' +import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' + +/** Initialize controlled provider configuration before the real application modules load. */ +vi.hoisted(() => { + if (process.env.KNOWLEDGE_SEARCH_PERFORMANCE_TEST === 'true') { + Object.assign(process.env, { + OPENAI_API_KEY: 'isolated-embedding-http-fixture', + CONFLUENCE_CLIENT_ID: 'isolated-confluence-fixture-client', + CONFLUENCE_CLIENT_SECRET: 'isolated-confluence-fixture-secret', + }) + } +}) + +const enabled = process.env.KNOWLEDGE_SEARCH_PERFORMANCE_TEST === 'true' +const chunkCount = Number(process.env.KNOWLEDGE_SEARCH_PERFORMANCE_CHUNKS ?? 20_000) +const dimensions = 1536 +const chunksPerDocument = 4 +const batchSize = 1000 +const logger = createLogger('SearchLatencyIntegration') +const fixtureSchema = z.object({ + aliceId: z.uuid(), + bobId: z.uuid(), + workspaceId: z.uuid(), + organizationId: z.uuid(), + knowledgeBaseId: z.uuid(), + connectorId: z.uuid(), + lockId: z.uuid(), + groups: z.array(z.string()).length(3), + groupIds: z.array(z.uuid()).length(3), +}) +const reuseFile = enabled ? process.env.KNOWLEDGE_SEARCH_PERFORMANCE_REUSE_REPORT_FILE : undefined +function readFixtureReport(file: string) { + if (statSync(file).size > 8 * 1024 * 1024) throw new Error('Fixture report exceeds 8 MiB') + return z + .object({ fixture: fixtureSchema, unrelatedFixture: fixtureSchema }) + .parse(JSON.parse(readFileSync(file, 'utf8'))) +} +const reused = reuseFile ? readFixtureReport(reuseFile) : undefined +const ids = reused?.fixture ?? createKnowledgeAclFixtureIds() +const unrelated = reused?.unrelatedFixture ?? createKnowledgeAclFixtureIds() +const organizationChatId = generateId() +const queryVector = Array.from({ length: dimensions }, (_, index) => (index === 0 ? 1 : 0)) +const captured: CapturedQuery[] = [] +const report: Record = { + fixture: ids, + unrelatedFixture: unrelated, + method: { + chunkCount, + dimensions, + chunksPerDocument, + sql: 'Captured from the real Assistant tool; no hand-written search query', + providers: + 'Embedding and source-permission HTTP responses are controlled; internal search and authorization code is real', + vectors: + 'Normalized topic clusters with deterministic dense noise; not semantic-quality evaluation', + cache: 'First and repeated samples; no claim of a cold operating-system cache', + }, +} +let capture = false +let embeddingCalls = 0 +let readerCalls = 0 +let readerRevoked = false +const previousDebug = db.$client.options.debug +let diagnosticLog: MockInstance | undefined + +interface CapturedQuery { + query: string + parameters: NonNullable[1]> +} + +interface ExplainNode { + 'Node Type': string + 'Actual Rows': number + 'Index Name'?: string + Plans?: ExplainNode[] +} + +const explainNodeSchema: z.ZodType = z.lazy(() => + z + .object({ + 'Node Type': z.string(), + 'Actual Rows': z.number(), + 'Index Name': z.string().optional(), + Plans: z.array(explainNodeSchema).optional(), + }) + .passthrough() +) +const explainSchema = z.array(z.object({ Plan: explainNodeSchema }).passthrough()).length(1) + +function usesVectorIndex(node: ExplainNode): boolean { + return ( + node['Index Name'] === 'embedding_vector_hnsw_idx' || + (node.Plans?.some(usesVectorIndex) ?? false) + ) +} + +function saveReport() { + const file = process.env.KNOWLEDGE_SEARCH_PERFORMANCE_REPORT_FILE + if (file) writeFileSync(file, JSON.stringify(report, null, 2), { mode: 0o600 }) +} + +const diagnosticSchema = z + .object({ + surface: z.enum(['dashboard', 'copilot']), + outcome: z.literal('success'), + elapsedMs: z.number(), + toolResultBytes: z.number().int().nonnegative().optional(), + passageBytes: z.number().int().nonnegative().optional(), + maxPassageBytes: z.number().int().nonnegative().optional(), + uniqueDocumentCount: z.number().int().nonnegative().optional(), + stages: z.record( + z.string(), + z.object({ + count: z.number(), + totalMs: z.number(), + maxMs: z.number(), + errors: z.number(), + }) + ), + }) + .passthrough() + +const resultSchema = z.object({ + success: z.literal(true), + data: z.object({ + results: z.array( + z.object({ documentId: z.string(), content: z.string(), knowledgeBaseId: z.string() }) + ), + }), +}) + +async function search( + userId = ids.aliceId, + query = 'Orion deployment', + filters: WorkspaceSearchFilters = {}, + organizationScope = false +) { + return resultSchema.parse( + await searchWorkspaceServerTool.execute( + { query, topK: 15, ...filters }, + { + userId, + ...(organizationScope + ? { organizationId: ids.organizationId, chatId: organizationChatId } + : { workspaceId: ids.workspaceId }), + requestMode: 'assistant', + toolCallId: generateId(), + copilotToolExecution: true, + resolvedSecretTraceRegistry: new ResolvedSecretTraceRegistry([], { + userId, + ...(organizationScope ? {} : { workspaceId: ids.workspaceId }), + }), + } + ) + ) +} + +async function sample(label: string, run: () => ReturnType) { + captured.length = 0 + diagnosticLog?.mockClear() + const start = performance.now() + capture = true + let result: Awaited> + try { + result = await run() + } finally { + capture = false + } + const milliseconds = performance.now() - start + const completed = + diagnosticLog?.mock.calls.filter(([message]) => message === 'Knowledge search completed') ?? [] + expect(completed).toHaveLength(1) + const diagnostics = diagnosticSchema.parse(completed[0][1]) + expect(diagnostics.stages.embedding.count).toBe(1) + expect(diagnostics.stages.retrieval.count).toBe(1) + if (diagnostics.surface === 'copilot') { + const passageBytes = result.data.results.map((row) => Buffer.byteLength(row.content)) + expect(diagnostics.passageBytes).toBe(passageBytes.reduce((total, bytes) => total + bytes, 0)) + expect(diagnostics.maxPassageBytes).toBe(Math.max(0, ...passageBytes)) + expect(diagnostics.uniqueDocumentCount).toBe( + new Set(result.data.results.map((row) => row.documentId)).size + ) + expect(diagnostics.toolResultBytes).toBeGreaterThan(diagnostics.passageBytes!) + } + expect(captured.length).toBeLessThan(300) + const searches = captured.filter( + (item) => + item.query.includes('from "embedding"') && + (item.query.includes('order by') || item.query.includes('limit')) + ) + const plans = [] + for (const query of searches) { + const plan = await db.$client.begin(async (tx) => { + await tx.unsafe("SET LOCAL hnsw.iterative_scan = 'relaxed_order'") + await tx.unsafe('SET LOCAL hnsw.max_scan_tuples = 20000') + return tx.unsafe(`EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) ${query.query}`, query.parameters) + }) + plans.push({ + kind: query.query.includes('keyword_rank') + ? 'keyword' + : query.query.includes('order by') + ? 'vector' + : 'probe', + query: query.query, + parameters: query.parameters, + plan: explainSchema.parse(plan[0]['QUERY PLAN']), + }) + } + report[label] = { + milliseconds, + diagnostics, + queryCount: captured.length, + resultCount: result.data.results.length, + plans, + } + saveReport() + logger.info(label, { + milliseconds, + queryCount: captured.length, + resultCount: result.data.results.length, + }) + return { result, plans, diagnostics } +} + +describe.skipIf(!enabled)('Assistant search latency on a realistic indexed corpus', () => { + beforeAll(async () => { + if ( + !Number.isInteger(chunkCount) || + chunkCount < 10_000 || + chunkCount > 200_000 || + chunkCount % batchSize !== 0 + ) + throw new Error( + 'KNOWLEDGE_SEARCH_PERFORMANCE_CHUNKS must be a multiple of 1000 from 10000 to 200000' + ) + vi.stubGlobal('fetch', async (input: string | URL | Request, init?: RequestInit) => { + const url = input instanceof Request ? input.url : String(input) + if ( + url === 'https://api.atlassian.com/ex/confluence/search-fixture/wiki/rest/api/user/current' + ) { + expect(new Headers(init?.headers).get('Authorization')).toBe('Bearer fixture-search-reader') + readerCalls++ + return readerRevoked + ? new Response(null, { status: 403 }) + : Response.json({ type: 'known', accountId: ids.aliceId }) + } + if (url !== 'https://api.openai.com/v1/embeddings') + throw new Error(`Unexpected outbound request in search fixture: ${new URL(url).origin}`) + const body = z + .object({ input: z.array(z.string()).length(1), encoding_format: z.literal('base64') }) + .parse(JSON.parse(String(init?.body))) + embeddingCalls += body.input.length + const bytes = Buffer.alloc(dimensions * 4) + queryVector.forEach((value, index) => bytes.writeFloatLE(value, index * 4)) + return Response.json({ + data: [{ embedding: bytes.toString('base64') }], + usage: { total_tokens: 4 }, + }) + }) + if (reused) { + const owners = await db + .select({ id: workspace.id, ownerId: workspace.ownerId }) + .from(workspace) + .where(inArray(workspace.id, [ids.workspaceId, unrelated.workspaceId])) + expect(owners).toEqual( + expect.arrayContaining([ + { id: ids.workspaceId, ownerId: ids.aliceId }, + { id: unrelated.workspaceId, ownerId: unrelated.aliceId }, + ]) + ) + for (const fixture of [ids, unrelated]) { + const [size] = await db.execute<{ count: number }>( + sql`SELECT count(*)::int AS count FROM embedding WHERE knowledge_base_id = ${fixture.knowledgeBaseId}` + ) + expect(size.count).toBe(fixture === ids ? chunkCount : chunkCount / 2) + } + await db + .update(knowledgeBase) + .set({ workspaceId: ids.workspaceId, organizationId: null }) + .where(eq(knowledgeBase.id, ids.knowledgeBaseId)) + await db + .update(knowledgeConnector) + .set({ connectorType: 'google_drive', credentialId: null, sourceConfig: {} }) + .where(eq(knowledgeConnector.id, ids.connectorId)) + await db.delete(credential).where(eq(credential.workspaceId, ids.workspaceId)) + await db.delete(credentialGroup).where(eq(credentialGroup.workspaceId, ids.workspaceId)) + await db + .delete(copilotChats) + .where( + and( + eq(copilotChats.organizationId, ids.organizationId), + eq(copilotChats.userId, ids.aliceId) + ) + ) + await db + .delete(member) + .where(and(eq(member.organizationId, ids.organizationId), eq(member.userId, ids.aliceId))) + await db.execute( + sql`UPDATE document SET acl = ARRAY[${`u:${ids.aliceId}@fixture.test`}], user_excluded = false, acl_verified_at = statement_timestamp() WHERE knowledge_base_id = ${ids.knowledgeBaseId}` + ) + } else { + await seedKnowledgeAclFixture(ids, { connectorType: 'google_drive' }) + await seedKnowledgeAclFixture(unrelated, { connectorType: 'google_drive' }) + await db + .update(knowledgeBase) + .set({ isSearchIndex: true }) + .where(eq(knowledgeBase.id, ids.knowledgeBaseId)) + const indexes = await db.execute<{ + indexname: string + indexdef: string + }>(sql`SELECT indexname, indexdef FROM pg_indexes + WHERE tablename = 'embedding' AND indexdef LIKE '% USING hnsw %'`) + for (const index of indexes) + await db.execute(sql`DROP INDEX ${sql.identifier(index.indexname)}`) + for (const fixture of [ids, unrelated]) { + const count = fixture === ids ? chunkCount : chunkCount / 2 + for (let first = 0; first < count / chunksPerDocument; first += batchSize) { + const last = Math.min(first + batchSize, count / chunksPerDocument) - 1 + await db.execute(sql`INSERT INTO document + (id, knowledge_base_id, connector_id, external_id, filename, file_url, file_size, mime_type, processing_status, acl, acl_verified_at) + SELECT ${fixture.workspaceId} || '-doc-' || n, ${fixture.knowledgeBaseId}, ${fixture.connectorId}, n::text, + 'Deployment guide ' || n, 'https://fixture.invalid/document/' || n, 12000, 'text/plain', 'completed', + ARRAY[${`u:${fixture.aliceId}@fixture.test`}]::text[], statement_timestamp() + FROM generate_series(${first}::int, ${last}::int) n`) + } + for (let first = 0; first < count; first += batchSize) { + const last = Math.min(first + batchSize, count) - 1 + await db.transaction(async (tx) => { + await tx.execute(sql`SET LOCAL jit = off`) + await tx.execute(sql`INSERT INTO embedding + (id, knowledge_base_id, document_id, chunk_index, chunk_hash, content, content_length, token_count, start_offset, end_offset, embedding) + SELECT ${fixture.workspaceId} || '-chunk-' || n, ${fixture.knowledgeBaseId}, ${fixture.workspaceId} || '-doc-' || (n / ${chunksPerDocument}), + n % ${chunksPerDocument}, 'hash-' || n, + CASE WHEN n % 8 = 0 THEN 'Orion deployment reference. ' ELSE 'Engineering operations reference. ' END || + (SELECT string_agg(md5(n::text || ':' || paragraph::text), ' ') FROM generate_series(1, 90) paragraph), + 3000, 750, 0, 3000, + l2_normalize(ARRAY(SELECT (CASE WHEN coordinate = n % 32 + 1 THEN 1 ELSE 0 END + + 0.025 * sin(n::double precision * coordinate * 12.9898 + coordinate * 78.233))::real + FROM generate_series(1, ${dimensions}) coordinate)::vector(1536)) + FROM generate_series(${first}::int, ${last}::int) n`) + }) + } + logger.info('Synthetic corpus loaded', { chunks: count }) + } + for (const index of indexes) await db.execute(sql.raw(index.indexdef)) + } + await db.execute(sql`ANALYZE document`) + await db.execute(sql`ANALYZE embedding`) + report.server = ( + await db.execute(sql`SELECT version(), current_setting('work_mem') AS work_mem, + (SELECT extversion FROM pg_extension WHERE extname = 'vector') AS pgvector`) + )[0] + db.$client.options.debug = (_connection, query, parameters) => { + if (capture && captured.length < 300) captured.push({ query, parameters: [...parameters] }) + } + diagnosticLog = vi.spyOn(Logger.prototype, 'info') + }, 60 * 60_000) + + afterAll(async () => { + diagnosticLog?.mockRestore() + db.$client.options.debug = previousDebug + vi.unstubAllGlobals() + saveReport() + for (const fixture of process.env.KNOWLEDGE_SEARCH_PERFORMANCE_KEEP_DATABASE === 'true' + ? [] + : [ids, unrelated]) { + await db.delete(workspace).where(eq(workspace.id, fixture.workspaceId)) + await db.delete(organization).where(eq(organization.id, fixture.organizationId)) + await db.delete(user).where(eq(user.id, fixture.aliceId)) + await db.delete(user).where(eq(user.id, fixture.bobId)) + } + await db.$client.end() + }, 120_000) + + it('records first and repeated application searches with the actual SQL plans', async () => { + for (let iteration = 0; iteration < 2; iteration++) { + const { result, plans } = await sample(`broad.${iteration}`, () => search()) + expect(result.data.results).toHaveLength(15) + expect(result.data.results.every((row) => row.knowledgeBaseId === ids.knowledgeBaseId)).toBe( + true + ) + expect(plans.length).toBeGreaterThanOrEqual(2) + const vectorPlans = plans.filter((plan) => plan.kind === 'vector') + expect(vectorPlans).toHaveLength(1) + expect(usesVectorIndex(vectorPlans[0].plan[0].Plan)).toBe(true) + } + expect(embeddingCalls).toBe(2) + }, 180_000) + + it('compares the Search tab and Assistant with the same person, query and index', async () => { + const dashboard = await sample('dashboard', async () => { + const result = await searchScopedKnowledge.execute({ + principal: { kind: 'session', userId: ids.aliceId, sessionId: 'fixture-dashboard' }, + input: { + workspaceId: ids.workspaceId, + query: 'Orion deployment', + topK: 15, + surface: 'dashboard', + }, + }) + return resultSchema.parse({ success: true, data: result }) + }) + const assistant = await sample('assistant.comparison', () => search()) + expect(dashboard.diagnostics.surface).toBe('dashboard') + expect(assistant.diagnostics.surface).toBe('copilot') + expect(dashboard.diagnostics.stages.result_provenance).toBeUndefined() + expect(assistant.diagnostics.stages.result_provenance.count).toBe(1) + expect(dashboard.result.data.results).toHaveLength(15) + expect(assistant.result.data.results).toHaveLength(15) + const dashboardVector = dashboard.plans.filter((plan) => plan.kind === 'vector') + const assistantVector = assistant.plans.filter((plan) => plan.kind === 'vector') + expect(dashboardVector).toHaveLength(1) + expect(assistantVector).toHaveLength(1) + expect(dashboardVector[0].query).toBe(assistantVector[0].query) + expect(dashboardVector[0].parameters).toEqual(assistantVector[0].parameters) + expect(usesVectorIndex(dashboardVector[0].plan[0].Plan)).toBe(true) + }, 180_000) + + it('keeps inaccessible content out of an otherwise identical search', async () => { + const { result } = await sample('denied', () => search(ids.bobId)) + expect(result.data.results).toEqual([]) + }, 180_000) + + it('ranks a small permission scope by its bounded IDs without a corpus-wide vector probe', async () => { + const documentIds = [0, 8, 16].map((index) => `${ids.workspaceId}-doc-${index}`) + await db + .update(document) + .set({ acl: [`u:${ids.aliceId}@fixture.test`, `u:${ids.bobId}@fixture.test`] }) + .where(inArray(document.id, documentIds)) + try { + const { result, plans } = await sample('small-scope', () => search(ids.bobId)) + expect(result.data.results.length).toBeGreaterThan(0) + expect(result.data.results.every((row) => documentIds.includes(row.documentId))).toBe(true) + const probe = plans.filter((plan) => plan.kind === 'probe') + expect(probe).toHaveLength(1) + expect(probe[0].query).not.toContain('<=>') + expect(probe[0].plan[0].Plan['Actual Rows']).toBe(12) + const vector = plans.filter((plan) => plan.kind === 'vector') + expect(vector).toHaveLength(1) + expect(vector[0].query).toContain('"embedding"."id" in') + } finally { + await db + .update(document) + .set({ acl: [`u:${ids.aliceId}@fixture.test`] }) + .where(inArray(document.id, documentIds)) + } + }, 180_000) + + it('applies selective document scope and exclusion before ranking', async () => { + const documentIds = [0, 8, 16, 24, 32].map((index) => `${ids.workspaceId}-doc-${index}`) + await db.update(document).set({ userExcluded: true }).where(eq(document.id, documentIds[0])) + try { + const { result } = await sample('selective', () => + search(ids.aliceId, 'Orion deployment', { documentIds }) + ) + expect(result.data.results.length).toBeGreaterThan(0) + expect(new Set(result.data.results.map((row) => row.documentId))).toEqual( + new Set(documentIds.slice(1)) + ) + expect( + result.data.results.every((row) => documentIds.slice(1).includes(row.documentId)) + ).toBe(true) + } finally { + await db.update(document).set({ userExcluded: false }).where(eq(document.id, documentIds[0])) + } + }, 180_000) + + it('runs two independent Assistant searches concurrently', async () => { + const start = performance.now() + const results = await Promise.all([search(), search(ids.aliceId, 'Engineering operations')]) + report.concurrent = { + milliseconds: performance.now() - start, + resultCounts: results.map((result) => result.data.results.length), + } + saveReport() + for (const result of results) expect(result.data.results).toHaveLength(15) + }, 180_000) + + it('checks live reader access on every search, including after revocation', async () => { + await seedSearchReaderFixture(ids) + const allowed = await sample('live.allowed', () => search()) + expect(allowed.result.data.results).toHaveLength(15) + expect(readerCalls).toBeGreaterThan(0) + readerRevoked = true + const before = readerCalls + const denied = await sample('live.revoked', () => search()) + for (const probe of denied.plans.filter((plan) => plan.kind === 'probe')) { + expect(probe.query).not.toContain('<=>') + } + expect(denied.result.data.results).toEqual([]) + expect(readerCalls).toBeGreaterThan(before) + readerRevoked = false + const restored = await sample('live.restored', () => search()) + expect(restored.result.data.results).toHaveLength(15) + }, 180_000) + + it('uses the same indexed retrieval through a persisted private organization Assistant chat', async () => { + await db.insert(member).values({ + id: generateId(), + organizationId: ids.organizationId, + userId: ids.aliceId, + role: 'owner', + }) + await db.insert(copilotChats).values({ + id: organizationChatId, + organizationId: ids.organizationId, + userId: ids.aliceId, + type: 'mothership', + }) + await db + .update(knowledgeBase) + .set({ workspaceId: null, organizationId: ids.organizationId }) + .where(eq(knowledgeBase.id, ids.knowledgeBaseId)) + await db + .update(knowledgeConnector) + .set({ connectorType: 'google_drive', credentialId: null, sourceConfig: {} }) + .where(eq(knowledgeConnector.id, ids.connectorId)) + await db.execute( + sql`UPDATE document SET acl = ARRAY[${`u:${ids.aliceId}@fixture.test`}] WHERE knowledge_base_id = ${ids.knowledgeBaseId}` + ) + await db.execute(sql`ANALYZE document`) + const { result } = await sample('organization', () => + search(ids.aliceId, 'Orion deployment', {}, true) + ) + expect(result.data.results).toHaveLength(15) + expect(result.data.results.every((row) => row.knowledgeBaseId === ids.knowledgeBaseId)).toBe( + true + ) + }, 180_000) +}) diff --git a/apps/sim/lib/knowledge/__integration__/seed-search-reader-fixture.ts b/apps/sim/lib/knowledge/__integration__/seed-search-reader-fixture.ts new file mode 100644 index 00000000000..8896912a40c --- /dev/null +++ b/apps/sim/lib/knowledge/__integration__/seed-search-reader-fixture.ts @@ -0,0 +1,107 @@ +import { createHash } from 'node:crypto' +import { db } from '@sim/db' +import { + credential, + credentialGroup, + credentialGroupEnrollment, + knowledgeConnector, +} from '@sim/db/schema' +import { generateId } from '@sim/utils/id' +import { eq, sql } from 'drizzle-orm' +import { encryptSecret } from '@/lib/core/security/encryption' +import { getCredentialGroupProviderAdapterByProviderId } from '@/lib/credential-groups/provider-registry' +import { encryptManagedOAuthTokenSet } from '@/lib/credentials/managed-oauth' +import type { createKnowledgeAclFixtureIds } from '@/lib/knowledge/__integration__/seed-source-access-fixture' + +/** Converts the existing synthetic corpus to a central crawl with a real managed reader. */ +export async function seedSearchReaderFixture( + ids: ReturnType +) { + const policy = await getCredentialGroupProviderAdapterByProviderId('confluence').getPolicy( + undefined, + { + workspaceId: ids.workspaceId, + } + ) + const groupId = generateId() + const optionId = generateId() + const crawlerId = generateId() + const enrollmentId = generateId() + await db.insert(credentialGroup).values({ + id: groupId, + workspaceId: ids.workspaceId, + publicId: generateId(), + name: 'Search benchmark readers', + options: [ + { + id: optionId, + provider: 'confluence', + label: 'Confluence fixture', + required: false, + status: 'active', + authorizationAppId: policy.authorizationAppId, + requiredScopes: policy.requiredScopes, + scopeVersion: policy.scopeVersion, + }, + ], + }) + await db.insert(credential).values({ + id: crawlerId, + workspaceId: ids.workspaceId, + type: 'service_account', + providerId: 'atlassian-service-account', + displayName: 'Synthetic crawler', + createdBy: ids.aliceId, + encryptedServiceAccountKey: ( + await encryptSecret( + JSON.stringify({ + type: 'atlassian_service_account', + cloudId: 'search-fixture', + domain: 'search-fixture.atlassian.net', + apiToken: 'fixture-crawler-never-used-for-reading', + }) + ) + ).encrypted, + }) + await db.insert(credentialGroupEnrollment).values({ + id: enrollmentId, + credentialGroupId: groupId, + userId: ids.aliceId, + email: `${ids.aliceId}@fixture.test`, + status: 'completed', + invitationTokenHash: createHash('sha256').update(generateId()).digest('hex'), + invitationExpiresAt: new Date(Date.now() + 3600000), + invitedAt: new Date(), + }) + await db.insert(credential).values({ + id: generateId(), + workspaceId: ids.workspaceId, + type: 'managed_oauth', + displayName: 'Synthetic search reader', + createdBy: ids.aliceId, + providerId: 'confluence', + providerSubjectId: ids.aliceId, + authorizationAppId: policy.authorizationAppId, + credentialGroupEnrollmentId: enrollmentId, + credentialGroupOptionId: optionId, + managedOauthScopeVersion: policy.scopeVersion, + managedOauthStatus: 'active', + grantedScopes: policy.requiredScopes, + grantedAt: new Date(), + encryptedOauthTokenSet: await encryptManagedOAuthTokenSet({ + accessToken: 'fixture-search-reader', + }), + accessTokenExpiresAt: new Date(Date.now() + 3600000), + }) + await db + .update(knowledgeConnector) + .set({ + connectorType: 'confluence', + credentialId: crawlerId, + sourceConfig: { domain: 'search-fixture.atlassian.net' }, + }) + .where(eq(knowledgeConnector.id, ids.connectorId)) + await db.execute(sql`UPDATE document SET acl = ARRAY[${`s:confluence:-:${ids.aliceId}`}], + acl_verified_at = statement_timestamp() WHERE knowledge_base_id = ${ids.knowledgeBaseId}`) + await db.execute(sql`ANALYZE document`) +} diff --git a/apps/sim/lib/knowledge/__integration__/seed-source-access-fixture.ts b/apps/sim/lib/knowledge/__integration__/seed-source-access-fixture.ts index 3c0749defc0..22e90269614 100644 --- a/apps/sim/lib/knowledge/__integration__/seed-source-access-fixture.ts +++ b/apps/sim/lib/knowledge/__integration__/seed-source-access-fixture.ts @@ -1,5 +1,6 @@ import { createHash } from 'node:crypto' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { credential, credentialGroup, @@ -11,6 +12,7 @@ import { knowledgeExternalGroup, knowledgeExternalGroupMember, organization, + organizationColumns, permissions, user, workspace, @@ -69,7 +71,7 @@ export async function seedKnowledgeAclFixture( updatedAt: now, }, ]) - await db.insert(organization).values({ + await db.insert(withInsertColumns(organization, organizationColumns)).values({ id: ids.organizationId, name: 'ACL integration organization', slug: ids.organizationId, diff --git a/apps/sim/lib/knowledge/__integration__/slack-search-turns.integration.ts b/apps/sim/lib/knowledge/__integration__/slack-search-turns.integration.ts index 740cfa9fc69..91e6301593a 100644 --- a/apps/sim/lib/knowledge/__integration__/slack-search-turns.integration.ts +++ b/apps/sim/lib/knowledge/__integration__/slack-search-turns.integration.ts @@ -1,10 +1,12 @@ /** Exercises real PostgreSQL locks and constraints using only isolated, explicitly cleaned fixtures. */ import type { OrganizationDelegatedPrincipal } from '@sim/auth/principal' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { copilotChats, credential, organization, + organizationColumns, outboxEvent, slackSearchInstallation, slackSearchTurn, @@ -72,7 +74,7 @@ describe('durable Slack Search turns in PostgreSQL', () => { })) ) await db - .insert(organization) + .insert(withInsertColumns(organization, organizationColumns)) .values({ id: organizationId, name: 'Slack queue fixture', slug: organizationId }) await db.insert(credential).values({ id: credentialId, diff --git a/apps/sim/lib/knowledge/application/search-diagnostics.ts b/apps/sim/lib/knowledge/application/search-diagnostics.ts new file mode 100644 index 00000000000..da404865994 --- /dev/null +++ b/apps/sim/lib/knowledge/application/search-diagnostics.ts @@ -0,0 +1,45 @@ +import type { AuthorizingUseCase } from '@/lib/core/application' +import type { knowledgeOperations } from '@/lib/knowledge/application/operations' +import type { SearchKnowledgeInput } from '@/lib/knowledge/application/search' +import { + measureSearchStage, + type SearchStage, + withSearchDiagnostics, +} from '@/lib/knowledge/search/diagnostics' + +/** Preserve the operation and authorization contract while timing the whole use case. */ +export function instrumentSearchUseCase< + I extends Omit & { + workspaceId?: string | null + organizationId?: string | null + }, + R, +>( + stage: Extract, + useCase: AuthorizingUseCase +): AuthorizingUseCase { + return { + ...useCase, + execute: (args) => + withSearchDiagnostics( + { + surface: args.input.surface ?? 'other', + principalKind: args.principal.kind, + scopeKind: args.input.organizationId + ? 'organization' + : args.input.workspaceId + ? 'workspace' + : undefined, + topK: args.input.topK, + documentFilterCount: args.input.filters?.documentIds?.length ?? 0, + hasSourceFilter: Boolean(args.input.filters?.source), + hasDateFilter: Boolean(args.input.filters?.modifiedAfter), + tagFilterCount: args.input.tagFilters?.length ?? 0, + hasProvenance: Boolean( + args.input.resultSecretRegistry || args.input.prepareModelInputProvenance + ), + }, + () => measureSearchStage(stage, () => useCase.execute(args)) + ), + } +} diff --git a/apps/sim/lib/knowledge/application/search.ts b/apps/sim/lib/knowledge/application/search.ts index 9568e817c9d..1bcd42670b8 100644 --- a/apps/sim/lib/knowledge/application/search.ts +++ b/apps/sim/lib/knowledge/application/search.ts @@ -33,6 +33,7 @@ import { resolveKnowledgeWorkspaceContext, } from '@/lib/knowledge/application/contexts' import { knowledgeOperations } from '@/lib/knowledge/application/operations' +import { instrumentSearchUseCase } from '@/lib/knowledge/application/search-diagnostics' import { ALL_TAG_SLOTS } from '@/lib/knowledge/constants' import { getEmbeddingModelInfo, toKbEmbeddingDimensions } from '@/lib/knowledge/embedding-models' import { generateSearchEmbedding, type KbEmbeddingTarget } from '@/lib/knowledge/embeddings' @@ -41,6 +42,7 @@ import { rerank } from '@/lib/knowledge/reranker' import type { RerankerStatus } from '@/lib/knowledge/reranker-models' import { recordOrganizationSearchActivity } from '@/lib/knowledge/search/activity' import { resolveKnowledgeSearchDefaults } from '@/lib/knowledge/search/defaults' +import { annotateSearchDiagnostics, measureSearchStage } from '@/lib/knowledge/search/diagnostics' import type { WorkspaceSearchFilters } from '@/lib/knowledge/search/filters' import { executeKnowledgeSearch, @@ -252,13 +254,20 @@ async function resolveKnowledgeSearchContext( } } -export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ +const searchKnowledgeUseCase = defineAuthorizedKnowledgeUseCase({ operation: knowledgeOperations.search, resolveContext: ({ principal, input }: { principal: Principal; input: SearchKnowledgeInput }) => - resolveKnowledgeSearchContext(input, principal), + measureSearchStage('knowledge_context', () => resolveKnowledgeSearchContext(input, principal)), async execute({ principal, input, context }) { + annotateSearchDiagnostics({ + scopeKind: context.organizationId ? 'organization' : 'workspace', + knowledgeBaseCount: context.knowledgeBases.length, + }) input.signal?.throwIfAborted() - if (context.organizationId) await requireOrganizationSearchAvailable(context.organizationId) + if (context.organizationId) + await measureSearchStage('availability', () => + requireOrganizationSearchAvailable(context.organizationId!) + ) const requestId = generateRequestId() const hasQuery = Boolean(input.query?.trim()) const filters = input.tagFilters ?? [] @@ -276,11 +285,17 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ ) const billingAttribution = hasQuery ? input.resolveBillingAttribution && context.workspaceId - ? await input.resolveBillingAttribution(context.workspaceId) - : await resolveKnowledgeBillingAttribution(principal, context) + ? await measureSearchStage('billing_attribution', () => + input.resolveBillingAttribution!(context.workspaceId!) + ) + : await measureSearchStage('billing_attribution', () => + resolveKnowledgeBillingAttribution(principal, context) + ) : undefined if (shouldMeter && billingAttribution) { - const usage = await checkAttributedUsageLimits(billingAttribution) + const usage = await measureSearchStage('usage_admission', () => + checkAttributedUsageLimits(billingAttribution) + ) if (usage.isExceeded) { throw new KnowledgeUsageLimitExceededError( usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.' @@ -292,7 +307,9 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ let structuredFilters: StructuredFilter[] = [] let definitionsByKnowledgeBase = new Map() if (filters.length > 0) { - const built = await resolveKnowledgeTagFilters(filters, knowledgeBaseIds) + const built = await measureSearchStage('tag_filters', () => + resolveKnowledgeTagFilters(filters, knowledgeBaseIds) + ) structuredFilters = built.structuredFilters definitionsByKnowledgeBase = built.definitionsByKnowledgeBase } @@ -333,32 +350,44 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ } : undefined const preparedRegistry = input.prepareModelInputProvenance - ? await input.prepareModelInputProvenance({ userId, workspaceId: context.workspaceId }) + ? await measureSearchStage('input_provenance', () => + input.prepareModelInputProvenance!({ userId, workspaceId: context.workspaceId }) + ) : undefined const resultSecretRegistry = preparedRegistry ?? input.resultSecretRegistry input.signal?.throwIfAborted() const [queryEmbedding, access, searchDefaults] = await Promise.all([ hasQuery - ? runWithKnowledgeModelInputProvenance(resultSecretRegistry, () => - generateSearchEmbedding( - input.query!, - embeddingTarget!, - context.workspaceId, - input.signal + ? measureSearchStage('embedding', () => + runWithKnowledgeModelInputProvenance(resultSecretRegistry, () => + generateSearchEmbedding( + input.query!, + embeddingTarget!, + context.workspaceId, + input.signal + ) ) ) : Promise.resolve(null), - context.access.get(), - resolveKnowledgeSearchDefaults({ - workspaceId: context.workspaceId, - organizationId: context.organizationId, + measureSearchStage('access_scope', () => context.access.get()), + measureSearchStage('defaults', () => + resolveKnowledgeSearchDefaults({ + workspaceId: context.workspaceId, + organizationId: context.organizationId, - /** The signed-in person, if any; never the billing owner or a key's creator. */ - userId: resolvePrincipalSubjectUserId(principal) ?? undefined, - requestedMode: input.searchMode, - }), + /** The signed-in person, if any; never the billing owner or a key's creator. */ + userId: resolvePrincipalSubjectUserId(principal) ?? undefined, + requestedMode: input.searchMode, + }) + ), ]) input.signal?.throwIfAborted() + annotateSearchDiagnostics({ + accessScopeKind: access.kind, + searchMode: searchDefaults.searchMode, + boostRecency: searchDefaults.boostRecency, + embeddingDimensions: embeddingTarget?.dimensions, + }) const useReranker = Boolean(input.rerankerEnabled && hasQuery) const candidateTopK = useReranker ? input.rerankerInputCount !== undefined @@ -368,24 +397,26 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ ) : Math.min(KNOWLEDGE_SEARCH_COST_POLICY.maxTopK, input.topK * 4) : input.topK - let rows = await executeKnowledgeSearch({ - knowledgeBaseIds, - topK: candidateTopK, - filters: input.filters, - access, - accessProvider: context.access, - signal: input.signal, - searchMode: searchDefaults.searchMode, - boostRecency: searchDefaults.boostRecency, - query: input.query, - queryVector: hasQuery - ? { - vector: JSON.stringify(queryEmbedding?.embedding ?? null), - dimensions: embeddingTarget!.dimensions, - } - : undefined, - structuredFilters: structuredFilters.length > 0 ? structuredFilters : undefined, - }) + let rows = await measureSearchStage('retrieval', () => + executeKnowledgeSearch({ + knowledgeBaseIds, + topK: candidateTopK, + filters: input.filters, + access, + accessProvider: context.access, + signal: input.signal, + searchMode: searchDefaults.searchMode, + boostRecency: searchDefaults.boostRecency, + query: input.query, + queryVector: hasQuery + ? { + vector: JSON.stringify(queryEmbedding?.embedding ?? null), + dimensions: embeddingTarget!.dimensions, + } + : undefined, + structuredFilters: structuredFilters.length > 0 ? structuredFilters : undefined, + }) + ) input.signal?.throwIfAborted() /** Public callers have no input envelope, but persisted reranker inputs still need provenance. */ @@ -404,10 +435,12 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ ReturnType > | null = null if (registry) { - provenanceSnapshot = await importKnowledgeSearchResultSecretProvenance({ - registry, - results: rows, - }) + provenanceSnapshot = await measureSearchStage('result_provenance', () => + importKnowledgeSearchResultSecretProvenance({ + registry, + results: rows, + }) + ) if (!provenanceSnapshot.imported) { registry.markIncomplete('knowledge-result-provenance-unavailable') if (useReranker) { @@ -456,18 +489,20 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ if (useReranker && input.rerankerModel && rows.length > 0) { const candidateCount = rows.length try { - const reranked = await runWithKnowledgeModelInputProvenance(registry, () => - rerank( - input.query!, - rows.map((row) => ({ id: row.id, text: row.content })), - { - model: input.rerankerModel!, - topN: input.topK, - workspaceId: context.workspaceId, + const reranked = await measureSearchStage('reranking', () => + runWithKnowledgeModelInputProvenance(registry, () => + rerank( + input.query!, + rows.map((row) => ({ id: row.id, text: row.content })), + { + model: input.rerankerModel!, + topN: input.topK, + workspaceId: context.workspaceId, - apiKey: input.rerankerApiKey, - signal: input.signal, - } + apiKey: input.rerankerApiKey, + signal: input.signal, + } + ) ) ) rerankerBilled = true @@ -537,29 +572,34 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ } if (shouldMeter && billingAttribution && baseCost && baseCost.total > 0) { try { - await recordUsage({ - userId, - ...(context.workspaceId ? { workspaceId: context.workspaceId } : {}), - ...toBillingContext(billingAttribution), - entries: [ - { - category: 'model', - source: 'knowledge-base', - description: embeddingModel, - cost: baseCost.total, - sourceReference: `kb-search:${requestId}`, - }, - ], - }) - await checkAndBillPayerOverageThreshold(billingAttribution.billingEntity) + await measureSearchStage('usage_recording', () => + recordUsage({ + userId, + ...(context.workspaceId ? { workspaceId: context.workspaceId } : {}), + ...toBillingContext(billingAttribution), + entries: [ + { + category: 'model', + source: 'knowledge-base', + description: embeddingModel, + cost: baseCost.total, + sourceReference: `kb-search:${requestId}`, + }, + ], + }) + ) + await measureSearchStage('overage_billing', () => + checkAndBillPayerOverageThreshold(billingAttribution.billingEntity) + ) } catch (error) { logger.error('Failed to record Knowledge search usage', { error }) } } if (filters.length === 0) { - definitionsByKnowledgeBase = - await getDocumentTagDefinitionsByKnowledgeBaseIds(knowledgeBaseIds) + definitionsByKnowledgeBase = await measureSearchStage('tag_definitions', () => + getDocumentTagDefinitionsByKnowledgeBaseIds(knowledgeBaseIds) + ) } const tagMaps = new Map( [...definitionsByKnowledgeBase].map(([knowledgeBaseId, definitions]) => [ @@ -572,11 +612,13 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ * a model may see, but the source card's modified time and connector type * are only carried here, under the same access predicate as the search. */ - const basicDocumentMetadata = await getDocumentMetadataByIds( - rows.map((row) => row.documentId), - access, - context.access, - input.signal + const basicDocumentMetadata = await measureSearchStage('metadata', () => + getDocumentMetadataByIds( + rows.map((row) => row.documentId), + access, + context.access, + input.signal + ) ) const results = rows .filter((row) => basicDocumentMetadata[row.documentId]) @@ -625,12 +667,14 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ if (renderedMetadata.length === 0) continue if (document.provenance.status === 'unknown' && !knowledgeEnforced) unrecordedCount += 1 if ( - !(await importDurableSecretProvenance( - registry, - document.provenance, - renderedMetadata, - 'knowledge', - { reportUnrecorded: false } + !(await measureSearchStage('metadata_provenance', () => + importDurableSecretProvenance( + registry, + document.provenance, + renderedMetadata, + 'knowledge', + { reportUnrecorded: false } + ) )) ) { registry.markIncomplete('knowledge-result-provenance-unavailable') @@ -652,6 +696,7 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ }) } } + annotateSearchDiagnostics({ resultCount: results.length }) const cost = baseCost ? { input: baseCost.input, @@ -691,12 +736,14 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ afterSuccess: async ({ principal, context, input, result }) => { const actorUserId = resolvePrincipalSubjectUserId(principal) if (context.organizationId && actorUserId) { - await recordOrganizationSearchActivity({ - organizationId: context.organizationId, - userId: actorUserId, - surface: input.surface ?? 'other', - results: result.results, - }) + await measureSearchStage('activity_recording', () => + recordOrganizationSearchActivity({ + organizationId: context.organizationId, + userId: actorUserId, + surface: input.surface ?? 'other', + results: result.results, + }) + ) } PlatformEvents.knowledgeBaseSearched({ knowledgeBaseId: result.knowledgeBaseId, @@ -720,3 +767,8 @@ export const searchKnowledge = defineAuthorizedKnowledgeUseCase({ }) }, }) + +export const searchKnowledge = instrumentSearchUseCase( + 'knowledge_application', + searchKnowledgeUseCase +) diff --git a/apps/sim/lib/knowledge/application/slack-search/installations.ts b/apps/sim/lib/knowledge/application/slack-search/installations.ts index cd53db31d22..801be121a54 100644 --- a/apps/sim/lib/knowledge/application/slack-search/installations.ts +++ b/apps/sim/lib/knowledge/application/slack-search/installations.ts @@ -15,7 +15,10 @@ import { resolveKnowledgeOrganizationContext } from '@/lib/knowledge/application import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { loadSlackSearchCredential } from '@/lib/knowledge/application/slack-search/repository' import { SLACK_CUSTOM_BOT_PROVIDER_ID } from '@/lib/oauth/types' -import { slackBotCredentialVersion } from '@/lib/slack-search/app-configuration' +import { + resolveSlackAppCredentials, + slackBotCredentialVersion, +} from '@/lib/slack-search/app-configuration' import { SLACK_SHARED_SEARCH_BOT_SCOPES } from '@/lib/slack-search/constants' import { readSharedSlackSearchApp, @@ -85,12 +88,15 @@ export const listSlackSearchInstallations = defineAuthorizedKnowledgeUseCase({ sharedAppAvailable: Boolean(sharedApp), installations: installations.map(({ credentialVersion, ...installation }) => { const bot = bots.find((bot) => bot.id === installation.credentialId) + const shared = installation.appKind === 'shared' + const appRevision = shared ? sharedApp?.revision : bot?.appRevision return { ...installation, appKind: installation.appKind ?? 'custom', needsValidation: + (shared && sharedApp?.id !== installation.appId) || !bot?.encryptedKey || - slackBotCredentialVersion(bot.encryptedKey, bot.appRevision ?? undefined) !== + slackBotCredentialVersion(bot.encryptedKey, appRevision ?? undefined) !== credentialVersion, } }), @@ -154,10 +160,12 @@ export const configureSlackSearchInstallation = defineAuthorizedKnowledgeUseCase if (input.enabled && current.slackAppId) await requireSlackSearchAppAvailable(current.slackAppId) if (current.slackAppId && !app) throw new Error('Slack app configuration is missing') + const appRevision = + app && secret ? (await resolveSlackAppCredentials(app)).revision : undefined if ( secret && (!current.encryptedServiceAccountKey || - slackBotCredentialVersion(current.encryptedServiceAccountKey, app?.revision) !== + slackBotCredentialVersion(current.encryptedServiceAccountKey, appRevision) !== secret.version) ) throw new OrchestrationError('conflict', 'The bot credential changed. Validate it again.') diff --git a/apps/sim/lib/knowledge/application/slack-search/setup.test.ts b/apps/sim/lib/knowledge/application/slack-search/setup.test.ts index 2f197b744b9..214f5f3a290 100644 --- a/apps/sim/lib/knowledge/application/slack-search/setup.test.ts +++ b/apps/sim/lib/knowledge/application/slack-search/setup.test.ts @@ -122,6 +122,7 @@ beforeEach(() => { values: m.values, set: m.set, onConflictDoUpdate: vi.fn(), + onConflictDoNothing: vi.fn(), returning: vi.fn().mockResolvedValue([{ id: 'credential1' }]), } for (const method of [ @@ -131,6 +132,7 @@ beforeEach(() => { txQuery.values, txQuery.set, txQuery.onConflictDoUpdate, + txQuery.onConflictDoNothing, ]) method.mockReturnValue(txQuery) const tx = { @@ -299,7 +301,15 @@ it('rejects a shared-app callback if the global configuration was disabled or ro }) describe('shared app completion', () => { - const sharedApp = { id: 'A1', revision: 'shared-revision', kind: 'shared', organizationId: null } + const sharedApp = { + id: 'A1', + revision: 'shared-revision', + kind: 'shared', + organizationId: null, + clientId: 'client', + clientSecret: 'environment-secret', + signingSecret: 'environment-signing', + } beforeEach(() => { m.shared.mockResolvedValue(sharedApp) m.consume.mockResolvedValue({ @@ -308,9 +318,27 @@ describe('shared app completion', () => { }) }) - it('commits the personal app configuration, bot credential and installation in one transaction', async () => { + it('starts shared OAuth without storing deployment secrets in the attempt', async () => { + const result = await startSlackSearchSetup.execute({ + principal, + input: { + organizationId: 'org1', + mode: 'shared', + name: 'Sim Search', + description: 'Search with sources', + }, + }) + expect(new URL(result.authorizationUrl).searchParams.get('client_id')).toBe('client') + const stored = m.store.mock.calls[0][0] + expect(stored.sharedApp).toEqual({ id: 'A1', revision: 'shared-revision' }) + expect(stored).not.toHaveProperty('encryptedClientSecret') + expect(stored).not.toHaveProperty('encryptedSigningSecret') + expect(JSON.stringify(stored)).not.toContain('environment-secret') + }) + + it('creates shared identity without app secrets and installs atomically without registration', async () => { m.rows - .mockResolvedValueOnce([sharedApp]) + .mockResolvedValueOnce([]) .mockResolvedValueOnce([]) .mockResolvedValueOnce([]) .mockResolvedValueOnce([ @@ -350,16 +378,27 @@ describe('shared app completion', () => { }) expect(configuration.slack).not.toHaveProperty('clientSecret') const rows = m.values.mock.calls.map(([value]) => value) - expect(rows).toHaveLength(2) - expect(rows[0]).toMatchObject({ + expect(rows).toHaveLength(3) + expect(rows[0]).toEqual({ + id: 'A1', + kind: 'shared', + organizationId: null, + revision: 'shared-revision', + }) + expect(m.exchange).toHaveBeenCalledWith( + expect.objectContaining({ clientSecret: 'environment-secret' }) + ) + expect(JSON.stringify(rows)).not.toContain('environment-secret') + expect(JSON.stringify(rows)).not.toContain('environment-signing') + expect(rows[1]).toMatchObject({ organizationId: 'org1', workspaceId: null, type: 'service_account', slackAppId: 'A1', }) - expect(rows[1]).toMatchObject({ + expect(rows[2]).toMatchObject({ organizationId: 'org1', - credentialId: rows[0].id, + credentialId: rows[1].id, slackAppId: 'A1', appId: 'A1', teamId: 'T1', @@ -382,13 +421,7 @@ describe('shared app completion', () => { }) it('revokes an unused shared grant after a database write fails', async () => { - m.rows - .mockResolvedValueOnce([sharedApp]) - .mockResolvedValueOnce([]) - .mockResolvedValueOnce([]) - .mockResolvedValueOnce([ - { id: 'accounts', options: [], encryptedProviderConfiguration: null }, - ]) + m.rows.mockResolvedValueOnce([sharedApp]).mockResolvedValueOnce([]).mockResolvedValueOnce([]) m.values.mockImplementationOnce(() => { throw new Error('write failed') }) diff --git a/apps/sim/lib/knowledge/application/slack-search/setup.ts b/apps/sim/lib/knowledge/application/slack-search/setup.ts index 4a6c46d2da5..b2e259a2a65 100644 --- a/apps/sim/lib/knowledge/application/slack-search/setup.ts +++ b/apps/sim/lib/knowledge/application/slack-search/setup.ts @@ -138,7 +138,8 @@ export const startSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ ) if (savedApp?.kind === 'custom' && savedApp.organizationId !== context.organizationId) throw new OrchestrationError('forbidden', 'Slack app ownership changed') - const app = shared ? await readSharedSlackSearchApp() : savedApp + const sharedApp = shared ? await readSharedSlackSearchApp() : null + const app = shared ? sharedApp : savedApp if (shared && (!app || input.clientId || input.clientSecret || input.signingSecret)) throw new OrchestrationError( 'validation', @@ -150,20 +151,29 @@ export const startSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ 'Remove the previous Slack source configuration before switching apps; members must reconnect' ) const clientId = input.clientId ?? app?.clientId - const encryptedClientSecret = input.clientSecret - ? (await encryptSecret(input.clientSecret)).encrypted - : app?.encryptedClientSecret - const encryptedSigningSecret = input.signingSecret - ? (await encryptSecret(input.signingSecret)).encrypted - : app?.encryptedSigningSecret - if (!clientId || !encryptedClientSecret || !encryptedSigningSecret) - throw new OrchestrationError( - 'validation', - 'Client ID, Client Secret, and Signing Secret are required for a new Slack app' - ) + if (!clientId) throw new OrchestrationError('validation', 'Slack Client ID is required') + let appCredentials: + | { sharedApp: { id: string; revision: string } } + | { encryptedClientSecret: string; encryptedSigningSecret: string } + if (sharedApp) { + appCredentials = { sharedApp: { id: sharedApp.id, revision: sharedApp.revision } } + } else { + const encryptedClientSecret = input.clientSecret + ? (await encryptSecret(input.clientSecret)).encrypted + : savedApp?.encryptedClientSecret + const encryptedSigningSecret = input.signingSecret + ? (await encryptSecret(input.signingSecret)).encrypted + : savedApp?.encryptedSigningSecret + if (!encryptedClientSecret || !encryptedSigningSecret) + throw new OrchestrationError( + 'validation', + 'Client ID, Client Secret, and Signing Secret are required for a new Slack app' + ) + appCredentials = { encryptedClientSecret, encryptedSigningSecret } + } const redirectUri = new URL(SLACK_SEARCH_CALLBACK_PATH, origin).href const state = await storeSlackSearchOAuthAttempt({ - ...(shared && app ? { sharedApp: { id: app.id, revision: app.revision } } : {}), + ...appCredentials, userId: principal.userId, sessionId: principal.sessionId, organizationId: context.organizationId, @@ -171,8 +181,6 @@ export const startSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ description: input.description, ...(member.app ? { memberApp: member.app } : {}), clientId, - encryptedClientSecret, - encryptedSigningSecret, redirectUri, createdAt: Date.now(), ...(installation @@ -256,15 +264,22 @@ export const completeSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ ) await requireOrganizationSearchAvailable(context.organizationId) const { attempt } = context + let clientSecret: string if (attempt.sharedApp) { const app = await readSharedSlackSearchApp() - if (app?.id !== attempt.sharedApp.id || app.revision !== attempt.sharedApp.revision) + if ( + app?.id !== attempt.sharedApp.id || + app.revision !== attempt.sharedApp.revision || + app.clientId !== attempt.clientId + ) throw new OrchestrationError( 'conflict', 'Shared Slack app configuration changed. Start again.' ) + clientSecret = app.clientSecret + } else { + clientSecret = (await decryptSecret(attempt.encryptedClientSecret)).decrypted } - const { decrypted: clientSecret } = await decryptSecret(attempt.encryptedClientSecret) const grant = await exchangeSlackBotAuthorization({ clientId: attempt.clientId, clientSecret, @@ -345,10 +360,7 @@ export const completeSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ .limit(1) if ( attempt.sharedApp - ? !existingApp || - existingApp.kind !== 'shared' || - existingApp.organizationId !== null || - existingApp.revision !== attempt.sharedApp.revision + ? existingApp && (existingApp.kind !== 'shared' || existingApp.organizationId !== null) : existingApp && (existingApp.kind !== 'custom' || existingApp.organizationId !== context.organizationId) @@ -358,6 +370,7 @@ export const completeSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ 'This Slack app belongs to another installation owner' ) if ( + !attempt.sharedApp && attempt.installation?.appRevision && existingApp?.revision !== attempt.installation.appRevision ) @@ -413,6 +426,12 @@ export const completeSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ 'This Slack workspace already has an active Search installation' ) if (attempt.sharedApp) { + const currentApp = await readSharedSlackSearchApp() + if ( + currentApp?.id !== attempt.sharedApp.id || + currentApp.revision !== attempt.sharedApp.revision + ) + throw new OrchestrationError('conflict', 'Shared Slack app configuration changed') /** A concurrent failed setup may have revoked an uncommitted grant while we waited. */ const current = await verifySlackSearchBot( grant.access_token, @@ -430,21 +449,33 @@ export const completeSlackSearchSetup = defineAuthorizedKnowledgeUseCase({ ) } const appRevision = attempt.sharedApp?.revision ?? generateId() - const appValues = { - id: identity.appId, - kind: 'custom' as const, - organizationId: context.organizationId, - clientId: attempt.clientId, - encryptedClientSecret: attempt.encryptedClientSecret, - encryptedSigningSecret: attempt.encryptedSigningSecret, - revision: appRevision, - updatedAt: new Date(), - } - if (!attempt.sharedApp) + if (attempt.sharedApp) { + /** The row supplies foreign-key identity only; shared secrets remain in the environment. */ + await tx + .insert(slackApp) + .values({ + id: identity.appId, + kind: 'shared', + organizationId: null, + revision: appRevision, + }) + .onConflictDoNothing() + } else { + const appValues = { + id: identity.appId, + kind: 'custom' as const, + organizationId: context.organizationId, + clientId: attempt.clientId, + encryptedClientSecret: attempt.encryptedClientSecret, + encryptedSigningSecret: attempt.encryptedSigningSecret, + revision: appRevision, + updatedAt: new Date(), + } await tx .insert(slackApp) .values(appValues) .onConflictDoUpdate({ target: slackApp.id, set: appValues }) + } await adoptOrganizationSlackMemberApp( tx, context.organizationId, diff --git a/apps/sim/lib/knowledge/application/workspace-search.ts b/apps/sim/lib/knowledge/application/workspace-search.ts index 4c44f62cf0e..983c7819011 100644 --- a/apps/sim/lib/knowledge/application/workspace-search.ts +++ b/apps/sim/lib/knowledge/application/workspace-search.ts @@ -9,7 +9,9 @@ import { } from '@/lib/knowledge/application/contexts' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { type SearchKnowledgeInput, searchKnowledge } from '@/lib/knowledge/application/search' +import { instrumentSearchUseCase } from '@/lib/knowledge/application/search-diagnostics' import { recordOrganizationSearchActivity } from '@/lib/knowledge/search/activity' +import { measureSearchStage } from '@/lib/knowledge/search/diagnostics' import { findSearchIndex, findWorkspaceSearchIndex } from '@/lib/knowledge/search/search-index' export type SearchWorkspaceKnowledgeInput = Omit< @@ -20,13 +22,15 @@ export type SearchWorkspaceKnowledgeInput = Omit< } /** Search and Assistant share the workspace's canonical Enterprise Search index. */ -export const searchWorkspaceKnowledge = defineAuthorizedKnowledgeUseCase({ +const searchWorkspaceKnowledgeUseCase = defineAuthorizedKnowledgeUseCase({ operation: knowledgeOperations.search, resolveContext: ({ input }: { input: SearchWorkspaceKnowledgeInput }) => - resolveKnowledgeWorkspaceContext(input), + measureSearchStage('scope_resolution', () => resolveKnowledgeWorkspaceContext(input)), async execute({ principal, input, context }) { input.signal?.throwIfAborted() - const index = await findWorkspaceSearchIndex(context.workspaceId) + const index = await measureSearchStage('index_resolution', () => + findWorkspaceSearchIndex(context.workspaceId) + ) if (!index) return { results: [], query: input.query ?? '', knowledgeBases: [] } return searchKnowledge.execute({ principal, @@ -35,22 +39,29 @@ export const searchWorkspaceKnowledge = defineAuthorizedKnowledgeUseCase({ }, }) +export const searchWorkspaceKnowledge = instrumentSearchUseCase( + 'workspace_application', + searchWorkspaceKnowledgeUseCase +) + export type SearchOrganizationKnowledgeInput = Omit< SearchWorkspaceKnowledgeInput, 'workspaceId' > & { organizationId: string } /** Organization Search and Assistant resolve the same index and provider ACLs. */ -export const searchOrganizationKnowledge = defineAuthorizedKnowledgeUseCase({ +const searchOrganizationKnowledgeUseCase = defineAuthorizedKnowledgeUseCase({ operation: knowledgeOperations.search, resolveContext: ({ input }: { input: SearchOrganizationKnowledgeInput }) => - resolveKnowledgeOrganizationContext(input), + measureSearchStage('scope_resolution', () => resolveKnowledgeOrganizationContext(input)), async execute({ principal, input, context }) { input.signal?.throwIfAborted() - const index = await findSearchIndex({ - kind: 'organization', - organizationId: context.organizationId, - }) + const index = await measureSearchStage('index_resolution', () => + findSearchIndex({ + kind: 'organization', + organizationId: context.organizationId, + }) + ) if (!index) { if (context.organizationId) { await requireOrganizationSearchAvailable(context.organizationId) @@ -66,10 +77,18 @@ export const searchOrganizationKnowledge = defineAuthorizedKnowledgeUseCase({ } return { results: [], query: input.query ?? '', knowledgeBases: [] } } - return searchKnowledge.execute({ principal, input: { ...input, knowledgeBaseIds: [index.id] } }) + return searchKnowledge.execute({ + principal, + input: { ...input, knowledgeBaseIds: [index.id] }, + }) }, }) +export const searchOrganizationKnowledge = instrumentSearchUseCase( + 'organization_application', + searchOrganizationKnowledgeUseCase +) + export type SearchScopedKnowledgeInput = Omit< SearchKnowledgeInput, 'knowledgeBaseIds' | 'workspaceId' | 'organizationId' @@ -77,13 +96,15 @@ export type SearchScopedKnowledgeInput = Omit< ResourceOwner /** The routed owner selects the index; current membership and provider ACLs select its documents. */ -export const searchScopedKnowledge = defineAuthorizedKnowledgeUseCase({ +const searchScopedKnowledgeUseCase = defineAuthorizedKnowledgeUseCase({ operation: knowledgeOperations.search, resolveContext: ({ input }: { input: SearchScopedKnowledgeInput }) => - resolveKnowledgeOwnerContext(input), + measureSearchStage('scope_resolution', () => resolveKnowledgeOwnerContext(input)), async execute({ principal, input, context }) { input.signal?.throwIfAborted() - const index = await findSearchIndex(resourceScopeFromOwner(context)) + const index = await measureSearchStage('index_resolution', () => + findSearchIndex(resourceScopeFromOwner(context)) + ) if (!index) { if (context.organizationId) { await requireOrganizationSearchAvailable(context.organizationId) @@ -110,3 +131,8 @@ export const searchScopedKnowledge = defineAuthorizedKnowledgeUseCase({ }) }, }) + +export const searchScopedKnowledge = instrumentSearchUseCase( + 'scoped_application', + searchScopedKnowledgeUseCase +) diff --git a/apps/sim/lib/knowledge/documents/document-processing-source.test.ts b/apps/sim/lib/knowledge/documents/document-processing-source.test.ts index 5b36cbdf3c2..24b77c321fa 100644 --- a/apps/sim/lib/knowledge/documents/document-processing-source.test.ts +++ b/apps/sim/lib/knowledge/documents/document-processing-source.test.ts @@ -420,6 +420,142 @@ describe('knowledge document processing source', () => { expect(mockGenerateEmbeddings).not.toHaveBeenCalled() }) + describe('execution source provenance', () => { + const executionKey = 'execution/workspace-1/workflow-1/run-1/source.pdf' + const executionUrl = `/api/files/serve/${encodeURIComponent(executionKey)}?context=workspace` + const executionBinding = { + ...SOURCE_BINDING, + key: executionKey, + context: 'execution', + secretProvenanceVersion: 1, + } + + beforeEach(() => { + dbChainMockFns.limit + .mockReset() + .mockResolvedValueOnce([{ ...PERSISTED_CONTEXT, fileUrl: executionUrl }]) + .mockResolvedValueOnce([{ ...PERSISTED_PROVENANCE_ROW, fileUrl: executionUrl }]) + .mockResolvedValueOnce([{ id: 'document-1' }]) + mockGetFileMetadataByKeys.mockImplementation(async (_keys: string[], context: string) => + context === 'execution' ? [executionBinding] : [] + ) + }) + + function process() { + return processDocumentAsync( + 'knowledge-base-1', + 'document-1', + { + filename: 'untrusted-queued-name.txt', + fileUrl: 'https://example.com/untrusted-queued-url.txt', + fileSize: 1, + mimeType: 'text/plain', + }, + {}, + BILLING_ATTRIBUTION + ) + } + + it('loads persisted execution lineage before parsing, ignoring the URL context label', async () => { + await process() + + expect(mockGetFileMetadataByKeys).toHaveBeenCalledWith( + [executionKey], + 'execution', + expect.anything(), + { includeDeleted: true } + ) + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).toHaveBeenCalledWith( + expect.anything(), + [executionBinding] + ) + expect(mockProcessDocument).toHaveBeenCalledWith( + executionUrl, + PERSISTED_CONTEXT.filename, + PERSISTED_CONTEXT.mimeType, + 1024, + 200, + 100, + expect.objectContaining({ userId: BILLING_ATTRIBUTION.actorUserId }), + PERSISTED_CONTEXT.workspaceId, + undefined, + undefined + ) + }) + + it.each(['unknown', 'missing'])( + 'refuses tracked execution sources with %s sidecars before parsing', + async (kind) => { + mockGetBoundWorkspaceFileSecretProvenanceByMetadata.mockResolvedValue( + new Map(kind === 'unknown' ? [[executionBinding.id, { status: 'unknown' }]] : []) + ) + + await expect(process()).rejects.toThrow( + 'Knowledge document secret provenance is unavailable' + ) + + expect(mockProcessDocument).not.toHaveBeenCalled() + expect(mockGenerateEmbeddings).not.toHaveBeenCalled() + } + ) + + it('refuses a soft-deleted tracked execution source before parsing', async () => { + const deletedBinding = { ...executionBinding, deletedAt: CONTENT_UPDATED_AT } + mockGetFileMetadataByKeys.mockImplementation( + async ( + _keys: string[], + context: string, + _executor: unknown, + options?: { includeDeleted?: boolean } + ) => (context === 'execution' && options?.includeDeleted ? [deletedBinding] : []) + ) + mockGetBoundWorkspaceFileSecretProvenanceByMetadata.mockResolvedValue( + new Map([[executionBinding.id, { status: 'unknown' }]]) + ) + + await expect(process()).rejects.toThrow('Knowledge document secret provenance is unavailable') + + expect(mockProcessDocument).not.toHaveBeenCalled() + expect(mockGenerateEmbeddings).not.toHaveBeenCalled() + }) + + it('preserves legacy null-marker behavior for a soft-deleted execution source', async () => { + mockGetFileMetadataByKeys.mockResolvedValue([ + { ...executionBinding, deletedAt: CONTENT_UPDATED_AT, secretProvenanceVersion: null }, + ]) + + await process() + + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).not.toHaveBeenCalled() + expect(mockProcessDocument).toHaveBeenCalled() + }) + + it.each(['missing', 'untracked'])( + 'retains legacy %s execution source behavior', + async (kind) => { + mockGetFileMetadataByKeys.mockResolvedValue( + kind === 'missing' ? [] : [{ ...executionBinding, secretProvenanceVersion: null }] + ) + + await process() + + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).not.toHaveBeenCalled() + expect(mockProcessDocument).toHaveBeenCalled() + } + ) + + it('refuses a source whose execution metadata belongs to another workspace', async () => { + mockGetFileMetadataByKeys.mockResolvedValue([ + { ...executionBinding, workspaceId: 'other-workspace' }, + ]) + + await expect(process()).rejects.toThrow('Document file is not owned by this knowledge base') + + expect(mockProcessDocument).not.toHaveBeenCalled() + expect(mockGenerateEmbeddings).not.toHaveBeenCalled() + }) + }) + it('takes over an existing processing attempt', async () => { dbChainMockFns.limit .mockReset() diff --git a/apps/sim/lib/knowledge/documents/service.ts b/apps/sim/lib/knowledge/documents/service.ts index d3dea4c4ecc..65eb4b14f95 100644 --- a/apps/sim/lib/knowledge/documents/service.ts +++ b/apps/sim/lib/knowledge/documents/service.ts @@ -184,7 +184,7 @@ const logger = createLogger('DocumentService') /** * Thrown when a knowledge-base document's `fileUrl` references an internal - * knowledge-base storage object not owned by the target knowledge base's workspace. + * knowledge-base or execution object not owned by the target knowledge base's workspace. * Routes map this to a 403. * * Deliberately carries no `details.code`. It belongs to the cross-tenant class @@ -214,12 +214,15 @@ function getKnowledgeBaseStorageKeys(fileUrls: readonly string[]): string[] { ] } -function getWorkspaceSourceStorageKeys(fileUrls: readonly string[]): string[] { +function getSourceStorageKeys( + fileUrls: readonly string[], + context: 'workspace' | 'execution' +): string[] { return [ ...new Set( fileUrls .map((url) => getKnowledgeBaseStorageKey(url)) - .filter((key): key is string => typeof key === 'string' && key.startsWith('workspace/')) + .filter((key): key is string => typeof key === 'string' && key.startsWith(`${context}/`)) ), ] } @@ -238,18 +241,39 @@ async function loadKnowledgeBaseFileBindings( return new Map(bindings.map((binding) => [binding.key, binding])) } -async function loadWorkspaceSourceFileBindings( +/** Execution metadata without a provenance marker predates stamping and remains a legacy source. */ +async function loadSourceFileBindings( fileUrls: readonly string[], + workspaceId: string | null, executor: DbExecutor = db ): Promise> { - const keys = getWorkspaceSourceStorageKeys(fileUrls) - if (keys.length === 0) return new Map() + const workspaceKeys = getSourceStorageKeys(fileUrls, 'workspace') + const executionKeys = getSourceStorageKeys(fileUrls, 'execution') + const workspaceBindings = + workspaceKeys.length > 0 + ? await getFileMetadataByKeys(workspaceKeys, 'workspace', executor) + : [] + const mothershipBindings = + workspaceKeys.length > 0 + ? await getFileMetadataByKeys(workspaceKeys, 'mothership', executor) + : [] + const executionBindings = + executionKeys.length > 0 + ? await getFileMetadataByKeys(executionKeys, 'execution', executor, { includeDeleted: true }) + : [] - const workspaceBindings = await getFileMetadataByKeys(keys, 'workspace', executor) - const mothershipBindings = await getFileMetadataByKeys(keys, 'mothership', executor) + for (const binding of executionBindings) { + if (!workspaceId || binding.workspaceId !== workspaceId) { + throw new KnowledgeBaseFileOwnershipError(binding.key) + } + } return new Map( - [...workspaceBindings, ...mothershipBindings].map((binding) => [binding.key, binding]) + [ + ...workspaceBindings, + ...mothershipBindings, + ...executionBindings.filter((binding) => binding.secretProvenanceVersion !== null), + ].map((binding) => [binding.key, binding]) ) } @@ -281,13 +305,14 @@ async function assertKnowledgeBaseFileUrlsOwnership( return bindingByKey } -async function loadCurrentWorkspaceSourceFileSecretProvenance(options: { +async function loadCurrentSourceFileSecretProvenance(options: { fileUrl: string + workspaceId: string | null }): Promise { const storageKey = getKnowledgeBaseStorageKey(options.fileUrl) - if (!storageKey?.startsWith('workspace/')) return undefined + if (!storageKey) return undefined - const bindingByKey = await loadWorkspaceSourceFileBindings([options.fileUrl]) + const bindingByKey = await loadSourceFileBindings([options.fileUrl], options.workspaceId) const binding = bindingByKey.get(storageKey) if (!binding) return undefined @@ -388,7 +413,7 @@ interface DocumentTagData { type TagDefinition = typeof knowledgeBaseTagDefinitions.$inferSelect type TagDefinitionsByName = Map -type DbExecutor = Pick +type DbExecutor = Pick async function loadTagDefinitions( knowledgeBaseId: string, @@ -1638,8 +1663,9 @@ export async function processDocumentAsync( let embeddingModelName = kbEmbeddingModel let embeddingPricingId = kbEmbeddingModel - const currentSourceFileProvenance = await loadCurrentWorkspaceSourceFileSecretProvenance({ + const currentSourceFileProvenance = await loadCurrentSourceFileSecretProvenance({ fileUrl: persistedDocData.fileUrl, + workspaceId: ctx.workspaceId, }) const documentSecretContext = await loadKnowledgeDocumentSecretRegistry( documentId, @@ -2287,8 +2313,9 @@ export async function createDocumentRecords( requestId, tx ) - const sourceBindingByKey = await loadWorkspaceSourceFileBindings( + const sourceBindingByKey = await loadSourceFileBindings( resolvedDocuments.map((docData) => docData.fileUrl), + admission.workspaceId, tx ) const trackedBindings = [ @@ -2961,8 +2988,9 @@ export async function createSingleDocument( requestId, tx ) - const sourceBindingByKey = await loadWorkspaceSourceFileBindings( + const sourceBindingByKey = await loadSourceFileBindings( [resolvedDocumentData.fileUrl], + admission.workspaceId, tx ) const storageKey = getKnowledgeBaseStorageKey(resolvedDocumentData.fileUrl) diff --git a/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts b/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts index ecbc5e5f543..4c860c16817 100644 --- a/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts +++ b/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts @@ -3,6 +3,7 @@ */ import { dbChainMockFns, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { WorkspaceFileSecretProvenance } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' const { mockCheckStorageQuotaForBillingContext, @@ -282,6 +283,137 @@ describe('knowledge workspace source provenance', () => { expect(findDocumentProvenanceWrite()).toBeUndefined() }) + describe('execution file sources', () => { + const executionKey = `execution/${WORKSPACE_ID}/workflow-1/run-1/source.pdf` + const executionUrl = `/api/files/serve/${encodeURIComponent(executionKey)}?context=workspace` + const executionBinding = { + ...SOURCE_BINDING, + id: 'execution-source-1', + key: executionKey, + context: 'execution', + } + const documentInput = { + filename: 'source.pdf', + fileUrl: executionUrl, + fileSize: 512, + mimeType: 'application/pdf', + } + + beforeEach(() => { + mockGetFileMetadataByKeys.mockImplementation(async (_keys: string[], context: string) => + context === 'execution' ? [executionBinding] : [] + ) + }) + + for (const mode of ['single', 'bulk'] as const) { + async function create() { + if (mode === 'single') { + await createSingleDocument(documentInput, KNOWLEDGE_BASE_ID, 'request-1', SOURCE_USER_ID) + } else { + await createDocumentRecords( + [documentInput], + KNOWLEDGE_BASE_ID, + 'request-1', + SOURCE_USER_ID + ) + } + } + + it.each([ + { status: 'exact', entries: [] }, + { + status: 'exact', + entries: [{ name: 'EXPORT_SECRET', encryptedValue: 'encrypted-export-secret' }], + }, + { status: 'unknown' }, + ] satisfies WorkspaceFileSecretProvenance[])( + `binds canonical execution byte provenance during ${mode} admission: %j`, + async (provenance) => { + mockGetBoundWorkspaceFileSecretProvenanceByMetadata.mockResolvedValue( + new Map([[executionBinding.id, provenance]]) + ) + + await create() + + expect(mockGetFileMetadataByKeys).toHaveBeenCalledWith( + [executionKey], + 'execution', + expect.anything(), + { includeDeleted: true } + ) + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).toHaveBeenCalledWith( + expect.anything(), + [executionBinding] + ) + expect(findDocumentProvenanceWrite()).toEqual( + expect.objectContaining({ + status: provenance.status, + entries: + provenance.status === 'exact' + ? provenance.entries.map((entry) => + expect.objectContaining({ + ...entry, + sourceUserId: SOURCE_USER_ID, + sourceWorkspaceId: WORKSPACE_ID, + sourceValueHash: expect.any(String), + }) + ) + : [], + }) + ) + } + ) + + it(`preserves soft-deleted execution taint during ${mode} admission`, async () => { + const deletedBinding = { ...executionBinding, deletedAt: CONTENT_UPDATED_AT } + mockGetFileMetadataByKeys.mockImplementation( + async ( + _keys: string[], + context: string, + _executor: unknown, + options?: { includeDeleted?: boolean } + ) => (context === 'execution' && options?.includeDeleted ? [deletedBinding] : []) + ) + mockGetBoundWorkspaceFileSecretProvenanceByMetadata.mockResolvedValue( + new Map([[executionBinding.id, { status: 'unknown' }]]) + ) + + await create() + + expect(findDocumentProvenanceWrite()).toMatchObject({ status: 'unknown', entries: [] }) + }) + + it.each(['missing', 'untracked'])( + `preserves legacy %s execution sources during ${mode} admission`, + async (kind) => { + mockGetFileMetadataByKeys.mockResolvedValue( + kind === 'missing' ? [] : [{ ...executionBinding, secretProvenanceVersion: null }] + ) + + await create() + + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).toHaveBeenCalledWith( + expect.anything(), + [] + ) + expect(findDocumentProvenanceWrite()).toBeUndefined() + } + ) + + it(`refuses another workspace's execution source before ${mode} admission`, async () => { + mockGetFileMetadataByKeys.mockResolvedValue([ + { ...executionBinding, workspaceId: 'other-workspace' }, + ]) + + await expect(create()).rejects.toThrow('Document file is not owned by this knowledge base') + + expect(mockGetBoundWorkspaceFileSecretProvenanceByMetadata).not.toHaveBeenCalled() + expect(findDocumentProvenanceWrite()).toBeUndefined() + expect(mockIncrementStorageUsageForBillingContextInTx).not.toHaveBeenCalled() + }) + } + }) + it('never deletes a referenced workspace source as knowledge-base storage', async () => { await deleteDocumentStorageFiles( [{ id: 'document-1', fileUrl: SOURCE_URL, workspaceId: WORKSPACE_ID }], diff --git a/apps/sim/lib/knowledge/search/diagnostics.test.ts b/apps/sim/lib/knowledge/search/diagnostics.test.ts new file mode 100644 index 00000000000..21cf17c8e07 --- /dev/null +++ b/apps/sim/lib/knowledge/search/diagnostics.test.ts @@ -0,0 +1,126 @@ +/** @vitest-environment node */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const logs = vi.hoisted(() => ({ info: vi.fn() })) +vi.mock('@sim/logger', () => ({ createLogger: () => logs })) + +import { + annotateSearchDiagnostics, + measureSearchStage, + withSearchDiagnostics, +} from '@/lib/knowledge/search/diagnostics' + +beforeEach(() => { + vi.clearAllMocks() + vi.useFakeTimers({ toFake: ['performance', 'setInterval', 'clearInterval'] }) +}) +afterEach(() => vi.useRealTimers()) + +function deferred() { + let resolve!: () => void + const promise = new Promise((done) => { + resolve = done + }) + return { promise, resolve } +} + +describe('search pipeline diagnostics', () => { + it('reports a stalled stage and clears its heartbeat on completion', async () => { + const gate = deferred() + const result = withSearchDiagnostics({ surface: 'copilot', toolCallId: 'fixture-tool' }, () => + measureSearchStage('embedding', () => gate.promise) + ) + await vi.advanceTimersByTimeAsync(5000) + expect(logs.info).toHaveBeenCalledWith( + 'Knowledge search still running', + expect.objectContaining({ + toolCallId: 'fixture-tool', + elapsedMs: 5000, + activeStages: [{ stage: 'embedding', elapsedMs: 5000 }], + }) + ) + gate.resolve() + await result + expect(logs.info).toHaveBeenLastCalledWith( + 'Knowledge search completed', + expect.objectContaining({ + outcome: 'success', + activeStages: [], + stages: { embedding: { count: 1, totalMs: 5000, maxMs: 5000, errors: 0 } }, + }) + ) + await vi.advanceTimersByTimeAsync(10_000) + expect(logs.info).toHaveBeenCalledTimes(2) + expect(vi.getTimerCount()).toBe(0) + }) + + it('keeps concurrent searches separate and emits only one summary for nested use cases', async () => { + const first = deferred() + const second = deferred() + const a = withSearchDiagnostics({ surface: 'copilot', toolCallId: 'first' }, () => + withSearchDiagnostics({ topK: 15 }, async () => { + await measureSearchStage('result_provenance', () => first.promise) + annotateSearchDiagnostics({ resultCount: 15 }) + }) + ) + const b = withSearchDiagnostics({ surface: 'dashboard' }, () => + measureSearchStage('retrieval', () => second.promise) + ) + await vi.advanceTimersByTimeAsync(25) + second.resolve() + await b + await vi.advanceTimersByTimeAsync(50) + first.resolve() + await a + expect(logs.info).toHaveBeenCalledTimes(2) + const dashboard = logs.info.mock.calls[0][1] + const assistant = logs.info.mock.calls[1][1] + expect(dashboard).toMatchObject({ surface: 'dashboard', elapsedMs: 25 }) + expect(dashboard).not.toHaveProperty('toolCallId') + expect(dashboard.stages).not.toHaveProperty('result_provenance') + expect(assistant).toMatchObject({ + toolCallId: 'first', + topK: 15, + resultCount: 15, + elapsedMs: 75, + }) + expect(assistant.searchId).not.toBe(dashboard.searchId) + expect(assistant.stages).not.toHaveProperty('retrieval') + }) + + it('preserves failures, records repeated stage errors, and does not log error content', async () => { + const error = new Error('private provider response') + await expect( + withSearchDiagnostics({ surface: 'copilot' }, async () => { + await measureSearchStage('vector.authorization', () => 'allowed') + return measureSearchStage('vector.authorization', () => { + throw error + }) + }) + ).rejects.toBe(error) + expect(logs.info).toHaveBeenLastCalledWith( + 'Knowledge search completed', + expect.objectContaining({ + outcome: 'error', + stages: { + 'vector.authorization': { count: 2, totalMs: 0, maxMs: 0, errors: 1 }, + }, + }) + ) + expect(JSON.stringify(logs.info.mock.calls)).not.toContain(error.message) + expect(vi.getTimerCount()).toBe(0) + }) + + it('reports tool failures returned as values without changing the result', async () => { + const result = { success: false, message: 'private error' } + expect(await withSearchDiagnostics({}, async () => result)).toBe(result) + expect(logs.info.mock.calls[0][1].outcome).toBe('error') + expect(JSON.stringify(logs.info.mock.calls)).not.toContain(result.message) + }) + + it('does not create diagnostics outside a search invocation', async () => { + expect(await measureSearchStage('embedding', () => 42)).toBe(42) + expect(logs.info).not.toHaveBeenCalled() + expect(vi.getTimerCount()).toBe(0) + }) +}) diff --git a/apps/sim/lib/knowledge/search/diagnostics.ts b/apps/sim/lib/knowledge/search/diagnostics.ts new file mode 100644 index 00000000000..09672e26a76 --- /dev/null +++ b/apps/sim/lib/knowledge/search/diagnostics.ts @@ -0,0 +1,173 @@ +import { AsyncLocalStorage } from 'node:async_hooks' +import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' + +const logger = createLogger('KnowledgeSearchDiagnostics', { logLevel: 'INFO' }) +const PROGRESS_INTERVAL_MS = 5000 + +type RetrievalLeg = 'vector' | 'keyword' | 'tags' +export type SearchStage = + | 'tool_input' + | 'tool_application' + | 'tool_presentation' + | 'workspace_application' + | 'organization_application' + | 'scoped_application' + | 'knowledge_application' + | 'scope_resolution' + | 'knowledge_context' + | 'index_resolution' + | 'availability' + | 'billing_attribution' + | 'usage_admission' + | 'tag_filters' + | 'input_provenance' + | 'embedding' + | 'access_scope' + | 'defaults' + | 'retrieval' + | 'result_provenance' + | 'reranking' + | 'usage_recording' + | 'overage_billing' + | 'tag_definitions' + | 'metadata' + | 'metadata.authorization' + | 'metadata.sql' + | 'metadata_provenance' + | 'activity_recording' + | RetrievalLeg + | `${RetrievalLeg}.candidates` + | `${RetrievalLeg}.authorization` + | `${RetrievalLeg}.hydration` + | 'vector.connection_acquire' + | 'vector.settings' + | 'vector.probe' + | 'vector.ann' + | 'vector.exact' + +/** Fixed, content-free fields. Never pass queries, filters, document identities, SQL, or errors. */ +export interface SearchDiagnosticMetadata { + surface?: 'dashboard' | 'mcp' | 'copilot' | 'workflow' | 'api' | 'slack' | 'other' + toolCallId?: string + executionId?: string + scopeKind?: 'workspace' | 'organization' + accessScopeKind?: 'workspace' | 'user' + principalKind?: string + topK?: number + knowledgeBaseCount?: number + documentFilterCount?: number + hasSourceFilter?: boolean + hasDateFilter?: boolean + tagFilterCount?: number + hasProvenance?: boolean + searchMode?: 'hybrid' | 'vector' + boostRecency?: boolean + embeddingDimensions?: number + resultCount?: number + /** Tool output before the executor's final egress projection; counts only, never content. */ + toolResultBytes?: number + passageBytes?: number + maxPassageBytes?: number + uniqueDocumentCount?: number +} + +interface StageTiming { + count: number + totalMs: number + maxMs: number + errors: number +} + +interface SearchTrace { + searchId: string + startedAt: number + metadata: SearchDiagnosticMetadata + stages: Partial> + active: Map +} + +const traces = new AsyncLocalStorage() +const roundMs = (value: number) => Math.round(value * 100) / 100 + +export function annotateSearchDiagnostics(metadata: SearchDiagnosticMetadata): void { + const trace = traces.getStore() + if (trace) Object.assign(trace.metadata, metadata) +} + +export function recordSearchStageDuration(stage: SearchStage, milliseconds: number): void { + const trace = traces.getStore() + if (!trace) return + const timing = (trace.stages[stage] ??= { count: 0, totalMs: 0, maxMs: 0, errors: 0 }) + timing.count++ + timing.totalMs = roundMs(timing.totalMs + milliseconds) + timing.maxMs = roundMs(Math.max(timing.maxMs, milliseconds)) +} + +/** Timings include waiting on the dependency; nested and parallel stages must not be summed. */ +export async function measureSearchStage( + stage: SearchStage, + run: () => T | PromiseLike +): Promise { + const trace = traces.getStore() + if (!trace) return run() + const span = Symbol(stage) + const startedAt = performance.now() + trace.active.set(span, { stage, startedAt }) + try { + return await run() + } catch (error) { + const timing = (trace.stages[stage] ??= { count: 0, totalMs: 0, maxMs: 0, errors: 0 }) + timing.errors++ + throw error + } finally { + trace.active.delete(span) + recordSearchStageDuration(stage, performance.now() - startedAt) + } +} + +/** One correlated summary per invocation, plus active stages every five seconds while stalled. */ +export async function withSearchDiagnostics( + metadata: SearchDiagnosticMetadata, + run: () => Promise +): Promise { + if (traces.getStore()) { + annotateSearchDiagnostics(metadata) + return run() + } + const trace: SearchTrace = { + searchId: generateId(), + startedAt: performance.now(), + metadata: { ...metadata }, + stages: {}, + active: new Map(), + } + return traces.run(trace, async () => { + const snapshot = () => ({ + searchId: trace.searchId, + ...trace.metadata, + elapsedMs: roundMs(performance.now() - trace.startedAt), + stages: structuredClone(trace.stages), + activeStages: [...trace.active.values()].map(({ stage, startedAt }) => ({ + stage, + elapsedMs: roundMs(performance.now() - startedAt), + })), + }) + const timer = setInterval(() => { + logger.info('Knowledge search still running', snapshot()) + }, PROGRESS_INTERVAL_MS) + timer.unref() + let outcome = 'error' + try { + const result = await run() + outcome = + result && typeof result === 'object' && 'success' in result && result.success === false + ? 'error' + : 'success' + return result + } finally { + clearInterval(timer) + logger.info('Knowledge search completed', { ...snapshot(), outcome }) + } + }) +} diff --git a/apps/sim/lib/knowledge/search/queries.test.ts b/apps/sim/lib/knowledge/search/queries.test.ts index bb56e09e2d7..e9beb8eccc7 100644 --- a/apps/sim/lib/knowledge/search/queries.test.ts +++ b/apps/sim/lib/knowledge/search/queries.test.ts @@ -523,6 +523,159 @@ describe('live repository authorization follows ranked candidates', () => { afterEach(() => vi.useRealTimers()) + it('bounds broad vector ranking before metadata and reorders relaxed candidates before trimming', async () => { + queueTableRows( + schemaMock.embedding, + Array.from({ length: 200 }, (_, index) => candidate(`probe-${index}`, 'allowed-source')) + ) + queueTableRows(schemaMock.embedding, [ + { ...candidate('far', 'allowed-source'), distance: 0.3 }, + { ...candidate('near', 'allowed-source'), distance: 0.1 }, + ...Array.from({ length: 18 }, (_, index) => candidate(`other-${index}`, 'allowed-source')), + ]) + queueTableRows(schemaMock.embedding, [ + { id: 'far', content: 'Far authorized passage', distance: 0.3 }, + { id: 'near', content: 'Near authorized passage', distance: 0.1 }, + ]) + const rows = await handleVectorOnlySearch({ + ...params, + structuredFilters: undefined, + topK: 1, + }) + expect(rows.map((row) => row.id)).toEqual(['near']) + expect(dbChainMockFns.orderBy.mock.calls[0]).toHaveLength(1) + expect(Object.keys(dbChainMockFns.select.mock.calls[1][0])).toEqual(['id', 'distance']) + expect(dbChainMockFns.limit.mock.invocationCallOrder[1]).toBeLessThan( + dbChainMockFns.select.mock.invocationCallOrder[2] + ) + expect(JSON.stringify(dbChainMockFns.where.mock.calls[1][0])).toContain('OFFSET 0') + expect(getForConnectors).toHaveBeenCalledExactlyOnceWith(['allowed-source'], undefined) + expect(JSON.stringify(dbChainMockFns.where.mock.calls[2][0])).toContain('github_read_grant') + }) + + it('finishes empty scopes after the bounded probe without scanning HNSW or calling providers', async () => { + queueTableRows(schemaMock.embedding, []) + expect(await handleVectorOnlySearch({ ...params, structuredFilters: undefined })).toEqual([]) + expect(dbChainMockFns.select).toHaveBeenCalledOnce() + expect(dbChainMockFns.limit).toHaveBeenCalledExactlyOnceWith(200) + expect(dbChainMockFns.orderBy).not.toHaveBeenCalled() + expect(getForConnectors).not.toHaveBeenCalled() + }) + + it('reads vectors only for the bounded IDs when a broad scope has few candidates', async () => { + queueTableRows(schemaMock.embedding, [candidate('selected', 'allowed-source')]) + queueTableRows(schemaMock.embedding, [candidate('selected', 'allowed-source')]) + queueTableRows(schemaMock.embedding, [ + { id: 'selected', content: 'Verified small scope', distance: 0.1 }, + ]) + expect(await handleVectorOnlySearch({ ...params, structuredFilters: undefined })).toEqual([ + { id: 'selected', content: 'Verified small scope', distance: 0.1 }, + ]) + expect(Object.keys(dbChainMockFns.select.mock.calls[0][0])).toEqual(['id']) + expect(JSON.stringify(dbChainMockFns.where.mock.calls[0][0])).not.toContain('<=>') + expect( + hasMockCondition( + dbChainMockFns.where.mock.calls[1][0], + (node) => + node.type === 'inArray' && + node.column === schemaMock.embedding.id && + Array.isArray(node.values) && + node.values.length === 1 && + node.values[0] === 'selected' + ) + ).toBe(true) + expect(getForConnectors).toHaveBeenCalledExactlyOnceWith(['allowed-source'], undefined) + }) + + it('falls back to exact ranking when the approximate page cannot fill its limit', async () => { + queueTableRows( + schemaMock.embedding, + Array.from({ length: 200 }, (_, index) => candidate(`probe-${index}`, 'allowed-source')) + ) + queueTableRows(schemaMock.embedding, [candidate('partial', 'allowed-source')]) + queueTableRows(schemaMock.embedding, [candidate('selected', 'allowed-source')]) + queueTableRows(schemaMock.embedding, [ + { id: 'selected', content: 'Verified fallback', distance: 0.1 }, + ]) + expect(await handleVectorOnlySearch({ ...params, structuredFilters: undefined })).toEqual([ + { id: 'selected', content: 'Verified fallback', distance: 0.1 }, + ]) + expect(render(dbChainMockFns.orderBy.mock.calls.at(-1)![0]).sql).toContain('+ 0') + expect(JSON.stringify(dbChainMockFns.where.mock.calls.at(-1)![0])).toContain( + 'github_read_grant' + ) + }) + + it('restarts exact ranking at zero and advances past already considered ANN candidates', async () => { + const probe = Array.from({ length: 200 }, (_, index) => + candidate(`probe-${index}`, 'allowed-source') + ) + const approximate = Array.from({ length: 20 }, (_, index) => + candidate(`approximate-${index}`, 'allowed-source') + ) + queueTableRows(schemaMock.embedding, probe) + queueTableRows(schemaMock.embedding, approximate) + queueTableRows(schemaMock.embedding, []) + queueTableRows(schemaMock.embedding, probe) + queueTableRows(schemaMock.embedding, []) + queueTableRows(schemaMock.embedding, approximate) + queueTableRows(schemaMock.embedding, probe) + queueTableRows(schemaMock.embedding, [candidate('selected', 'allowed-source')]) + queueTableRows(schemaMock.embedding, [ + { id: 'selected', content: 'Reachable after the exact restart', distance: 0.1 }, + ]) + + const rows = await handleVectorOnlySearch({ ...params, structuredFilters: undefined }) + + expect(rows.map((row) => row.id)).toEqual(['selected']) + expect(dbChainMockFns.offset.mock.calls).toEqual([[0], [20], [0], [20]]) + expect(getForConnectors).toHaveBeenCalledTimes(2) + expect(dbChainMockFns.transaction).toHaveBeenCalledTimes(3) + }) + + it('keeps the nearest exact results when an earlier ANN page hydrated only a farther result', async () => { + const probe = Array.from({ length: 200 }, (_, index) => + candidate(`probe-${index}`, 'allowed-source') + ) + queueTableRows(schemaMock.embedding, probe) + queueTableRows(schemaMock.embedding, [ + { ...candidate('far', 'allowed-source'), distance: 0.7 }, + ...Array.from({ length: 19 }, (_, index) => candidate(`hidden-${index}`, 'allowed-source')), + ]) + queueTableRows(schemaMock.embedding, [{ id: 'far', content: 'Far result', distance: 0.7 }]) + queueTableRows(schemaMock.embedding, probe) + queueTableRows(schemaMock.embedding, []) + queueTableRows(schemaMock.embedding, [ + candidate('near', 'allowed-source'), + candidate('nearer', 'allowed-source'), + candidate('far', 'allowed-source'), + ]) + queueTableRows(schemaMock.embedding, [ + { id: 'near', content: 'Near result', distance: 0.2 }, + { id: 'nearer', content: 'Nearest result', distance: 0.1 }, + ]) + + const rows = await handleVectorOnlySearch({ + ...params, + topK: 2, + structuredFilters: undefined, + }) + + expect(rows.map((row) => row.id)).toEqual(['nearer', 'near']) + expect(dbChainMockFns.offset.mock.calls).toEqual([[0], [20], [0]]) + expect( + hasMockCondition( + dbChainMockFns.where.mock.calls.at(-1)![0], + (node) => + node.type === 'inArray' && + node.column === schemaMock.embedding.id && + Array.isArray(node.values) && + node.values.length === 2 && + !node.values.includes('far') + ) + ).toBe(true) + }) + it.each(['vector', 'tag-vector', 'tags', 'keyword'] as const)( '%s ranks identifiers before verification and loads content under the full predicate', async (mode) => { diff --git a/apps/sim/lib/knowledge/search/queries.ts b/apps/sim/lib/knowledge/search/queries.ts index fc7b69969ab..17becb53e74 100644 --- a/apps/sim/lib/knowledge/search/queries.ts +++ b/apps/sim/lib/knowledge/search/queries.ts @@ -9,6 +9,7 @@ import { } from '@/lib/knowledge/access/predicate' import type { KnowledgeAccessProvider, KnowledgeAccessScope } from '@/lib/knowledge/access/types' import type { KbEmbeddingDimensions } from '@/lib/knowledge/embedding-models' +import { measureSearchStage, recordSearchStageDuration } from '@/lib/knowledge/search/diagnostics' import { workspaceSearchFilterConditions } from '@/lib/knowledge/search/filter-conditions' import type { WorkspaceSearchFilters } from '@/lib/knowledge/search/filters' import { applyRecencyBoost, RRF_K } from '@/lib/knowledge/search/recency' @@ -44,12 +45,16 @@ async function withVectorScanSettings( run: (executor: SearchExecutor) => Promise ): Promise { if (Date.now() < hnswSettingsUnsupportedUntil) return run(db) + const acquireStarted = performance.now() let applyingSettings = false try { return await db.transaction(async (tx) => { + recordSearchStageDuration('vector.connection_acquire', performance.now() - acquireStarted) applyingSettings = true - await tx.execute( - sql`SELECT set_config('hnsw.iterative_scan', 'relaxed_order', true), set_config('hnsw.max_scan_tuples', ${HNSW_MAX_SCAN_TUPLES}, true)` + await measureSearchStage('vector.settings', () => + tx.execute( + sql`SELECT set_config('hnsw.iterative_scan', 'relaxed_order', true), set_config('hnsw.max_scan_tuples', ${HNSW_MAX_SCAN_TUPLES}, true)` + ) ) applyingSettings = false return run(tx) @@ -92,28 +97,31 @@ export async function getDocumentMetadataByIds( const uniqueIds = [...new Set(documentIds)] const authorizedAccess = accessProvider - ? await accessProvider.getForDocuments(uniqueIds, signal) + ? await measureSearchStage('metadata.authorization', () => + accessProvider.getForDocuments(uniqueIds, signal) + ) : access - const documents = await db - .select({ - id: document.id, - filename: document.filename, - sourceUrl: document.sourceUrl, - sourceModifiedAt: document.sourceModifiedAt, - connectorType: knowledgeConnector.connectorType, - }) - .from(document) - .leftJoin(knowledgeConnector, eq(knowledgeConnector.id, document.connectorId)) - .where( - and( - inArray(document.id, uniqueIds), - eq(document.userExcluded, false), - isNull(document.archivedAt), - isNull(document.deletedAt), - knowledgeAccessCondition(authorizedAccess) + const documents = await measureSearchStage('metadata.sql', () => + db + .select({ + id: document.id, + filename: document.filename, + sourceUrl: document.sourceUrl, + sourceModifiedAt: document.sourceModifiedAt, + connectorType: knowledgeConnector.connectorType, + }) + .from(document) + .leftJoin(knowledgeConnector, eq(knowledgeConnector.id, document.connectorId)) + .where( + and( + inArray(document.id, uniqueIds), + eq(document.userExcluded, false), + isNull(document.archivedAt), + isNull(document.deletedAt), + knowledgeAccessCondition(authorizedAccess) + ) ) - ) - + ) const map: Record = {} documents.forEach((doc) => { map[doc.id] = { @@ -424,6 +432,12 @@ function getVisibilityConditions( ] } +/** Each ranking strategy owns its cursor; ANN offsets cannot paginate exact ordering. */ +interface SearchReadCandidatePage { + candidates: SearchReadCandidate[] + nextOffset: number +} + interface SearchReadCandidate { id: string documentId: string @@ -457,6 +471,7 @@ const LIVE_SEARCH_BUDGET_MS = 8000 * consume every result slot. The existing vector tuple budget also bounds candidate work. */ async function selectAuthorizedSearchResults(input: { + leg: 'vector' | 'keyword' | 'tags' accessProvider: KnowledgeAccessProvider filters?: WorkspaceSearchFilters signal?: AbortSignal @@ -465,13 +480,15 @@ async function selectAuthorizedSearchResults(input: { limit: number, offset: number, excludedSources: readonly string[] - ) => Promise + ) => Promise + compareResults?: (a: SearchResult, b: SearchResult) => number hydrate: (ids: string[], access: KnowledgeAccessScope) => Promise }): Promise { const deadline = Date.now() + LIVE_SEARCH_BUDGET_MS const pageSize = Math.min(LIVE_SEARCH_PAGE_SIZE, Math.max(input.topK, 20)) const results = new Map() const excludedSources = new Set() + const considered = new Set() let scanned = 0 let offset = 0 while ( @@ -480,9 +497,18 @@ async function selectAuthorizedSearchResults(input: { Date.now() < deadline ) { input.signal?.throwIfAborted() - const candidates = await input.selectPage(pageSize, offset, [...excludedSources]) - if (!candidates.length) break - scanned += candidates.length + const page = await measureSearchStage(`${input.leg}.candidates`, () => + input.selectPage(pageSize, offset, [...excludedSources]) + ) + if (!page.candidates.length) break + scanned += page.candidates.length + offset = page.nextOffset + const candidates = page.candidates.filter((candidate) => !considered.has(candidate.id)) + for (const candidate of candidates) considered.add(candidate.id) + if (!candidates.length) { + if (page.candidates.length < pageSize) break + continue + } /** Candidate and hydration queries enforce this source filter; connector types are immutable. */ const connectorIds = input.filters?.source && @@ -496,7 +522,9 @@ async function selectAuthorizedSearchResults(input: { ) ), ] - const access = await input.accessProvider.getForConnectors(connectorIds, input.signal) + const access = await measureSearchStage(`${input.leg}.authorization`, () => + input.accessProvider.getForConnectors(connectorIds, input.signal) + ) input.signal?.throwIfAborted() const grantedSources = new Set( access.kind === 'user' @@ -515,21 +543,25 @@ async function selectAuthorizedSearchResults(input: { ) excludedSources.add(candidate.connectorId) } - const hydrated = await input.hydrate( - candidates.map((candidate) => candidate.id), - access + const hydrated = await measureSearchStage(`${input.leg}.hydration`, () => + input.hydrate( + candidates.map((candidate) => candidate.id), + access + ) ) const byId = new Map(hydrated.map((row) => [row.id, row])) for (const candidate of candidates) { const row = byId.get(candidate.id) if (row) results.set(row.id, row) - if (results.size === input.topK) break + if (!input.compareResults && results.size === input.topK) break } - if (excludedSources.size > excludedBefore) offset = 0 - else { - offset += candidates.length - if (candidates.length < pageSize) break + if (input.compareResults) { + const ranked = [...results.values()].sort(input.compareResults).slice(0, input.topK) + results.clear() + for (const row of ranked) results.set(row.id, row) } + if (excludedSources.size > excludedBefore) offset = 0 + else if (page.candidates.length < pageSize) break } input.signal?.throwIfAborted() return [...results.values()] @@ -593,12 +625,13 @@ export async function handleTagOnlySearch(params: SearchParams): Promise - db + selectPage: async (limit, offset, excludedSources) => { + const candidates = await db .select(SEARCH_READ_CANDIDATE_FIELDS) .from(embedding) .innerJoin(document, eq(embedding.documentId, document.id)) @@ -615,7 +648,9 @@ export async function handleTagOnlySearch(params: SearchParams): Promise hydrateSearchCandidates( ids, @@ -712,43 +747,115 @@ export async function handleVectorOnlySearch(params: SearchParams): Promise a.distance - b.distance) } -/** The vector transaction ends after ranking, before any provider authorization request starts. */ -function selectLiveVectorResults( +/** + * Keep broad candidate visibility correlated with the vector scan. Flattening + * the document join can make PostgreSQL prefer sorting every vector (whose + * TOAST reads it undercosts) before checking access. OFFSET 0 keeps that + * visibility check inside the scan, before LIMIT. Only bounded identities join + * back for source metadata; content still requires live authorization below. + * Small scopes and underfilled approximate pages use exact ranking, so selective + * permissions do not force a fruitless index walk or lose reachable matches. + */ +async function selectLiveVectorResults( params: SearchParams, accessProvider: KnowledgeAccessProvider, distance: SQL, filters: (SQL | undefined)[] ): Promise { const conditions = [inArray(embedding.knowledgeBaseId, params.knowledgeBaseIds), ...filters] - return selectAuthorizedSearchResults({ + let useExactRanking = false + const rows = await selectAuthorizedSearchResults({ + leg: 'vector', accessProvider, filters: params.filters, signal: params.signal, topK: params.topK, + compareResults: (a, b) => a.distance - b.distance, selectPage: (limit, offset, excludedSources) => - withVectorScanSettings((executor) => - executor - .select({ ...SEARCH_READ_CANDIDATE_FIELDS, distance: distance.as('distance') }) + withVectorScanSettings(async (executor) => { + const visibility = [ + ...getVisibilityConditions( + params.access, + params.filters, + knowledgeMetadataCandidateAccessCondition(params.access) + ), + excludeSearchSources(excludedSources), + ] + /** Adding zero prevents an underfilled HNSW scan from being chosen again for fallback. */ + const exactPage = async (candidateIds?: string[]) => { + const exactOffset = useExactRanking ? offset : 0 + useExactRanking = true + const candidates = await measureSearchStage('vector.exact', () => + executor + .select({ ...SEARCH_READ_CANDIDATE_FIELDS, distance: distance.as('distance') }) + .from(embedding) + .innerJoin(document, eq(embedding.documentId, document.id)) + .where( + and( + ...conditions, + ...visibility, + candidateIds ? inArray(embedding.id, candidateIds) : undefined + ) + ) + .orderBy(sql`(${distance}) + 0`, embedding.id) + .limit(limit) + .offset(exactOffset) + ) + return { candidates, nextOffset: exactOffset + candidates.length } + } + if (params.filters?.documentIds?.length || params.structuredFilters?.length) { + return exactPage() + } + /** Probe visibility without vector reads; revoked scopes must not detoast the corpus. */ + const probe = await measureSearchStage('vector.probe', () => + executor + .select({ id: embedding.id }) + .from(embedding) + .innerJoin(document, eq(embedding.documentId, document.id)) + .where(and(inArray(embedding.knowledgeBaseId, params.knowledgeBaseIds), ...visibility)) + .limit(LIVE_SEARCH_PAGE_SIZE) + ) + if (probe.length === 0) return { candidates: [], nextOffset: offset } + if (probe.length < LIVE_SEARCH_PAGE_SIZE) { + return exactPage(probe.map((candidate) => candidate.id)) + } + if (useExactRanking) return exactPage() + const ranked = executor + .select({ id: embedding.id, distance: distance.as('distance') }) .from(embedding) - .innerJoin(document, eq(embedding.documentId, document.id)) .where( and( ...conditions, - ...getVisibilityConditions( - params.access, - params.filters, - knowledgeMetadataCandidateAccessCondition(params.access) - ), - excludeSearchSources(excludedSources) + sql`EXISTS ( + SELECT 1 FROM ${document} + WHERE ${and(eq(document.id, embedding.documentId), ...visibility)} + OFFSET 0 + )` ) ) - .orderBy(distance, embedding.id) + .orderBy(distance) .limit(limit) .offset(offset) - ), + .as('ranked_search_candidates') + const page = await measureSearchStage('vector.ann', () => + executor + .select({ ...SEARCH_READ_CANDIDATE_FIELDS, distance: ranked.distance }) + .from(ranked) + .innerJoin(embedding, eq(embedding.id, ranked.id)) + .innerJoin(document, eq(document.id, embedding.documentId)) + .orderBy(ranked.distance, ranked.id) + ) + if (page.length < limit) return exactPage() + return { + candidates: page.sort((a, b) => a.distance - b.distance), + nextOffset: offset + page.length, + } + }), hydrate: (ids, authorized) => hydrateSearchCandidates(ids, authorized, distance.as('distance'), params.filters, conditions), }) + /** Relaxed HNSW scans can return adjacent pages out of distance order. */ + return rows.sort((a, b) => a.distance - b.distance) } /** @@ -835,12 +942,13 @@ export async function executeKeywordSearch(params: KeywordSearchParams): Promise ...tagFilterConditions, ] return selectAuthorizedSearchResults({ + leg: 'keyword', accessProvider: params.accessProvider, filters: params.filters, signal: params.signal, topK, - selectPage: (limit, offset, excludedSources) => - db + selectPage: async (limit, offset, excludedSources) => { + const candidates = await db .select({ ...SEARCH_READ_CANDIDATE_FIELDS, keywordRank: rankExpr.as('keyword_rank') }) .from(embedding) .innerJoin(document, eq(embedding.documentId, document.id)) @@ -857,7 +965,9 @@ export async function executeKeywordSearch(params: KeywordSearchParams): Promise ) .orderBy(sql`${rankExpr} DESC`, embedding.id) .limit(limit) - .offset(offset), + .offset(offset) + return { candidates, nextOffset: offset + candidates.length } + }, hydrate: (ids, authorized) => hydrateSearchCandidates( ids, @@ -1087,15 +1197,17 @@ export async function executeKnowledgeSearch( if (!hasFilters) { throw new Error('A search query or tag filters are required') } - return await handleTagOnlySearch({ - knowledgeBaseIds, - topK, - structuredFilters, - access, - accessProvider: params.accessProvider, - signal: params.signal, - filters: params.filters, - }) + return await measureSearchStage('tags', () => + handleTagOnlySearch({ + knowledgeBaseIds, + topK, + structuredFilters, + access, + accessProvider: params.accessProvider, + signal: params.signal, + filters: params.filters, + }) + ) } if (!queryVector) { @@ -1110,29 +1222,30 @@ export async function executeKnowledgeSearch( */ const legTopK = searchMode === 'hybrid' ? hybridCandidateCount(topK) : topK - const vectorSearch = hasFilters - ? handleTagAndVectorSearch({ - knowledgeBaseIds, - topK: legTopK, - structuredFilters, - queryVector, - distanceThreshold, - access, - accessProvider: params.accessProvider, - signal: params.signal, - filters: params.filters, - }) - : handleVectorOnlySearch({ - knowledgeBaseIds, - topK: legTopK, - queryVector, - distanceThreshold, - access, - accessProvider: params.accessProvider, - signal: params.signal, - filters: params.filters, - }) - + const vectorSearch = measureSearchStage('vector', () => + hasFilters + ? handleTagAndVectorSearch({ + knowledgeBaseIds, + topK: legTopK, + structuredFilters, + queryVector, + distanceThreshold, + access, + accessProvider: params.accessProvider, + signal: params.signal, + filters: params.filters, + }) + : handleVectorOnlySearch({ + knowledgeBaseIds, + topK: legTopK, + queryVector, + distanceThreshold, + access, + accessProvider: params.accessProvider, + signal: params.signal, + filters: params.filters, + }) + ) if (searchMode === 'vector') { const results = await vectorSearch return boostRecency ? applyRecencyBoost(results) : results @@ -1142,17 +1255,19 @@ export async function executeKnowledgeSearch( * The lexical leg is best-effort: a failure there falls back to vector-only * results rather than failing the whole search. */ - const keywordSearch = executeKeywordSearch({ - knowledgeBaseIds, - topK: legTopK, - query: query!, - queryVector, - structuredFilters, - access, - accessProvider: params.accessProvider, - signal: params.signal, - filters: params.filters, - }).catch((error) => { + const keywordSearch = measureSearchStage('keyword', () => + executeKeywordSearch({ + knowledgeBaseIds, + topK: legTopK, + query: query!, + queryVector, + structuredFilters, + access, + accessProvider: params.accessProvider, + signal: params.signal, + filters: params.filters, + }) + ).catch((error) => { logger.warn('Keyword search leg failed; falling back to vector-only results', { error: getErrorMessage(error, 'Unknown error'), }) diff --git a/apps/sim/lib/logs/execution/logger.ts b/apps/sim/lib/logs/execution/logger.ts index 840f5804247..45d53d4764c 100644 --- a/apps/sim/lib/logs/execution/logger.ts +++ b/apps/sim/lib/logs/execution/logger.ts @@ -1,4 +1,5 @@ import { db, dbFor } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { organization, usageLog, @@ -698,7 +699,7 @@ export class ExecutionLogger implements IExecutionLoggerService { const startTime = new Date() const [workflowLog] = await execDb - .insert(workflowExecutionLogs) + .insert(withInsertColumns(workflowExecutionLogs, workflowExecutionLogColumns)) .values({ id: generateId(), workflowId, diff --git a/apps/sim/lib/mothership/events.ts b/apps/sim/lib/mothership/events.ts index 1ee0b53150c..0d3bdd13d65 100644 --- a/apps/sim/lib/mothership/events.ts +++ b/apps/sim/lib/mothership/events.ts @@ -61,7 +61,7 @@ export function sendMothershipMessage( assistantSearch?: WorkspaceSearchFilters ): boolean { const trimmed = message.trim() - if (!trimmed) { + if (!trimmed && !fileAttachments?.length) { logger.warn('sendMothershipMessage called with empty message') return false } diff --git a/apps/sim/lib/organizations/settings-access.test.ts b/apps/sim/lib/organizations/settings-access.test.ts index b7564da047b..acd91c2d60f 100644 --- a/apps/sim/lib/organizations/settings-access.test.ts +++ b/apps/sim/lib/organizations/settings-access.test.ts @@ -46,6 +46,17 @@ describe('organization settings access', () => { }) }) + it('allows recovery only for current members of the target organization', async () => { + queueTableRows(member, [{ role: 'member' }]) + await expect( + canOpenOrganizationSettingsSection('organization-route', 'viewer', 'recently-deleted') + ).resolves.toBe(true) + queueTableRows(member, []) + await expect( + canOpenOrganizationSettingsSection('organization-route', 'viewer', 'recently-deleted') + ).resolves.toBe(false) + }) + it('fails closed when a stored membership has a non-canonical role', async () => { queueTableRows(member, [{ role: 'billing-owner' }]) diff --git a/apps/sim/lib/permission-groups/block-successors.generated.ts b/apps/sim/lib/permission-groups/block-successors.generated.ts index d4f7c30c858..6892fa1b402 100644 --- a/apps/sim/lib/permission-groups/block-successors.generated.ts +++ b/apps/sim/lib/permission-groups/block-successors.generated.ts @@ -9,9 +9,12 @@ */ export const BLOCK_ACCESS_SUCCESSORS: Record = { api_trigger: 'start_trigger', + box: 'box_v2', chat_trigger: 'start_trigger', confluence: 'confluence_v2', cursor: 'cursor_v2', + dropbox: 'dropbox_v2', + dub: 'dub_v2', extend: 'extend_v2', file: 'file_v5', file_v2: 'file_v5', @@ -28,20 +31,26 @@ export const BLOCK_ACCESS_SUCCESSORS: Record = { image_generator: 'image_generator_v2', input_trigger: 'start_trigger', intercom: 'intercom_v2', + jupyter: 'jupyter_v2', kalshi: 'kalshi_v2', linear: 'linear_v2', logs: 'logs_v2', manual_trigger: 'start_trigger', + microsoft_dataverse: 'microsoft_dataverse_v2', microsoft_excel: 'microsoft_excel_v2', mistral_parse: 'mistral_parse_v3', mistral_parse_v2: 'mistral_parse_v3', notion: 'notion_v2', openai: 'embeddings', pulse: 'pulse_v2', + quiver: 'quiver_v2', reducto: 'reducto_v2', router: 'router_v2', + servicenow: 'servicenow_v2', + sftp: 'sftp_v2', sharepoint: 'sharepoint_v2', slack: 'slack_v2', + ssh: 'ssh_v2', starter: 'start_trigger', stt: 'stt_v2', table: 'table_v2', diff --git a/apps/sim/lib/slack-search/app-configuration.test.ts b/apps/sim/lib/slack-search/app-configuration.test.ts new file mode 100644 index 00000000000..bde6f6a6a71 --- /dev/null +++ b/apps/sim/lib/slack-search/app-configuration.test.ts @@ -0,0 +1,108 @@ +/** @vitest-environment node */ +import { db } from '@sim/db' +import { slackApp } from '@sim/db/schema' +import { queueTableRows, resetDbChainMock } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const m = vi.hoisted(() => ({ + env: { + SLACK_SEARCH_APP_ID: 'ASHARED', + SLACK_SEARCH_CLIENT_ID: '123.456', + SLACK_SEARCH_CLIENT_SECRET: 'environment-client-secret', + SLACK_SEARCH_SIGNING_SECRET: 'environment-signing-secret', + }, + decrypt: vi.fn(async (value: string) => ({ decrypted: value.replace('encrypted:', '') })), +})) +vi.mock('@/lib/core/config/env', () => ({ env: m.env })) +vi.mock('@/lib/core/security/encryption', () => ({ decryptSecret: m.decrypt })) + +import { + loadSlackAppConfiguration, + resolveSlackAppCredentials, + slackBotCredentialVersion, +} from '@/lib/slack-search/app-configuration' +import { getSharedSlackSearchAppConfiguration } from '@/lib/slack-search/shared-app-env' + +const stored = { + id: 'ASHARED', + kind: 'shared' as const, + organizationId: null, + clientId: 'old-client', + encryptedClientSecret: 'encrypted:old-client-secret', + encryptedSigningSecret: 'encrypted:old-signing-secret', + revision: 'old-revision', + createdAt: new Date(0), + updatedAt: new Date(0), +} +const credentialKeys = [ + 'SLACK_SEARCH_CLIENT_ID', + 'SLACK_SEARCH_CLIENT_SECRET', + 'SLACK_SEARCH_SIGNING_SECRET', +] as const + +beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + Object.assign(m.env, { + SLACK_SEARCH_APP_ID: 'ASHARED', + SLACK_SEARCH_CLIENT_ID: '123.456', + SLACK_SEARCH_CLIENT_SECRET: 'environment-client-secret', + SLACK_SEARCH_SIGNING_SECRET: 'environment-signing-secret', + }) +}) + +describe('deployment-owned Slack app configuration', () => { + it('authenticates shared ingress without a database registration', async () => { + await expect(loadSlackAppConfiguration('ASHARED')).resolves.toMatchObject({ + signingSecret: 'environment-signing-secret', + app: { id: 'ASHARED', kind: 'shared', organizationId: null }, + }) + expect(db.select).not.toHaveBeenCalled() + expect(m.decrypt).not.toHaveBeenCalled() + }) + + it('ignores previously registered credentials for the configured company app', async () => { + await expect(resolveSlackAppCredentials(stored)).resolves.toMatchObject({ + clientId: '123.456', + clientSecret: 'environment-client-secret', + }) + expect(m.decrypt).not.toHaveBeenCalled() + }) + + it.each(credentialKeys)('does not use stored secrets when %s is missing', async (key) => { + m.env[key] = '' + queueTableRows(slackApp, [stored]) + await expect(loadSlackAppConfiguration('ASHARED')).rejects.toThrow('Configure SLACK_SEARCH') + await expect(resolveSlackAppCredentials(stored)).rejects.toThrow('Configure SLACK_SEARCH') + expect(m.decrypt).not.toHaveBeenCalled() + }) + + it.each(credentialKeys)('invalidates queued work when %s rotates', (key) => { + const previous = getSharedSlackSearchAppConfiguration()?.revision + expect(getSharedSlackSearchAppConfiguration()?.revision).toBe(previous) + m.env[key] = 'rotated' + const current = getSharedSlackSearchAppConfiguration()?.revision + expect(current).not.toBe(previous) + expect(slackBotCredentialVersion('token', current)).not.toBe( + slackBotCredentialVersion('token', previous) + ) + }) + + it('preserves custom ingress when shared credentials are incomplete', async () => { + m.env.SLACK_SEARCH_CLIENT_SECRET = '' + queueTableRows(slackApp, [{ ...stored, id: 'ACUSTOM', kind: 'custom', organizationId: 'org' }]) + await expect(loadSlackAppConfiguration('ACUSTOM')).resolves.toMatchObject({ + signingSecret: 'old-signing-secret', + }) + }) + + it('rejects company identity colliding with a custom app owner', async () => { + await expect( + resolveSlackAppCredentials({ ...stored, kind: 'custom', organizationId: 'org' }) + ).rejects.toThrow('belongs to a custom installation') + }) + + it('never assigns the shared signing key to an unknown app', async () => { + await expect(loadSlackAppConfiguration('AUNKNOWN')).resolves.toBeNull() + }) +}) diff --git a/apps/sim/lib/slack-search/app-configuration.ts b/apps/sim/lib/slack-search/app-configuration.ts index 197d1f3010c..83171eb9ab0 100644 --- a/apps/sim/lib/slack-search/app-configuration.ts +++ b/apps/sim/lib/slack-search/app-configuration.ts @@ -3,11 +3,41 @@ import { slackApp } from '@sim/db/schema' import { sha256Hex } from '@sim/security/hash' import { eq } from 'drizzle-orm' import { decryptSecret } from '@/lib/core/security/encryption' +import { getSharedSlackSearchAppConfiguration } from '@/lib/slack-search/shared-app-env' + +/** Shared credentials always come from the deployment, even for previously registered apps. */ +export async function resolveSlackAppCredentials(app: typeof slackApp.$inferSelect) { + const shared = getSharedSlackSearchAppConfiguration(app.id) + if (shared?.id === app.id) { + if (app.kind !== 'shared' || app.organizationId !== null) + throw new Error('The configured shared Slack app belongs to a custom installation') + return shared + } + if (!app.clientId || !app.encryptedClientSecret || !app.encryptedSigningSecret) + throw new Error('Slack app credentials are missing') + const [client, signing] = await Promise.all([ + decryptSecret(app.encryptedClientSecret), + decryptSecret(app.encryptedSigningSecret), + ]) + if (!client.decrypted || !signing.decrypted) throw new Error('Slack app credentials are empty') + return { + id: app.id, + kind: app.kind, + organizationId: app.organizationId, + clientId: app.clientId, + clientSecret: client.decrypted, + signingSecret: signing.decrypted, + revision: app.revision, + } +} /** Authentication lookup only: an app ID selects a key, never grants organization access. */ export async function loadSlackAppConfiguration(appId: string) { + const shared = getSharedSlackSearchAppConfiguration(appId) + if (shared?.id === appId) return { app: shared, signingSecret: shared.signingSecret } const [app] = await db.select().from(slackApp).where(eq(slackApp.id, appId)).limit(1) if (!app) return null + if (!app.encryptedSigningSecret) throw new Error('Slack app signing secret is missing') const { decrypted: signingSecret } = await decryptSecret(app.encryptedSigningSecret) if (!signingSecret) throw new Error('Slack app signing secret is empty') return { app, signingSecret } diff --git a/apps/sim/lib/slack-search/manifest.test.ts b/apps/sim/lib/slack-search/manifest.test.ts index 12c578edd47..6f434da4f74 100644 --- a/apps/sim/lib/slack-search/manifest.test.ts +++ b/apps/sim/lib/slack-search/manifest.test.ts @@ -92,7 +92,7 @@ describe('Search app manifest', () => { }) }) -it('official app uses the existing personal indexing grants with bot commands', () => { +it('official app declares expanded permissions without subscribing to member message events', () => { const manifest = createSharedSlackSearchManifest('https://www.sim.ai') expect(manifest.oauth_config.scopes.user).toEqual([ 'channels:history', @@ -105,9 +105,44 @@ it('official app uses the existing personal indexing grants with bot commands', 'mpim:read', 'users:read', 'users:read.email', + 'canvases:read', + 'canvases:write', + 'chat:write', + 'files:read', + 'search:read.files', + 'search:read.im', + 'search:read.mpim', + 'search:read.private', + 'search:read.public', + 'search:read.users', + 'team:read', + 'usergroups:read', + ]) + expect(manifest.oauth_config.scopes.bot).toEqual([ + 'assistant:write', + 'chat:write', + 'im:history', + 'im:write', + 'app_mentions:read', + 'users:read', + 'users:read.email', + 'commands', + 'channels:history', + 'channels:manage', + 'channels:read', + 'channels:write.invites', + 'chat:write.public', + 'groups:history', + 'groups:read', + 'groups:write', + 'groups:write.invites', + 'links:read', + 'links:write', + 'mpim:history', + 'mpim:read', + 'mpim:write', + 'reactions:write', ]) - expect(manifest.oauth_config.scopes.bot).toContain('commands') - expect(manifest.oauth_config.scopes.bot).not.toContain('groups:history') expect(manifest.features.slash_commands.map((command) => command.command)).toEqual([ '/query', '/connect', diff --git a/apps/sim/lib/slack-search/manifest.ts b/apps/sim/lib/slack-search/manifest.ts index cadeba498c0..f8135416f95 100644 --- a/apps/sim/lib/slack-search/manifest.ts +++ b/apps/sim/lib/slack-search/manifest.ts @@ -62,7 +62,10 @@ export function createSlackSearchManifest( } } -/** The official app combines personal source indexing with bot conversations and commands. */ +/** + * Declares the company app's permissions, including planned capabilities. + * Runtime OAuth validation continues to require only scopes used by implemented features. + */ export function createSharedSlackSearchManifest(origin: string) { const manifest = createSlackSearchManifest( SLACK_SEARCH_DEFAULT_NAME, @@ -94,8 +97,39 @@ export function createSharedSlackSearchManifest(origin: string) { oauth_config: { ...manifest.oauth_config, scopes: { - bot: [...SLACK_SHARED_SEARCH_BOT_SCOPES], - user: [...SLACK_SEARCH_USER_SCOPES], + bot: [ + ...SLACK_SHARED_SEARCH_BOT_SCOPES, + 'channels:history', + 'channels:manage', + 'channels:read', + 'channels:write.invites', + 'chat:write.public', + 'groups:history', + 'groups:read', + 'groups:write', + 'groups:write.invites', + 'links:read', + 'links:write', + 'mpim:history', + 'mpim:read', + 'mpim:write', + 'reactions:write', + ], + user: [ + ...SLACK_SEARCH_USER_SCOPES, + 'canvases:read', + 'canvases:write', + 'chat:write', + 'files:read', + 'search:read.files', + 'search:read.im', + 'search:read.mpim', + 'search:read.private', + 'search:read.public', + 'search:read.users', + 'team:read', + 'usergroups:read', + ], }, }, settings: { diff --git a/apps/sim/lib/slack-search/oauth-state.test.ts b/apps/sim/lib/slack-search/oauth-state.test.ts index 74f85cf2c95..3bf69859e8f 100644 --- a/apps/sim/lib/slack-search/oauth-state.test.ts +++ b/apps/sim/lib/slack-search/oauth-state.test.ts @@ -35,6 +35,14 @@ describe('Slack OAuth state', () => { expect(JSON.parse(value)).toEqual(attempt) expect([expiryMode, ttl, condition]).toEqual(['EX', 600, 'NX']) }) + it('stores only shared identity and revision without app secrets', async () => { + const { encryptedClientSecret, encryptedSigningSecret, ...common } = attempt + const shared = { ...common, sharedApp: { id: 'ASHARED', revision: 'env-revision' } } + await storeSlackSearchOAuthAttempt(shared) + expect(JSON.parse(redis.set.mock.calls[0][1])).toEqual(shared) + redis.eval.mockResolvedValueOnce(JSON.stringify(shared)) + await expect(consumeSlackSearchOAuthAttempt('state', principal)).resolves.toEqual(shared) + }) it('consumes only for the initiating admin session and rejects replay', async () => { redis.eval.mockResolvedValueOnce(JSON.stringify(attempt)) expect(await consumeSlackSearchOAuthAttempt('state', principal)).toEqual(attempt) diff --git a/apps/sim/lib/slack-search/oauth-state.ts b/apps/sim/lib/slack-search/oauth-state.ts index a1257f0582b..7074500722e 100644 --- a/apps/sim/lib/slack-search/oauth-state.ts +++ b/apps/sim/lib/slack-search/oauth-state.ts @@ -6,30 +6,38 @@ import { getRedisClient } from '@/lib/core/config/redis' import { OrchestrationError } from '@/lib/core/orchestration/types' const TTL_SECONDS = 600 -const attemptSchema = z.object({ - userId: z.string().min(1), - sessionId: z.string().min(1), - organizationId: z.string().min(1), - name: z.string().min(1), - description: z.string().min(1), - sharedApp: z.object({ id: z.string().min(1), revision: z.string().min(1) }).optional(), - memberApp: z.object({ appId: z.string().min(1), teamId: z.string().min(1) }).optional(), - clientId: z.string().min(1), - encryptedClientSecret: z.string().min(1), - encryptedSigningSecret: z.string().min(1), - redirectUri: z.string().url(), - createdAt: z.number(), - installation: z - .object({ - id: z.string(), - revision: z.string(), - credentialId: z.string(), - appId: z.string(), - teamId: z.string(), - appRevision: z.string().optional(), - }) - .optional(), -}) +const attemptSchema = z + .object({ + userId: z.string().min(1), + sessionId: z.string().min(1), + organizationId: z.string().min(1), + name: z.string().min(1), + description: z.string().min(1), + memberApp: z.object({ appId: z.string().min(1), teamId: z.string().min(1) }).optional(), + clientId: z.string().min(1), + redirectUri: z.string().url(), + createdAt: z.number(), + installation: z + .object({ + id: z.string(), + revision: z.string(), + credentialId: z.string(), + appId: z.string(), + teamId: z.string(), + appRevision: z.string().optional(), + }) + .optional(), + }) + .and( + z.union([ + z.object({ sharedApp: z.object({ id: z.string().min(1), revision: z.string().min(1) }) }), + z.object({ + sharedApp: z.undefined().optional(), + encryptedClientSecret: z.string().min(1), + encryptedSigningSecret: z.string().min(1), + }), + ]) + ) export type SlackSearchOAuthAttempt = z.infer const CONSUME = ` local value = redis.call('GET', KEYS[1]) diff --git a/apps/sim/lib/slack-search/shared-app-env.ts b/apps/sim/lib/slack-search/shared-app-env.ts new file mode 100644 index 00000000000..c1bebbcb1b3 --- /dev/null +++ b/apps/sim/lib/slack-search/shared-app-env.ts @@ -0,0 +1,24 @@ +import { sha256Hex } from '@sim/security/hash' +import { env } from '@/lib/core/config/env' + +/** Deployment-owned credentials; callers apply Search availability separately. */ +export function getSharedSlackSearchAppConfiguration(appId?: string) { + const id = env.SLACK_SEARCH_APP_ID + if (!id || (appId !== undefined && appId !== id)) return null + const clientId = env.SLACK_SEARCH_CLIENT_ID + const clientSecret = env.SLACK_SEARCH_CLIENT_SECRET + const signingSecret = env.SLACK_SEARCH_SIGNING_SECRET + if (!/^A[A-Z0-9]{1,199}$/.test(id) || !clientId || !clientSecret || !signingSecret) + throw new Error( + 'Configure SLACK_SEARCH_APP_ID, SLACK_SEARCH_CLIENT_ID, SLACK_SEARCH_CLIENT_SECRET, and SLACK_SEARCH_SIGNING_SECRET for the shared Slack app' + ) + return { + id, + kind: 'shared' as const, + organizationId: null, + clientId, + clientSecret, + signingSecret, + revision: sha256Hex(JSON.stringify([id, clientId, clientSecret, signingSecret])), + } +} diff --git a/apps/sim/lib/slack-search/shared-app.test.ts b/apps/sim/lib/slack-search/shared-app.test.ts index ce541a84e61..9bce83397a7 100644 --- a/apps/sim/lib/slack-search/shared-app.test.ts +++ b/apps/sim/lib/slack-search/shared-app.test.ts @@ -1,9 +1,18 @@ /** @vitest-environment node */ +import { db } from '@sim/db' import { slackApp, slackSearchInstallation } from '@sim/db/schema' import { queueTableRows, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' -const m = vi.hoisted(() => ({ flag: vi.fn(), env: { SLACK_SEARCH_APP_ID: 'A1' } })) +const m = vi.hoisted(() => ({ + flag: vi.fn(), + env: { + SLACK_SEARCH_APP_ID: 'A1', + SLACK_SEARCH_CLIENT_ID: 'client', + SLACK_SEARCH_CLIENT_SECRET: 'secret', + SLACK_SEARCH_SIGNING_SECRET: 'signing', + }, +})) vi.mock('@/lib/core/config/env', () => ({ env: m.env })) vi.mock('@/lib/core/config/feature-flags', () => ({ isFeatureEnabled: m.flag })) @@ -16,7 +25,12 @@ import { beforeEach(() => { vi.clearAllMocks() resetDbChainMock() - m.env.SLACK_SEARCH_APP_ID = 'A1' + Object.assign(m.env, { + SLACK_SEARCH_APP_ID: 'A1', + SLACK_SEARCH_CLIENT_ID: 'client', + SLACK_SEARCH_CLIENT_SECRET: 'secret', + SLACK_SEARCH_SIGNING_SECRET: 'signing', + }) m.flag.mockResolvedValue(true) }) describe('shared Slack rollout', () => { @@ -25,15 +39,22 @@ describe('shared Slack rollout', () => { if (flag) m.env.SLACK_SEARCH_APP_ID = '' await expect(readSharedSlackSearchApp()).resolves.toBeNull() }) - it.each( - [ - [], - [{ id: 'A1', kind: 'custom', organizationId: 'org' }], - [{ id: 'A1', kind: 'shared', organizationId: 'org' }], - ].map((rows) => ({ rows })) - )('fails closed for invalid registration %#', async ({ rows }) => { - queueTableRows(slackApp, rows) - await expect(readSharedSlackSearchApp()).rejects.toThrow('not registered') + it.each([ + 'SLACK_SEARCH_CLIENT_ID', + 'SLACK_SEARCH_CLIENT_SECRET', + 'SLACK_SEARCH_SIGNING_SECRET', + ] as const)('fails closed without %s', async (key) => { + m.env[key] = '' + await expect(readSharedSlackSearchApp()).rejects.toThrow('Configure SLACK_SEARCH_APP_ID') + }) + it('uses deployment credentials without requiring a registered database row', async () => { + await expect(readSharedSlackSearchApp()).resolves.toMatchObject({ + id: 'A1', + clientId: 'client', + clientSecret: 'secret', + signingSecret: 'signing', + }) + expect(db.select).not.toHaveBeenCalled() }) it('preserves custom bot handling while the shared flag is off', async () => { m.flag.mockResolvedValue(false) diff --git a/apps/sim/lib/slack-search/shared-app.ts b/apps/sim/lib/slack-search/shared-app.ts index 8b87c0e2c22..9431b4565ab 100644 --- a/apps/sim/lib/slack-search/shared-app.ts +++ b/apps/sim/lib/slack-search/shared-app.ts @@ -1,25 +1,24 @@ import { db } from '@sim/db' import { slackApp, slackSearchInstallation } from '@sim/db/schema' import { and, eq } from 'drizzle-orm' -import { env } from '@/lib/core/config/env' import { isFeatureEnabled } from '@/lib/core/config/feature-flags' import { OrchestrationError } from '@/lib/core/orchestration/types' +import { getSharedSlackSearchAppConfiguration } from '@/lib/slack-search/shared-app-env' /** Called only inside authorized installation/member operations; never returns secrets to a surface. */ export async function readSharedSlackSearchApp() { - if (!(await isFeatureEnabled('slack-search-shared-app')) || !env.SLACK_SEARCH_APP_ID) return null - const [app] = await db - .select() - .from(slackApp) - .where(eq(slackApp.id, env.SLACK_SEARCH_APP_ID)) - .limit(1) - if (!app || app.kind !== 'shared' || app.organizationId !== null) - throw new Error('The configured shared Slack Search app is not registered') - return app + if (!(await isFeatureEnabled('slack-search-shared-app'))) return null + return getSharedSlackSearchAppConfiguration() } /** Existing custom bots remain independent of the shared-app rollout. */ export async function requireSlackSearchAppAvailable(appId: string) { + const shared = getSharedSlackSearchAppConfiguration(appId) + if (shared?.id === appId) { + if (!(await readSharedSlackSearchApp())) + throw new OrchestrationError('forbidden', 'The shared Slack Search app is unavailable') + return + } const [app] = await db .select({ kind: slackApp.kind }) .from(slackApp) diff --git a/apps/sim/lib/uploads/client/admission.ts b/apps/sim/lib/uploads/client/admission.ts index 670a570a2fd..489323c1c2a 100644 --- a/apps/sim/lib/uploads/client/admission.ts +++ b/apps/sim/lib/uploads/client/admission.ts @@ -34,6 +34,7 @@ interface UploadAdmissionFile { interface MultiFileUploadAdmissionOptions { existingFiles?: ArrayLike + maxFiles?: number maxFileBytes?: number maxTotalBytes?: number } @@ -48,9 +49,13 @@ export function assertMultiFileUploadAdmission( options: MultiFileUploadAdmissionOptions = {} ): void { const existingFiles = options.existingFiles + const maxFiles = options.maxFiles ?? MULTI_FILE_UPLOAD_MAX_FILES const maxFileBytes = options.maxFileBytes ?? MULTI_FILE_UPLOAD_MAX_FILE_BYTES const maxTotalBytes = options.maxTotalBytes ?? MULTI_FILE_UPLOAD_MAX_TOTAL_FILE_EQUIVALENTS * maxFileBytes + if (!Number.isSafeInteger(maxFiles) || maxFiles < 1) { + throw new Error('Invalid upload file count limit') + } if (!Number.isSafeInteger(maxFileBytes) || maxFileBytes < 1) { throw new Error('Invalid per-file upload limit') } @@ -59,9 +64,9 @@ export function assertMultiFileUploadAdmission( } const existingCount = existingFiles?.length ?? 0 const totalCount = existingCount + files.length - if (totalCount > MULTI_FILE_UPLOAD_MAX_FILES) { + if (totalCount > maxFiles) { throw new MultiFileUploadAdmissionError( - `Select up to ${MULTI_FILE_UPLOAD_MAX_FILES} files at a time.`, + `Select up to ${maxFiles} files at a time.`, 'UPLOAD_FILE_COUNT_EXCEEDED' ) } diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.test.ts b/apps/sim/lib/uploads/client/logo-file.test.ts similarity index 51% rename from apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.test.ts rename to apps/sim/lib/uploads/client/logo-file.test.ts index 888fe0b2bac..f2912f707c2 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file.test.ts +++ b/apps/sim/lib/uploads/client/logo-file.test.ts @@ -1,8 +1,5 @@ import { describe, expect, it } from 'vitest' -import { - validateWorkspaceLogoFile, - WORKSPACE_LOGO_ACCEPT_ATTRIBUTE, -} from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks/workspace-logo-file' +import { LOGO_ACCEPT_ATTRIBUTE, validateLogoFile } from '@/lib/uploads/client/logo-file' function file(overrides: Partial> = {}) { return { @@ -13,20 +10,20 @@ function file(overrides: Partial> = {}) { } } -describe('workspace logo files', () => { +describe('logo files', () => { it('advertises and accepts GIF images', () => { - expect(WORKSPACE_LOGO_ACCEPT_ATTRIBUTE.split(',')).toContain('image/gif') - expect(validateWorkspaceLogoFile(file({ name: 'animated.gif', type: 'image/gif' }))).toBeNull() + expect(LOGO_ACCEPT_ATTRIBUTE.split(',')).toContain('image/gif') + expect(validateLogoFile(file({ name: 'animated.gif', type: 'image/gif' }))).toBeNull() }) it('rejects files larger than 5MB', () => { - expect(validateWorkspaceLogoFile(file({ size: 5 * 1024 * 1024 + 1 }))).toBe( + expect(validateLogoFile(file({ size: 5 * 1024 * 1024 + 1 }))).toBe( 'File "logo.png" is too large. Maximum size is 5MB.' ) }) it('lists GIF among the supported formats in validation errors', () => { - expect(validateWorkspaceLogoFile(file({ name: 'logo.bmp', type: 'image/bmp' }))).toBe( + expect(validateLogoFile(file({ name: 'logo.bmp', type: 'image/bmp' }))).toBe( 'File "logo.bmp" is not a supported image format. Please use PNG, JPEG, GIF, SVG, or WebP.' ) }) diff --git a/apps/sim/lib/uploads/client/logo-file.ts b/apps/sim/lib/uploads/client/logo-file.ts new file mode 100644 index 00000000000..eb053c0d8ff --- /dev/null +++ b/apps/sim/lib/uploads/client/logo-file.ts @@ -0,0 +1,24 @@ +const MAX_LOGO_SIZE = 5 * 1024 * 1024 + +const LOGO_IMAGE_TYPES = [ + 'image/png', + 'image/jpeg', + 'image/jpg', + 'image/gif', + 'image/svg+xml', + 'image/webp', +] as const + +const LOGO_IMAGE_TYPE_SET = new Set(LOGO_IMAGE_TYPES) + +export const LOGO_ACCEPT_ATTRIBUTE = LOGO_IMAGE_TYPES.join(',') + +export function validateLogoFile(file: Pick): string | null { + if (file.size > MAX_LOGO_SIZE) { + return `File "${file.name}" is too large. Maximum size is 5MB.` + } + if (!LOGO_IMAGE_TYPE_SET.has(file.type)) { + return `File "${file.name}" is not a supported image format. Please use PNG, JPEG, GIF, SVG, or WebP.` + } + return null +} diff --git a/apps/sim/lib/uploads/client/session-upload.test.ts b/apps/sim/lib/uploads/client/session-upload.test.ts index 235e6892d36..264408e75c0 100644 --- a/apps/sim/lib/uploads/client/session-upload.test.ts +++ b/apps/sim/lib/uploads/client/session-upload.test.ts @@ -122,4 +122,48 @@ describe('session upload domain clients', () => { size: 100, }) }) + + it('uploads an organization logo through the shared internal session', async () => { + const result = { + path: '/api/files/serve/logo.png', + key: 'organization-logos/logo.png', + name: 'logo.png', + size: 100, + type: 'image/png', + } + mockRequestJson + .mockResolvedValueOnce({ + data: { + session: { id: 'upload-2', purpose: 'organization_logo' }, + uploadToken: 'token', + transfer: { + method: 'put', + url: 'https://storage.example/upload', + headers: { 'Content-Type': 'image/png' }, + }, + }, + }) + .mockResolvedValueOnce({ + data: { id: 'upload-2', purpose: 'organization_logo', result }, + }) + mockUploadFileSession.mockImplementation( + async (params: UploadClientMockParams) => params.complete() + ) + + await expect( + uploadInternalFileSession({ + purpose: 'organization_logo', + organizationId: 'org-1', + file: { name: 'logo.png', type: 'image/png', size: 100 } as File, + }) + ).resolves.toEqual(result) + + expect(mockRequestJson.mock.calls[0][1].body).toEqual({ + purpose: 'organization_logo', + organizationId: 'org-1', + name: 'logo.png', + contentType: 'image/png', + size: 100, + }) + }) }) diff --git a/apps/sim/lib/uploads/client/session-upload.ts b/apps/sim/lib/uploads/client/session-upload.ts index 55d1a7466f6..c222a637fb5 100644 --- a/apps/sim/lib/uploads/client/session-upload.ts +++ b/apps/sim/lib/uploads/client/session-upload.ts @@ -39,7 +39,9 @@ type InternalUploadContext = | { purpose: 'workspace_file'; workspaceId: string; folderId?: string | null } | { purpose: 'profile_picture' } | { purpose: 'workspace_logo'; workspaceId: string } - | { purpose: 'mothership_attachment'; workspaceId: string } + | { purpose: 'organization_logo'; organizationId: string } + | { purpose: 'mothership_attachment'; workspaceId: string; organizationId?: never } + | { purpose: 'mothership_attachment'; organizationId: string; workspaceId?: never } | { purpose: 'execution_attachment' workspaceId: string @@ -162,12 +164,25 @@ function internalUploadBody(params: UploadInternalFileSessionParams): CreateInte case 'profile_picture': return { purpose: params.purpose, ...fileFields } case 'workspace_logo': - case 'mothership_attachment': return { purpose: params.purpose, workspaceId: params.workspaceId, ...fileFields, } + case 'organization_logo': + return { + purpose: params.purpose, + organizationId: params.organizationId, + ...fileFields, + } + case 'mothership_attachment': + return { + purpose: params.purpose, + ...(params.organizationId + ? { organizationId: params.organizationId } + : { workspaceId: params.workspaceId }), + ...fileFields, + } case 'execution_attachment': return { purpose: params.purpose, diff --git a/apps/sim/lib/uploads/config.ts b/apps/sim/lib/uploads/config.ts index e527b6f605e..ab49401e8c3 100644 --- a/apps/sim/lib/uploads/config.ts +++ b/apps/sim/lib/uploads/config.ts @@ -221,6 +221,7 @@ function getS3Config(context: StorageContext): StorageConfig { case 'mothership': case 'workspace': case 'table-import': + case 'organization-logos': return { bucket: S3_CONFIG.bucket, region: S3_CONFIG.region, @@ -284,6 +285,7 @@ function getBlobConfig(context: StorageContext): StorageConfig { case 'mothership': case 'workspace': case 'table-import': + case 'organization-logos': return { accountName: BLOB_CONFIG.accountName, accountKey: BLOB_CONFIG.accountKey, @@ -344,6 +346,7 @@ function getGcsConfig(context: StorageContext): StorageConfig { case 'mothership': case 'workspace': case 'table-import': + case 'organization-logos': return { bucket: GCS_CONFIG.bucket } case 'profile-pictures': return { bucket: GCS_PROFILE_PICTURES_CONFIG.bucket || GCS_CONFIG.bucket } diff --git a/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.test.ts b/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.test.ts new file mode 100644 index 00000000000..538bde91cae --- /dev/null +++ b/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.test.ts @@ -0,0 +1,56 @@ +/** @vitest-environment node */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockUploadFile } = vi.hoisted(() => ({ mockUploadFile: vi.fn() })) + +vi.mock('@/lib/uploads/core/storage-service', () => ({ + uploadFile: mockUploadFile, + downloadFile: vi.fn(), +})) +vi.mock('@/lib/core/utils/urls', () => ({ getBaseUrl: () => 'https://sim.example' })) + +import { uploadCopilotFile } from '@/lib/uploads/contexts/copilot/copilot-file-manager' +import type { UploadFileOptions } from '@/lib/uploads/shared/types' + +describe('Copilot output key allocation', () => { + beforeEach(() => { + vi.clearAllMocks() + mockUploadFile.mockImplementation(async (options: UploadFileOptions) => ({ + key: options.customKey, + path: `/api/files/serve/${encodeURIComponent(options.customKey!)}`, + name: options.customKey, + type: options.contentType, + size: options.file.length, + })) + }) + + it('gives concurrent same-named files unique owned keys and preserves their display names', async () => { + const upload = (userId: string) => + uploadCopilotFile({ + buffer: Buffer.from(userId), + fileName: 'report.xlsx', + contentType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', + userId, + }) + const [first, second] = await Promise.all([upload('user-1'), upload('user-2')]) + + expect(first.key).not.toBe(second.key) + expect(first.key).toMatch(/^copilot\/[0-9a-f-]+\/report\.xlsx$/) + for (const stored of [first, second]) { + expect(stored.name).toBe('report.xlsx') + expect(stored.context).toBe('copilot') + expect(stored.url).toBe( + `https://sim.example/api/files/serve/${encodeURIComponent(stored.key)}` + ) + } + for (const [options] of mockUploadFile.mock.calls) { + expect(options.preserveKey).toBe(true) + expect(options.cleanupOnMetadataFailure).toBe(true) + expect(options.fileName).toBe('report.xlsx') + expect(options.metadata.originalName).toBe('report.xlsx') + expect(options.context).toBe('copilot') + } + expect(mockUploadFile.mock.calls[0][0].metadata.userId).toBe('user-1') + expect(mockUploadFile.mock.calls[1][0].metadata.userId).toBe('user-2') + }) +}) diff --git a/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts b/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts index 9f182f09eb3..c6f537a099a 100644 --- a/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts +++ b/apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts @@ -1,5 +1,7 @@ import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' import { getBaseUrl } from '@/lib/core/utils/urls' +import { buildStorageKeySegment } from '@/lib/uploads/core/storage-key' import { downloadFile, uploadFile } from '@/lib/uploads/core/storage-service' const logger = createLogger('CopilotFileManager') @@ -52,11 +54,15 @@ export async function uploadCopilotFile(options: { contentType: string userId: string }): Promise { + const storageKey = `copilot/${generateId()}/${buildStorageKeySegment('', options.fileName)}` const fileInfo = await uploadFile({ file: options.buffer, fileName: options.fileName, contentType: options.contentType, context: 'copilot', + customKey: storageKey, + preserveKey: true, + cleanupOnMetadataFailure: true, metadata: { userId: options.userId, originalName: options.fileName, @@ -77,7 +83,7 @@ export async function uploadCopilotFile(options: { id: fileInfo.key, key: fileInfo.key, context: 'copilot', - name: fileInfo.name, + name: options.fileName, url, size: fileInfo.size, type: fileInfo.type, diff --git a/apps/sim/lib/uploads/contexts/execution/execution-file-manager.test.ts b/apps/sim/lib/uploads/contexts/execution/execution-file-manager.test.ts index 4f0c5d007c5..9e4a6b0f7b8 100644 --- a/apps/sim/lib/uploads/contexts/execution/execution-file-manager.test.ts +++ b/apps/sim/lib/uploads/contexts/execution/execution-file-manager.test.ts @@ -4,9 +4,10 @@ import { dbChainMockFns, resetDbChainMock } from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockUploadToS3, mockGetPresignedUrlWithConfig } = vi.hoisted(() => ({ +const { mockUploadToS3, mockGetPresignedUrlWithConfig, mockDeleteFromS3 } = vi.hoisted(() => ({ mockUploadToS3: vi.fn(), mockGetPresignedUrlWithConfig: vi.fn(), + mockDeleteFromS3: vi.fn(), })) vi.mock('@/lib/uploads/config', () => ({ @@ -19,6 +20,7 @@ vi.mock('@/lib/uploads/config', () => ({ vi.mock('@/lib/uploads/providers/s3/client', () => ({ uploadToS3: mockUploadToS3, getPresignedUrlWithConfig: mockGetPresignedUrlWithConfig, + deleteFromS3: mockDeleteFromS3, })) import { uploadExecutionFile } from '@/lib/uploads/contexts/execution/execution-file-manager' @@ -41,6 +43,7 @@ describe('uploadExecutionFile key allocation', () => { type: contentType, })) mockGetPresignedUrlWithConfig.mockResolvedValue('https://example.com/download') + mockDeleteFromS3.mockResolvedValue(undefined) dbChainMockFns.limit.mockResolvedValue([]) dbChainMockFns.returning.mockResolvedValue([{ id: 'file-1' }]) }) @@ -64,4 +67,138 @@ describe('uploadExecutionFile key allocation', () => { expect(first.key).not.toBe(second.key) expect(dbChainMockFns.insert).toHaveBeenCalledTimes(2) }) + + it('commits tracked provenance with the canonical file before returning its URL', async () => { + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + dbChainMockFns.returning.mockImplementation(async () => { + const values = dbChainMockFns.values.mock.calls.at(-1)?.[0] + return [{ ...values, id: values?.id ?? values?.fileId, contentUpdatedAt }] + }) + const file = await uploadExecutionFile( + context, + Buffer.from('archive'), + 'report.zip', + 'application/zip', + 'user-1', + { status: 'exact', entries: [] } + ) + + expect(dbChainMockFns.transaction).toHaveBeenCalledTimes(1) + expect(dbChainMockFns.values).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ id: file.id, key: file.key, context: 'execution' }) + ) + expect(dbChainMockFns.values).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ fileId: file.id, contentUpdatedAt, status: 'exact', entries: [] }) + ) + expect(dbChainMockFns.set).toHaveBeenCalledWith({ secretProvenanceVersion: 1 }) + expect(dbChainMockFns.set.mock.invocationCallOrder[0]).toBeLessThan( + mockGetPresignedUrlWithConfig.mock.invocationCallOrder[0] + ) + expect(file).not.toHaveProperty('secretProvenance') + }) + + it('removes uploaded bytes when their provenance cannot be committed', async () => { + const failure = new Error('Provenance commit failed') + dbChainMockFns.returning + .mockResolvedValueOnce([ + { + id: 'recorded-file', + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + }, + ]) + .mockRejectedValueOnce(failure) + + await expect( + uploadExecutionFile( + context, + Buffer.from('archive'), + 'report.zip', + 'application/zip', + 'user-1', + { status: 'unknown' } + ) + ).rejects.toThrow('Provenance commit failed') + + expect(mockDeleteFromS3).toHaveBeenCalledWith( + mockUploadToS3.mock.calls[0][1], + expect.any(Object), + undefined + ) + expect(mockGetPresignedUrlWithConfig).not.toHaveBeenCalled() + }) + + it('rejects tracked uploads without an owner before writing bytes', async () => { + await expect( + uploadExecutionFile( + context, + Buffer.from('archive'), + 'report.zip', + 'application/zip', + undefined, + { + status: 'exact', + entries: [], + } + ) + ).rejects.toThrow('requires an owner and workspace') + expect(mockUploadToS3).not.toHaveBeenCalled() + }) + + it('cleans both committed metadata and bytes when its download URL cannot be issued', async () => { + const contentUpdatedAt = new Date('2026-01-01T00:00:00Z') + dbChainMockFns.returning.mockImplementation(async () => { + const values = dbChainMockFns.values.mock.calls.at(-1)?.[0] + return [{ ...values, id: values?.id ?? values?.fileId, contentUpdatedAt }] + }) + mockGetPresignedUrlWithConfig.mockRejectedValueOnce(new Error('Signing failed')) + + await expect( + uploadExecutionFile( + context, + Buffer.from('archive'), + 'report.zip', + 'application/zip', + 'user-1', + { status: 'unknown' } + ) + ).rejects.toThrow('Signing failed') + expect(mockDeleteFromS3).toHaveBeenCalledTimes(1) + expect(dbChainMockFns.set).toHaveBeenCalledWith({ deletedAt: expect.any(Date) }) + }) + + it('removes the uploaded object and metadata when creating its download URL fails', async () => { + mockGetPresignedUrlWithConfig.mockRejectedValueOnce(new Error('Presigning failed')) + + await expect( + uploadExecutionFile(context, Buffer.from('file'), 'file.txt', 'text/plain', 'user-1') + ).rejects.toThrow('Presigning failed') + + expect(mockDeleteFromS3.mock.calls[0]?.[0]).toBe(mockUploadToS3.mock.calls[0]?.[1]) + expect(dbChainMockFns.update).toHaveBeenCalledTimes(1) + }) + + it('removes its unique object when metadata insertion fails before uploadFile returns', async () => { + dbChainMockFns.returning.mockRejectedValueOnce(new Error('Metadata persistence failed')) + + await expect( + uploadExecutionFile(context, Buffer.from('file'), 'file.txt', 'text/plain', 'user-1') + ).rejects.toThrow('Metadata persistence failed') + + expect(mockDeleteFromS3.mock.calls[0]?.[0]).toBe(mockUploadToS3.mock.calls[0]?.[1]) + expect(mockDeleteFromS3).toHaveBeenCalledOnce() + expect(mockGetPresignedUrlWithConfig).not.toHaveBeenCalled() + }) + + it('keeps the original upload error if cleanup also fails', async () => { + mockGetPresignedUrlWithConfig.mockRejectedValueOnce(new Error('Presigning failed')) + mockDeleteFromS3.mockRejectedValueOnce(new Error('Deletion failed')) + + await expect( + uploadExecutionFile(context, Buffer.from('file'), 'file.txt', 'text/plain', 'user-1') + ).rejects.toThrow('Presigning failed') + + expect(mockDeleteFromS3).toHaveBeenCalledTimes(1) + }) }) diff --git a/apps/sim/lib/uploads/contexts/execution/execution-file-manager.ts b/apps/sim/lib/uploads/contexts/execution/execution-file-manager.ts index d4f4bea9cb9..8cef00a6767 100644 --- a/apps/sim/lib/uploads/contexts/execution/execution-file-manager.ts +++ b/apps/sim/lib/uploads/contexts/execution/execution-file-manager.ts @@ -1,5 +1,7 @@ +import { db } from '@sim/db' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import { generateId } from '@sim/utils/id' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { isUserFileWithMetadata } from '@/lib/core/utils/user-file' import type { ExecutionContext } from '@/lib/uploads/contexts/execution/utils' @@ -7,6 +9,16 @@ import { generateFileId, generateUniqueExecutionFileKey, } from '@/lib/uploads/contexts/execution/utils' +import { + initializeWorkspaceFileSecretProvenanceInTx, + type WorkspaceFileSecretProvenance, +} from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' +import { + deleteFileMetadata, + deleteFileMetadataByIdentity, + type FileMetadataRecord, + insertImmutableFileMetadata, +} from '@/lib/uploads/server/metadata' import type { UserFile } from '@/executor/types' const logger = createLogger('ExecutionFileStorage') @@ -69,8 +81,12 @@ export async function uploadExecutionFile( fileBuffer: Buffer, fileName: string, contentType: string, - userId?: string + userId?: string, + secretProvenance?: WorkspaceFileSecretProvenance ): Promise { + if (secretProvenance && (!userId || !context.workspaceId)) { + throw new Error('Execution file provenance requires an owner and workspace') + } logger.info(`Uploading execution file: ${fileName} for execution ${context.executionId}`) logger.debug(`File upload context:`, { workspaceId: context.workspaceId, @@ -82,7 +98,7 @@ export async function uploadExecutionFile( }) const storageKey = generateUniqueExecutionFileKey(context, fileName) - const fileId = generateFileId() + const fileId = secretProvenance ? generateId() : generateFileId() logger.info(`Generated storage key: "${storageKey}" for file: ${fileName}`) @@ -97,8 +113,10 @@ export async function uploadExecutionFile( metadata.userId = userId } + const StorageService = await getStorageService() + let uploadedKey: string | undefined + let recordedFile: FileMetadataRecord | undefined try { - const StorageService = await getStorageService() const fileInfo = await StorageService.uploadFile({ file: fileBuffer, fileName: storageKey, @@ -106,8 +124,36 @@ export async function uploadExecutionFile( context: 'execution', preserveKey: true, // Don't add timestamp prefix customKey: storageKey, // Use exact execution-scoped key + cleanupOnMetadataFailure: true, metadata, // Pass metadata for cloud storage and database tracking + ...(secretProvenance ? { persistMetadata: false } : {}), }) + uploadedKey = fileInfo.key + + if (secretProvenance && userId) { + recordedFile = await db.transaction(async (tx) => { + const record = await insertImmutableFileMetadata( + { + id: fileId, + key: fileInfo.key, + userId, + workspaceId: context.workspaceId, + context: 'execution', + originalName: fileName, + contentType, + size: fileBuffer.length, + }, + tx + ) + await initializeWorkspaceFileSecretProvenanceInTx( + tx, + record.id, + record.contentUpdatedAt, + secretProvenance + ) + return record + }) + } const presignedUrl = await StorageService.generatePresignedDownloadUrl( fileInfo.key, @@ -130,6 +176,26 @@ export async function uploadExecutionFile( }) return userFile } catch (error) { + if (uploadedKey) { + try { + await StorageService.deleteFile({ key: uploadedKey, context: 'execution' }) + if (recordedFile) { + await deleteFileMetadataByIdentity({ + id: recordedFile.id, + key: recordedFile.key, + context: 'execution', + contentUpdatedAt: recordedFile.contentUpdatedAt, + }) + } else if (!secretProvenance) { + await deleteFileMetadata(uploadedKey) + } + } catch (cleanupError) { + logger.error('Failed to clean up an unpublished execution file', { + key: uploadedKey, + error: getErrorMessage(cleanupError), + }) + } + } logger.error(`Failed to upload execution file ${fileName}:`, error) throw new Error(`Failed to upload file: ${getErrorMessage(error, 'Unknown error')}`) } diff --git a/apps/sim/lib/uploads/contexts/organization-assistant/application.test.ts b/apps/sim/lib/uploads/contexts/organization-assistant/application.test.ts new file mode 100644 index 00000000000..dca45cbf243 --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-assistant/application.test.ts @@ -0,0 +1,226 @@ +/** @vitest-environment node */ +import { dbChainMockFns, resetDbChainMock } from '@sim/testing' +import sharp from 'sharp' +import { beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ download: vi.fn(), config: vi.fn(), create: vi.fn() })) +vi.mock('@/lib/uploads/core/storage-service', () => ({ downloadFile: mocks.download })) +vi.mock('@/lib/uploads/upload-session/service', () => ({ createUploadSession: mocks.create })) +vi.mock('@/lib/permission-groups/resolve.server', () => ({ + getUserPermissionConfigForOrganization: mocks.config, +})) +vi.mock('@/lib/uploads/config', () => ({ getServeStoragePrefix: () => 's3' })) + +import { + authorizeOrganizationAttachmentControl, + createOrganizationAssistantAttachment, + finalizeOrganizationAssistantAttachment, + readOrganizationAssistantImage, +} from '@/lib/uploads/contexts/organization-assistant/application' +import { ASSISTANT_IMAGE_MAX_BYTES } from '@/lib/uploads/shared/assistant-images' +import type { UploadSessionRecord } from '@/lib/uploads/upload-session/service' + +const principal = { kind: 'session', userId: 'user-1', sessionId: 'session-1' } as const +const key = 'assistant/org-1/user-1/upload-1/image.png' +const session: UploadSessionRecord = { + id: 'upload-1', + purpose: 'mothership_attachment', + workspaceId: null, + userId: 'user-1', + metadata: { + organizationAttachment: { organizationId: 'org-1', userId: 'user-1', sessionId: 'session-1' }, + }, + finalKey: key, + storageKey: key, + fileName: 'image.png', + contentType: 'image/png', + fileSize: 100, + storageContext: 'mothership', + storageProvider: 's3', + status: 'completed', + method: 'put', + knowledgeBaseId: null, + workflowId: null, + executionId: null, + providerUploadId: null, + providerObjectVersion: 'v1', + partSize: null, + partCount: null, + uploadToken: '', + createdAt: new Date(), + expiresAt: new Date(), + completedFileId: null, + error: null, + completedAt: new Date(), + updatedAt: new Date(), +} +let png: Buffer +beforeAll(async () => { + png = await sharp({ create: { width: 4, height: 4, channels: 3, background: '#ff0000' } }) + .png() + .toBuffer() +}) +beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + mocks.config.mockResolvedValue(null) + mocks.download.mockResolvedValue(png) + dbChainMockFns.limit.mockResolvedValue([{ role: 'member' }]) +}) + +function read(overrides: Partial[0]> = {}) { + return readOrganizationAssistantImage({ principal, organizationId: 'org-1', key, ...overrides }) +} + +describe('private organization Assistant images', () => { + it('creates uploads as the actual current member with no workspace fallback', async () => { + await createOrganizationAssistantAttachment(principal, { + organizationId: 'org-1', + name: 'image.png', + contentType: 'image/png', + size: 100, + localOrigin: 'http://localhost', + }) + expect(mocks.create).toHaveBeenCalledWith( + expect.objectContaining({ + principal, + userId: 'user-1', + organizationId: 'org-1', + purpose: 'mothership_attachment', + }) + ) + expect(mocks.create.mock.calls[0][0]).not.toHaveProperty('workspaceId') + }) + + it('rejects removed members before creating an upload', async () => { + dbChainMockFns.limit.mockResolvedValue([]) + await expect( + createOrganizationAssistantAttachment(principal, { + organizationId: 'org-1', + name: 'image.png', + contentType: 'image/png', + size: 100, + localOrigin: 'http://localhost', + }) + ).rejects.toMatchObject({ code: 'not_found' }) + expect(mocks.create).not.toHaveBeenCalled() + }) + + it('reads canonical completed uploads after a new login and emits bounded decoded bytes', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([session]) + const signal = new AbortController().signal + const image = await read({ principal: { ...principal, sessionId: 'new-session' }, signal }) + expect(image).toMatchObject({ + id: 'upload-1', + key, + name: 'image.png', + contentType: 'image/webp', + }) + expect((await sharp(image.buffer).metadata()).format).toBe('webp') + expect(mocks.download).toHaveBeenCalledWith({ + key, + context: 'mothership', + maxBytes: ASSISTANT_IMAGE_MAX_BYTES, + signal, + }) + }) + + it.each([ + { key: 'https://example.com/image.png' }, + { key: 'assistant/org-1/user-1/../image.png' }, + { principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const }, + ])('rejects invalid references or non-session callers before loading', async (input) => { + await expect(read(input)).rejects.toMatchObject({ code: 'not_found' }) + expect(dbChainMockFns.limit).not.toHaveBeenCalled() + expect(mocks.download).not.toHaveBeenCalled() + }) + + it.each([ + { organizationId: 'other-org' }, + { principal: { ...principal, userId: 'other-user' } }, + { key: 'assistant/other-org/user-1/upload-1/image.png' }, + ])('rejects a mismatched asserted owner', async (input) => { + dbChainMockFns.limit.mockResolvedValueOnce([session]) + await expect(read(input)).rejects.toMatchObject({ code: 'not_found' }) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('refuses absent, incomplete, or purged records', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([]) + await expect(read()).rejects.toMatchObject({ code: 'not_found' }) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('rechecks membership for every preview/model read', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([session]).mockResolvedValueOnce([]) + await expect(read()).rejects.toMatchObject({ code: 'not_found' }) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('rejects missing immutable scope metadata', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([{ ...session, metadata: {} }]) + await expect(read()).rejects.toMatchObject({ code: 'not_found' }) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('refuses metadata above the byte cap without downloading', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([ + { ...session, fileSize: ASSISTANT_IMAGE_MAX_BYTES + 1 }, + ]) + await expect(read()).rejects.toMatchObject({ code: 'payload_too_large' }) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it.each([ + '', + '', + ])('rejects active content with a forged image MIME', async (content) => { + dbChainMockFns.limit.mockResolvedValueOnce([session]) + mocks.download.mockResolvedValue(Buffer.from(content)) + await expect(read()).rejects.toMatchObject({ code: 'validation' }) + }) + + it('propagates storage infrastructure failures unchanged', async () => { + const error = new Error('storage unavailable') + dbChainMockFns.limit.mockResolvedValueOnce([session]) + mocks.download.mockRejectedValue(error) + await expect(read()).rejects.toBe(error) + }) + + it('rejects compressed images above the 25 megapixel decode budget', async () => { + const largePng = await sharp({ + create: { width: 5001, height: 5000, channels: 3, background: '#000' }, + }) + .png() + .toBuffer() + expect(largePng.length).toBeLessThan(ASSISTANT_IMAGE_MAX_BYTES) + dbChainMockFns.limit.mockResolvedValueOnce([session]) + mocks.download.mockResolvedValue(largePng) + await expect(read()).rejects.toMatchObject({ code: 'validation', cause: expect.any(Error) }) + }) + + it('binds upload controls to the exact creating session', async () => { + await expect( + authorizeOrganizationAttachmentControl({ ...principal, sessionId: 'other-session' }, session) + ).rejects.toMatchObject({ code: 'not_found' }) + expect(dbChainMockFns.limit).not.toHaveBeenCalled() + }) + + it('reauthorizes finalization after decoding before returning an attachment', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([{ role: 'member' }]).mockResolvedValueOnce([]) + await expect(finalizeOrganizationAssistantAttachment(principal, session)).rejects.toMatchObject( + { code: 'not_found' } + ) + expect(mocks.download).toHaveBeenCalledTimes(1) + }) + + it('returns the same durable metadata on completion replay', async () => { + const first = await finalizeOrganizationAssistantAttachment(principal, session) + const second = await finalizeOrganizationAssistantAttachment(principal, session) + expect(second).toEqual(first) + expect(first).toMatchObject({ + key, + path: `/api/files/serve/s3/${encodeURIComponent(key)}?context=mothership`, + }) + }) +}) diff --git a/apps/sim/lib/uploads/contexts/organization-assistant/application.ts b/apps/sim/lib/uploads/contexts/organization-assistant/application.ts new file mode 100644 index 00000000000..ea894123634 --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-assistant/application.ts @@ -0,0 +1,181 @@ +import type { Principal } from '@sim/auth/principal' +import { db } from '@sim/db' +import { uploadSession } from '@sim/db/schema' +import { and, eq, isNull } from 'drizzle-orm' +import sharp from 'sharp' +import { authorizeOrganizationOperation } from '@/lib/core/application/organization-authorization' +import { defineOrganizationOperation } from '@/lib/core/application/organization-operation' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { getServeStoragePrefix } from '@/lib/uploads/config' +import { + assertOrganizationAttachmentControlBinding, + organizationAttachmentBinding, +} from '@/lib/uploads/contexts/organization-assistant/binding' +import { downloadFile } from '@/lib/uploads/core/storage-service' +import { + ASSISTANT_IMAGE_MAX_BYTES, + isAssistantImageType, +} from '@/lib/uploads/shared/assistant-images' +import { createUploadSession, type UploadSessionRecord } from '@/lib/uploads/upload-session/service' + +const organizationAttachmentOperation = defineOrganizationOperation({ + id: 'organization.assistant.attachments.use', + minimumRole: 'member', + principalKinds: ['session'], + capability: 'copilot.use', +}) + +const MAX_ASSISTANT_IMAGE_PIXELS = 25_000_000 + +export interface CreateOrganizationAssistantAttachmentInput { + organizationId: string + name: string + contentType: string + size: number + localOrigin: string +} + +export async function createOrganizationAssistantAttachment( + principal: Principal, + input: CreateOrganizationAssistantAttachmentInput +) { + const context = await authorizeOrganizationOperation( + principal, + organizationAttachmentOperation, + input + ) + return createUploadSession({ + purpose: 'mothership_attachment', + principal, + organizationId: context.organizationId, + userId: context.userId, + fileName: input.name, + contentType: input.contentType, + fileSize: input.size, + localOrigin: input.localOrigin, + }) +} + +export async function authorizeOrganizationAttachmentControl( + principal: Principal, + session: UploadSessionRecord +): Promise { + const binding = assertOrganizationAttachmentControlBinding(session, principal) + await authorizeOrganizationOperation(principal, organizationAttachmentOperation, binding) +} + +/** A bounded decode removes active content, metadata, and animation before preview or model use. */ +async function readImageBytes(key: string, contentType: string, signal?: AbortSignal) { + if (!isAssistantImageType(contentType)) + throw new OrchestrationError('validation', 'Unsupported image type') + const buffer = await downloadFile({ + key, + context: 'mothership', + maxBytes: ASSISTANT_IMAGE_MAX_BYTES, + signal, + }) + try { + const image = sharp(buffer, { limitInputPixels: MAX_ASSISTANT_IMAGE_PIXELS, pages: 1 }) + const metadata = await image.metadata() + if (!metadata.format || !['jpeg', 'png', 'gif', 'webp'].includes(metadata.format)) { + throw new OrchestrationError( + 'validation', + 'Attachment must contain a PNG, JPEG, GIF, or WebP image' + ) + } + const normalized = await image + .rotate() + .resize(1568, 1568, { fit: 'inside', withoutEnlargement: true }) + .webp({ quality: 85 }) + .toBuffer() + if (normalized.length > ASSISTANT_IMAGE_MAX_BYTES) + throw new OrchestrationError('payload_too_large', 'Image exceeds the 5 MB limit') + return normalized + } catch (cause) { + if (cause instanceof OrchestrationError) throw cause + const error = new OrchestrationError('validation', 'Attachment is not a valid supported image') + error.cause = cause + throw error + } +} + +export async function finalizeOrganizationAssistantAttachment( + principal: Principal, + session: UploadSessionRecord +) { + await authorizeOrganizationAttachmentControl(principal, session) + await readImageBytes(session.finalKey, session.contentType) + await authorizeOrganizationAttachmentControl(principal, session) + return { + path: `/api/files/serve/${getServeStoragePrefix()}/${encodeURIComponent(session.finalKey)}?context=mothership`, + key: session.finalKey, + name: session.fileName, + size: session.fileSize, + type: session.contentType, + } +} + +/** Resolves only completed images owned by the current user and their current organization. */ +export async function readOrganizationAssistantImage(input: { + principal: Principal + organizationId?: string + key: string + signal?: AbortSignal +}) { + if (input.principal.kind !== 'session') + throw new OrchestrationError('not_found', 'Attachment not found') + const keyParts = input.key.split('/') + if ( + keyParts.length !== 5 || + keyParts[0] !== 'assistant' || + keyParts.some((part) => !part || part === '.' || part === '..') + ) { + throw new OrchestrationError('not_found', 'Attachment not found') + } + const [session] = await db + .select({ + id: uploadSession.id, + purpose: uploadSession.purpose, + workspaceId: uploadSession.workspaceId, + userId: uploadSession.userId, + metadata: uploadSession.metadata, + fileName: uploadSession.fileName, + contentType: uploadSession.contentType, + fileSize: uploadSession.fileSize, + finalKey: uploadSession.finalKey, + }) + .from(uploadSession) + .where( + and( + eq(uploadSession.id, keyParts[3]), + eq(uploadSession.finalKey, input.key), + eq(uploadSession.userId, input.principal.userId), + eq(uploadSession.purpose, 'mothership_attachment'), + eq(uploadSession.status, 'completed'), + isNull(uploadSession.workspaceId) + ) + ) + .limit(1) + if (!session) throw new OrchestrationError('not_found', 'Attachment not found') + const binding = organizationAttachmentBinding(session) + if ( + session.userId !== input.principal.userId || + keyParts[1] !== binding.organizationId || + keyParts[2] !== binding.userId || + (input.organizationId && input.organizationId !== binding.organizationId) + ) { + throw new OrchestrationError('not_found', 'Attachment not found') + } + await authorizeOrganizationOperation(input.principal, organizationAttachmentOperation, binding) + if (session.fileSize > ASSISTANT_IMAGE_MAX_BYTES) + throw new OrchestrationError('payload_too_large', 'Image exceeds the 5 MB limit') + const buffer = await readImageBytes(session.finalKey, session.contentType, input.signal) + return { + id: session.id, + key: session.finalKey, + name: session.fileName, + size: buffer.length, + contentType: 'image/webp', + buffer, + } +} diff --git a/apps/sim/lib/uploads/contexts/organization-assistant/binding.ts b/apps/sim/lib/uploads/contexts/organization-assistant/binding.ts new file mode 100644 index 00000000000..df2743c533c --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-assistant/binding.ts @@ -0,0 +1,56 @@ +import type { Principal } from '@sim/auth/principal' +import { isRecordLike } from '@sim/utils/object' +import { OrchestrationError } from '@/lib/core/orchestration/types' + +export interface OrganizationAttachmentBinding { + organizationId: string + userId: string + sessionId: string +} + +interface OrganizationAttachmentSession { + purpose: string + workspaceId: string | null + userId: string + metadata: Record +} + +/** Organization attachments remain private to their uploader across new login sessions. */ +export function organizationAttachmentBinding( + session: OrganizationAttachmentSession +): OrganizationAttachmentBinding { + const binding = session.metadata.organizationAttachment + if ( + session.purpose !== 'mothership_attachment' || + session.workspaceId !== null || + !isRecordLike(binding) || + typeof binding.organizationId !== 'string' || + !binding.organizationId || + binding.userId !== session.userId || + typeof binding.sessionId !== 'string' || + !binding.sessionId + ) { + throw new OrchestrationError('not_found', 'Attachment not found') + } + return { + organizationId: binding.organizationId, + userId: session.userId, + sessionId: binding.sessionId, + } +} + +/** A byte-transfer token cannot replace the session that initiated the upload. */ +export function assertOrganizationAttachmentControlBinding( + session: OrganizationAttachmentSession, + principal: Principal +): OrganizationAttachmentBinding { + const binding = organizationAttachmentBinding(session) + if ( + principal.kind !== 'session' || + principal.userId !== binding.userId || + principal.sessionId !== binding.sessionId + ) { + throw new OrchestrationError('not_found', 'Upload session not found') + } + return binding +} diff --git a/apps/sim/lib/uploads/contexts/organization-logo/application.integration.ts b/apps/sim/lib/uploads/contexts/organization-logo/application.integration.ts new file mode 100644 index 00000000000..bd97af65748 --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-logo/application.integration.ts @@ -0,0 +1,192 @@ +/** Real PostgreSQL verifies cross-session registration and durable logo retention. */ +import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' +import { member, organization, organizationColumns, uploadSession } from '@sim/db/schema' +import { generateId } from '@sim/utils/id' +import { eq } from 'drizzle-orm' +import type { Sql } from 'postgres' +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' + +const fixture = vi.hoisted(() => ({ + schema: '', + connection: undefined as Sql | undefined, + deleteObject: vi.fn(), + headObject: vi.fn(), +})) + +vi.mock('@sim/db', async () => { + const { drizzle } = await import('drizzle-orm/postgres-js') + const { default: postgres } = await import('postgres') + const { withUtcTimestamps } = await import('@sim/db/timestamps') + const { generateId } = await import('@sim/utils/id') + fixture.schema = `logo_test_${generateId().replaceAll('-', '')}` + fixture.connection = postgres( + process.env.KNOWLEDGE_ACL_TEST_DATABASE_URL!, + withUtcTimestamps({ + max: 4, + prepare: false, + fetch_types: false, + connection: { search_path: fixture.schema }, + onnotice: () => {}, + }) + ) + const database = drizzle(fixture.connection) + return { db: database, dbFor: () => database } +}) + +vi.mock('@/lib/permission-groups/resolve.server', () => ({ + getUserPermissionConfigForOrganization: async () => null, +})) +vi.mock('@sim/audit', async (importOriginal) => ({ + ...(await importOriginal()), + recordAudit: vi.fn(), +})) +vi.mock('@/lib/uploads/upload-session/cleanup', () => ({ + maybeCleanupLocalUploadArtifacts: async () => ({ scanned: 0, removed: 0 }), +})) +vi.mock('@/lib/uploads/upload-session/provider', async (importOriginal) => ({ + ...(await importOriginal()), + createPutProviderTransfer: async () => ({ + method: 'put', + url: 'http://localhost/upload', + headers: {}, + }), + headProviderObject: fixture.headObject, + deleteProviderObjectVersion: fixture.deleteObject, +})) + +import { + createOrganizationLogoUpload, + finalizeOrganizationLogoUpload, +} from '@/lib/uploads/contexts/organization-logo/application' +import { cleanupExpiredUploadSessions } from '@/lib/uploads/upload-session/service' + +describe('organization logo concurrency and retention', () => { + const organizationId = generateId() + const principals = [ + { kind: 'session', userId: generateId(), sessionId: generateId() }, + { kind: 'session', userId: generateId(), sessionId: generateId() }, + ] as const + const request = { headers: new Headers() } + + beforeAll(async () => { + const connection = fixture.connection! + await connection`CREATE SCHEMA ${connection(fixture.schema)}` + for (const table of ['user', 'member', 'organization', 'upload_session']) { + await connection`CREATE TABLE ${connection(table)} (LIKE ${connection(`public.${table}`)} INCLUDING ALL)` + } + await db.insert(withInsertColumns(organization, organizationColumns)).values({ + id: organizationId, + name: 'Logo test organization', + slug: generateId(), + createdAt: new Date(), + }) + await db.insert(member).values( + principals.map((principal) => ({ + id: generateId(), + organizationId, + userId: principal.userId, + role: 'admin', + })) + ) + }) + + beforeEach(async () => { + vi.clearAllMocks() + await db.delete(uploadSession) + await db.update(organization).set({ logo: null }).where(eq(organization.id, organizationId)) + }) + + afterAll(async () => { + const connection = fixture.connection + if (!connection) return + try { + await connection`DROP SCHEMA ${connection(fixture.schema)} CASCADE` + } finally { + await connection.end() + } + }) + + async function start(principal = principals[0]) { + const session = await createOrganizationLogoUpload(principal, { + organizationId, + name: 'logo.png', + contentType: 'image/png', + size: 100, + localOrigin: 'http://localhost', + }) + await db + .update(uploadSession) + .set({ status: 'finalizing' }) + .where(eq(uploadSession.id, session.id)) + return session + } + + async function currentLogo() { + const [row] = await db + .select({ logo: organization.logo }) + .from(organization) + .where(eq(organization.id, organizationId)) + return row.logo + } + + it('rejects an older upload after a different administrator completes a newer one', async () => { + const older = await start() + const newer = await start(principals[1]) + const result = await finalizeOrganizationLogoUpload(principals[1], newer, request) + await expect( + finalizeOrganizationLogoUpload(principals[0], older, request) + ).rejects.toMatchObject({ code: 'conflict' }) + expect(await currentLogo()).toBe(result.value.path) + }) + + it('allows only one concurrent completion from the same starting logo', async () => { + const sessions = await Promise.all(principals.map((principal) => start(principal))) + const results = await Promise.allSettled( + sessions.map((session, index) => + finalizeOrganizationLogoUpload(principals[index], session, request) + ) + ) + expect(results.filter((result) => result.status === 'fulfilled')).toHaveLength(1) + const rejected = results.find((result) => result.status === 'rejected') + expect(rejected?.status === 'rejected' && rejected.reason).toMatchObject({ code: 'conflict' }) + }) + + it('retains the active logo and retries deletion of a replaced logo before purging its record', async () => { + const first = await start() + await finalizeOrganizationLogoUpload(principals[0], first, request) + const second = await start(principals[1]) + const current = await finalizeOrganizationLogoUpload(principals[1], second, request) + await finalizeOrganizationLogoUpload(principals[0], first, request) + expect(await currentLogo()).toBe(current.value.path) + await db.update(uploadSession).set({ + status: 'completed', + completedAt: new Date(Date.now() - 8 * 24 * 60 * 60 * 1000), + }) + fixture.headObject.mockImplementation(async ({ key }: { key: string }) => { + expect(key).toBe(first.finalKey) + return { + size: first.fileSize, + contentType: first.contentType, + uploadId: first.id, + version: 'v1', + } + }) + fixture.deleteObject.mockRejectedValueOnce(new Error('Storage unavailable')) + expect(await cleanupExpiredUploadSessions()).toEqual({ expired: 0, failed: 1, purged: 0 }) + expect(await db.select({ id: uploadSession.id }).from(uploadSession)).toHaveLength(2) + fixture.deleteObject.mockResolvedValue(undefined) + expect(await cleanupExpiredUploadSessions()).toEqual({ expired: 0, failed: 0, purged: 1 }) + expect(await db.select({ id: uploadSession.id }).from(uploadSession)).toEqual([ + { id: second.id }, + ]) + expect(await currentLogo()).toBe(current.value.path) + expect(fixture.deleteObject).toHaveBeenLastCalledWith( + expect.objectContaining({ + key: first.finalKey, + context: 'organization-logos', + version: 'v1', + }) + ) + }) +}) diff --git a/apps/sim/lib/uploads/contexts/organization-logo/application.test.ts b/apps/sim/lib/uploads/contexts/organization-logo/application.test.ts new file mode 100644 index 00000000000..cd836098476 --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-logo/application.test.ts @@ -0,0 +1,238 @@ +/** @vitest-environment node */ +import { recordAudit } from '@sim/audit' +import { db } from '@sim/db' +import { dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ config: vi.fn(), create: vi.fn() })) +vi.mock('@/lib/uploads/upload-session/service', () => ({ createUploadSession: mocks.create })) +vi.mock('@/lib/permission-groups/resolve.server', () => ({ + getUserPermissionConfigForOrganization: mocks.config, +})) +vi.mock('@/lib/uploads/config', () => ({ getServeStoragePrefix: () => 's3' })) +vi.mock('@sim/audit', async (importOriginal) => ({ + ...(await importOriginal()), + recordAudit: vi.fn(), +})) + +import { createInternalFileUploadBodySchema } from '@/lib/api/contracts/upload-sessions' +import { + authorizeOrganizationLogoControl, + createOrganizationLogoUpload, + finalizeOrganizationLogoUpload, + organizationLogoOperation, +} from '@/lib/uploads/contexts/organization-logo/application' +import type { UploadSessionRecord } from '@/lib/uploads/upload-session/service' + +const principal = { kind: 'session', userId: 'user-1', sessionId: 'session-1' } as const +const input = { + organizationId: 'org-1', + name: 'logo.png', + contentType: 'image/png', + size: 100, + localOrigin: 'http://localhost', +} +const key = 'organization-logos/org-1/upload-1-logo.png' +const session = { + id: 'upload-1', + purpose: 'organization_logo', + workspaceId: null, + userId: 'user-1', + metadata: { + organizationLogo: { + organizationId: 'org-1', + expectedLogo: null, + userId: 'user-1', + sessionId: 'session-1', + }, + }, + finalKey: key, + storageKey: key, + fileName: 'logo.png', + contentType: 'image/png', + fileSize: 100, +} as UploadSessionRecord +const request = { headers: new Headers() } + +beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + mocks.config.mockResolvedValue(null) + dbChainMockFns.limit.mockResolvedValue([{ role: 'admin' }]) +}) + +describe('organization logo uploads', () => { + it('defines a session-only organization administrator operation', () => { + expect(organizationLogoOperation).toMatchObject({ + minimumRole: 'admin', + principalKinds: ['session'], + }) + }) + + it.each(['owner', 'admin'])( + 'allows the current %s and uses their real organization identity', + async (role) => { + dbChainMockFns.limit.mockResolvedValueOnce([{ role }]).mockResolvedValueOnce([{ logo: null }]) + await createOrganizationLogoUpload(principal, input) + expect(mocks.create).toHaveBeenCalledWith({ + purpose: 'organization_logo', + principal, + organizationId: 'org-1', + expectedLogo: null, + userId: 'user-1', + fileName: 'logo.png', + contentType: 'image/png', + fileSize: 100, + localOrigin: 'http://localhost', + }) + } + ) + + it.each([ + ['member', 'forbidden'], + ['invalid', 'not_found'], + [undefined, 'not_found'], + ])('rejects %s before storage is initialized', async (role, code) => { + dbChainMockFns.limit.mockResolvedValue(role ? [{ role }] : []) + await expect(createOrganizationLogoUpload(principal, input)).rejects.toMatchObject({ code }) + expect(mocks.create).not.toHaveBeenCalled() + }) + + it.each([ + { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const, + { kind: 'workspace_api_key', workspaceId: 'workspace-1', keyId: 'key-1' } as const, + ])('rejects other principal kinds before protected reads', async (caller) => { + await expect(createOrganizationLogoUpload(caller, input)).rejects.toMatchObject({ + code: 'forbidden', + }) + expect(dbChainMockFns.limit).not.toHaveBeenCalled() + expect(mocks.create).not.toHaveBeenCalled() + }) + + it.each([ + { ...principal, sessionId: 'other-session' }, + { ...principal, userId: 'other-user' }, + ])('binds all upload controls to the creating credential', async (caller) => { + await expect(authorizeOrganizationLogoControl(caller, session)).rejects.toMatchObject({ + code: 'not_found', + }) + expect(dbChainMockFns.limit).not.toHaveBeenCalled() + }) + + it('rejects forged workspace scope and missing organization binding', async () => { + await expect( + authorizeOrganizationLogoControl(principal, { ...session, workspaceId: 'workspace-1' }) + ).rejects.toMatchObject({ code: 'not_found' }) + await expect( + authorizeOrganizationLogoControl(principal, { ...session, metadata: {} }) + ).rejects.toMatchObject({ code: 'not_found' }) + expect(dbChainMockFns.limit).not.toHaveBeenCalled() + }) + + it('persists the logo and completion marker in one transaction, with one organization audit', async () => { + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ completedFileId: null, status: 'finalizing' }]) + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ logo: null }]) + dbChainMockFns.returning + .mockResolvedValueOnce([{ id: 'org-1', name: 'Test org' }]) + .mockResolvedValueOnce([{ id: 'upload-1' }]) + const first = await finalizeOrganizationLogoUpload(principal, session, request) + expect(db.transaction).toHaveBeenCalledOnce() + expect(dbChainMockFns.set).toHaveBeenCalledWith({ logo: first.value.path }) + expect(dbChainMockFns.set).toHaveBeenCalledWith({ + completedFileId: 'upload-1', + metadata: { ...session.metadata, organizationLogoPath: first.value.path }, + updatedAt: expect.any(Date), + }) + expect(recordAudit).toHaveBeenCalledWith( + expect.objectContaining({ + action: 'organization.updated', + resourceType: 'organization', + resourceId: 'org-1', + actorId: 'user-1', + metadata: expect.objectContaining({ + organizationId: 'org-1', + operation: 'organization.logo.update', + }), + }) + ) + dbChainMockFns.set.mockClear() + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ completedFileId: 'upload-1', status: 'completed' }]) + expect(await finalizeOrganizationLogoUpload(principal, session, request)).toEqual(first) + expect(dbChainMockFns.set).not.toHaveBeenCalled() + expect(recordAudit).toHaveBeenCalledOnce() + }) + + it('rechecks administrator membership under a transaction lock before changing the organization', async () => { + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ completedFileId: null, status: 'finalizing' }]) + .mockResolvedValueOnce([{ role: 'member' }]) + await expect(finalizeOrganizationLogoUpload(principal, session, request)).rejects.toMatchObject( + { code: 'forbidden' } + ) + expect(dbChainMockFns.set).not.toHaveBeenCalled() + expect(recordAudit).not.toHaveBeenCalled() + }) + + it('rejects an organization deleted before registration without marking or auditing completion', async () => { + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ completedFileId: null, status: 'finalizing' }]) + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([]) + await expect(finalizeOrganizationLogoUpload(principal, session, request)).rejects.toMatchObject( + { code: 'not_found' } + ) + expect(dbChainMockFns.set).not.toHaveBeenCalled() + expect(recordAudit).not.toHaveBeenCalled() + }) + + it('rejects an unfinished upload after another session replaces its starting logo', async () => { + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ completedFileId: null, status: 'finalizing' }]) + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ logo: '/api/files/serve/s3/newer-logo.png' }]) + await expect(finalizeOrganizationLogoUpload(principal, session, request)).rejects.toMatchObject( + { + code: 'conflict', + } + ) + expect(dbChainMockFns.set).not.toHaveBeenCalled() + expect(recordAudit).not.toHaveBeenCalled() + }) + + it('captures an existing logo as server-authored concurrency state', async () => { + dbChainMockFns.limit + .mockResolvedValueOnce([{ role: 'admin' }]) + .mockResolvedValueOnce([{ logo: '/existing-logo.png' }]) + await createOrganizationLogoUpload(principal, input) + expect(mocks.create).toHaveBeenCalledWith( + expect.objectContaining({ + expectedLogo: '/existing-logo.png', + }) + ) + }) + + it('propagates infrastructure errors without reporting a successful logo update', async () => { + dbChainMockFns.limit.mockRejectedValueOnce(new Error('database unavailable')) + await expect(finalizeOrganizationLogoUpload(principal, session, request)).rejects.toThrow( + 'database unavailable' + ) + expect(dbChainMockFns.set).not.toHaveBeenCalled() + expect(recordAudit).not.toHaveBeenCalled() + }) + + it.each([ + { ...input, purpose: 'organization_logo', organizationId: undefined }, + { ...input, purpose: 'organization_logo', workspaceId: 'workspace-1' }, + { ...input, purpose: 'organization_logo', size: 5 * 1024 * 1024 + 1 }, + ])('rejects invalid organization logo contracts', ({ localOrigin: _localOrigin, ...body }) => { + expect(createInternalFileUploadBodySchema.safeParse(body).success).toBe(false) + }) +}) diff --git a/apps/sim/lib/uploads/contexts/organization-logo/application.ts b/apps/sim/lib/uploads/contexts/organization-logo/application.ts new file mode 100644 index 00000000000..5b7914b28fc --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-logo/application.ts @@ -0,0 +1,169 @@ +import { AuditAction, AuditResourceType } from '@sim/audit' +import type { Principal } from '@sim/auth/principal' +import { db } from '@sim/db' +import { member, organization, uploadSession } from '@sim/db/schema' +import { isOrgAdminRole } from '@sim/platform-authz/workspace' +import { and, eq, isNull } from 'drizzle-orm' +import { recordProjectedUseCaseAuditEntries } from '@/lib/core/application/authorized-workspace-use-case' +import { authorizeOrganizationOperation } from '@/lib/core/application/organization-authorization' +import { defineOrganizationOperation } from '@/lib/core/application/organization-operation' +import { + OrchestrationError, + type OrchestrationRequestContext, +} from '@/lib/core/orchestration/types' +import { getServeStoragePrefix } from '@/lib/uploads/config' +import { assertOrganizationLogoControlBinding } from '@/lib/uploads/contexts/organization-logo/binding' +import { createUploadSession, type UploadSessionRecord } from '@/lib/uploads/upload-session/service' + +/** + * permission-group-exempt: organization appearance is governed by organization administrator membership. + */ +export const organizationLogoOperation = defineOrganizationOperation({ + id: 'organization.logo.update', + minimumRole: 'admin', + principalKinds: ['session'], + capability: 'none', +}) + +export interface CreateOrganizationLogoUploadInput { + organizationId: string + name: string + contentType: string + size: number + localOrigin: string +} + +export async function createOrganizationLogoUpload( + principal: Principal, + input: CreateOrganizationLogoUploadInput +) { + const context = await authorizeOrganizationOperation(principal, organizationLogoOperation, input) + const [current] = await db + .select({ logo: organization.logo }) + .from(organization) + .where(eq(organization.id, context.organizationId)) + .limit(1) + if (!current) throw new OrchestrationError('not_found', 'Organization not found') + return createUploadSession({ + purpose: 'organization_logo', + principal, + organizationId: context.organizationId, + expectedLogo: current.logo, + userId: context.userId, + fileName: input.name, + contentType: input.contentType, + fileSize: input.size, + localOrigin: input.localOrigin, + }) +} + +export async function authorizeOrganizationLogoControl( + principal: Principal, + session: UploadSessionRecord +): Promise { + const binding = assertOrganizationLogoControlBinding(session, principal) + await authorizeOrganizationOperation(principal, organizationLogoOperation, binding) +} + +export function organizationLogoUploadResult(session: UploadSessionRecord) { + return { + path: `/api/files/serve/${getServeStoragePrefix()}/${encodeURIComponent(session.finalKey)}?context=organization-logos`, + key: session.finalKey, + name: session.fileName, + size: session.fileSize, + type: session.contentType, + } +} + +/** Registers the logo and durable completion marker together, so retries cannot restore an old logo. */ +export async function finalizeOrganizationLogoUpload( + principal: Principal, + session: UploadSessionRecord, + request: OrchestrationRequestContext +) { + await authorizeOrganizationLogoControl(principal, session) + const binding = assertOrganizationLogoControlBinding(session, principal) + const value = organizationLogoUploadResult(session) + const changed = await db.transaction(async (tx) => { + const [current] = await tx + .select({ completedFileId: uploadSession.completedFileId, status: uploadSession.status }) + .from(uploadSession) + .where( + and( + eq(uploadSession.id, session.id), + eq(uploadSession.purpose, 'organization_logo'), + eq(uploadSession.finalKey, session.finalKey), + isNull(uploadSession.workspaceId) + ) + ) + .for('update') + .limit(1) + if (!current) throw new OrchestrationError('not_found', 'Upload session not found') + if (current.completedFileId) return null + if (current.status !== 'finalizing') { + throw new OrchestrationError('conflict', 'Upload session is not ready for finalization') + } + + const [membership] = await tx + .select({ role: member.role }) + .from(member) + .where( + and(eq(member.organizationId, binding.organizationId), eq(member.userId, binding.userId)) + ) + .for('share') + .limit(1) + if (!membership) throw new OrchestrationError('not_found', 'Organization not found') + if (!isOrgAdminRole(membership.role)) { + throw new OrchestrationError('forbidden', 'Organization administrator access is required') + } + const [currentOrganization] = await tx + .select({ logo: organization.logo }) + .from(organization) + .where(eq(organization.id, binding.organizationId)) + .for('update') + .limit(1) + if (!currentOrganization) throw new OrchestrationError('not_found', 'Organization not found') + if (currentOrganization.logo !== binding.expectedLogo) { + throw new OrchestrationError( + 'conflict', + 'The organization logo changed while this upload was in progress. Please upload it again.' + ) + } + const [updated] = await tx + .update(organization) + .set({ logo: value.path }) + .where(eq(organization.id, binding.organizationId)) + .returning({ id: organization.id, name: organization.name }) + if (!updated) throw new OrchestrationError('not_found', 'Organization not found') + const [registered] = await tx + .update(uploadSession) + .set({ + completedFileId: session.id, + metadata: { ...session.metadata, organizationLogoPath: value.path }, + updatedAt: new Date(), + }) + .where(and(eq(uploadSession.id, session.id), eq(uploadSession.status, 'finalizing'))) + .returning({ id: uploadSession.id }) + if (!registered) throw new Error('Organization logo registration marker could not be persisted') + return updated + }) + if (changed) { + recordProjectedUseCaseAuditEntries( + organizationLogoOperation, + null, + principal, + request, + [ + { + action: AuditAction.ORGANIZATION_UPDATED, + resourceType: AuditResourceType.ORGANIZATION, + resourceId: changed.id, + resourceName: changed.name, + description: 'Updated organization logo', + }, + ], + binding.organizationId + ) + } + return { value, completedFileId: session.id } +} diff --git a/apps/sim/lib/uploads/contexts/organization-logo/binding.ts b/apps/sim/lib/uploads/contexts/organization-logo/binding.ts new file mode 100644 index 00000000000..c137efe2257 --- /dev/null +++ b/apps/sim/lib/uploads/contexts/organization-logo/binding.ts @@ -0,0 +1,45 @@ +import type { Principal } from '@sim/auth/principal' +import { isRecordLike } from '@sim/utils/object' +import { OrchestrationError } from '@/lib/core/orchestration/types' + +export interface OrganizationLogoBinding { + organizationId: string + expectedLogo: string | null + userId: string + sessionId: string +} + +interface OrganizationLogoSession { + purpose: string + workspaceId: string | null + userId: string + metadata: Record +} + +/** The upload's organization and credential are immutable server-authored scope. */ +export function assertOrganizationLogoControlBinding( + session: OrganizationLogoSession, + principal: Principal +): OrganizationLogoBinding { + const binding = session.metadata.organizationLogo + if ( + session.purpose !== 'organization_logo' || + session.workspaceId !== null || + !isRecordLike(binding) || + typeof binding.organizationId !== 'string' || + !binding.organizationId || + (binding.expectedLogo !== null && typeof binding.expectedLogo !== 'string') || + binding.userId !== session.userId || + principal.kind !== 'session' || + principal.userId !== binding.userId || + principal.sessionId !== binding.sessionId + ) { + throw new OrchestrationError('not_found', 'Upload session not found') + } + return { + organizationId: binding.organizationId, + expectedLogo: binding.expectedLogo, + userId: principal.userId, + sessionId: principal.sessionId, + } +} diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts index df414d185b1..33221719dea 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts @@ -5,6 +5,7 @@ import { randomBytes } from 'crypto' import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { uploadSession, type WorkspaceFileRow, @@ -266,7 +267,7 @@ async function insertWorkspaceFileMetadataInTx( metadata: WorkspaceFileMetadataInsert ): Promise { const [inserted] = await tx - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ ...omit(metadata, ['size']), sizeBytes: metadata.size, @@ -1056,7 +1057,7 @@ export async function trackChatUpload( await db.transaction(async (tx) => { const [inserted] = await tx - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ id: fileId, key: s3Key, diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts index a1b84a68099..b6bdbd17523 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts @@ -21,11 +21,16 @@ vi.mock('@/lib/execution/durable-secret-provenance-enforcement', () => ({ })) import type { DbTransaction } from '@/lib/db/types' +import { + PROVENANCE_MAX_ENTRIES, + PROVENANCE_MAX_SERIALIZED_BYTES, +} from '@/lib/execution/provenance-limits' import { areModelSafeWorkspaceFileKeys, copyWorkspaceFileSecretProvenanceInTx, createWorkspaceFileSecretProvenanceFromRegistry, filterModelSafeWorkspaceFileAttachments, + getBoundWorkspaceFileSecretProvenance, importWorkspaceFileSecretProvenanceForModelView, importWorkspaceFileSecretProvenanceForRuntime, initializeWorkspaceFileSecretProvenanceInTx, @@ -39,6 +44,64 @@ import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secr const CONTENT_UPDATED_AT = new Date('2026-08-04T00:00:00.000Z') +describe('execution file sidecars at model boundaries', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + }) + + it.each([ + { status: 'exact', version: 1, stale: false, entries: [], safe: true }, + { status: 'unknown', version: 1, stale: false, entries: [], safe: false }, + { status: 'exact', version: 1, stale: true, entries: [], safe: false }, + { status: null, version: 1, stale: false, entries: null, safe: false }, + { status: 'unknown', version: null, stale: true, entries: [], safe: true }, + { + status: 'exact', + version: 1, + stale: false, + entries: [{ name: 'KEY', encryptedValue: 'ciphertext', sourceUserId: 'writer' }], + safe: false, + }, + ])( + 'classifies execution bytes consistently: %j', + async ({ status, version, stale, entries, safe }) => { + const key = 'execution/workspace-1/workflow-1/execution-1/file.zip' + const row = { + key, + workspaceId: 'workspace-1', + context: 'execution', + fileContentUpdatedAt: CONTENT_UPDATED_AT, + secretProvenanceVersion: version, + provenanceContentUpdatedAt: stale ? new Date(0) : CONTENT_UPDATED_AT, + status, + entries, + } + for (const enforced of [false, true]) { + mockIsEnforced.mockReturnValue(enforced) + queueTableRows(workspaceFiles, [row]) + expect(await isModelSafeWorkspaceFileKey(key, { workspaceId: 'workspace-1' })).toBe(safe) + queueTableRows(workspaceFiles, [row]) + expect( + await filterModelSafeWorkspaceFileAttachments([{ id: 'invented-id', key }], { + workspaceId: 'workspace-1', + }) + ).toEqual(safe ? [{ id: 'invented-id', key }] : []) + queueTableRows(workspaceFiles, [row]) + const bound = await getBoundWorkspaceFileSecretProvenance('workspace-1', { + fileId: 'canonical-id', + key, + context: 'execution', + contentUpdatedAt: CONTENT_UPDATED_AT, + }) + expect(bound.status).toBe( + version === null || (status === 'exact' && !stale) ? 'exact' : 'unknown' + ) + } + } + ) +}) + describe('workspace file secret provenance', () => { beforeEach(() => { vi.clearAllMocks() @@ -256,7 +319,11 @@ describe('workspace file secret provenance', () => { left: 'workspaceFiles.contentUpdatedAt', right: new Date(CONTENT_UPDATED_AT.getTime() + 1), }, - { type: 'inArray', column: 'workspaceFiles.context', values: ['workspace', 'mothership'] }, + { + type: 'inArray', + column: 'workspaceFiles.context', + values: ['workspace', 'mothership', 'execution'], + }, { type: 'or', conditions: [ @@ -436,7 +503,6 @@ describe('workspace file secret provenance', () => { */ { id: 'unrecorded-id', key: 'unrecorded-key' }, { id: 'pre-marker-sidecar-id', key: 'pre-marker-sidecar-key' }, - { id: 'synthetic-execution-id', key: 'untracked-context-key' }, { id: 'legacy-id', key: 'legacy-key' }, { id: 'inline-file' }, ]) @@ -996,14 +1062,20 @@ describe('workspace file secret provenance', () => { it('merges exact byte contributors and propagates unknown classifications', () => { expect( mergeWorkspaceFileSecretProvenance( - { status: 'exact', entries: [{ name: 'A', encryptedValue: 'encrypted-a' }] }, - { status: 'exact', entries: [{ name: 'B', encryptedValue: 'encrypted-b' }] } + { + status: 'exact', + entries: [{ name: 'A', encryptedValue: 'encrypted-a', sourceUserId: 'user-1' }], + }, + { + status: 'exact', + entries: [{ name: 'B', encryptedValue: 'encrypted-b', sourceUserId: 'user-1' }], + } ) ).toEqual({ status: 'exact', entries: [ - { name: 'A', encryptedValue: 'encrypted-a' }, - { name: 'B', encryptedValue: 'encrypted-b' }, + { name: 'A', encryptedValue: 'encrypted-a', sourceUserId: 'user-1' }, + { name: 'B', encryptedValue: 'encrypted-b', sourceUserId: 'user-1' }, ], }) expect( @@ -1011,6 +1083,86 @@ describe('workspace file secret provenance', () => { ).toEqual({ status: 'unknown' }) }) + it('deduplicates only identical scoped entries across repeated contributors', () => { + const base = { + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + name: 'TOKEN', + encryptedValue: 'ciphertext', + } + const entries = [ + base, + { ...base, sourceUserId: 'user-2' }, + { ...base, sourceWorkspaceId: 'workspace-2' }, + { ...base, name: 'OTHER_TOKEN' }, + { sourceUserId: base.sourceUserId, encryptedValue: base.encryptedValue }, + { ...base, encryptedValue: 'different-ciphertext' }, + ] + const contributors = Array.from({ length: 1_000 }, () => ({ + status: 'exact' as const, + entries, + })) + + expect(mergeWorkspaceFileSecretProvenance(...contributors)).toEqual({ + status: 'exact', + entries, + }) + }) + + it('counts distinct merged entries at the actual entry boundary and refuses overflow', () => { + const entries = Array.from({ length: PROVENANCE_MAX_ENTRIES }, (_, index) => ({ + sourceUserId: 'user-1', + encryptedValue: `ciphertext-${index}`, + })) + const full = { status: 'exact' as const, entries } + expect(mergeWorkspaceFileSecretProvenance(full, full)).toEqual(full) + expect( + mergeWorkspaceFileSecretProvenance(full, { + status: 'exact', + entries: [{ sourceUserId: 'user-1', encryptedValue: 'one-more-secret' }], + }) + ).toEqual({ status: 'unknown' }) + }) + + it('deduplicates before charging the actual byte boundary and refuses a larger union', () => { + const sourceUserId = 'user-1' + const name = 'TOKEN' + const overhead = Buffer.byteLength(sourceUserId + name, 'utf8') + const entry = { + sourceUserId, + name, + encryptedValue: 'x'.repeat(PROVENANCE_MAX_SERIALIZED_BYTES - overhead), + } + const full = { status: 'exact' as const, entries: [entry] } + expect(mergeWorkspaceFileSecretProvenance(full, full)).toEqual(full) + expect( + mergeWorkspaceFileSecretProvenance(full, { + status: 'exact', + entries: [{ sourceUserId, encryptedValue: 'one-more-secret' }], + }) + ).toEqual({ status: 'unknown' }) + expect( + mergeWorkspaceFileSecretProvenance({ + status: 'exact', + entries: [{ ...entry, encryptedValue: `${entry.encryptedValue}é` }], + }) + ).toEqual({ status: 'unknown' }) + }) + + it('stops reading entries once the merged envelope cannot be represented', () => { + const entries = [ + { sourceUserId: 'user-1', encryptedValue: 'x'.repeat(PROVENANCE_MAX_SERIALIZED_BYTES) }, + ] + Object.defineProperty(entries, 1, { + get: () => { + throw new Error('overflow must stop the merge') + }, + }) + expect(mergeWorkspaceFileSecretProvenance({ status: 'exact', entries })).toEqual({ + status: 'unknown', + }) + }) + it('does not discard known secret entries when another contributor is unrecorded', () => { const known = { status: 'exact' as const, diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts index e89f258199e..88d1b89f889 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts @@ -5,7 +5,7 @@ import { workspaceFileSecretProvenance, workspaceFiles, } from '@sim/db/schema' -import { and, eq, gte, inArray, isNull, lt, or } from 'drizzle-orm' +import { and, desc, eq, gte, inArray, isNull, lt, or, sql } from 'drizzle-orm' import { encryptSecret } from '@/lib/core/security/encryption' import type { DbTransaction } from '@/lib/db/types' import { @@ -81,7 +81,7 @@ interface WorkspaceFileAttachmentIdentity { export interface WorkspaceFileSecretProvenanceIdentity { fileId: string key: string - context: 'workspace' | 'mothership' + context: 'workspace' | 'mothership' | 'execution' contentUpdatedAt?: Date } @@ -138,12 +138,35 @@ export function mergeWorkspaceFileSecretProvenance( : { status: 'unrecorded' } } - return { - status: 'exact', - entries: provenances.flatMap((provenance) => - provenance.status === 'exact' ? provenance.entries : [] - ), + const entries = new Map() + let bytes = 0 + for (const provenance of provenances) { + if (provenance.status !== 'exact') continue + for (const entry of provenance.entries) { + if ( + !entry.encryptedValue || + !entry.sourceUserId || + (entry.name !== undefined && entry.name.length === 0) + ) { + return { status: 'unknown' } + } + const entryBytes = exactEntryByteSize(entry) + if (entryBytes > PROVENANCE_MAX_SERIALIZED_BYTES) return { status: 'unknown' } + const key = JSON.stringify([ + entry.sourceUserId, + entry.sourceWorkspaceId ?? '', + entry.name ?? '', + entry.encryptedValue, + ]) + if (entries.has(key)) continue + bytes += entryBytes + if (entries.size >= PROVENANCE_MAX_ENTRIES || bytes > PROVENANCE_MAX_SERIALIZED_BYTES) { + return { status: 'unknown' } + } + entries.set(key, entry) + } } + return { status: 'exact', entries: [...entries.values()] } } function compareStrings(left: string, right: string): number { @@ -422,7 +445,7 @@ async function markWorkspaceFileSecretProvenanceTrackedInTx( eq(workspaceFiles.id, fileId), gte(workspaceFiles.contentUpdatedAt, contentUpdatedAt), lt(workspaceFiles.contentUpdatedAt, nextContentMillisecond), - inArray(workspaceFiles.context, ['workspace', 'mothership']), + inArray(workspaceFiles.context, ['workspace', 'mothership', 'execution']), or( isNull(workspaceFiles.secretProvenanceVersion), eq(workspaceFiles.secretProvenanceVersion, 1) @@ -859,7 +882,7 @@ export async function getBoundWorkspaceFileSecretProvenanceByMetadata( * absence this covers. Closing the surface again is a matter of naming it in * `DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES`. */ -function mayReadUnrecordedWorkspaceFile( +export function mayReadUnrecordedWorkspaceFile( workspaceId: string | undefined, count = 1, actorUserId?: string @@ -1017,7 +1040,8 @@ export async function importWorkspaceFileSecretProvenanceForRuntime(args: { /** * Removes model attachments whose canonical workspace-file record is tainted or unknown. * Missing legacy records remain compatible; persisted records are classified by their unique - * active storage-key binding and private provenance row. Attachment ids are deliberately ignored: + * storage-key binding (active first, newest archived execution revision otherwise) and private + * provenance row. Attachment ids are deliberately ignored: * older persisted workflows omit them and file normalization may synthesize a runtime-only id. * This classification is not file authorization; callers still enforce storage access before * reading bytes or issuing a provider URL. @@ -1051,7 +1075,13 @@ export async function filterModelSafeWorkspaceFileAttachments< if (typeof attachment.key !== 'string' || attachment.key.length === 0) return true const row = rowByKey.get(attachment.key) if (!row) return true - if (row.context !== 'workspace' && row.context !== 'mothership') return true + if ( + row.context !== 'workspace' && + row.context !== 'mothership' && + row.context !== 'execution' + ) { + return true + } const classification = classifyModelSafeWorkspaceFileRow(row, options.workspaceId) if (classification === 'safe') return true if (classification === 'unsafe') { @@ -1098,7 +1128,7 @@ async function loadModelSafeWorkspaceFileRows( keys: readonly string[] ): Promise { return db - .select({ + .selectDistinctOn([workspaceFiles.key], { key: workspaceFiles.key, workspaceId: workspaceFiles.workspaceId, context: workspaceFiles.context, @@ -1113,7 +1143,18 @@ async function loadModelSafeWorkspaceFileRows( workspaceFileSecretProvenance, eq(workspaceFileSecretProvenance.fileId, workspaceFiles.id) ) - .where(and(inArray(workspaceFiles.key, [...keys]), isNull(workspaceFiles.deletedAt))) + .where( + and( + inArray(workspaceFiles.key, [...keys]), + or(isNull(workspaceFiles.deletedAt), eq(workspaceFiles.context, 'execution')) + ) + ) + .orderBy( + workspaceFiles.key, + sql`${workspaceFiles.deletedAt} IS NULL DESC`, + desc(workspaceFiles.contentUpdatedAt), + workspaceFiles.id + ) } /** @@ -1131,8 +1172,8 @@ export async function isModelSafeWorkspaceFileKey( /** * Batch variant for server-authorized storage keys crossing the same model boundary. Missing keys - * and non-workspace contexts retain their legacy/raw behavior; canonical workspace and mothership - * rows are accepted only when every current content version has exact-empty provenance. + * retain their legacy behavior; tracked workspace, mothership, and execution files must satisfy + * the same classification before their bytes leave private storage. */ export async function areModelSafeWorkspaceFileKeys( keys: readonly string[], @@ -1148,7 +1189,13 @@ export async function areModelSafeWorkspaceFileKeys( let unrecorded = 0 for (const row of rows) { - if (row.context !== 'workspace' && row.context !== 'mothership') continue + if ( + row.context !== 'workspace' && + row.context !== 'mothership' && + row.context !== 'execution' + ) { + continue + } const classification = classifyModelSafeWorkspaceFileRow(row, options.workspaceId) if (classification === 'unsafe') { return refuseWorkspaceFileProvenance( diff --git a/apps/sim/lib/uploads/core/storage-service.local.test.ts b/apps/sim/lib/uploads/core/storage-service.local.test.ts index 230d16086b1..49c9e59a78d 100644 --- a/apps/sim/lib/uploads/core/storage-service.local.test.ts +++ b/apps/sim/lib/uploads/core/storage-service.local.test.ts @@ -6,10 +6,13 @@ import { join } from 'node:path' import { resetDbChainMock } from '@sim/testing' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' -const { testDirectory, mockInsertMetadata } = vi.hoisted(() => ({ - testDirectory: `/tmp/sim-knowledge-upload-compensation-${process.pid}`, - mockInsertMetadata: vi.fn(), -})) +const { testDirectory, mockInsertMetadata, mockInsertFileMetadata, mockDeleteFileMetadata } = + vi.hoisted(() => ({ + testDirectory: `/tmp/sim-knowledge-upload-compensation-${process.pid}`, + mockInsertMetadata: vi.fn(), + mockInsertFileMetadata: vi.fn(), + mockDeleteFileMetadata: vi.fn(), + })) vi.mock('@/lib/uploads/core/setup.server', () => ({ UPLOAD_DIR_SERVER: testDirectory })) vi.mock('@/lib/uploads/config', () => ({ @@ -19,7 +22,8 @@ vi.mock('@/lib/uploads/config', () => ({ getStorageConfig: () => ({}), })) vi.mock('@/lib/uploads/server/metadata', () => ({ - insertFileMetadata: vi.fn(), + insertFileMetadata: mockInsertFileMetadata, + deleteFileMetadata: mockDeleteFileMetadata, insertImmutableFileMetadata: mockInsertMetadata, })) @@ -63,6 +67,8 @@ describe('local cache upload compensation', () => { vi.clearAllMocks() resetDbChainMock() mockInsertMetadata.mockReset().mockResolvedValue({ id: 'file-1' }) + mockInsertFileMetadata.mockReset().mockResolvedValue({ id: 'file-1' }) + mockDeleteFileMetadata.mockReset().mockResolvedValue(undefined) await rm(testDirectory, { recursive: true, force: true }) await mkdir(testDirectory, { recursive: true }) }) @@ -118,6 +124,28 @@ describe('local cache upload compensation', () => { ).rejects.toMatchObject({ code: 'ENOENT' }) }) + it.each(['execution', 'copilot'] as const)( + 'removes owned new local %s uploads if metadata persistence fails', + async (context) => { + const key = `${context}/unique-id/file.txt` + mockInsertFileMetadata.mockRejectedValueOnce(ORIGINAL_ERROR) + await expect( + uploadFile({ + file: Buffer.from('hello'), + fileName: 'file.txt', + customKey: key, + preserveKey: true, + cleanupOnMetadataFailure: true, + context, + contentType: 'text/plain', + metadata: { userId: 'user-1' }, + }) + ).rejects.toBe(ORIGINAL_ERROR) + await expect(stat(join(testDirectory, key))).rejects.toMatchObject({ code: 'ENOENT' }) + expect(mockDeleteFileMetadata).toHaveBeenCalledExactlyOnceWith(key) + } + ) + it('does not replace or delete a preexisting object', async () => { await writeOtherAttempt() diff --git a/apps/sim/lib/uploads/core/storage-service.test.ts b/apps/sim/lib/uploads/core/storage-service.test.ts index fe307baf115..43af9e525ac 100644 --- a/apps/sim/lib/uploads/core/storage-service.test.ts +++ b/apps/sim/lib/uploads/core/storage-service.test.ts @@ -11,6 +11,7 @@ const { mockUploadToS3, mockDeleteFromS3, mockInsertFileMetadata, + mockDeleteFileMetadata, mockInsertImmutableFileMetadata, mockCleanupUnboundKnowledgeUpload, mockGetSignedUrl, @@ -26,6 +27,7 @@ const { mockUploadToS3: vi.fn(), mockDeleteFromS3: vi.fn(), mockInsertFileMetadata: vi.fn(), + mockDeleteFileMetadata: vi.fn(), mockInsertImmutableFileMetadata: vi.fn(), mockCleanupUnboundKnowledgeUpload: vi.fn(), mockGetSignedUrl: vi.fn(), @@ -63,6 +65,7 @@ vi.mock('@/lib/uploads/providers/s3/client', () => ({ vi.mock('@/lib/uploads/server/metadata', () => ({ insertFileMetadata: mockInsertFileMetadata, + deleteFileMetadata: mockDeleteFileMetadata, insertImmutableFileMetadata: mockInsertImmutableFileMetadata, })) @@ -87,6 +90,8 @@ describe('createMultipartUpload', () => { mockAbort.mockResolvedValue(undefined) mockUploadToS3.mockResolvedValue({ key: 'k', path: 'p', name: 'k', size: 0, type: 'text/csv' }) mockInsertFileMetadata.mockResolvedValue({ id: 'file-1' }) + mockDeleteFileMetadata.mockResolvedValue(undefined) + mockDeleteFromS3.mockResolvedValue(undefined) mockInsertImmutableFileMetadata.mockResolvedValue({ id: 'file-1' }) mockCleanupUnboundKnowledgeUpload.mockResolvedValue(undefined) mockGetSignedUrl.mockResolvedValue('https://s3.example/create-only') @@ -252,6 +257,101 @@ describe('createMultipartUpload', () => { expect(mockCleanupUnboundKnowledgeUpload).not.toHaveBeenCalled() }) + it.each(['execution', 'copilot'] as const)( + 'removes an owned new %s object if metadata persistence fails, even after cancellation', + async (context) => { + const key = `${context}/new-file-id/file.txt` + const failure = new Error('metadata unavailable') + const controller = new AbortController() + mockUploadToS3.mockResolvedValueOnce({ key }) + mockInsertFileMetadata.mockImplementationOnce(async () => { + controller.abort(new Error('cancelled')) + throw failure + }) + + await expect( + uploadFile({ + file: Buffer.from('hello'), + fileName: 'file.txt', + customKey: key, + preserveKey: true, + cleanupOnMetadataFailure: true, + contentType: 'text/plain', + context, + metadata: { userId: 'user-1', workspaceId: 'workspace-1' }, + signal: controller.signal, + }) + ).rejects.toBe(failure) + + expect(mockDeleteFromS3).toHaveBeenCalledExactlyOnceWith( + key, + { bucket: 'b', region: 'r' }, + undefined + ) + expect(mockDeleteFileMetadata).toHaveBeenCalledExactlyOnceWith(key) + } + ) + + it('preserves the metadata error when cleanup of an owned object also fails', async () => { + const failure = new Error('metadata unavailable') + mockInsertFileMetadata.mockRejectedValueOnce(failure) + mockDeleteFromS3.mockRejectedValueOnce(new Error('storage unavailable')) + + await expect( + uploadFile({ + file: Buffer.from('hello'), + fileName: 'file.txt', + customKey: 'execution/new-file-id/file.txt', + preserveKey: true, + cleanupOnMetadataFailure: true, + contentType: 'text/plain', + context: 'execution', + metadata: { workspaceId: 'workspace-1' }, + }) + ).rejects.toBe(failure) + expect(mockDeleteFromS3).toHaveBeenCalledOnce() + expect(mockDeleteFileMetadata).not.toHaveBeenCalled() + }) + + it.each(['workspace', 'execution', 'copilot'] as const)( + 'leaves existing %s replacement keys untouched without explicit new-key ownership', + async (context) => { + const failure = new Error('metadata unavailable') + mockInsertFileMetadata.mockRejectedValueOnce(failure) + await expect( + uploadFile({ + file: Buffer.from('hello'), + fileName: 'existing.txt', + customKey: `${context}/existing.txt`, + preserveKey: true, + contentType: 'text/plain', + context, + metadata: { userId: 'user-1', workspaceId: 'workspace-1' }, + }) + ).rejects.toBe(failure) + expect(mockDeleteFromS3).not.toHaveBeenCalled() + expect(mockDeleteFileMetadata).not.toHaveBeenCalled() + } + ) + + it.each([ + { context: 'workspace', customKey: 'workspace/file.txt', preserveKey: true }, + { context: 'execution', customKey: undefined, preserveKey: true }, + { context: 'copilot', customKey: 'copilot/file.txt', preserveKey: false }, + ] as const)('rejects cleanup without an explicitly owned ephemeral key: %j', async (scope) => { + await expect( + uploadFile({ + ...scope, + file: Buffer.from('hello'), + fileName: 'file.txt', + contentType: 'text/plain', + cleanupOnMetadataFailure: true, + metadata: { userId: 'user-1' }, + }) + ).rejects.toThrow('newly allocated execution or Copilot key') + expect(mockUploadToS3).not.toHaveBeenCalled() + }) + it('takes the single-shot PutObject path for a payload smaller than one part', async () => { const handle = await createMultipartUpload({ key: 'k', diff --git a/apps/sim/lib/uploads/core/storage-service.ts b/apps/sim/lib/uploads/core/storage-service.ts index dfd2edc83c1..ee67df9704a 100644 --- a/apps/sim/lib/uploads/core/storage-service.ts +++ b/apps/sim/lib/uploads/core/storage-service.ts @@ -98,7 +98,8 @@ async function insertFileMetadataHelper( fileName: string, contentType: string, fileSize: number, - uploadId?: string + uploadId?: string, + cleanupOnMetadataFailure = false ): Promise { const { insertFileMetadata, insertImmutableFileMetadata } = await import( '@/lib/uploads/server/metadata' @@ -131,6 +132,18 @@ async function insertFileMetadataHelper( error: cleanupError, }) } + } else if (cleanupOnMetadataFailure) { + try { + await deleteFile({ key, context }) + const { deleteFileMetadata } = await import('@/lib/uploads/server/metadata') + await deleteFileMetadata(key) + } catch (cleanupError) { + logger.error('Failed to clean up an unpublished tool output upload', { + key, + context, + error: getErrorMessage(cleanupError), + }) + } } throw error } @@ -149,6 +162,7 @@ export async function uploadFile(options: UploadFileOptions): Promise customKey, metadata, persistMetadata = true, + cleanupOnMetadataFailure = false, createOnlyUploadId, signal, } = options @@ -156,6 +170,12 @@ export async function uploadFile(options: UploadFileOptions): Promise if (createOnlyUploadId && (context !== 'knowledge-base' || !metadata)) { throw new Error('Reserved create-only uploads require knowledge-base ownership metadata') } + if ( + cleanupOnMetadataFailure && + ((context !== 'execution' && context !== 'copilot') || !preserveKey || !customKey) + ) { + throw new Error('Upload cleanup requires a newly allocated execution or Copilot key') + } logger.info(`Uploading file to ${context} storage: ${fileName}`) @@ -190,7 +210,8 @@ export async function uploadFile(options: UploadFileOptions): Promise fileName, contentType, file.length, - uploadId + uploadId, + cleanupOnMetadataFailure ) } @@ -219,7 +240,8 @@ export async function uploadFile(options: UploadFileOptions): Promise fileName, contentType, file.length, - uploadId + uploadId, + cleanupOnMetadataFailure ) } @@ -248,7 +270,8 @@ export async function uploadFile(options: UploadFileOptions): Promise fileName, contentType, file.length, - uploadId + uploadId, + cleanupOnMetadataFailure ) } @@ -295,7 +318,8 @@ export async function uploadFile(options: UploadFileOptions): Promise fileName, contentType, file.length, - uploadId + uploadId, + cleanupOnMetadataFailure ) } diff --git a/apps/sim/lib/uploads/server/metadata.ts b/apps/sim/lib/uploads/server/metadata.ts index 0df26c0cf2c..97ab3565e38 100644 --- a/apps/sim/lib/uploads/server/metadata.ts +++ b/apps/sim/lib/uploads/server/metadata.ts @@ -1,8 +1,9 @@ import { db } from '@sim/db' +import { withInsertColumns } from '@sim/db/insert-columns' import { type WorkspaceFileRow, workspaceFileColumns, workspaceFiles } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' -import { and, eq, inArray, isNotNull, isNull, sql } from 'drizzle-orm' +import { and, desc, eq, inArray, isNotNull, isNull, sql } from 'drizzle-orm' import type { DbOrTx, DbTransaction } from '@/lib/db/types' import { getWorkspaceFileSize, @@ -179,7 +180,7 @@ async function insertFileMetadataWithExecutor( try { const [inserted] = await executor - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ id: fileId, key, @@ -235,7 +236,7 @@ async function insertImmutableFileMetadataWithExecutor( } = options assertFileMetadataOrganizationOwner(options) const [inserted] = await executor - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values({ id: id || generateId(), key, @@ -316,7 +317,7 @@ export async function insertFileMetadataMany( const uniqueRows = [...uniqueRowsByKey.values()] const inserted = await db - .insert(workspaceFiles) + .insert(withInsertColumns(workspaceFiles, workspaceFileColumns)) .values( uniqueRows.map((row) => ({ id: row.id || generateId(), @@ -423,18 +424,31 @@ export async function resolveStoredFileContext(key: string): Promise = db, - options?: { lock?: 'share' } + executor: Pick = db, + options?: { lock?: 'share'; includeDeleted?: false } | { lock?: never; includeDeleted: true } ): Promise { if (keys.length === 0) { return [] } + if (options?.includeDeleted) { + return executor + .selectDistinctOn([workspaceFiles.key], workspaceFileColumns) + .from(workspaceFiles) + .where(and(inArray(workspaceFiles.key, keys), eq(workspaceFiles.context, context))) + .orderBy( + workspaceFiles.key, + sql`${workspaceFiles.deletedAt} IS NULL DESC`, + desc(workspaceFiles.contentUpdatedAt), + workspaceFiles.id + ) + } const query = executor .select(workspaceFileColumns) .from(workspaceFiles) diff --git a/apps/sim/lib/uploads/shared/assistant-images.ts b/apps/sim/lib/uploads/shared/assistant-images.ts new file mode 100644 index 00000000000..deae05e9a5d --- /dev/null +++ b/apps/sim/lib/uploads/shared/assistant-images.ts @@ -0,0 +1,15 @@ +/** Inline image limits shared by Assistant upload, preview, and model preparation. */ +export const ASSISTANT_IMAGE_MAX_BYTES = 5 * 1024 * 1024 +export const ASSISTANT_IMAGE_MAX_COUNT = 5 +export const ASSISTANT_IMAGE_MAX_TOTAL_BYTES = ASSISTANT_IMAGE_MAX_BYTES * ASSISTANT_IMAGE_MAX_COUNT +export const ASSISTANT_IMAGE_CONTENT_TYPES = [ + 'image/jpeg', + 'image/png', + 'image/gif', + 'image/webp', +] as const +export const ASSISTANT_IMAGE_ACCEPT_ATTRIBUTE = ASSISTANT_IMAGE_CONTENT_TYPES.join(',') + +export function isAssistantImageType(contentType: string): boolean { + return ASSISTANT_IMAGE_CONTENT_TYPES.some((type) => type === contentType) +} diff --git a/apps/sim/lib/uploads/shared/types.ts b/apps/sim/lib/uploads/shared/types.ts index f8d58b86e0f..ad9eb00585b 100644 --- a/apps/sim/lib/uploads/shared/types.ts +++ b/apps/sim/lib/uploads/shared/types.ts @@ -74,6 +74,7 @@ export type StorageContext = | 'og-images' | 'logs' | 'workspace-logos' + | 'organization-logos' /** * The contexts stored under the `workspace/` key prefix. They share a bucket and @@ -127,6 +128,8 @@ export interface UploadFileOptions { * Disable when a caller finalizes metadata in its own database transaction. */ persistMetadata?: boolean + /** Only for newly allocated, unique execution or Copilot keys; never enable for replacements. */ + cleanupOnMetadataFailure?: boolean /** Internal create-only upload identity when metadata and cleanup were reserved before writing bytes. */ createOnlyUploadId?: string signal?: AbortSignal diff --git a/apps/sim/lib/uploads/upload-session/application.test.ts b/apps/sim/lib/uploads/upload-session/application.test.ts index 70ae6b01c61..0208b2c6c9c 100644 --- a/apps/sim/lib/uploads/upload-session/application.test.ts +++ b/apps/sim/lib/uploads/upload-session/application.test.ts @@ -12,6 +12,18 @@ const mocks = vi.hoisted(() => ({ getPrincipalSession: vi.fn(), reauthorizeWorkspacePurpose: vi.fn(), getWorkspaceFile: vi.fn(), + authorizeOrganizationAttachment: vi.fn(), + authorizeOrganizationLogo: vi.fn(), +})) + +vi.mock('@/lib/uploads/contexts/organization-assistant/application', () => ({ + authorizeOrganizationAttachmentControl: mocks.authorizeOrganizationAttachment, + createOrganizationAssistantAttachment: vi.fn(), +})) + +vi.mock('@/lib/uploads/contexts/organization-logo/application', () => ({ + authorizeOrganizationLogoControl: mocks.authorizeOrganizationLogo, + createOrganizationLogoUpload: vi.fn(), })) vi.mock('@/lib/uploads/contexts/workspace', () => ({ @@ -43,7 +55,9 @@ vi.mock('@/app/api/files/uploads/purposes', () => ({ })) import { + abortInternalUploadSession, completeInternalUploadSession, + issueInternalUploadPartUrls, readWorkspaceUploadSession, } from '@/lib/uploads/upload-session/application' import type { UploadSessionRecord } from '@/lib/uploads/upload-session/service' @@ -58,6 +72,8 @@ const actor = { id: 'user-1', name: 'Ada', email: 'ada@example.com' } describe('upload session application', () => { beforeEach(() => { vi.clearAllMocks() + mocks.authorizeOrganizationAttachment.mockResolvedValue(undefined) + mocks.authorizeOrganizationLogo.mockResolvedValue(undefined) const session = workspaceUploadSession() mocks.getOwnedSession.mockResolvedValue(session) mocks.finalizePurpose.mockResolvedValue({ @@ -75,6 +91,52 @@ describe('upload session application', () => { }) }) + it.each(['complete', 'abort', 'parts'] as const)( + 'rechecks organization administrator membership before the logo %s control leg', + async (control) => { + const session = { + ...workspaceUploadSession(), + purpose: 'organization_logo' as const, + workspaceId: null, + } + mocks.getOwnedSession.mockResolvedValue(session) + mocks.authorizeOrganizationLogo.mockRejectedValue( + new Error('Organization administrator access is required') + ) + const request = new NextRequest('http://localhost/api/files/uploads/upload-1/complete') + const input = { uploadId: 'upload-1', uploadToken: 'upload-token', partNumbers: [1] } + const result = + control === 'complete' + ? completeInternalUploadSession(principal, input, request) + : control === 'abort' + ? abortInternalUploadSession(principal, input) + : issueInternalUploadPartUrls(principal, input, request) + await expect(result).rejects.toThrow('Organization administrator access is required') + expect(mocks.assertAuthBinding).toHaveBeenCalledWith(session, principal) + expect(mocks.authorizeOrganizationLogo).toHaveBeenCalledWith(principal, session) + expect(mocks.finalizePurpose).not.toHaveBeenCalled() + } + ) + + it('does not finalize a logo when organization access is revoked after claim', async () => { + mocks.getOwnedSession.mockResolvedValue({ + ...workspaceUploadSession(), + purpose: 'organization_logo', + workspaceId: null, + }) + mocks.authorizeOrganizationLogo + .mockResolvedValueOnce(undefined) + .mockRejectedValueOnce(new Error('Organization not found')) + await expect( + completeInternalUploadSession( + principal, + { uploadId: 'upload-1', uploadToken: 'upload-token' }, + new NextRequest('http://localhost/api/files/uploads/upload-1/complete') + ) + ).rejects.toThrow('Organization not found') + expect(mocks.finalizePurpose).not.toHaveBeenCalled() + }) + it('preserves the authenticated actor metadata through internal finalization', async () => { const request = new NextRequest('http://localhost/api/files/uploads/upload-1/complete', { method: 'POST', @@ -91,6 +153,48 @@ describe('upload session application', () => { ) }) + it.each(['complete', 'abort', 'parts'] as const)( + 'rechecks organization membership before the %s control leg', + async (control) => { + const session = { + ...workspaceUploadSession(), + purpose: 'mothership_attachment' as const, + workspaceId: null, + } + mocks.getOwnedSession.mockResolvedValue(session) + const request = new NextRequest('http://localhost/api/files/uploads/upload-1/complete', { + headers: { host: 'localhost' }, + }) + const input = { uploadId: 'upload-1', uploadToken: 'upload-token', partNumbers: [1] } + if (control === 'complete') await completeInternalUploadSession(principal, input, request) + else if (control === 'abort') await abortInternalUploadSession(principal, input) + else await issueInternalUploadPartUrls(principal, input, request) + expect(mocks.assertAuthBinding).toHaveBeenCalledWith(session, principal) + expect(mocks.authorizeOrganizationAttachment).toHaveBeenCalledWith(principal, session) + expect(mocks.reauthorizeWorkspacePurpose).not.toHaveBeenCalled() + } + ) + + it('does not finalize when organization access is revoked after the session is claimed', async () => { + const session = { + ...workspaceUploadSession(), + purpose: 'mothership_attachment' as const, + workspaceId: null, + } + mocks.getOwnedSession.mockResolvedValue(session) + mocks.authorizeOrganizationAttachment + .mockResolvedValueOnce(undefined) + .mockRejectedValueOnce(new Error('Organization not found')) + await expect( + completeInternalUploadSession( + principal, + { uploadId: 'upload-1', uploadToken: 'upload-token' }, + new NextRequest('http://localhost/api/files/uploads/upload-1/complete') + ) + ).rejects.toThrow('Organization not found') + expect(mocks.finalizePurpose).not.toHaveBeenCalled() + }) + /** * The read is a control leg, so it re-authorizes the caller's present * workspace permission rather than trusting the session lookup alone. @@ -119,7 +223,11 @@ describe('upload session application', () => { * had created. */ it('returns the registered file once the session has completed', async () => { - const completed = { ...workspaceUploadSession(), status: 'completed' as const, completedFileId: 'file-1' } + const completed = { + ...workspaceUploadSession(), + status: 'completed' as const, + completedFileId: 'file-1', + } mocks.getOwnedSession.mockResolvedValue(completed) mocks.getPrincipalSession.mockResolvedValue(completed) mocks.getWorkspaceFile.mockResolvedValue({ id: 'file-1', name: 'file.txt' }) @@ -154,7 +262,11 @@ describe('upload session application', () => { * there was nothing. A failed read is not the same answer as no file. */ it('surfaces a failed file read instead of reporting the upload fileless', async () => { - const completed = { ...workspaceUploadSession(), status: 'completed' as const, completedFileId: 'file-1' } + const completed = { + ...workspaceUploadSession(), + status: 'completed' as const, + completedFileId: 'file-1', + } mocks.getOwnedSession.mockResolvedValue(completed) mocks.getPrincipalSession.mockResolvedValue(completed) mocks.getWorkspaceFile.mockRejectedValue(new Error('connection terminated')) @@ -169,7 +281,11 @@ describe('upload session application', () => { }) it('reads the completed file with throwOnError so a fault cannot read as absence', async () => { - const completed = { ...workspaceUploadSession(), status: 'completed' as const, completedFileId: 'file-1' } + const completed = { + ...workspaceUploadSession(), + status: 'completed' as const, + completedFileId: 'file-1', + } mocks.getOwnedSession.mockResolvedValue(completed) mocks.getPrincipalSession.mockResolvedValue(completed) mocks.getWorkspaceFile.mockResolvedValue({ id: 'file-1', name: 'file.txt' }) @@ -187,7 +303,11 @@ describe('upload session application', () => { /** A completed session whose file was since deleted has nothing to address. */ it('answers null when the completed file is gone', async () => { - const gone = { ...workspaceUploadSession(), status: 'completed' as const, completedFileId: 'file-1' } + const gone = { + ...workspaceUploadSession(), + status: 'completed' as const, + completedFileId: 'file-1', + } mocks.getOwnedSession.mockResolvedValue(gone) mocks.getPrincipalSession.mockResolvedValue(gone) mocks.getWorkspaceFile.mockResolvedValue(null) diff --git a/apps/sim/lib/uploads/upload-session/application.ts b/apps/sim/lib/uploads/upload-session/application.ts index 61929b1dd77..ab53a62f72d 100644 --- a/apps/sim/lib/uploads/upload-session/application.ts +++ b/apps/sim/lib/uploads/upload-session/application.ts @@ -3,6 +3,14 @@ import type { CreateInternalFileUploadBody } from '@/lib/api/contracts/upload-se import type { OrchestrationRequestContext } from '@/lib/core/orchestration/types' import { OrchestrationError } from '@/lib/core/orchestration/types' import { loadActiveFolderPathIndex, resolveFolderPathFromIndex } from '@/lib/folders/queries' +import { + authorizeOrganizationAttachmentControl, + createOrganizationAssistantAttachment, +} from '@/lib/uploads/contexts/organization-assistant/application' +import { + authorizeOrganizationLogoControl, + createOrganizationLogoUpload, +} from '@/lib/uploads/contexts/organization-logo/application' import { getWorkspaceFile, type WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace' import { abortUploadSession, @@ -82,6 +90,16 @@ export async function createInternalPurposeUploadSession( body: CreateInternalFileUploadBody, request: OrchestrationRequestContext ): Promise>> { + if (body.purpose === 'organization_logo') { + return createOrganizationLogoUpload(principal, { ...body, localOrigin: requestOrigin(request) }) + } + if (body.purpose === 'mothership_attachment' && body.organizationId) { + return createOrganizationAssistantAttachment(principal, { + ...body, + organizationId: body.organizationId, + localOrigin: requestOrigin(request), + }) + } return createPurposeUploadSession(principal, body, requestOrigin(request)) } @@ -96,7 +114,11 @@ export async function loadAuthorizedInternalUploadSession( uploadToken: input.uploadToken, userId: principalUserId(principal), }) - if (session.purpose === 'workspace_file') assertUploadSessionAuthBinding(session, principal) + if (session.purpose === 'workspace_file' || session.purpose === 'organization_logo') + assertUploadSessionAuthBinding(session, principal) + if (session.purpose === 'mothership_attachment' && session.workspaceId === null) { + assertUploadSessionAuthBinding(session, principal) + } return session } @@ -108,6 +130,10 @@ export async function issueInternalUploadPartUrls( const session = await loadAuthorizedInternalUploadSession(principal, input) if (session.purpose === 'workspace_file') { await reauthorizeWorkspaceUploadPurpose(principal, session, fileOperations.uploadParts) + } else if (session.purpose === 'organization_logo') { + await authorizeOrganizationLogoControl(principal, session) + } else if (session.purpose === 'mothership_attachment' && session.workspaceId === null) { + await authorizeOrganizationAttachmentControl(principal, session) } else { await reauthorizeUploadPurpose(principalUserId(principal), session) } @@ -127,6 +153,10 @@ export async function abortInternalUploadSession( const session = await loadAuthorizedInternalUploadSession(principal, input) if (session.purpose === 'workspace_file') { await reauthorizeWorkspaceUploadPurpose(principal, session, fileOperations.uploadCancel) + } else if (session.purpose === 'organization_logo') { + await authorizeOrganizationLogoControl(principal, session) + } else if (session.purpose === 'mothership_attachment' && session.workspaceId === null) { + await authorizeOrganizationAttachmentControl(principal, session) } else { await reauthorizeUploadPurpose(principalUserId(principal), session) } @@ -146,6 +176,10 @@ export async function completeInternalUploadSession( const authorize = async (claimed: UploadSessionRecord) => { if (claimed.purpose === 'workspace_file') { await reauthorizeWorkspaceUploadPurpose(principal, claimed, fileOperations.uploadComplete) + } else if (claimed.purpose === 'organization_logo') { + await authorizeOrganizationLogoControl(principal, claimed) + } else if (claimed.purpose === 'mothership_attachment' && claimed.workspaceId === null) { + await authorizeOrganizationAttachmentControl(principal, claimed) } else { await reauthorizeUploadPurpose(principalUserId(principal), claimed) } diff --git a/apps/sim/lib/uploads/upload-session/service.test.ts b/apps/sim/lib/uploads/upload-session/service.test.ts index 558b1fca924..f779a62fca0 100644 --- a/apps/sim/lib/uploads/upload-session/service.test.ts +++ b/apps/sim/lib/uploads/upload-session/service.test.ts @@ -153,6 +153,127 @@ describe('upload sessions', () => { }) }) + it('stores organization logos under their own scope and replaces forged credential metadata', async () => { + const finalKey = 'organization-logos/org-1/upload-1-logo.png' + dbChainMockFns.returning.mockResolvedValueOnce([ + uploadRow({ + purpose: 'organization_logo', + workspaceId: null, + storageContext: 'organization-logos', + finalKey, + contentType: 'image/png', + }), + ]) + await createUploadSession({ + id: 'upload-1', + userId: 'user-1', + purpose: 'organization_logo', + organizationId: 'org-1', + expectedLogo: null, + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + fileName: 'logo.png', + contentType: 'image/png', + fileSize: 100, + metadata: { organizationLogo: { organizationId: 'forged' } }, + }) + expect(dbChainMockFns.values).toHaveBeenCalledWith( + expect.objectContaining({ + workspaceId: null, + finalKey, + storageContext: 'organization-logos', + metadata: { + organizationLogo: { + organizationId: 'org-1', + expectedLogo: null, + userId: 'user-1', + sessionId: 'session-1', + }, + }, + }) + ) + expect(mockCreatePutTransfer).toHaveBeenCalledWith( + expect.objectContaining({ context: 'organization-logos', fileSize: 100 }) + ) + }) + + it.each([ + { contentType: 'text/html', fileSize: 100 }, + { contentType: 'image/png', fileSize: 5 * 1024 * 1024 + 1 }, + { contentType: 'image/png', fileSize: 0 }, + ])('rejects invalid organization logos before initializing storage', async (file) => { + await expect( + createUploadSession({ + purpose: 'organization_logo', + organizationId: 'org-1', + expectedLogo: null, + userId: 'user-1', + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + fileName: 'logo.png', + ...file, + }) + ).rejects.toMatchObject({ code: 'validation' }) + expect(mockCreatePutTransfer).not.toHaveBeenCalled() + expect(dbChainMockFns.values).not.toHaveBeenCalled() + }) + + it('binds organization images to the creating session and stores them without a workspace', async () => { + dbChainMockFns.returning.mockResolvedValueOnce([ + uploadRow({ + purpose: 'mothership_attachment', + workspaceId: null, + storageContext: 'mothership', + finalKey: 'assistant/org-1/user-1/upload-1/image.png', + contentType: 'image/png', + }), + ]) + await createUploadSession({ + id: 'upload-1', + userId: 'user-1', + purpose: 'mothership_attachment', + organizationId: 'org-1', + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + fileName: 'image.png', + contentType: 'image/png', + fileSize: 100, + metadata: { organizationAttachment: { organizationId: 'forged' } }, + }) + expect(dbChainMockFns.values).toHaveBeenCalledWith( + expect.objectContaining({ + workspaceId: null, + finalKey: 'assistant/org-1/user-1/upload-1/image.png', + metadata: { + organizationAttachment: { + organizationId: 'org-1', + userId: 'user-1', + sessionId: 'session-1', + }, + }, + }) + ) + expect(mockCreatePutTransfer).toHaveBeenCalledWith( + expect.objectContaining({ context: 'mothership', fileSize: 100 }) + ) + }) + + it.each([ + { contentType: 'text/html', fileSize: 100 }, + { contentType: 'image/png', fileSize: 5 * 1024 * 1024 + 1 }, + ])('rejects invalid organization images before storage initialization', async (file) => { + await expect( + createUploadSession({ + id: 'upload-1', + userId: 'user-1', + purpose: 'mothership_attachment', + organizationId: 'org-1', + principal: { kind: 'session', userId: 'user-1', sessionId: 'session-1' }, + fileName: 'image.png', + ...file, + }) + ).rejects.toThrow('Assistant attachments must be') + expect(mockCreatePutTransfer).not.toHaveBeenCalled() + expect(dbChainMockFns.values).not.toHaveBeenCalled() + }) + // Local storage stores an object's metadata sidecar beside it, under the // object's own name, so the whole key + suffix must fit one path component. // Three purposes built their key by hand and admitted a 255-character name @@ -1031,6 +1152,96 @@ describe('upload sessions', () => { expect.objectContaining({ key: FINAL_KEY, version: 'version-1' }) ) }) + + it.each(['mothership_attachment', 'organization_logo'] as const)( + 'reclaims an unreferenced completed %s object', + async (purpose) => { + const image = uploadRow({ + purpose, + workspaceId: null, + storageContext: purpose === 'organization_logo' ? 'organization-logos' : 'mothership', + finalKey: + purpose === 'organization_logo' + ? 'organization-logos/org-1/upload-1-logo.png' + : 'assistant/org-1/user-1/upload-1/image.png', + status: 'completed', + completedAt: new Date(Date.now() - 8 * 24 * 60 * 60 * 1000), + }) + queueTableRows(schemaMock.uploadSession, []) + queueTableRows(schemaMock.uploadSession, [image]) + mockHeadObject.mockResolvedValue(providerObject(sessionRecord(image), 'version-1')) + dbChainMockFns.returning + .mockResolvedValueOnce([image]) + .mockResolvedValueOnce([{ id: image.id }]) + + await expect(cleanupExpiredUploadSessions()).resolves.toEqual({ + expired: 0, + failed: 0, + purged: 1, + }) + expect(mockDeleteObjectVersion).toHaveBeenCalledWith({ + provider: 's3', + key: image.finalKey, + context: image.storageContext, + version: 'version-1', + }) + expect(mockDeleteObjectVersion.mock.invocationCallOrder[0]).toBeLessThan( + dbChainMockFns.delete.mock.invocationCallOrder[0] + ) + } + ) + + it.each(['mothership_attachment', 'organization_logo'] as const)( + 'retains %s ownership records after deletion failure so cleanup can retry', + async (purpose) => { + const image = uploadRow({ + purpose, + workspaceId: null, + storageContext: purpose === 'organization_logo' ? 'organization-logos' : 'mothership', + status: 'completed', + completedAt: new Date(Date.now() - 8 * 24 * 60 * 60 * 1000), + }) + queueTableRows(schemaMock.uploadSession, []) + queueTableRows(schemaMock.uploadSession, [image]) + mockHeadObject.mockResolvedValue(providerObject(sessionRecord(image), 'version-1')) + mockDeleteObjectVersion.mockRejectedValueOnce(new Error('Storage unavailable')) + dbChainMockFns.returning.mockResolvedValueOnce([image]) + + await expect(cleanupExpiredUploadSessions()).resolves.toEqual({ + expired: 0, + failed: 1, + purged: 0, + }) + expect(dbChainMockFns.delete).not.toHaveBeenCalled() + expect(dbChainMockFns.set).toHaveBeenLastCalledWith( + expect.objectContaining({ + processingLeaseId: null, + processingLeaseExpiresAt: null, + error: 'Storage unavailable', + }) + ) + } + ) + + it('purges completed workspace attachment sessions without deleting their registered objects', async () => { + const attachment = uploadRow({ + purpose: 'mothership_attachment', + status: 'completed', + completedAt: new Date(Date.now() - 8 * 24 * 60 * 60 * 1000), + }) + queueTableRows(schemaMock.uploadSession, []) + queueTableRows(schemaMock.uploadSession, [attachment]) + dbChainMockFns.returning + .mockResolvedValueOnce([attachment]) + .mockResolvedValueOnce([{ id: attachment.id }]) + + await expect(cleanupExpiredUploadSessions()).resolves.toEqual({ + expired: 0, + failed: 0, + purged: 1, + }) + expect(mockDeleteObjectVersion).not.toHaveBeenCalled() + }) }) async function createWorkspaceUpload(fileSize: number) { diff --git a/apps/sim/lib/uploads/upload-session/service.ts b/apps/sim/lib/uploads/upload-session/service.ts index 2c9aed76a7c..b54e6313553 100644 --- a/apps/sim/lib/uploads/upload-session/service.ts +++ b/apps/sim/lib/uploads/upload-session/service.ts @@ -5,13 +5,13 @@ import { requirePrincipalSubjectUserId, } from '@sim/auth/principal' import { db, dbFor } from '@sim/db' -import { uploadSession } from '@sim/db/schema' +import { organization, uploadSession, user } from '@sim/db/schema' import { safeCompare } from '@sim/security/compare' import { sha256Hex } from '@sim/security/hash' import { generateSecureToken } from '@sim/security/tokens' import { getErrorMessage } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, asc, eq, inArray, isNull, lt, or } from 'drizzle-orm' +import { and, asc, eq, inArray, isNull, lt, or, sql } from 'drizzle-orm' import { checkStorageQuotaForBillingContext, resolveStorageBillingContext, @@ -19,8 +19,14 @@ import { import { OrchestrationError } from '@/lib/core/orchestration/types' import { generateUniqueExecutionFileKey } from '@/lib/uploads/contexts/execution/utils' import { generateKnowledgeBaseFileKey } from '@/lib/uploads/contexts/knowledge-base/knowledge-base-file-manager' +import { assertOrganizationAttachmentControlBinding } from '@/lib/uploads/contexts/organization-assistant/binding' +import { assertOrganizationLogoControlBinding } from '@/lib/uploads/contexts/organization-logo/binding' import { generateWorkspaceFileKey } from '@/lib/uploads/contexts/workspace' import { buildStorageKeySegment } from '@/lib/uploads/core/storage-key' +import { + ASSISTANT_IMAGE_MAX_BYTES, + isAssistantImageType, +} from '@/lib/uploads/shared/assistant-images' import { MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE, MAX_WORKSPACE_FILE_SIZE, @@ -47,6 +53,7 @@ import type { UploadStorageProvider, UploadTransferMethod, } from '@/lib/uploads/upload-session/types' +import { isImageFileType } from '@/lib/uploads/utils/file-utils' export const UPLOAD_SESSION_PUT_MAX_BYTES = 50 * 1024 * 1024 export const UPLOAD_SESSION_PART_SIZE = 8 * 1024 * 1024 @@ -189,7 +196,20 @@ export type CreateUploadSessionParams = CreateUploadSessionBaseParams & principal: Principal } | { purpose: 'profile_picture'; workspaceId?: null } + | { + purpose: 'organization_logo' + organizationId: string + expectedLogo: string | null + principal: Principal + workspaceId?: never + } | { purpose: 'workspace_logo' | 'mothership_attachment'; workspaceId: string } + | { + purpose: 'mothership_attachment' + organizationId: string + principal: Principal + workspaceId?: never + } | { purpose: 'execution_attachment' workspaceId: string @@ -206,8 +226,32 @@ export async function createUploadSession( validateFile(params) const id = params.id ?? generateId() const uploadToken = generateSecureToken(32) - const workspaceId = params.purpose === 'profile_picture' ? null : params.workspaceId + const workspaceId = params.purpose === 'profile_picture' ? null : (params.workspaceId ?? null) const metadata = { ...(params.metadata ?? {}) } + if (params.purpose === 'organization_logo') { + if (params.principal.kind !== 'session' || params.principal.userId !== params.userId) { + throw new UploadSessionError('forbidden', 'Organization logos require the uploading session') + } + metadata.organizationLogo = { + organizationId: params.organizationId, + expectedLogo: params.expectedLogo, + userId: params.userId, + sessionId: params.principal.sessionId, + } + } + if (params.purpose === 'mothership_attachment' && 'organizationId' in params) { + if (params.principal.kind !== 'session' || params.principal.userId !== params.userId) { + throw new UploadSessionError( + 'forbidden', + 'Organization attachments require the uploading session' + ) + } + metadata.organizationAttachment = { + organizationId: params.organizationId, + userId: params.userId, + sessionId: params.principal.sessionId, + } + } if (params.purpose === 'workspace_file' || params.purpose === 'knowledge_document') { if (!workspaceId) throw new Error(`${params.purpose} upload is missing workspaceId`) if (!params.principal) { @@ -500,6 +544,14 @@ export function assertUploadSessionAuthBinding( session: UploadSessionRecord, principal: Principal ): void { + if (session.purpose === 'organization_logo') { + assertOrganizationLogoControlBinding(session, principal) + return + } + if (session.purpose === 'mothership_attachment' && session.workspaceId === null) { + assertOrganizationAttachmentControlBinding(session, principal) + return + } if (!isPrincipalBoundUploadPurpose(session.purpose)) return const candidate = session.metadata.authBinding if (candidate === undefined) { @@ -913,6 +965,27 @@ export async function cleanupExpiredUploadSessions(): Promise<{ .where( and( inArray(uploadSession.status, ['completed', 'aborted', 'expired']), + /** Retain ownership records for active logos so replacements remain eligible for cleanup. */ + sql`NOT ( + ${uploadSession.status} = 'completed' + AND ${uploadSession.purpose} = 'organization_logo' + AND EXISTS ( + SELECT 1 FROM ${organization} + WHERE ${organization.id} = ${uploadSession.metadata}->'organizationLogo'->>'organizationId' + AND ${organization.logo} = ${uploadSession.metadata}->>'organizationLogoPath' + ) + )`, + /** Keep private images while both their uploader and organization exist. */ + sql`NOT ( + ${uploadSession.status} = 'completed' + AND ${uploadSession.purpose} = 'mothership_attachment' + AND ${uploadSession.workspaceId} IS NULL + AND EXISTS (SELECT 1 FROM ${user} WHERE ${user.id} = ${uploadSession.userId}) + AND EXISTS ( + SELECT 1 FROM ${organization} + WHERE ${organization.id} = ${uploadSession.metadata}->'organizationAttachment'->>'organizationId' + ) + )`, lt(uploadSession.completedAt, terminalCutoff), or( isNull(uploadSession.processingLeaseId), @@ -934,7 +1007,12 @@ export async function cleanupExpiredUploadSessions(): Promise<{ candidate.status, cleanupDb ) - if (claimed.status === 'aborted' || claimed.status === 'expired') { + if ( + claimed.status === 'aborted' || + claimed.status === 'expired' || + claimed.purpose === 'organization_logo' || + (claimed.purpose === 'mothership_attachment' && claimed.workspaceId === null) + ) { await deleteOwnedFinalObject(claimed) } else if (claimed.status !== 'completed') { throw new Error(`Invalid terminal upload status ${claimed.status}`) @@ -1202,7 +1280,34 @@ function validateFile(params: CreateUploadSessionParams): void { if (params.fileSize > maximum) { throw new UploadSessionError('validation', `File size exceeds maximum of ${maximum} bytes`) } - if (params.purpose !== 'profile_picture' && !params.workspaceId.trim()) { + if ( + params.purpose === 'organization_logo' && + (!params.organizationId.trim() || !isImageFileType(params.contentType)) + ) { + throw new UploadSessionError( + 'validation', + 'Organization logos must be image files with an organizationId' + ) + } + const organizationAttachment = + params.purpose === 'mothership_attachment' && 'organizationId' in params + if ( + organizationAttachment && + (!params.organizationId.trim() || + params.fileSize > ASSISTANT_IMAGE_MAX_BYTES || + !isAssistantImageType(params.contentType)) + ) { + throw new UploadSessionError( + 'validation', + 'Assistant attachments must be PNG, JPEG, GIF, or WebP images up to 5 MB' + ) + } + if ( + params.purpose !== 'profile_picture' && + params.purpose !== 'organization_logo' && + !organizationAttachment && + !params.workspaceId?.trim() + ) { throw new UploadSessionError('validation', 'workspaceId must not be empty') } if (params.purpose === 'knowledge_document' && !params.knowledgeBaseId.trim()) { @@ -1218,7 +1323,11 @@ function validateFile(params: CreateUploadSessionParams): void { function maximumFileSize(purpose: UploadSessionPurpose): number { if (purpose === 'knowledge_document') return MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE - if (purpose === 'profile_picture' || purpose === 'workspace_logo') { + if ( + purpose === 'profile_picture' || + purpose === 'workspace_logo' || + purpose === 'organization_logo' + ) { return UPLOAD_SESSION_ASSET_MAX_BYTES } if (purpose === 'execution_attachment') return MAX_WORKSPACE_FORMDATA_FILE_SIZE @@ -1231,7 +1340,11 @@ function requiresStorageQuota(purpose: UploadSessionPurpose): boolean { function isPrincipalBoundUploadPurpose(purpose: UploadSessionPurpose): boolean { return ( - purpose === 'workspace_file' || purpose === 'knowledge_document' || purpose === 'table_import' + purpose === 'workspace_file' || + purpose === 'knowledge_document' || + purpose === 'table_import' || + purpose === 'mothership_attachment' || + purpose === 'organization_logo' ) } @@ -1260,12 +1373,23 @@ function resolveUploadStorage( storageContext: 'profile-pictures', finalKey: `profile-pictures/${buildStorageKeySegment(`${id}-`, params.fileName)}`, } + case 'organization_logo': + return { + storageContext: 'organization-logos', + finalKey: `organization-logos/${params.organizationId}/${buildStorageKeySegment(`${id}-`, params.fileName)}`, + } case 'workspace_logo': return { storageContext: 'workspace-logos', finalKey: `workspace-logos/${params.workspaceId}/${buildStorageKeySegment(`${id}-`, params.fileName)}`, } case 'mothership_attachment': + if ('organizationId' in params) { + return { + storageContext: 'mothership', + finalKey: `assistant/${params.organizationId}/${params.userId}/${id}/${buildStorageKeySegment('', params.fileName)}`, + } + } return { storageContext: 'mothership', finalKey: generateWorkspaceFileKey(params.workspaceId, params.fileName), @@ -1292,6 +1416,7 @@ function isStorageContext(value: string): value is StorageContext { 'knowledge-base', 'profile-pictures', 'workspace-logos', + 'organization-logos', 'mothership', 'execution', ].includes(value) diff --git a/apps/sim/lib/uploads/upload-session/types.ts b/apps/sim/lib/uploads/upload-session/types.ts index 9468ab2a02e..e418047a3dc 100644 --- a/apps/sim/lib/uploads/upload-session/types.ts +++ b/apps/sim/lib/uploads/upload-session/types.ts @@ -4,6 +4,7 @@ export type UploadSessionPurpose = | 'knowledge_document' | 'profile_picture' | 'workspace_logo' + | 'organization_logo' | 'mothership_attachment' | 'execution_attachment' diff --git a/apps/sim/lib/uploads/utils/attachment-download-budget.test.ts b/apps/sim/lib/uploads/utils/attachment-download-budget.test.ts new file mode 100644 index 00000000000..c68fa225e6c --- /dev/null +++ b/apps/sim/lib/uploads/utils/attachment-download-budget.test.ts @@ -0,0 +1,61 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' +import { + AttachmentDownloadBudget, + readAttachmentJson, +} from '@/lib/uploads/utils/attachment-download-budget' + +describe('AttachmentDownloadBudget', () => { + it('defaults to the shared 100 MiB transfer bound', () => { + expect(new AttachmentDownloadBudget().remainingBytes).toBe(MAX_BUFFERED_TRANSFER_BYTES) + }) + + it('shares the remaining bytes across downloads and accepts the exact boundary', async () => { + const budget = new AttachmentDownloadBudget({ maxBytes: 6 }) + await budget.read(new Response('abc'), 'attachments') + await budget.read(new Response('def'), 'attachments') + expect(budget.remainingBytes).toBe(0) + await expect(budget.read(new Response('g'), 'attachments')).rejects.toBeInstanceOf( + PayloadSizeLimitError + ) + expect(budget.remainingBytes).toBe(0) + }) + + it('enforces actual bytes even with a false content-length', async () => { + const budget = new AttachmentDownloadBudget({ maxBytes: 3 }) + await expect( + budget.read(new Response('four', { headers: { 'content-length': '1' } }), 'attachments') + ).rejects.toBeInstanceOf(PayloadSizeLimitError) + }) + + it('accepts zero-byte files at the exact aggregate boundary', async () => { + const budget = new AttachmentDownloadBudget({ maxBytes: 0 }) + expect((await budget.read(new Response(''), 'attachments')).byteLength).toBe(0) + }) + + it('propagates cancellation while reading a body', async () => { + const controller = new AbortController() + const budget = new AttachmentDownloadBudget({ signal: controller.signal }) + const stream = new ReadableStream({ + pull() { + controller.abort(new DOMException('cancelled', 'AbortError')) + }, + }) + await expect(budget.read(new Response(stream), 'attachments')).rejects.toMatchObject({ + name: 'AbortError', + }) + }) + + it('keeps metadata responses bounded separately from file content', async () => { + await expect( + readAttachmentJson( + new Response('{}', { headers: { 'content-length': String(10 * 1024 * 1024 + 1) } }), + 'metadata' + ) + ).rejects.toBeInstanceOf(PayloadSizeLimitError) + }) +}) diff --git a/apps/sim/lib/uploads/utils/attachment-download-budget.ts b/apps/sim/lib/uploads/utils/attachment-download-budget.ts new file mode 100644 index 00000000000..15cdb5520bb --- /dev/null +++ b/apps/sim/lib/uploads/utils/attachment-download-budget.ts @@ -0,0 +1,70 @@ +import { + assertKnownSizeWithinLimit, + DEFAULT_MAX_ERROR_BODY_BYTES, + isPayloadSizeLimitError, + readResponseTextWithLimit, + readResponseToBufferWithLimit, +} from '@/lib/core/utils/stream-limits' +import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' + +const MAX_ATTACHMENT_METADATA_BYTES = 10 * 1024 * 1024 + +/** A shared byte budget for sequential attachment downloads in one tool call. */ +export class AttachmentDownloadBudget { + private downloadedBytes = 0 + readonly signal?: AbortSignal + private readonly maxBytes: number + + constructor(options: { signal?: AbortSignal; maxBytes?: number } = {}) { + this.signal = options.signal + this.maxBytes = options.maxBytes ?? MAX_BUFFERED_TRANSFER_BYTES + } + + get remainingBytes(): number { + return this.maxBytes - this.downloadedBytes + } + + assertSize(size: number, label: string, maxFileBytes = this.maxBytes): void { + this.signal?.throwIfAborted() + assertKnownSizeWithinLimit(size, Math.min(this.remainingBytes, maxFileBytes), label) + } + + consume(buffer: Buffer, label: string): Buffer { + this.assertSize(buffer.byteLength, label) + this.downloadedBytes += buffer.byteLength + return buffer + } + + async read(response: Response, label: string, maxFileBytes = this.maxBytes): Promise { + this.signal?.throwIfAborted() + const buffer = await readResponseToBufferWithLimit(response, { + maxBytes: Math.min(this.remainingBytes, maxFileBytes), + label, + signal: this.signal, + }) + return this.consume(buffer, label) + } +} + +/** Provider metadata and error bodies remain bounded independently of file content. */ +export async function readAttachmentJson( + response: Response, + label: string, + signal?: AbortSignal, + maxBytes = MAX_ATTACHMENT_METADATA_BYTES +): Promise { + signal?.throwIfAborted() + const text = await readResponseTextWithLimit(response, { + maxBytes: response.ok ? maxBytes : DEFAULT_MAX_ERROR_BODY_BYTES, + label, + signal, + }) + signal?.throwIfAborted() + return JSON.parse(text) as T +} + +/** Partial attachment failures may be skipped, but cancellation and byte limits must stop the call. */ +export function rethrowAttachmentDownloadError(error: unknown, signal?: AbortSignal): void { + signal?.throwIfAborted() + if (isPayloadSizeLimitError(error)) throw error +} diff --git a/apps/sim/lib/uploads/utils/file-utils.server.test.ts b/apps/sim/lib/uploads/utils/file-utils.server.test.ts index 91670aeeb25..f3463527fa8 100644 --- a/apps/sim/lib/uploads/utils/file-utils.server.test.ts +++ b/apps/sim/lib/uploads/utils/file-utils.server.test.ts @@ -3,13 +3,13 @@ */ import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockDownloadFile, mockParseWorkspaceFileKey, mockResolveServableDocBytes } = vi.hoisted( - () => ({ +const { mockDownloadFile, mockParseWorkspaceFileKey, mockResolveServableDocBytes, mockRenderPage } = + vi.hoisted(() => ({ mockDownloadFile: vi.fn(), mockParseWorkspaceFileKey: vi.fn(), mockResolveServableDocBytes: vi.fn(), - }) -) + mockRenderPage: vi.fn(), + })) vi.mock('@/lib/uploads/core/storage-service', () => ({ downloadFile: mockDownloadFile, @@ -28,6 +28,10 @@ vi.mock('@/lib/copilot/tools/server/files/doc-compile', () => ({ resolveServableDocBytes: mockResolveServableDocBytes, })) +vi.mock('@/lib/workspace-files/page-document.server', () => ({ + renderSimPageDocumentWithContributors: mockRenderPage, +})) + vi.mock('@/app/api/files/authorization', () => ({ verifyFileAccess: vi.fn(), })) @@ -220,3 +224,43 @@ describe('downloadServableFilesWithinBudget', () => { expect(mockDownloadFile).toHaveBeenCalledTimes(1) }) }) + +describe('servable page provenance', () => { + it('preserves the inlined image identity for execution-stored pages', async () => { + const workspaceId = '2f1d8c3e-5b6a-4c7d-8e9f-0a1b2c3d4e5f' + const contributor = { + fileId: 'image-file', + key: `workspace/${workspaceId}/image.png`, + context: 'workspace' as const, + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), + } + mockParseWorkspaceFileKey.mockReturnValue(null) + mockDownloadFile.mockResolvedValue(Buffer.from('---\ntitle: Example\n---\nPage body')) + mockRenderPage.mockResolvedValue({ + html: 'rendered image', + contributingFiles: [contributor], + }) + + const rendered = await downloadServableFileFromStorage( + { + id: 'page-file', + name: 'page.html', + key: `execution/${workspaceId}/3f2e9d4c-6a7b-4d8e-9f0a-1b2c3d4e5f6a/4a3b2c1d-7e8f-4a9b-8c0d-1e2f3a4b5c6d/page.html`, + url: '', + type: 'text/x-sim-page', + size: 100, + context: 'execution', + }, + 'request', + createLogger('test'), + { maxBytes: 1024 } + ) + + expect(mockRenderPage).toHaveBeenCalledWith(expect.any(String), { workspaceId }) + expect(rendered).toEqual({ + buffer: Buffer.from('rendered image'), + contentType: 'text/html', + contributingFiles: [contributor], + }) + }) +}) diff --git a/apps/sim/lib/uploads/utils/file-utils.server.ts b/apps/sim/lib/uploads/utils/file-utils.server.ts index b78bff2b00b..625dcddc262 100644 --- a/apps/sim/lib/uploads/utils/file-utils.server.ts +++ b/apps/sim/lib/uploads/utils/file-utils.server.ts @@ -36,7 +36,7 @@ import { resolveTrustedFileContext, } from '@/lib/uploads/utils/file-utils' import { isSimPageSource, SIM_PAGE_CONTENT_TYPE } from '@/lib/workspace-files/page-compile' -import { renderSimPageDocumentWithAssets } from '@/lib/workspace-files/page-document.server' +import { renderSimPageDocumentWithContributors } from '@/lib/workspace-files/page-document.server' import { type KnowledgeFileAccess, verifyFileAccess } from '@/app/api/files/authorization' import type { UserFile } from '@/executor/types' @@ -461,16 +461,20 @@ export async function downloadServableFileFromStorage( const text = buffer.toString('utf8') if (isSimPageSource(text)) { const workspaceId = userFile.key - ? (parseWorkspaceFileKey(userFile.key) ?? undefined) + ? (parseWorkspaceFileKey(userFile.key) ?? + extractWorkspaceIdFromExecutionKey(userFile.key) ?? + undefined) : undefined - const rendered = Buffer.from( - await renderSimPageDocumentWithAssets(text, { workspaceId }), - 'utf8' - ) + const page = await renderSimPageDocumentWithContributors(text, { workspaceId }) + const rendered = Buffer.from(page.html, 'utf8') // Rendering inlines referenced assets, so a source well under the ceiling can // resolve to a document well over it. assertKnownSizeWithinLimit(rendered.length, options.maxBytes, 'servable page render') - return { buffer: rendered, contentType: 'text/html' } + return { + buffer: rendered, + contentType: 'text/html', + contributingFiles: page.contributingFiles, + } } } diff --git a/apps/sim/lib/uploads/utils/file-utils.test.ts b/apps/sim/lib/uploads/utils/file-utils.test.ts index fbc7a69c376..69e58d96b5f 100644 --- a/apps/sim/lib/uploads/utils/file-utils.test.ts +++ b/apps/sim/lib/uploads/utils/file-utils.test.ts @@ -113,6 +113,7 @@ describe('inferContextFromKey', () => { expect(inferContextFromKey('profile-pictures/x')).toBe('profile-pictures') expect(inferContextFromKey('og-images/x')).toBe('og-images') expect(inferContextFromKey('workspace-logos/x')).toBe('workspace-logos') + expect(inferContextFromKey('organization-logos/x')).toBe('organization-logos') expect(inferContextFromKey('logs/x')).toBe('logs') }) @@ -158,6 +159,7 @@ describe('resolveTrustedFileContext', () => { 'workspace' ) expect(resolveTrustedFileContext('chat/x', 'workspace-logos')).toBe('chat') + expect(resolveTrustedFileContext('chat/x', 'organization-logos')).toBe('chat') expect(resolveTrustedFileContext('workspace/ws/x', 'mothership')).toBe('workspace') }) diff --git a/apps/sim/lib/uploads/utils/file-utils.ts b/apps/sim/lib/uploads/utils/file-utils.ts index b00ac1ab7ee..546e95d05f4 100644 --- a/apps/sim/lib/uploads/utils/file-utils.ts +++ b/apps/sim/lib/uploads/utils/file-utils.ts @@ -775,7 +775,7 @@ export function inferContextFromKey(key: string): StorageContext { if (!context) { throw new Error( key - ? `File key must start with a context prefix (kb/, knowledge-base/, chat/, copilot/, execution/, workspace/, profile-pictures/, og-images/, workspace-logos/, or logs/). Got: ${key}` + ? `File key must start with a context prefix (kb/, knowledge-base/, chat/, copilot/, execution/, workspace/, profile-pictures/, og-images/, workspace-logos/, organization-logos/, or logs/). Got: ${key}` : 'Cannot infer context from empty key' ) } @@ -801,9 +801,11 @@ export function tryInferContextFromKey(key: string): StorageContext | null { if (key.startsWith('copilot/')) return 'copilot' if (key.startsWith('execution/')) return 'execution' if (key.startsWith('workspace/')) return 'workspace' + if (key.startsWith('assistant/')) return 'mothership' if (key.startsWith('profile-pictures/')) return 'profile-pictures' if (key.startsWith('og-images/')) return 'og-images' if (key.startsWith('workspace-logos/')) return 'workspace-logos' + if (key.startsWith('organization-logos/')) return 'organization-logos' if (key.startsWith('logs/')) return 'logs' return null @@ -819,6 +821,7 @@ const PUBLIC_STORAGE_CONTEXTS = new Set([ 'profile-pictures', 'og-images', 'workspace-logos', + 'organization-logos', ]) /** Whether a trusted storage context is world-readable. */ diff --git a/apps/sim/lib/uploads/utils/stored-file-metadata.ts b/apps/sim/lib/uploads/utils/stored-file-metadata.ts new file mode 100644 index 00000000000..d1cee88a972 --- /dev/null +++ b/apps/sim/lib/uploads/utils/stored-file-metadata.ts @@ -0,0 +1,33 @@ +import { sniffImageContentType } from '@/lib/uploads/utils/validation' + +const IMAGE_FILE_EXTENSIONS: Record = { + 'image/gif': 'gif', + 'image/jpeg': 'jpg', + 'image/png': 'png', + 'image/webp': 'webp', +} + +/** Derives stored image metadata from its bytes rather than trusting a provider's MIME type. */ +export function resolveStoredFileMetadata( + fileName: string, + declaredMimeType: string, + buffer: Buffer +): { fileName: string; mimeType: string } { + if (!declaredMimeType.startsWith('image/')) { + return { fileName, mimeType: declaredMimeType } + } + + const mimeType = sniffImageContentType(buffer) + if (!mimeType) { + return { + fileName: `${fileName.replace(/\.[^.]+$/, '')}.bin`, + mimeType: 'application/octet-stream', + } + } + + const extension = IMAGE_FILE_EXTENSIONS[mimeType] + return { + fileName: extension ? `${fileName.replace(/\.[^.]+$/, '')}.${extension}` : fileName, + mimeType, + } +} diff --git a/apps/sim/lib/users/account-deletion-attachments.test.ts b/apps/sim/lib/users/account-deletion-attachments.test.ts new file mode 100644 index 00000000000..5b99873ead4 --- /dev/null +++ b/apps/sim/lib/users/account-deletion-attachments.test.ts @@ -0,0 +1,234 @@ +/** + * @vitest-environment node + */ +import { + dbChainMockFns, + hasMockCondition, + queueTableRows, + resetDbChainMock, + schemaMock, +} from '@sim/testing' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + isSoleOwnerOfPaidOrganization: vi.fn(), + getPersonalSubscription: vi.fn(), + isUsingCloudStorage: vi.fn(), + deleteFiles: vi.fn(), +})) + +vi.mock('@/lib/billing/organizations/membership', () => ({ + isSoleOwnerOfPaidOrganization: mocks.isSoleOwnerOfPaidOrganization, +})) +vi.mock('@/lib/billing/core/plan', () => ({ + getHighestPriorityPersonalSubscription: mocks.getPersonalSubscription, +})) +vi.mock('@/lib/uploads', () => ({ + isUsingCloudStorage: mocks.isUsingCloudStorage, + StorageService: { deleteFiles: mocks.deleteFiles }, +})) +vi.mock('@/lib/workspaces/utils', () => ({ + reassignBilledAccountForUser: vi.fn(async () => ({ unresolved: [] })), + reassignOwnedWorkspacesForUser: vi.fn(async () => ({ unresolved: [] })), +})) +vi.mock('@/lib/table/events', () => ({ appendTableEvent: vi.fn() })) +vi.mock('@/lib/table/rows/executions', () => ({ + cancelPendingMarkersForGovernedSubject: vi.fn(async () => []), +})) + +import { deleteUserAccount } from '@/lib/users/account-deletion' + +const IMAGE_KEY = 'assistant/org-1/user-1/upload-1/photo.png' +const FAILED_IMAGE_KEY = 'assistant/org-2/user-1/upload-2/photo.png' +const NOW = new Date('2026-09-11T12:00:00Z') + +function imageDeletionFilter() { + return dbChainMockFns.where.mock.calls + .map(([condition]) => condition) + .find((condition) => + hasMockCondition( + condition, + (node) => node.type === 'inArray' && node.column === schemaMock.uploadSession.finalKey + ) + ) +} + +describe('account deletion of private organization Assistant images', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + vi.useFakeTimers() + vi.setSystemTime(NOW) + mocks.isSoleOwnerOfPaidOrganization.mockResolvedValue({ isBlocker: false }) + mocks.getPersonalSubscription.mockResolvedValue(null) + mocks.isUsingCloudStorage.mockReturnValue(true) + mocks.deleteFiles.mockResolvedValue({ deleted: 1, failed: [] }) + }) + + afterEach(() => { + vi.useRealTimers() + }) + + it.each([true, false])( + 'purges image objects and ownership records after deleting an account without workspaces (cloud: %s)', + async (cloudStorage) => { + mocks.isUsingCloudStorage.mockReturnValue(cloudStorage) + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1', key: IMAGE_KEY }]) + + const plan = await deleteUserAccount('user-1') + + expect(plan.workspacesToDelete).toEqual([]) + expect(mocks.deleteFiles).toHaveBeenCalledWith([IMAGE_KEY], 'mothership') + expect(dbChainMockFns.delete).toHaveBeenCalledWith(schemaMock.uploadSession) + const userDeleteIndex = dbChainMockFns.delete.mock.calls.findIndex( + ([table]) => table === schemaMock.user + ) + const imageDeleteIndex = dbChainMockFns.delete.mock.calls.findIndex( + ([table]) => table === schemaMock.uploadSession + ) + expect(dbChainMockFns.delete.mock.invocationCallOrder[userDeleteIndex]).toBeLessThan( + mocks.deleteFiles.mock.invocationCallOrder[0] + ) + expect(mocks.deleteFiles.mock.invocationCallOrder[0]).toBeLessThan( + dbChainMockFns.delete.mock.invocationCallOrder[imageDeleteIndex] + ) + } + ) + + it('scopes both collection and ownership deletion to this uploader’s completed organization images', async () => { + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1', key: IMAGE_KEY }]) + + await deleteUserAccount('user-1') + + const imageFilters = dbChainMockFns.where.mock.calls + .map(([condition]) => condition) + .filter((condition) => + hasMockCondition(condition, (node) => node.left === schemaMock.uploadSession.userId) + ) + expect(imageFilters).toHaveLength(2) + for (const filter of imageFilters) { + for (const [column, value] of [ + [schemaMock.uploadSession.userId, 'user-1'], + [schemaMock.uploadSession.purpose, 'mothership_attachment'], + [schemaMock.uploadSession.status, 'completed'], + ]) { + expect( + hasMockCondition( + filter, + (node) => node.type === 'eq' && node.left === column && node.right === value + ) + ).toBe(true) + } + expect( + hasMockCondition( + filter, + (node) => node.type === 'isNull' && node.column === schemaMock.uploadSession.workspaceId + ) + ).toBe(true) + } + }) + + it('retains ownership while an issued upload URL could recreate a purged object', async () => { + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1', key: IMAGE_KEY }]) + + await deleteUserAccount('user-1') + + expect( + hasMockCondition( + imageDeletionFilter(), + (node) => + node.type === 'lte' && + node.left === schemaMock.uploadSession.expiresAt && + node.right instanceof Date && + node.right.getTime() === NOW.getTime() + ) + ).toBe(true) + }) + + it('retains failed objects’ ownership records for the upload-session sweep', async () => { + queueTableRows(schemaMock.uploadSession, [ + { id: 'upload-1', key: IMAGE_KEY }, + { id: 'upload-2', key: FAILED_IMAGE_KEY }, + ]) + mocks.deleteFiles.mockResolvedValue({ + deleted: 1, + failed: [{ key: FAILED_IMAGE_KEY, error: 'Storage unavailable' }], + }) + + await deleteUserAccount('user-1') + + expect( + hasMockCondition( + imageDeletionFilter(), + (node) => + node.type === 'inArray' && + node.column === schemaMock.uploadSession.finalKey && + Array.isArray(node.values) && + node.values.length === 1 && + node.values[0] === IMAGE_KEY + ) + ).toBe(true) + }) + + it.each(['batch', 'object'])( + 'keeps ownership records when all %s deletions fail', + async (failure) => { + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1', key: IMAGE_KEY }]) + if (failure === 'batch') { + mocks.deleteFiles.mockRejectedValueOnce(new Error('Storage unavailable')) + } else { + mocks.deleteFiles.mockResolvedValueOnce({ + deleted: 0, + failed: [{ key: IMAGE_KEY, error: 'Storage unavailable' }], + }) + } + + await expect(deleteUserAccount('user-1')).resolves.toMatchObject({ blockers: [] }) + + expect(dbChainMockFns.delete).not.toHaveBeenCalledWith(schemaMock.uploadSession) + } + ) + + it('does not purge images or ownership records if account deletion rolls back', async () => { + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1', key: IMAGE_KEY }]) + dbChainMockFns.transaction.mockRejectedValueOnce(new Error('Transaction rolled back')) + + await expect(deleteUserAccount('user-1')).rejects.toThrow('Transaction rolled back') + + expect(mocks.deleteFiles).not.toHaveBeenCalled() + expect(dbChainMockFns.delete).not.toHaveBeenCalledWith(schemaMock.uploadSession) + }) + + it('leaves storage untouched when deletion is blocked for an active account', async () => { + mocks.getPersonalSubscription.mockResolvedValueOnce({ plan: 'pro' }) + + await expect(deleteUserAccount('user-1')).rejects.toMatchObject({ code: 'conflict' }) + + expect(dbChainMockFns.from).not.toHaveBeenCalledWith(schemaMock.uploadSession) + expect(mocks.deleteFiles).not.toHaveBeenCalled() + }) + + it('collects and purges image keys in bounded pages', async () => { + const firstPage = Array.from({ length: 1000 }, (_, index) => ({ + id: `upload-${String(index).padStart(4, '0')}`, + key: `assistant/org-1/user-1/upload-${index}/photo.png`, + })) + queueTableRows(schemaMock.uploadSession, firstPage) + queueTableRows(schemaMock.uploadSession, [{ id: 'upload-1000', key: IMAGE_KEY }]) + + await deleteUserAccount('user-1') + + expect(mocks.deleteFiles.mock.calls.map(([keys]) => keys.length)).toEqual([1000, 1]) + expect( + dbChainMockFns.where.mock.calls.some(([condition]) => + hasMockCondition( + condition, + (node) => + node.type === 'gt' && + node.left === schemaMock.uploadSession.id && + node.right === firstPage[999].id + ) + ) + ).toBe(true) + }) +}) diff --git a/apps/sim/lib/users/account-deletion.ts b/apps/sim/lib/users/account-deletion.ts index 447beba1b30..1b81d20988c 100644 --- a/apps/sim/lib/users/account-deletion.ts +++ b/apps/sim/lib/users/account-deletion.ts @@ -7,6 +7,7 @@ import { organization, permissions, tableRunDispatches, + uploadSession, user, workspaceFile, workspaceFiles, @@ -14,7 +15,7 @@ import { } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { formatQuotedNameList } from '@sim/utils/string' -import { and, eq, gt, inArray, isNotNull, ne, notExists, or, sql } from 'drizzle-orm' +import { and, eq, gt, inArray, isNotNull, isNull, lte, ne, notExists, or, sql } from 'drizzle-orm' import type { AccountDeletionBlocker, AccountDeletionPlan, @@ -415,7 +416,7 @@ export function extractProfilePictureKey(image: string | null): string | null { } /** - * Collects every stored object held by workspaces that go with the account. + * Collects the account's private images and stored objects in workspaces that go with it. * * This has to run *before* the rows are deleted: they disappear with the * workspace through `ON DELETE CASCADE`, and the retention sweep that normally @@ -429,6 +430,27 @@ async function collectAccountStorageKeys( workspaceIds: string[] ): Promise { const batches: StorageKeyBatch[] = [] + + await collectPages( + (afterId) => + db + .select({ id: uploadSession.id, key: uploadSession.finalKey }) + .from(uploadSession) + .where( + and( + eq(uploadSession.userId, userId), + eq(uploadSession.purpose, 'mothership_attachment'), + isNull(uploadSession.workspaceId), + eq(uploadSession.status, 'completed'), + gt(uploadSession.id, afterId) + ) + ) + .orderBy(uploadSession.id) + .limit(STORAGE_PAGE_SIZE), + batches, + () => 'mothership' + ) + if (!isUsingCloudStorage()) return batches const [profile] = await db @@ -497,7 +519,7 @@ async function collectAccountStorageKeys( * failure for work that cannot be undone. An orphaned object is recoverable from * the log; a deletion the caller believes failed is not. */ -async function purgeStorageObjects(batches: StorageKeyBatch[]): Promise { +async function purgeStorageObjects(userId: string, batches: StorageKeyBatch[]): Promise { for (const { context, keys } of batches) { if (keys.length === 0) continue try { @@ -509,8 +531,34 @@ async function purgeStorageObjects(batches: StorageKeyBatch[]): Promise { error, }) } + + if (context === 'mothership') { + const failedKeys = new Set(failed.map(({ key }) => key)) + const deletedImageKeys = keys.filter( + (key) => key.startsWith('assistant/') && !failedKeys.has(key) + ) + if (deletedImageKeys.length > 0) { + /** + * Keep ownership records while a signed PUT can recreate the object. + * The upload-session sweep retries these and failed object deletions + * after the deleted uploader and transfer expiry are confirmed. + */ + await db + .delete(uploadSession) + .where( + and( + eq(uploadSession.userId, userId), + eq(uploadSession.purpose, 'mothership_attachment'), + isNull(uploadSession.workspaceId), + eq(uploadSession.status, 'completed'), + lte(uploadSession.expiresAt, new Date()), + inArray(uploadSession.finalKey, deletedImageKeys) + ) + ) + } + } } catch (error) { - logger.error('Storage batch deletion failed during account deletion', { context, error }) + logger.error('Storage cleanup failed during account deletion', { context, error }) } } } @@ -743,7 +791,7 @@ export async function deleteUserAccount(userId: string): Promise ({ clearExecutionCancellation: clearExecutionCancellationMock, })) +const { connectExecutionSignalHubMock } = vi.hoisted(() => ({ + connectExecutionSignalHubMock: vi.fn(), +})) + +vi.mock('@/lib/execution/execution-signal', () => ({ + connectExecutionSignalHub: connectExecutionSignalHubMock, +})) + vi.mock('@/lib/core/security/encryption', () => ({ decryptSecret: decryptSecretMock, })) @@ -376,6 +384,21 @@ describe('executeWorkflowCore terminal finalization sequencing', () => { expect(executorConstructorMock).toHaveBeenCalledTimes(1) }) + it('begins connecting the signal subscriber synchronously, before the first await', async () => { + const executionPromise = executeWorkflowCore({ + snapshot: createSnapshot() as unknown as ExecutionSnapshot, + callbacks: {}, + loggingSession: loggingSession as unknown as LoggingSession, + }) + + // Asserted with no await in between: the handshake has to start ahead of + // the custom-block read, or it stops overlapping the work that precedes the + // cancellation subscribe and is paid inside that subscribe's budget instead. + expect(connectExecutionSignalHubMock).toHaveBeenCalledOnce() + + await executionPromise + }) + it('routes onBlockStart through logging session persistence path', async () => { executorExecuteMock.mockResolvedValue({ success: true, diff --git a/apps/sim/lib/workflows/executor/execution-core.ts b/apps/sim/lib/workflows/executor/execution-core.ts index 82ba64adbc1..a465c6b48e4 100644 --- a/apps/sim/lib/workflows/executor/execution-core.ts +++ b/apps/sim/lib/workflows/executor/execution-core.ts @@ -22,6 +22,7 @@ import { import { withDatabaseReadRetry } from '@/lib/db/read-retry' import { getExecutionEnvironment } from '@/lib/environment/utils' import { clearExecutionCancellation } from '@/lib/execution/cancellation' +import { connectExecutionSignalHub } from '@/lib/execution/execution-signal' import { warmLargeValueRefs } from '@/lib/execution/payloads/hydration' import { parseLargeExecutionValue } from '@/lib/execution/payloads/large-execution-value' import type { LoggingSession } from '@/lib/logs/execution/logging-session' @@ -377,10 +378,19 @@ async function finalizeExecutionError(params: { * the background job — puts `custom_block_*` types in scope for serialization, * execution, and any nested child-workflow serialization (ALS propagates to the * whole async subtree). + * + * Also begins the execution-signal subscriber's connection first: every + * execution subscribes to cancellation signals once its engine starts, so + * starting that handshake here — the one path all of them share — lets it + * overlap the reads and preprocessing ahead of the subscribe instead of being + * paid inside its readiness budget. Connecting on intent rather than at worker + * start keeps the tasks that never execute a workflow, most of the fleet by + * volume, from opening a connection they would never use. */ export async function executeWorkflowCore( options: ExecuteWorkflowCoreOptions ): Promise { + connectExecutionSignalHub() const workspaceId = options.snapshot.metadata.workspaceId const rows = workspaceId ? await withDatabaseReadRetry(() => getCustomBlockRowsForWorkspace(workspaceId), { diff --git a/apps/sim/lib/workspace-files/page-document.server.test.ts b/apps/sim/lib/workspace-files/page-document.server.test.ts index 14375b70a16..5eba3bbd766 100644 --- a/apps/sim/lib/workspace-files/page-document.server.test.ts +++ b/apps/sim/lib/workspace-files/page-document.server.test.ts @@ -21,7 +21,10 @@ vi.mock('@/lib/workspace-files/page-document', () => ({ renderSimPageDocument: mockRenderSimPageDocument, })) -import { renderSimPageDocumentWithAssets } from '@/lib/workspace-files/page-document.server' +import { + renderSimPageDocumentWithAssets, + renderSimPageDocumentWithContributors, +} from '@/lib/workspace-files/page-document.server' const WORKSPACE_ID = 'ws-1' const MB = 1024 * 1024 @@ -35,6 +38,7 @@ function imageRecord(id: string, size: number) { contentType: 'image/png', size, sizeBytes: size, + contentUpdatedAt: new Date('2026-01-01T00:00:00Z'), } } @@ -119,3 +123,73 @@ describe('renderSimPageDocumentWithAssets memory bounds', () => { expect(html).toContain('src="/api/files/view/theirs"') }) }) + +describe('rendered page contributors', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('reports only the canonical revisions whose bytes were embedded', async () => { + mockRenderSimPageDocument.mockReturnValue( + documentReferencing(['mine', 'failed', 'foreign', 'missing', 'mine']) + ) + const record = imageRecord('mine', 5) + mockGetFileMetadataById.mockImplementation(async (id: string) => { + if (id === 'missing') return null + if (id === 'foreign') return { ...imageRecord(id, 5), workspaceId: 'other-workspace' } + return id === 'mine' ? record : imageRecord(id, 5) + }) + mockDownloadFile.mockImplementation(async ({ key }: { key: string }) => { + if (key.includes('failed')) throw new Error('unavailable') + return Buffer.from('image') + }) + + const rendered = await renderSimPageDocumentWithContributors('source', { + workspaceId: WORKSPACE_ID, + }) + + expect(rendered.contributingFiles).toEqual([ + { + fileId: record.id, + key: record.key, + context: 'workspace', + contentUpdatedAt: record.contentUpdatedAt, + }, + ]) + expect(rendered.html).toContain('data:image/png;base64,aW1hZ2U=') + expect(rendered.html).toContain('/api/files/view/failed') + expect(rendered.html).toContain('/api/files/view/foreign') + expect(mockGetFileMetadataById).toHaveBeenCalledTimes(4) + }) + + it('bounds metadata reads for missing images', async () => { + mockRenderSimPageDocument.mockReturnValue( + documentReferencing(Array.from({ length: 300 }, (_, i) => `missing-${i}`)) + ) + mockGetFileMetadataById.mockResolvedValue(null) + + const rendered = await renderSimPageDocumentWithContributors('source', { + workspaceId: WORKSPACE_ID, + }) + + expect(mockGetFileMetadataById).toHaveBeenCalledTimes(256) + expect(mockDownloadFile).not.toHaveBeenCalled() + expect(rendered.contributingFiles).toEqual([]) + }) + + it('charges repeated image occurrences against the rendered byte budget', async () => { + const source = documentReferencing(Array(12).fill('image')) + mockRenderSimPageDocument.mockReturnValue(source) + mockGetFileMetadataById.mockResolvedValue(imageRecord('image', 8 * MB)) + mockDownloadFile.mockResolvedValue(Buffer.alloc(8 * MB)) + + const rendered = await renderSimPageDocumentWithContributors('source', { + workspaceId: WORKSPACE_ID, + }) + + expect(mockDownloadFile).toHaveBeenCalledTimes(1) + expect(rendered.html.length).toBeLessThanOrEqual(source.length + Math.ceil((32 * MB * 4) / 3)) + expect(rendered.html).toContain('/api/files/view/image') + expect(rendered.contributingFiles).toHaveLength(1) + }) +}) diff --git a/apps/sim/lib/workspace-files/page-document.server.ts b/apps/sim/lib/workspace-files/page-document.server.ts index 4c4954e33dc..7abcdfa74d4 100644 --- a/apps/sim/lib/workspace-files/page-document.server.ts +++ b/apps/sim/lib/workspace-files/page-document.server.ts @@ -1,3 +1,4 @@ +import type { WorkspaceFileSecretProvenanceIdentity } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' import { downloadFile } from '@/lib/uploads/core/storage-service' import { getFileMetadataById } from '@/lib/uploads/server/metadata' import { renderSimPageDocument } from '@/lib/workspace-files/page-document' @@ -12,6 +13,9 @@ const MAX_INLINE_IMAGE_BYTES = 8 * 1024 * 1024 */ const MAX_INLINE_TOTAL_BYTES = 32 * 1024 * 1024 +/** Bounds metadata reads even when the page references many missing or empty images. */ +const MAX_INLINE_IMAGE_REFERENCES = 256 + const IMAGE_SRC = /src="[^"]*\/api\/files\/view\/([^"]+)"/g /** @@ -27,30 +31,32 @@ export async function renderSimPageDocumentWithAssets( source: string, options: { workspaceId?: string } ): Promise { - const documentHtml = renderSimPageDocument(source, options) - const ids = [...new Set([...documentHtml.matchAll(IMAGE_SRC)].map((match) => match[1]))] - if (ids.length === 0 || !options.workspaceId) return documentHtml + return (await renderSimPageDocumentWithContributors(source, options)).html +} - const candidates = await Promise.all( - ids.map(async (id) => { - const record = await getFileMetadataById(id).catch(() => null) - if (!record || record.context !== 'workspace' || record.workspaceId !== options.workspaceId) - return null - return { id, record } - }) - ) +/** Servable page bytes and the exact stored image revisions actually embedded in them. */ +export async function renderSimPageDocumentWithContributors( + source: string, + options: { workspaceId?: string } +): Promise<{ html: string; contributingFiles: readonly WorkspaceFileSecretProvenanceIdentity[] }> { + const documentHtml = renderSimPageDocument(source, options) + if (!options.workspaceId) return { html: documentHtml, contributingFiles: [] } - // One image at a time, charged against the budget by what each download actually - // delivered. Fetching them concurrently made the peak the sum of every image rather - // than the largest one, and the ceiling on the finished document could only observe - // that after the fact. Each download is given whatever the budget has left, so an - // image that does not fit is refused by the read itself instead of after it lands. - const inlined = new Map() + const visited = new Set() + const inlined = new Map< + string, + { dataUri: string; identity: WorkspaceFileSecretProvenanceIdentity } + >() let remaining = MAX_INLINE_TOTAL_BYTES - for (const candidate of candidates) { - if (!candidate) continue - if (remaining === 0) break - const { id, record } = candidate + for (const match of documentHtml.matchAll(IMAGE_SRC)) { + const id = match[1] + if (visited.has(id)) continue + if (remaining === 0 || visited.size >= MAX_INLINE_IMAGE_REFERENCES) break + visited.add(id) + const record = await getFileMetadataById(id).catch(() => null) + if (!record || record.context !== 'workspace' || record.workspaceId !== options.workspaceId) { + continue + } try { const bytes = await downloadFile({ key: record.key, @@ -61,14 +67,28 @@ export async function renderSimPageDocumentWithAssets( const mime = record.contentType?.startsWith('image/') ? record.contentType : 'application/octet-stream' - inlined.set(id, `data:${mime};base64,${bytes.toString('base64')}`) + inlined.set(id, { + dataUri: `data:${mime};base64,${bytes.toString('base64')}`, + identity: { + fileId: record.id, + key: record.key, + context: 'workspace', + contentUpdatedAt: record.contentUpdatedAt, + }, + }) } catch { - // A missing, unreadable or too-large image keeps its URL reference. + /** A missing, unreadable or too-large image keeps its URL reference. */ } } - if (inlined.size === 0) return documentHtml - return documentHtml.replace(IMAGE_SRC, (match, id: string) => { - const dataUri = inlined.get(id) - return dataUri ? `src="${dataUri}"` : match + /** Charge each occurrence: repeating one image must not multiply the rendered byte budget. */ + let remainingEncodedBytes = Math.ceil((MAX_INLINE_TOTAL_BYTES * 4) / 3) + const contributors = new Map() + const html = documentHtml.replace(IMAGE_SRC, (match, id: string) => { + const image = inlined.get(id) + if (!image || image.dataUri.length > remainingEncodedBytes) return match + remainingEncodedBytes -= image.dataUri.length + contributors.set(id, image.identity) + return `src="${image.dataUri}"` }) + return { html, contributingFiles: [...contributors.values()] } } diff --git a/apps/sim/lib/workspaces/admin-move-source-impact.ts b/apps/sim/lib/workspaces/admin-move-source-impact.ts index 2674334b9a6..687f89ec9ac 100644 --- a/apps/sim/lib/workspaces/admin-move-source-impact.ts +++ b/apps/sim/lib/workspaces/admin-move-source-impact.ts @@ -49,12 +49,12 @@ import { getCustomBlockUsageCounts } from '@/lib/workflows/custom-blocks/operati * failure mode a downgrade disclosure cannot have. Now adding a section to * that union fails the build here until somebody decides whether it is gated. * - * The gating mirrors `isOrganizationSettingsSectionAvailable`: on hosted every - * section except `members` and `billing` resolves to `hasEnterprisePlan`. The - * type is imported type-only so this domain module stays free of the settings + * The gating mirrors `isOrganizationSettingsSectionAvailable`. The type is + * imported type-only so this domain module stays free of the settings * navigation module's React and icon imports. */ const ENTERPRISE_GATED_SECTION_LABELS: Record = { + 'recently-deleted': null, integrations: 'Sim Search source setup', 'search-mcp': null, 'search-slack': 'Sim Search in Slack', diff --git a/apps/sim/lib/workspaces/constants.ts b/apps/sim/lib/workspaces/constants.ts new file mode 100644 index 00000000000..7a3247a6149 --- /dev/null +++ b/apps/sim/lib/workspaces/constants.ts @@ -0,0 +1,2 @@ +/** Search becomes available when the workspace menu exceeds five entries. */ +export const WORKSPACE_SEARCH_THRESHOLD = 6 diff --git a/apps/sim/lib/workspaces/utils.test.ts b/apps/sim/lib/workspaces/utils.test.ts index 2f2474c785b..f0d240c0d9a 100644 --- a/apps/sim/lib/workspaces/utils.test.ts +++ b/apps/sim/lib/workspaces/utils.test.ts @@ -278,7 +278,13 @@ describe('listAccessibleWorkspaceRowsForUser', () => { }) it('elevates an org admin to admin on an org workspace where they hold a lower explicit grant', async () => { - const orgWorkspace = { id: 'ws-1', name: 'Shared', ownerId: 'owner-x', organizationId: 'org-1' } + const orgWorkspace = { + id: 'ws-1', + name: 'Shared', + ownerId: 'owner-x', + organizationId: 'org-1', + createdAt: new Date('2026-01-01'), + } dbChainMockFns.select .mockReturnValueOnce(createMockChain([{ workspace: orgWorkspace, permissionType: 'write' }])) @@ -292,12 +298,19 @@ describe('listAccessibleWorkspaceRowsForUser', () => { it('keeps a lower explicit grant on a workspace owned by a different organization', async () => { const externalWorkspace = { + createdAt: new Date('2026-02-01'), id: 'ws-ext', name: 'External', ownerId: 'owner-y', organizationId: 'org-2', } - const orgWorkspace = { id: 'ws-1', name: 'Shared', ownerId: 'owner-x', organizationId: 'org-1' } + const orgWorkspace = { + id: 'ws-1', + name: 'Shared', + ownerId: 'owner-x', + organizationId: 'org-1', + createdAt: new Date('2026-01-01'), + } dbChainMockFns.select .mockReturnValueOnce( @@ -325,4 +338,19 @@ describe('listAccessibleWorkspaceRowsForUser', () => { expect(rows).toEqual([{ workspace: ownWorkspace, permissionType: 'admin', viaOrgAdmin: false }]) }) + it('globally orders combined explicit and derived access by newest creation date', async () => { + const explicit = { id: 'ws-explicit', createdAt: new Date('2026-01-01') } + const derived = { id: 'ws-derived', createdAt: new Date('2026-02-01') } + dbChainMockFns.select + .mockReturnValueOnce(createMockChain([{ workspace: explicit, permissionType: 'write' }])) + .mockReturnValueOnce(createMockChain([{ organizationId: 'org-1', role: 'admin' }])) + .mockReturnValueOnce(createMockChain([explicit, derived])) + + const rows = await listAccessibleWorkspaceRowsForUser('user-1', 'active') + expect(rows.map(({ workspace }) => workspace.id)).toEqual(['ws-derived', 'ws-explicit']) + expect(rows).toEqual([ + { workspace: derived, permissionType: 'admin', viaOrgAdmin: true }, + { workspace: explicit, permissionType: 'admin', viaOrgAdmin: true }, + ]) + }) }) diff --git a/apps/sim/lib/workspaces/utils.ts b/apps/sim/lib/workspaces/utils.ts index 760dcb9b1ce..e225516d12e 100644 --- a/apps/sim/lib/workspaces/utils.ts +++ b/apps/sim/lib/workspaces/utils.ts @@ -150,7 +150,9 @@ export async function listAccessibleWorkspaceRowsForUser( .filter((ws) => !seen.has(ws.id)) .map((ws) => ({ workspace: ws, permissionType: 'admin' as const, viaOrgAdmin: true })) - return [...elevatedExplicit, ...derived] + return [...elevatedExplicit, ...derived].sort( + (a, b) => b.workspace.createdAt.getTime() - a.workspace.createdAt.getTime() + ) } export async function listUserWorkspaces(userId: string, scope: WorkspaceScope = 'active') { diff --git a/apps/sim/public/library/ai-workflow-automation-platform-buyers-checklist/cover.jpg b/apps/sim/public/library/ai-workflow-automation-platform-buyers-checklist/cover.jpg new file mode 100644 index 00000000000..efe0522fc5e Binary files /dev/null and b/apps/sim/public/library/ai-workflow-automation-platform-buyers-checklist/cover.jpg differ diff --git a/apps/sim/public/library/why-no-code-ai-agents-need-live-web-access/cover.jpg b/apps/sim/public/library/why-no-code-ai-agents-need-live-web-access/cover.jpg new file mode 100644 index 00000000000..050cf7d87ce Binary files /dev/null and b/apps/sim/public/library/why-no-code-ai-agents-need-live-web-access/cover.jpg differ diff --git a/apps/sim/scripts/register-platform-slack-app.ts b/apps/sim/scripts/register-platform-slack-app.ts index 5e0376cbb09..185a7b9aa8a 100644 --- a/apps/sim/scripts/register-platform-slack-app.ts +++ b/apps/sim/scripts/register-platform-slack-app.ts @@ -10,17 +10,16 @@ const logger = createLogger('RegisterPlatformSlackApp') /** Explicit deployment preparation; never chooses an app identity from an unauthenticated event. */ async function main() { const appId = process.argv[2] - const searchApp = process.argv.includes('--search') - const clientId = searchApp ? process.env.SLACK_SEARCH_CLIENT_ID : process.env.SLACK_CLIENT_ID - const clientSecret = searchApp - ? process.env.SLACK_SEARCH_CLIENT_SECRET - : process.env.SLACK_CLIENT_SECRET - const signingSecret = searchApp - ? process.env.SLACK_SEARCH_SIGNING_SECRET - : process.env.SLACK_SIGNING_SECRET + if (process.argv.includes('--search')) + throw new Error( + 'Slack Search reads its app credentials directly from SLACK_SEARCH_* environment variables' + ) + const clientId = process.env.SLACK_CLIENT_ID + const clientSecret = process.env.SLACK_CLIENT_SECRET + const signingSecret = process.env.SLACK_SIGNING_SECRET if (!appId || !/^A[A-Z0-9]+$/.test(appId) || !clientId || !clientSecret || !signingSecret) throw new Error( - 'Supply a verified app ID and client/signing secrets. With --search use SLACK_SEARCH_CLIENT_ID, SLACK_SEARCH_CLIENT_SECRET, SLACK_SEARCH_SIGNING_SECRET; otherwise use SLACK_CLIENT_ID, SLACK_CLIENT_SECRET, SLACK_SIGNING_SECRET.' + 'Supply a verified app ID, SLACK_CLIENT_ID, SLACK_CLIENT_SECRET, and SLACK_SIGNING_SECRET.' ) const [client, signing] = await Promise.all([ encryptSecret(clientSecret), diff --git a/apps/sim/tools/agiloft/retrieve_attachment.test.ts b/apps/sim/tools/agiloft/retrieve_attachment.test.ts new file mode 100644 index 00000000000..611f53b8abc --- /dev/null +++ b/apps/sim/tools/agiloft/retrieve_attachment.test.ts @@ -0,0 +1,23 @@ +/** @vitest-environment node */ +import { describe, expect, it } from 'vitest' +import { agiloftRetrieveAttachmentTool } from '@/tools/agiloft/retrieve_attachment' + +describe('Agiloft attachment file output', () => { + it('preserves the canonical stored file descriptor without requiring inline data', async () => { + const file = { + id: 'stored-file', + name: 'attachment.pdf', + size: 12 * 1024 * 1024, + type: 'application/pdf', + url: '/api/files/stored', + key: 'execution/attachment.pdf', + context: 'execution', + } + const result = await agiloftRetrieveAttachmentTool.transformResponse!( + Response.json({ success: true, output: { file } }) + ) + expect(result).toEqual({ success: true, output: { file } }) + expect(result.output.file).not.toHaveProperty('data') + expect(result.output.file).not.toHaveProperty('mimeType') + }) +}) diff --git a/apps/sim/tools/agiloft/retrieve_attachment.ts b/apps/sim/tools/agiloft/retrieve_attachment.ts index ad2b2ab1953..b7f02e0f87f 100644 --- a/apps/sim/tools/agiloft/retrieve_attachment.ts +++ b/apps/sim/tools/agiloft/retrieve_attachment.ts @@ -93,12 +93,7 @@ export const agiloftRetrieveAttachmentTool: InternalToolConfig< return { success: true, output: { - file: { - name: data.output.file.name, - mimeType: data.output.file.mimeType, - data: data.output.file.data, - size: data.output.file.size, - }, + file: data.output.file, }, } }, diff --git a/apps/sim/tools/agiloft/types.ts b/apps/sim/tools/agiloft/types.ts index f21eb5c265c..2ef86af87e6 100644 --- a/apps/sim/tools/agiloft/types.ts +++ b/apps/sim/tools/agiloft/types.ts @@ -1,4 +1,5 @@ -import type { ToolResponse } from '@/tools/types' +import type { UserFile } from '@/executor/types' +import type { ToolFileData, ToolResponse } from '@/tools/types' /** * Connection and credentials. `table` is optional here because EWLogin is @@ -149,12 +150,7 @@ export interface AgiloftRetrieveAttachmentParams extends AgiloftBaseParams { export interface AgiloftRetrieveAttachmentResponse extends ToolResponse { output: { - file: { - name: string - mimeType: string - data: string - size: number - } + file: UserFile | ToolFileData } } diff --git a/apps/sim/tools/binary-downloads.test.ts b/apps/sim/tools/binary-downloads.test.ts new file mode 100644 index 00000000000..d08f4a94893 --- /dev/null +++ b/apps/sim/tools/binary-downloads.test.ts @@ -0,0 +1,235 @@ +/** @vitest-environment node */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { boxDownloadFileTool, boxDownloadFileV2Tool } from '@/tools/box/download_file' +import { daytonaDownloadFileTool } from '@/tools/daytona/download_file' +import { dropboxDownloadTool, dropboxDownloadV2Tool } from '@/tools/dropbox/download' +import { getQrCodeTool, getQrCodeV2Tool } from '@/tools/dub/get_qr_code' +import { + dataverseDownloadFileTool, + dataverseDownloadFileV2Tool, +} from '@/tools/microsoft_dataverse/download_file' +import { personaPrintInquiryPdfTool } from '@/tools/persona/print_inquiry_pdf' +import { s3GetObjectTool } from '@/tools/s3/get_object' +import { + downloadAttachmentTool, + downloadAttachmentV2Tool, +} from '@/tools/servicenow/download_attachment' +import { storageDownloadTool } from '@/tools/supabase/storage_download' + +interface BinaryResult { + success: boolean + output: Record +} + +interface DownloadCase { + tool: { id: string; request: { responseType?: 'binary' }; outputs?: Record } + transform: (response: Response) => Promise + name: string + mimeType?: string + outputKeys?: string[] + checkMetadata?: (output: Record, size: number) => void +} + +const DOWNLOAD_CASES: DownloadCase[] = [ + { + tool: boxDownloadFileV2Tool, + transform: (response) => boxDownloadFileV2Tool.transformResponse!(response), + name: 'download.pdf', + outputKeys: ['file'], + }, + { + tool: dropboxDownloadV2Tool, + transform: (response) => + dropboxDownloadV2Tool.transformResponse!(response, { path: '/download.pdf' }), + name: 'download.pdf', + outputKeys: ['file', 'metadata', 'temporaryLink'], + checkMetadata: (output, size) => { + expect(output.metadata).toEqual({ id: 'file-1', name: 'download.pdf', size }) + expect(output.temporaryLink).toBeUndefined() + }, + }, + { + tool: s3GetObjectTool, + transform: (response) => + s3GetObjectTool.transformResponse!(response, { + accessKeyId: 'test-access-key', + secretAccessKey: 'test-secret-key', + bucketName: 'test-bucket', + region: 'us-east-1', + objectKey: 'folder/download.pdf', + }), + name: 'download.pdf', + checkMetadata: (output, size) => { + expect(output.metadata).toEqual({ + fileType: 'application/pdf', + size, + name: 'download.pdf', + lastModified: 'Fri, 11 Sep 2026 12:00:00 GMT', + }) + expect(output.url).toMatch( + /^https:\/\/test-bucket\.s3\.us-east-1\.amazonaws\.com\/folder\/download\.pdf\?/ + ) + }, + }, + { + tool: storageDownloadTool, + transform: (response) => + storageDownloadTool.transformResponse!(response, { + projectId: 'project-1', + apiKey: 'test-key', + bucket: 'documents', + path: 'folder/original.pdf', + fileName: 'renamed.pdf', + }), + name: 'renamed.pdf', + }, + { + tool: daytonaDownloadFileTool, + transform: (response) => + daytonaDownloadFileTool.transformResponse!(response, { + apiKey: 'test-key', + sandboxId: 'sandbox-1', + filePath: '/workspace/download.pdf', + }), + name: 'download.pdf', + checkMetadata: (output, size) => { + expect(output.name).toBe('download.pdf') + expect(output.mimeType).toBe('application/pdf') + expect(output.size).toBe(size) + }, + }, + { + tool: dataverseDownloadFileV2Tool, + transform: (response) => + dataverseDownloadFileV2Tool.transformResponse!(response, { + accessToken: 'test-token', + environmentUrl: 'https://test.crm.dynamics.com', + entitySetName: 'accounts', + recordId: 'record-1', + fileColumn: 'cr_document', + }), + name: 'download.pdf', + outputKeys: ['file', 'fileColumn'], + checkMetadata: (output) => { + expect(output.fileColumn).toBe('cr_document') + }, + }, + { + tool: personaPrintInquiryPdfTool, + transform: (response) => + personaPrintInquiryPdfTool.transformResponse!(response, { + apiKey: 'test-key', + inquiryId: 'inq_test', + }), + name: 'inq_test.pdf', + }, + { + tool: downloadAttachmentV2Tool, + transform: (response) => downloadAttachmentV2Tool.transformResponse!(response), + name: 'download.pdf', + outputKeys: ['file'], + }, + { + tool: getQrCodeV2Tool, + transform: (response) => getQrCodeV2Tool.transformResponse!(response), + name: 'qrcode.png', + mimeType: 'image/png', + outputKeys: ['file'], + }, +] + +function binaryResponse(buffer: Buffer, mimeType = 'application/pdf'): Response { + return new Response(buffer, { + headers: { + 'content-type': mimeType, + 'content-length': String(buffer.length), + 'content-disposition': 'attachment; filename="download.pdf"', + 'last-modified': 'Fri, 11 Sep 2026 12:00:00 GMT', + 'dropbox-api-result': JSON.stringify({ + id: 'file-1', + name: 'download.pdf', + size: buffer.length, + }), + 'x-ms-file-name': 'download.pdf', + 'x-ms-file-size': String(buffer.length), + }, + }) +} + +function expectBinaryFile(result: BinaryResult, buffer: Buffer, provider: DownloadCase): void { + expect(result.success).toBe(true) + const file = result.output.file as { + name: unknown + mimeType: unknown + size: unknown + data: unknown + } + expect(file.name).toBe(provider.name) + expect(file.mimeType).toBe(provider.mimeType ?? 'application/pdf') + expect(file.size).toBe(buffer.length) + expect(Buffer.isBuffer(file.data)).toBe(true) + if (!Buffer.isBuffer(file.data)) throw new Error('Expected raw file bytes') + expect(file.data.length).toBe(buffer.length) + expect(file.data.equals(buffer)).toBe(true) + expect(result.output).not.toHaveProperty('content') + expect(result.output).not.toHaveProperty('fileContent') + if (provider.outputKeys) { + expect(Object.keys(result.output).sort()).toEqual(provider.outputKeys) + } + provider.checkMetadata?.(result.output, buffer.length) +} + +beforeEach(() => { + vi.stubGlobal('fetch', vi.fn()) +}) + +afterEach(() => { + expect(fetch).not.toHaveBeenCalled() + vi.unstubAllGlobals() +}) + +describe.each(DOWNLOAD_CASES)('$tool.id binary download', (provider) => { + it('opts into the bounded binary transfer budget', () => { + expect(provider.tool.request.responseType).toBe('binary') + expect(provider.tool.outputs).not.toHaveProperty('content') + expect(provider.tool.outputs).not.toHaveProperty('fileContent') + if (provider.outputKeys) { + expect(Object.keys(provider.tool.outputs!).sort()).toEqual(provider.outputKeys) + } + }) + + it('returns raw bytes and file metadata above the former 10 MiB cap', async () => { + const buffer = Buffer.alloc(11 * 1024 * 1024, 65) + const result = await provider.transform(binaryResponse(buffer, provider.mimeType)) + expectBinaryFile(result, buffer, provider) + }) + + it('returns small downloads without inline content', async () => { + const buffer = Buffer.from('small file contents') + const result = await provider.transform(binaryResponse(buffer, provider.mimeType)) + expectBinaryFile(result, buffer, provider) + }) +}) + +const LEGACY_DOWNLOAD_CASES = [ + { tool: boxDownloadFileTool, content: 'content' }, + { tool: dropboxDownloadTool, content: 'content' }, + { tool: getQrCodeTool, content: 'content' }, + { tool: dataverseDownloadFileTool, content: 'fileContent' }, + { tool: downloadAttachmentTool, content: 'content' }, +] as const + +describe.each(LEGACY_DOWNLOAD_CASES)( + '$tool.id legacy download compatibility', + ({ tool, content }) => { + it('retains the original response budget and inline base64 contract', async () => { + expect(tool.request).not.toHaveProperty('responseType') + const buffer = Buffer.from('saved workflow content') + const result = await tool.transformResponse(binaryResponse(buffer)) + expect(result.success).toBe(true) + expect(result.output).toHaveProperty(content, buffer.toString('base64')) + expect(result.output.file?.data).toBe(buffer.toString('base64')) + expect(tool.outputs).toHaveProperty(content) + }) + } +) diff --git a/apps/sim/tools/box/download_file.ts b/apps/sim/tools/box/download_file.ts index 24366105742..20ea1ffdf46 100644 --- a/apps/sim/tools/box/download_file.ts +++ b/apps/sim/tools/box/download_file.ts @@ -1,7 +1,50 @@ -import type { ToolConfig } from '@/tools/types' -import type { BoxDownloadFileParams, BoxDownloadFileResponse } from './types' +import { omit } from '@sim/utils/object' +import type { + BoxDownloadFileParams, + BoxDownloadFileResponse, + BoxDownloadFileV2Response, +} from '@/tools/box/types' +import type { ToolConfig, ToolFileData } from '@/tools/types' -export const boxDownloadFileTool: ToolConfig = { +async function transformDownloadResponse(response: Response) { + if (response.status === 202) { + const retryAfter = response.headers.get('retry-after') || 'a few' + throw new Error(`File is not yet ready for download. Retry after ${retryAfter} seconds.`) + } + + if (!response.ok) { + const errorText = await response.text() + throw new Error(errorText || `Failed to download file: ${response.status}`) + } + + const contentType = response.headers.get('content-type') || 'application/octet-stream' + const contentDisposition = response.headers.get('content-disposition') + let fileName = 'download' + + if (contentDisposition) { + const match = contentDisposition.match(/filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/) + if (match?.[1]) { + fileName = match[1].replace(/['"]/g, '') + } + } + + const arrayBuffer = await response.arrayBuffer() + const buffer = Buffer.from(arrayBuffer) + + return { + success: true, + output: { + file: { + name: fileName, + mimeType: contentType, + data: buffer, + size: buffer.length, + }, + }, + } +} + +export const boxDownloadFileTool = { id: 'box_download_file', name: 'Box Download File', description: 'Download a file from Box', @@ -36,40 +79,15 @@ export const boxDownloadFileTool: ToolConfig { - if (response.status === 202) { - const retryAfter = response.headers.get('retry-after') || 'a few' - throw new Error(`File is not yet ready for download. Retry after ${retryAfter} seconds.`) - } - - if (!response.ok) { - const errorText = await response.text() - throw new Error(errorText || `Failed to download file: ${response.status}`) - } - - const contentType = response.headers.get('content-type') || 'application/octet-stream' - const contentDisposition = response.headers.get('content-disposition') - let fileName = 'download' - - if (contentDisposition) { - const match = contentDisposition.match(/filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/) - if (match?.[1]) { - fileName = match[1].replace(/['"]/g, '') - } - } - - const arrayBuffer = await response.arrayBuffer() - const buffer = Buffer.from(arrayBuffer) - + const result = await transformDownloadResponse(response) + const file = result.output.file + const content = file.data.toString('base64') return { - success: true, + ...result, output: { - file: { - name: fileName, - mimeType: contentType, - data: buffer.toString('base64'), - size: buffer.length, - }, - content: buffer.toString('base64'), + ...result.output, + file: { ...file, data: content }, + content, }, } }, @@ -84,4 +102,16 @@ export const boxDownloadFileTool: ToolConfig + +export const boxDownloadFileV2Tool: ToolConfig< + BoxDownloadFileParams, + BoxDownloadFileV2Response +> = { + ...boxDownloadFileTool, + id: 'box_download_file_v2', + version: '2.0.0', + request: { ...boxDownloadFileTool.request, responseType: 'binary' }, + transformResponse: transformDownloadResponse, + outputs: omit(boxDownloadFileTool.outputs, ['content']), } diff --git a/apps/sim/tools/box/index.ts b/apps/sim/tools/box/index.ts index 2e3b748fb42..a9bd6e3cbf5 100644 --- a/apps/sim/tools/box/index.ts +++ b/apps/sim/tools/box/index.ts @@ -2,7 +2,7 @@ export { boxCopyFileTool } from '@/tools/box/copy_file' export { boxCreateFolderTool } from '@/tools/box/create_folder' export { boxDeleteFileTool } from '@/tools/box/delete_file' export { boxDeleteFolderTool } from '@/tools/box/delete_folder' -export { boxDownloadFileTool } from '@/tools/box/download_file' +export { boxDownloadFileTool, boxDownloadFileV2Tool } from '@/tools/box/download_file' export { boxGetFileInfoTool } from '@/tools/box/get_file_info' export { boxListFolderItemsTool } from '@/tools/box/list_folder_items' export { boxSearchTool } from '@/tools/box/search' diff --git a/apps/sim/tools/box/types.ts b/apps/sim/tools/box/types.ts index 07368f79984..c5463b3089c 100644 --- a/apps/sim/tools/box/types.ts +++ b/apps/sim/tools/box/types.ts @@ -1,3 +1,4 @@ +import type { UserFile } from '@/executor/types' import type { OutputProperty, ToolResponse } from '@/tools/types' export interface BoxUploadFileParams { @@ -95,6 +96,12 @@ export interface BoxDownloadFileResponse extends ToolResponse { } } +export interface BoxDownloadFileV2Response extends ToolResponse { + output: Omit & { + file: File + } +} + export interface BoxFileInfoResponse extends ToolResponse { output: { id: string diff --git a/apps/sim/tools/cursor/download_artifact.test.ts b/apps/sim/tools/cursor/download_artifact.test.ts new file mode 100644 index 00000000000..dc32b3f1953 --- /dev/null +++ b/apps/sim/tools/cursor/download_artifact.test.ts @@ -0,0 +1,33 @@ +/** @vitest-environment node */ +import { describe, expect, it } from 'vitest' +import { downloadArtifactTool, downloadArtifactV2Tool } from '@/tools/cursor/download_artifact' + +describe('Cursor artifact output versions', () => { + it('preserves legacy inline metadata', async () => { + const file = { name: 'index.ts', mimeType: 'text/plain', data: 'YQ==', size: 1 } + const result = await downloadArtifactTool.transformResponse!( + Response.json({ success: true, output: { file } }) + ) + expect(result).toEqual({ + success: true, + output: { content: 'Downloaded artifact: index.ts', metadata: file }, + }) + }) + + it('preserves only the canonical stored file in v2', async () => { + const file = { + id: 'stored-file', + name: 'index.ts', + size: 12 * 1024 * 1024, + type: 'text/plain', + url: '/api/files/stored', + key: 'execution/index.ts', + context: 'execution', + } + const result = await downloadArtifactV2Tool.transformResponse!( + Response.json({ success: true, output: { file } }) + ) + expect(result).toEqual({ success: true, output: { file } }) + expect(Object.keys(downloadArtifactV2Tool.outputs!)).toEqual(['file']) + }) +}) diff --git a/apps/sim/tools/cursor/types.ts b/apps/sim/tools/cursor/types.ts index 2dfcf38fa20..31159e509bd 100644 --- a/apps/sim/tools/cursor/types.ts +++ b/apps/sim/tools/cursor/types.ts @@ -1,3 +1,4 @@ +import type { UserFile } from '@/executor/types' import type { ToolResponse } from '@/tools/types' interface BaseCursorParams { @@ -218,12 +219,7 @@ export interface DownloadArtifactResponse extends ToolResponse { export interface DownloadArtifactV2Response extends ToolResponse { output: { - file: { - name: string - mimeType: string - data: string - size: number - } + file: UserFile } } diff --git a/apps/sim/tools/daytona/download_file.ts b/apps/sim/tools/daytona/download_file.ts index 201bd6fa5af..1b2c5402840 100644 --- a/apps/sim/tools/daytona/download_file.ts +++ b/apps/sim/tools/daytona/download_file.ts @@ -40,6 +40,7 @@ export const daytonaDownloadFileTool: ToolConfig< }, request: { + responseType: 'binary', url: (params) => daytonaToolboxUrl( params.sandboxId, @@ -75,7 +76,7 @@ export const daytonaDownloadFileTool: ToolConfig< file: { name: fileName, mimeType, - data: buffer.toString('base64'), + data: buffer, size: buffer.length, }, name: fileName, diff --git a/apps/sim/tools/dropbox/download.ts b/apps/sim/tools/dropbox/download.ts index 011e1a64a83..dbd57e8894c 100644 --- a/apps/sim/tools/dropbox/download.ts +++ b/apps/sim/tools/dropbox/download.ts @@ -1,8 +1,66 @@ +import { omit } from '@sim/utils/object' import { httpHeaderSafeJson } from '@/lib/core/utils/validation' -import type { DropboxDownloadParams, DropboxDownloadResponse } from '@/tools/dropbox/types' -import type { ToolConfig } from '@/tools/types' +import type { + DropboxDownloadParams, + DropboxDownloadResponse, + DropboxDownloadV2Response, +} from '@/tools/dropbox/types' +import type { ToolConfig, ToolFileData } from '@/tools/types' -export const dropboxDownloadTool: ToolConfig = { +async function transformDownloadResponse(response: Response, params?: DropboxDownloadParams) { + if (!response.ok) { + const errorText = await response.text() + return { + success: false, + error: errorText || 'Failed to download file', + output: {}, + } + } + + const apiResultHeader = + response.headers.get('dropbox-api-result') || response.headers.get('Dropbox-API-Result') + const metadata = apiResultHeader ? JSON.parse(apiResultHeader) : undefined + const contentType = response.headers.get('content-type') || 'application/octet-stream' + const arrayBuffer = await response.arrayBuffer() + const buffer = Buffer.from(arrayBuffer) + const resolvedName = metadata?.name || params?.path?.split('/').pop() || 'download' + + let temporaryLink: string | undefined + if (params?.accessToken) { + try { + const linkResponse = await fetch('https://api.dropboxapi.com/2/files/get_temporary_link', { + method: 'POST', + headers: { + Authorization: `Bearer ${params.accessToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ path: params.path.trim() }), + }) + if (linkResponse.ok) { + const linkData = await linkResponse.json() + temporaryLink = linkData.link + } + } catch { + temporaryLink = undefined + } + } + + return { + success: true, + output: { + file: { + name: resolvedName, + mimeType: contentType, + data: buffer, + size: buffer.length, + }, + metadata, + temporaryLink, + }, + } +} + +export const dropboxDownloadTool = { id: 'dropbox_download', name: 'Dropbox Download File', description: 'Download a file from Dropbox with metadata and content', @@ -38,55 +96,16 @@ export const dropboxDownloadTool: ToolConfig { - if (!response.ok) { - const errorText = await response.text() - return { - success: false, - error: errorText || 'Failed to download file', - output: {}, - } - } - - const apiResultHeader = - response.headers.get('dropbox-api-result') || response.headers.get('Dropbox-API-Result') - const metadata = apiResultHeader ? JSON.parse(apiResultHeader) : undefined - const contentType = response.headers.get('content-type') || 'application/octet-stream' - const arrayBuffer = await response.arrayBuffer() - const buffer = Buffer.from(arrayBuffer) - const resolvedName = metadata?.name || params?.path?.split('/').pop() || 'download' - - let temporaryLink: string | undefined - if (params?.accessToken) { - try { - const linkResponse = await fetch('https://api.dropboxapi.com/2/files/get_temporary_link', { - method: 'POST', - headers: { - Authorization: `Bearer ${params.accessToken}`, - 'Content-Type': 'application/json', - }, - body: JSON.stringify({ path: params.path.trim() }), - }) - if (linkResponse.ok) { - const linkData = await linkResponse.json() - temporaryLink = linkData.link - } - } catch { - temporaryLink = undefined - } - } - + const result = await transformDownloadResponse(response, params) + if (!result.success || !result.output.file) return result + const file = result.output.file + const content = file.data.toString('base64') return { - success: true, + ...result, output: { - file: { - name: resolvedName, - mimeType: contentType, - data: buffer.toString('base64'), - size: buffer.length, - }, - content: buffer.toString('base64'), - metadata, - temporaryLink, + ...result.output, + file: { ...file, data: content }, + content, }, } }, @@ -109,4 +128,17 @@ export const dropboxDownloadTool: ToolConfig + +export const dropboxDownloadV2Tool: ToolConfig< + DropboxDownloadParams, + DropboxDownloadV2Response +> = { + ...dropboxDownloadTool, + id: 'dropbox_download_v2', + description: 'Download a file from Dropbox with metadata', + version: '2.0.0', + request: { ...dropboxDownloadTool.request, responseType: 'binary' }, + transformResponse: transformDownloadResponse, + outputs: omit(dropboxDownloadTool.outputs, ['content']), } diff --git a/apps/sim/tools/dropbox/index.ts b/apps/sim/tools/dropbox/index.ts index c113cc121af..6e9dc450113 100644 --- a/apps/sim/tools/dropbox/index.ts +++ b/apps/sim/tools/dropbox/index.ts @@ -2,7 +2,7 @@ import { dropboxCopyTool } from '@/tools/dropbox/copy' import { dropboxCreateFolderTool } from '@/tools/dropbox/create_folder' import { dropboxCreateSharedLinkTool } from '@/tools/dropbox/create_shared_link' import { dropboxDeleteTool } from '@/tools/dropbox/delete' -import { dropboxDownloadTool } from '@/tools/dropbox/download' +import { dropboxDownloadTool, dropboxDownloadV2Tool } from '@/tools/dropbox/download' import { dropboxGetMetadataTool } from '@/tools/dropbox/get_metadata' import { dropboxListFolderTool } from '@/tools/dropbox/list_folder' import { dropboxListRevisionsTool } from '@/tools/dropbox/list_revisions' @@ -18,6 +18,7 @@ export { dropboxCreateSharedLinkTool, dropboxDeleteTool, dropboxDownloadTool, + dropboxDownloadV2Tool, dropboxGetMetadataTool, dropboxListFolderTool, dropboxListRevisionsTool, diff --git a/apps/sim/tools/dropbox/types.ts b/apps/sim/tools/dropbox/types.ts index bfb5cf1c8ae..05c9594b9a4 100644 --- a/apps/sim/tools/dropbox/types.ts +++ b/apps/sim/tools/dropbox/types.ts @@ -1,4 +1,5 @@ import type { UserFileLike } from '@/lib/core/utils/user-file' +import type { UserFile } from '@/executor/types' import type { ToolFileData, ToolResponse } from '@/tools/types' interface DropboxFileMetadata { @@ -93,6 +94,12 @@ export interface DropboxDownloadResponse extends ToolResponse { } } +export interface DropboxDownloadV2Response extends ToolResponse { + output: Omit & { + file?: File + } +} + export interface DropboxListFolderParams extends DropboxBaseParams { path: string recursive?: boolean diff --git a/apps/sim/tools/dub/get_qr_code.ts b/apps/sim/tools/dub/get_qr_code.ts index 443ca643e38..24bf57d60a1 100644 --- a/apps/sim/tools/dub/get_qr_code.ts +++ b/apps/sim/tools/dub/get_qr_code.ts @@ -1,7 +1,42 @@ -import type { DubGetQrCodeParams, DubGetQrCodeResponse } from '@/tools/dub/types' -import type { ToolConfig } from '@/tools/types' +import { omit } from '@sim/utils/object' +import type { + DubGetQrCodeParams, + DubGetQrCodeResponse, + DubGetQrCodeV2Response, +} from '@/tools/dub/types' +import type { ToolConfig, ToolFileData } from '@/tools/types' -export const getQrCodeTool: ToolConfig = { +async function transformDownloadResponse(response: Response) { + if (!response.ok) { + const errorText = await response.text() + let message = errorText || `Failed to generate QR code: ${response.status}` + try { + const parsed = JSON.parse(errorText) + message = parsed.error?.message || parsed.error || message + } catch { + /** Non-JSON error body; use the raw text. */ + } + throw new Error(message) + } + + const arrayBuffer = await response.arrayBuffer() + const buffer = Buffer.from(arrayBuffer) + const mimeType = response.headers.get('content-type') || 'image/png' + + return { + success: true, + output: { + file: { + name: 'qrcode.png', + mimeType, + data: buffer, + size: buffer.length, + }, + }, + } +} + +export const getQrCodeTool = { id: 'dub_get_qr_code', name: 'Dub Get QR Code', description: @@ -85,33 +120,16 @@ export const getQrCodeTool: ToolConfig }), }, - transformResponse: async (response: Response) => { - if (!response.ok) { - const errorText = await response.text() - let message = errorText || `Failed to generate QR code: ${response.status}` - try { - const parsed = JSON.parse(errorText) - message = parsed.error?.message || parsed.error || message - } catch { - // Non-JSON error body; use the raw text - } - throw new Error(message) - } - - const arrayBuffer = await response.arrayBuffer() - const buffer = Buffer.from(arrayBuffer) - const mimeType = response.headers.get('content-type') || 'image/png' - + transformResponse: async (response) => { + const result = await transformDownloadResponse(response) + const file = result.output.file + const content = file.data.toString('base64') return { - success: true, + ...result, output: { - file: { - name: 'qrcode.png', - mimeType, - data: buffer.toString('base64'), - size: buffer.length, - }, - content: buffer.toString('base64'), + ...result.output, + file: { ...file, data: content }, + content, }, } }, @@ -126,4 +144,16 @@ export const getQrCodeTool: ToolConfig description: 'Base64-encoded PNG image data', }, }, +} satisfies ToolConfig + +export const getQrCodeV2Tool: ToolConfig< + DubGetQrCodeParams, + DubGetQrCodeV2Response +> = { + ...getQrCodeTool, + id: 'dub_get_qr_code_v2', + version: '2.0.0', + request: { ...getQrCodeTool.request, responseType: 'binary' }, + transformResponse: transformDownloadResponse, + outputs: omit(getQrCodeTool.outputs, ['content']), } diff --git a/apps/sim/tools/dub/index.ts b/apps/sim/tools/dub/index.ts index 09030412b92..7b7b957873f 100644 --- a/apps/sim/tools/dub/index.ts +++ b/apps/sim/tools/dub/index.ts @@ -8,7 +8,7 @@ import { getAnalyticsTool } from '@/tools/dub/get_analytics' import { getEventsTool } from '@/tools/dub/get_events' import { getLinkTool } from '@/tools/dub/get_link' import { getLinksCountTool } from '@/tools/dub/get_links_count' -import { getQrCodeTool } from '@/tools/dub/get_qr_code' +import { getQrCodeTool, getQrCodeV2Tool } from '@/tools/dub/get_qr_code' import { listDomainsTool } from '@/tools/dub/list_domains' import { listFoldersTool } from '@/tools/dub/list_folders' import { listLinksTool } from '@/tools/dub/list_links' @@ -29,6 +29,7 @@ export const dubBulkCreateLinksTool = bulkCreateLinksTool export const dubBulkUpdateLinksTool = bulkUpdateLinksTool export const dubBulkDeleteLinksTool = bulkDeleteLinksTool export const dubGetQrCodeTool = getQrCodeTool +export const dubGetQrCodeV2Tool = getQrCodeV2Tool export const dubListDomainsTool = listDomainsTool export const dubListTagsTool = listTagsTool export const dubCreateTagTool = createTagTool diff --git a/apps/sim/tools/dub/types.ts b/apps/sim/tools/dub/types.ts index 6270a26f3f8..9fea3443f39 100644 --- a/apps/sim/tools/dub/types.ts +++ b/apps/sim/tools/dub/types.ts @@ -1,3 +1,4 @@ +import type { UserFile } from '@/executor/types' import type { ToolResponse } from '@/tools/types' interface DubBaseParams { @@ -303,6 +304,12 @@ export interface DubGetQrCodeResponse extends ToolResponse { } } +export interface DubGetQrCodeV2Response extends ToolResponse { + output: Omit & { + file: File + } +} + export interface DubListDomainsResponse extends ToolResponse { output: { domains: Record[] diff --git a/apps/sim/tools/file-message-operation-security.test.ts b/apps/sim/tools/file-message-operation-security.test.ts index 7015ce8ce55..5dfd134da78 100644 --- a/apps/sim/tools/file-message-operation-security.test.ts +++ b/apps/sim/tools/file-message-operation-security.test.ts @@ -1,7 +1,7 @@ /** * @vitest-environment node */ -import { beforeEach, describe, expect, it, vi } from 'vitest' +import { assert, beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ assertToolFileAccess: vi.fn(), @@ -68,6 +68,7 @@ import { executeDataverseUploadFile } from '@/lib/internal/microsoft-dataverse/o import { executePipedriveGetFiles } from '@/lib/internal/pipedrive/operations' import type { ServiceNowOperationError } from '@/lib/internal/servicenow/errors' import { executeServiceNowUploadAttachment } from '@/lib/internal/servicenow/operations' +import { isInternalToolFileResult } from '@/lib/internal/tool-operations/file-result' import { MAX_BUFFERED_TRANSFER_BYTES } from '@/lib/uploads/shared/types' const FILE = { @@ -248,6 +249,25 @@ describe('file and message operation security', () => { MAX_BUFFERED_TRANSFER_BYTES, MAX_BUFFERED_TRANSFER_BYTES - 3, ]) - expect(result.output.downloadedFiles).toHaveLength(2) + assert(isInternalToolFileResult(result)) + expect(result.files).toEqual([ + { name: 'one.txt', mimeType: 'text/plain', buffer: Buffer.from('abc') }, + { name: 'two.txt', mimeType: 'text/plain', buffer: Buffer.from('defg') }, + ]) + const storedFiles = [ + { ...FILE, name: 'one.txt', mimeType: 'text/plain' }, + { + ...FILE, + id: 'file-2', + key: 'execution/file-2', + name: 'two.txt', + size: 4, + mimeType: 'text/plain', + }, + ] + expect(result.present(storedFiles)).toMatchObject({ + success: true, + output: { downloadedFiles: storedFiles, has_more: true, next_start: 2 }, + }) }) }) diff --git a/apps/sim/tools/file/parser.test.ts b/apps/sim/tools/file/parser.test.ts index 8a0e3e1ac91..18f794e0228 100644 --- a/apps/sim/tools/file/parser.test.ts +++ b/apps/sim/tools/file/parser.test.ts @@ -2,9 +2,20 @@ * @vitest-environment node */ import { describe, expect, it } from 'vitest' -import { fileFetchTool, fileParserTool, fileParserV3Tool } from '@/tools/file/parser' +import { + fileFetchTool, + fileParserTool, + fileParserV2Tool, + fileParserV3Tool, +} from '@/tools/file/parser' describe('fileParserTool', () => { + it.each([fileFetchTool, fileParserTool, fileParserV2Tool, fileParserV3Tool])( + '$id negotiates stored source provenance before exposing parsed content', + (tool) => { + expect(tool.operation.secretProvenance?.response).toEqual({ incomplete: 'reject' }) + } + ) it('maps the public File Fetch URL to the internal parser path', () => { expect( fileFetchTool.operation.input({ diff --git a/apps/sim/tools/file/parser.ts b/apps/sim/tools/file/parser.ts index fb4d8cf869f..a17a1c9a03c 100644 --- a/apps/sim/tools/file/parser.ts +++ b/apps/sim/tools/file/parser.ts @@ -197,6 +197,7 @@ export const fileParserTool: InternalToolConfig { logger.info('Request parameters received by tool body:', params) @@ -384,6 +385,7 @@ export const fileFetchTool: InternalToolConfig fileParserTool.operation.input({ ...params, diff --git a/apps/sim/tools/generated/tool-ids.ts b/apps/sim/tools/generated/tool-ids.ts index ca9a23f2d24..67aa132eb1a 100644 --- a/apps/sim/tools/generated/tool-ids.ts +++ b/apps/sim/tools/generated/tool-ids.ts @@ -3,7 +3,7 @@ /** Every registered tool id, including versioned variants. */ const toolIds: string[] = JSON.parse( - '["a2a_cancel_task","a2a_get_agent_card","a2a_get_task","a2a_send_message","affinity_batch_update_entity_fields","affinity_batch_update_list_entry_fields","affinity_create_list","affinity_create_list_field_dropdown_option","affinity_create_merge","affinity_create_note","affinity_create_reminder","affinity_delete_list_field_dropdown_option","affinity_delete_note","affinity_get_company","affinity_get_current_user","affinity_get_entity_field_value","affinity_get_list","affinity_get_list_entry","affinity_get_list_entry_field","affinity_get_list_field_dropdown_option","affinity_get_merge","affinity_get_merge_task","affinity_get_note","affinity_get_opportunity","affinity_get_person","affinity_get_saved_view","affinity_get_transcript","affinity_get_user","affinity_list_calls","affinity_list_chat_messages","affinity_list_companies","affinity_list_coworker_connections","affinity_list_emails","affinity_list_entity_field_values","affinity_list_entity_list_entries","affinity_list_entity_lists","affinity_list_entity_notes","affinity_list_entity_relationships","affinity_list_field_dropdown_options","affinity_list_field_metadata","affinity_list_field_value_changes","affinity_list_investor_executive_connections","affinity_list_list_entries","affinity_list_list_entry_field_value_changes","affinity_list_list_entry_fields","affinity_list_list_field_dropdown_options","affinity_list_list_fields","affinity_list_lists","affinity_list_meetings","affinity_list_merge_tasks","affinity_list_merges","affinity_list_note_attached_companies","affinity_list_note_attached_opportunities","affinity_list_note_attached_persons","affinity_list_note_replies","affinity_list_notes","affinity_list_opportunities","affinity_list_persons","affinity_list_reminders","affinity_list_saved_view_entries","affinity_list_saved_views","affinity_list_transcript_fragments","affinity_list_transcripts","affinity_list_users","affinity_search_companies","affinity_search_files","affinity_search_list_entries","affinity_search_notes","affinity_search_persons","affinity_semantic_search","affinity_update_entity_field_value","affinity_update_list_entry_field","affinity_update_list_field_dropdown_option","affinity_update_note","agentmail_create_draft","agentmail_create_inbox","agentmail_delete_draft","agentmail_delete_inbox","agentmail_delete_thread","agentmail_forward_message","agentmail_get_draft","agentmail_get_inbox","agentmail_get_message","agentmail_get_thread","agentmail_list_drafts","agentmail_list_inboxes","agentmail_list_messages","agentmail_list_threads","agentmail_reply_message","agentmail_send_draft","agentmail_send_message","agentmail_update_draft","agentmail_update_inbox","agentmail_update_message","agentmail_update_thread","agentphone_create_call","agentphone_create_contact","agentphone_create_number","agentphone_delete_contact","agentphone_get_call","agentphone_get_call_transcript","agentphone_get_contact","agentphone_get_conversation","agentphone_get_conversation_messages","agentphone_get_number_messages","agentphone_get_usage","agentphone_get_usage_daily","agentphone_get_usage_monthly","agentphone_list_calls","agentphone_list_contacts","agentphone_list_conversations","agentphone_list_numbers","agentphone_react_to_message","agentphone_release_number","agentphone_send_message","agentphone_update_contact","agentphone_update_conversation","agiloft_async_status","agiloft_attach_file","agiloft_attachment_info","agiloft_create_record","agiloft_delete_record","agiloft_get_choice_line_id","agiloft_list_tables","agiloft_lock_record","agiloft_nlp_search","agiloft_read_record","agiloft_remove_attachment","agiloft_retrieve_attachment","agiloft_run_action_button","agiloft_saved_search","agiloft_search_records","agiloft_select_records","agiloft_update_record","agiloft_upsert_record","ahrefs_anchors","ahrefs_backlinks","ahrefs_backlinks_stats","ahrefs_batch_analysis","ahrefs_broken_backlinks","ahrefs_domain_rating","ahrefs_domain_rating_history","ahrefs_keyword_overview","ahrefs_keywords_history","ahrefs_metrics","ahrefs_metrics_history","ahrefs_organic_competitors","ahrefs_organic_keywords","ahrefs_paid_pages","ahrefs_rank_tracker_competitors_overview","ahrefs_rank_tracker_competitors_stats","ahrefs_rank_tracker_overview","ahrefs_rank_tracker_serp_overview","ahrefs_refdomains_history","ahrefs_referring_domains","ahrefs_related_terms","ahrefs_site_audit_page_explorer","ahrefs_top_pages","airtable_create_records","airtable_delete_records","airtable_get_base_schema","airtable_get_record","airtable_list_bases","airtable_list_records","airtable_list_tables","airtable_update_multiple_records","airtable_update_record","airtable_upsert_records","airweave_search","algolia_add_record","algolia_batch_operations","algolia_browse_records","algolia_clear_records","algolia_copy_move_index","algolia_delete_by_filter","algolia_delete_index","algolia_delete_record","algolia_get_record","algolia_get_records","algolia_get_settings","algolia_get_task_status","algolia_list_indices","algolia_partial_update_record","algolia_search","algolia_update_settings","amplitude_event_segmentation","amplitude_funnels","amplitude_get_active_users","amplitude_get_revenue","amplitude_group_identify","amplitude_identify_user","amplitude_list_events","amplitude_realtime_active_users","amplitude_retention","amplitude_send_event","amplitude_user_activity","amplitude_user_profile","amplitude_user_search","apify_get_dataset_items","apify_get_run","apify_run_actor_async","apify_run_actor_sync","apify_run_task","apollo_account_bulk_create","apollo_account_bulk_update","apollo_account_create","apollo_account_search","apollo_account_update","apollo_contact_bulk_create","apollo_contact_bulk_update","apollo_contact_create","apollo_contact_search","apollo_contact_update","apollo_email_accounts","apollo_opportunity_create","apollo_opportunity_get","apollo_opportunity_search","apollo_opportunity_update","apollo_organization_bulk_enrich","apollo_organization_enrich","apollo_organization_search","apollo_people_bulk_enrich","apollo_people_enrich","apollo_people_search","apollo_sequence_add_contacts","apollo_sequence_search","apollo_task_create","apollo_task_search","appconfig_create_application","appconfig_create_configuration_profile","appconfig_create_environment","appconfig_create_hosted_configuration_version","appconfig_delete_application","appconfig_delete_configuration_profile","appconfig_delete_environment","appconfig_delete_hosted_configuration_version","appconfig_get_application","appconfig_get_configuration","appconfig_get_configuration_profile","appconfig_get_deployment","appconfig_get_environment","appconfig_get_hosted_configuration_version","appconfig_list_applications","appconfig_list_configuration_profiles","appconfig_list_deployment_strategies","appconfig_list_deployments","appconfig_list_environments","appconfig_list_hosted_configuration_versions","appconfig_start_deployment","appconfig_stop_deployment","appconfig_update_application","appconfig_update_configuration_profile","appconfig_update_environment","arxiv_get_author_papers","arxiv_get_paper","arxiv_search","asana_add_comment","asana_add_followers","asana_create_project","asana_create_section","asana_create_subtask","asana_create_task","asana_delete_task","asana_get_project","asana_get_projects","asana_get_task","asana_list_sections","asana_list_workspaces","asana_search_tasks","asana_update_task","ashby_add_candidate_tag","ashby_anonymize_candidate","ashby_change_application_source","ashby_change_application_stage","ashby_create_application","ashby_create_candidate","ashby_create_note","ashby_delete_application","ashby_get_application","ashby_get_candidate","ashby_get_job","ashby_get_job_posting","ashby_get_offer","ashby_get_opening","ashby_list_application_feedback","ashby_list_application_history","ashby_list_applications","ashby_list_archive_reasons","ashby_list_candidate_tags","ashby_list_candidates","ashby_list_custom_fields","ashby_list_departments","ashby_list_interview_plans","ashby_list_interview_stages","ashby_list_interviews","ashby_list_job_postings","ashby_list_jobs","ashby_list_locations","ashby_list_notes","ashby_list_offers","ashby_list_openings","ashby_list_sources","ashby_list_users","ashby_remove_candidate_tag","ashby_search_candidates","ashby_search_jobs","ashby_search_openings","ashby_search_users","ashby_set_custom_field_value","ashby_set_custom_field_values","ashby_transfer_application","ashby_update_candidate","ashby_upload_candidate_file","ashby_upload_resume","athena_batch_get_named_query","athena_batch_get_prepared_statement","athena_batch_get_query_execution","athena_create_named_query","athena_create_prepared_statement","athena_delete_named_query","athena_delete_prepared_statement","athena_get_data_catalog","athena_get_database","athena_get_named_query","athena_get_prepared_statement","athena_get_query_execution","athena_get_query_results","athena_get_query_runtime_statistics","athena_get_table_metadata","athena_get_work_group","athena_list_data_catalogs","athena_list_databases","athena_list_named_queries","athena_list_prepared_statements","athena_list_query_executions","athena_list_table_metadata","athena_list_work_groups","athena_start_query","athena_stop_query","athena_update_named_query","athena_update_prepared_statement","attio_assert_record","attio_create_attribute","attio_create_comment","attio_create_list","attio_create_list_entry","attio_create_note","attio_create_object","attio_create_record","attio_create_task","attio_create_webhook","attio_delete_comment","attio_delete_list_entry","attio_delete_note","attio_delete_record","attio_delete_task","attio_delete_webhook","attio_get_attribute","attio_get_comment","attio_get_list","attio_get_list_entry","attio_get_member","attio_get_note","attio_get_object","attio_get_record","attio_get_task","attio_get_thread","attio_get_webhook","attio_list_attributes","attio_list_lists","attio_list_members","attio_list_notes","attio_list_objects","attio_list_records","attio_list_tasks","attio_list_threads","attio_list_webhooks","attio_query_list_entries","attio_search_records","attio_update_attribute","attio_update_list","attio_update_list_entry","attio_update_object","attio_update_record","attio_update_task","attio_update_webhook","azure_data_explorer_create_table","azure_data_explorer_drop_table","azure_data_explorer_ingest_from_query","azure_data_explorer_ingest_inline","azure_data_explorer_list_databases","azure_data_explorer_list_functions","azure_data_explorer_list_tables","azure_data_explorer_management","azure_data_explorer_query","azure_data_explorer_show_database_schema","azure_data_explorer_show_ingestion_failures","azure_data_explorer_show_operations","azure_data_explorer_show_table_details","azure_data_explorer_show_table_schema","azure_devops_add_comment","azure_devops_create_work_item","azure_devops_get_build_log","azure_devops_get_build_timeline","azure_devops_get_comments","azure_devops_get_pipeline","azure_devops_get_pipeline_run","azure_devops_get_work_item","azure_devops_get_work_items_batch","azure_devops_get_work_items_between_builds","azure_devops_list_build_logs","azure_devops_list_builds","azure_devops_list_pipeline_runs","azure_devops_list_pipelines","azure_devops_query_work_items","azure_devops_update_work_item","bitbucket_approve_pull_request","bitbucket_create_branch","bitbucket_create_pull_request","bitbucket_create_pull_request_comment","bitbucket_decline_pull_request","bitbucket_delete_branch","bitbucket_get_commit","bitbucket_get_file","bitbucket_get_file_metadata","bitbucket_get_pipeline","bitbucket_get_pipeline_step_log","bitbucket_get_pull_request","bitbucket_get_pull_request_diff","bitbucket_get_pull_request_diffstat","bitbucket_get_pull_request_merge_task_status","bitbucket_get_repository","bitbucket_list_branches","bitbucket_list_commits","bitbucket_list_directory","bitbucket_list_pipeline_steps","bitbucket_list_pipelines","bitbucket_list_pull_request_comments","bitbucket_list_pull_request_commit_statuses","bitbucket_list_pull_requests","bitbucket_list_repositories","bitbucket_list_workspaces","bitbucket_merge_pull_request","bitbucket_request_pull_request_changes","bitbucket_stop_pipeline","bitbucket_trigger_pipeline","box_copy_file","box_create_folder","box_delete_file","box_delete_folder","box_download_file","box_get_file_info","box_list_folder_items","box_search","box_sign_cancel_request","box_sign_create_request","box_sign_get_request","box_sign_list_requests","box_sign_resend_request","box_update_file","box_upload_file","brandfetch_get_brand","brandfetch_search","brex_archive_budget","brex_create_budget","brex_create_spend_limit","brex_create_transfer","brex_create_vendor","brex_get_budget","brex_get_cash_account","brex_get_company","brex_get_current_user","brex_get_expense","brex_get_spend_limit","brex_get_transfer","brex_get_user","brex_get_vendor","brex_list_budgets","brex_list_card_accounts","brex_list_card_statements","brex_list_card_transactions","brex_list_cards","brex_list_cash_accounts","brex_list_cash_statements","brex_list_cash_transactions","brex_list_departments","brex_list_expenses","brex_list_locations","brex_list_spend_limits","brex_list_titles","brex_list_transfers","brex_list_users","brex_list_vendors","brex_match_receipt","brex_update_expense","brex_update_vendor","brex_upload_receipt","brightdata_cancel_snapshot","brightdata_discover","brightdata_download_snapshot","brightdata_scrape_dataset","brightdata_scrape_url","brightdata_serp_search","brightdata_snapshot_status","brightdata_sync_scrape","browser_use_run_task","buffer_create_idea","buffer_create_post","buffer_delete_post","buffer_edit_post","buffer_get_account","buffer_get_channels","buffer_get_idea_groups","buffer_get_ideas","buffer_get_post","buffer_get_posts","calcom_cancel_booking","calcom_confirm_booking","calcom_create_booking","calcom_create_event_type","calcom_create_schedule","calcom_decline_booking","calcom_delete_event_type","calcom_delete_schedule","calcom_get_booking","calcom_get_default_schedule","calcom_get_event_type","calcom_get_schedule","calcom_get_slots","calcom_list_bookings","calcom_list_event_types","calcom_list_schedules","calcom_reschedule_booking","calcom_update_event_type","calcom_update_schedule","calendly_cancel_event","calendly_create_event_invitee","calendly_create_invitee_no_show","calendly_create_scheduling_link","calendly_create_webhook","calendly_delete_invitee_no_show","calendly_delete_webhook","calendly_get_current_user","calendly_get_event_invitee","calendly_get_event_type","calendly_get_scheduled_event","calendly_get_user","calendly_list_event_invitees","calendly_list_event_type_available_times","calendly_list_event_types","calendly_list_organization_memberships","calendly_list_routing_form_submissions","calendly_list_routing_forms","calendly_list_scheduled_events","calendly_list_user_availability_schedules","calendly_list_user_busy_times","calendly_list_webhooks","cbinsights_chat","cbinsights_get_commercial_maturity_history","cbinsights_get_exit_probability_history","cbinsights_get_mosaic_history","cbinsights_get_org_business_relationships","cbinsights_get_org_funding_window","cbinsights_get_org_fundings","cbinsights_get_org_investments","cbinsights_get_org_management_and_board","cbinsights_get_org_outlook","cbinsights_get_org_portfolio_exits","cbinsights_get_org_revenue","cbinsights_get_scouting_report","cbinsights_get_strategy_map","cbinsights_list_business_relationships","cbinsights_list_funding_window","cbinsights_list_fundings","cbinsights_list_investments","cbinsights_list_management_and_board","cbinsights_list_outlook","cbinsights_list_portfolio_exits","cbinsights_list_revenue","cbinsights_lookup_organizations","cbinsights_rag","cbinsights_search_firmographics","circleback_add_tag_to_meetings","circleback_create_tag","circleback_delete_action_item","circleback_delete_meeting","circleback_delete_tag","circleback_get_company","circleback_get_meeting","circleback_get_person","circleback_get_transcript","circleback_list_action_items","circleback_list_calendar_events","circleback_list_companies","circleback_list_meetings","circleback_list_people","circleback_list_tags","circleback_remove_tag_from_meetings","circleback_search_meetings","circleback_update_action_item","circleback_update_meeting","circleback_update_tag","clay_populate","clerk_add_organization_member","clerk_ban_user","clerk_create_actor_token","clerk_create_allowlist_identifier","clerk_create_blocklist_identifier","clerk_create_organization","clerk_create_organization_invitation","clerk_create_user","clerk_delete_allowlist_identifier","clerk_delete_blocklist_identifier","clerk_delete_organization","clerk_delete_user","clerk_get_jwt_template","clerk_get_organization","clerk_get_session","clerk_get_user","clerk_get_user_oauth_token","clerk_list_allowlist_identifiers","clerk_list_blocklist_identifiers","clerk_list_jwt_templates","clerk_list_organization_invitations","clerk_list_organization_memberships","clerk_list_organizations","clerk_list_sessions","clerk_list_users","clerk_lock_user","clerk_remove_organization_member","clerk_revoke_actor_token","clerk_revoke_session","clerk_unban_user","clerk_unlock_user","clerk_update_organization","clerk_update_organization_membership","clerk_update_user","clickhouse_count_rows","clickhouse_create_database","clickhouse_create_table","clickhouse_delete","clickhouse_describe_table","clickhouse_drop_database","clickhouse_drop_partition","clickhouse_drop_table","clickhouse_execute","clickhouse_insert","clickhouse_insert_rows","clickhouse_introspect","clickhouse_kill_query","clickhouse_list_clusters","clickhouse_list_databases","clickhouse_list_mutations","clickhouse_list_partitions","clickhouse_list_running_queries","clickhouse_list_tables","clickhouse_optimize_table","clickhouse_query","clickhouse_rename_table","clickhouse_show_create_table","clickhouse_table_stats","clickhouse_truncate_table","clickhouse_update","clickup_add_tag_to_task","clickup_create_checklist","clickup_create_checklist_item","clickup_create_comment","clickup_create_folder","clickup_create_list","clickup_create_task","clickup_create_time_entry","clickup_delete_checklist","clickup_delete_checklist_item","clickup_delete_comment","clickup_delete_task","clickup_delete_time_entry","clickup_get_comments","clickup_get_custom_fields","clickup_get_folders","clickup_get_list_members","clickup_get_lists","clickup_get_running_timer","clickup_get_space_tags","clickup_get_spaces","clickup_get_task","clickup_get_task_members","clickup_get_tasks","clickup_get_time_entries","clickup_get_workspaces","clickup_remove_custom_field_value","clickup_remove_tag_from_task","clickup_search_tasks","clickup_set_custom_field_value","clickup_start_timer","clickup_stop_timer","clickup_update_checklist","clickup_update_checklist_item","clickup_update_comment","clickup_update_task","clickup_update_time_entry","clickup_upload_attachment","cloudflare_create_access_application","cloudflare_create_access_policy","cloudflare_create_access_service_token","cloudflare_create_dns_record","cloudflare_create_r2_bucket","cloudflare_create_rate_limit_rule","cloudflare_create_ruleset","cloudflare_create_ruleset_rule","cloudflare_create_zone","cloudflare_delete_access_application","cloudflare_delete_access_policy","cloudflare_delete_dns_record","cloudflare_delete_r2_bucket","cloudflare_delete_ruleset_rule","cloudflare_delete_zone","cloudflare_dns_analytics","cloudflare_get_access_application","cloudflare_get_r2_bucket","cloudflare_get_ruleset","cloudflare_get_ruleset_entrypoint","cloudflare_get_tunnel","cloudflare_get_tunnel_configuration","cloudflare_get_worker_script_settings","cloudflare_get_zone","cloudflare_get_zone_settings","cloudflare_list_access_applications","cloudflare_list_access_groups","cloudflare_list_access_identity_providers","cloudflare_list_access_policies","cloudflare_list_access_service_tokens","cloudflare_list_certificates","cloudflare_list_dns_records","cloudflare_list_managed_ruleset_overrides","cloudflare_list_r2_buckets","cloudflare_list_rate_limit_rules","cloudflare_list_rulesets","cloudflare_list_tunnels","cloudflare_list_worker_routes","cloudflare_list_worker_scripts","cloudflare_list_zones","cloudflare_purge_cache","cloudflare_revoke_access_service_token","cloudflare_update_access_application","cloudflare_update_access_policy","cloudflare_update_dns_record","cloudflare_update_rate_limit_rule","cloudflare_update_ruleset_rule","cloudflare_update_zone_setting","cloudformation_cancel_update_stack","cloudformation_create_change_set","cloudformation_create_stack","cloudformation_delete_stack","cloudformation_describe_change_set","cloudformation_describe_stack_drift_detection_status","cloudformation_describe_stack_events","cloudformation_describe_stacks","cloudformation_detect_stack_drift","cloudformation_execute_change_set","cloudformation_get_template","cloudformation_get_template_summary","cloudformation_list_stack_resources","cloudformation_update_stack","cloudformation_validate_template","cloudtrail_cancel_query","cloudtrail_describe_query","cloudtrail_describe_trails","cloudtrail_get_event_data_store","cloudtrail_get_event_selectors","cloudtrail_get_insight_selectors","cloudtrail_get_query_results","cloudtrail_get_trail","cloudtrail_get_trail_status","cloudtrail_list_event_data_stores","cloudtrail_list_tags","cloudtrail_list_trails","cloudtrail_lookup_events","cloudtrail_start_query","cloudwatch_describe_alarm_history","cloudwatch_describe_alarms","cloudwatch_describe_log_groups","cloudwatch_describe_log_streams","cloudwatch_filter_log_events","cloudwatch_get_log_events","cloudwatch_get_metric_statistics","cloudwatch_list_metrics","cloudwatch_mute_alarm","cloudwatch_put_log_group_retention","cloudwatch_put_metric_data","cloudwatch_query_logs","cloudwatch_unmute_alarm","codepipeline_disable_stage_transition","codepipeline_enable_stage_transition","codepipeline_get_pipeline","codepipeline_get_pipeline_execution","codepipeline_get_pipeline_state","codepipeline_list_action_executions","codepipeline_list_pipeline_executions","codepipeline_list_pipelines","codepipeline_put_approval_result","codepipeline_retry_stage_execution","codepipeline_start_execution","codepipeline_stop_execution","confluence_add_label","confluence_create_blogpost","confluence_create_comment","confluence_create_page","confluence_create_page_property","confluence_create_space","confluence_create_space_property","confluence_delete_attachment","confluence_delete_blogpost","confluence_delete_comment","confluence_delete_label","confluence_delete_page","confluence_delete_page_property","confluence_delete_space","confluence_delete_space_property","confluence_get_blogpost","confluence_get_page_ancestors","confluence_get_page_children","confluence_get_page_descendants","confluence_get_page_version","confluence_get_pages_by_label","confluence_get_space","confluence_get_task","confluence_get_user","confluence_list_attachments","confluence_list_blogposts","confluence_list_blogposts_in_space","confluence_list_comments","confluence_list_labels","confluence_list_page_properties","confluence_list_page_versions","confluence_list_pages_in_space","confluence_list_space_labels","confluence_list_space_permissions","confluence_list_space_properties","confluence_list_spaces","confluence_list_tasks","confluence_retrieve","confluence_search","confluence_search_in_space","confluence_update","confluence_update_blogpost","confluence_update_comment","confluence_update_space","confluence_update_task","confluence_upload_attachment","context_dev_classify_naics","context_dev_classify_sic","context_dev_crawl","context_dev_extract","context_dev_extract_product","context_dev_extract_products","context_dev_get_brand","context_dev_get_brand_by_email","context_dev_get_brand_by_name","context_dev_get_brand_by_ticker","context_dev_identify_transaction","context_dev_map","context_dev_scrape_fonts","context_dev_scrape_html","context_dev_scrape_images","context_dev_scrape_markdown","context_dev_scrape_styleguide","context_dev_screenshot","context_dev_search","convex_action","convex_document_deltas","convex_list_documents","convex_list_tables","convex_mutation","convex_query","convex_run_function","crowdstrike_create_indicators","crowdstrike_delete_indicators","crowdstrike_delete_rtr_session","crowdstrike_execute_rtr_command","crowdstrike_get_alert_details","crowdstrike_get_case_details","crowdstrike_get_host_group_details","crowdstrike_get_indicator_details","crowdstrike_get_rtr_command_status","crowdstrike_get_sensor_aggregates","crowdstrike_get_sensor_details","crowdstrike_get_vulnerability_details","crowdstrike_init_rtr_session","crowdstrike_perform_host_action","crowdstrike_perform_host_group_action","crowdstrike_query_alerts","crowdstrike_query_cases","crowdstrike_query_host_groups","crowdstrike_query_indicators","crowdstrike_query_sensors","crowdstrike_query_vulnerabilities","crowdstrike_update_alerts","crowdstrike_update_indicators","crunchbase_autocomplete","crunchbase_get_acquisition","crunchbase_get_entity","crunchbase_get_entity_card","crunchbase_get_fields_metadata","crunchbase_get_funding_round","crunchbase_get_organization","crunchbase_get_person","crunchbase_list_deleted_entities","crunchbase_search_acquisitions","crunchbase_search_entities","crunchbase_search_funding_rounds","crunchbase_search_organizations","crunchbase_search_people","cursor_add_followup","cursor_add_followup_v2","cursor_delete_agent","cursor_delete_agent_v2","cursor_download_artifact","cursor_download_artifact_v2","cursor_get_agent","cursor_get_agent_v2","cursor_get_api_key_info","cursor_get_api_key_info_v2","cursor_get_conversation","cursor_get_conversation_v2","cursor_launch_agent","cursor_launch_agent_v2","cursor_list_agents","cursor_list_agents_v2","cursor_list_artifacts","cursor_list_artifacts_v2","cursor_list_models","cursor_list_models_v2","cursor_list_repositories","cursor_list_repositories_v2","cursor_stop_agent","cursor_stop_agent_v2","dagster_delete_run","dagster_get_asset","dagster_get_run","dagster_get_run_logs","dagster_launch_run","dagster_list_assets","dagster_list_jobs","dagster_list_runs","dagster_list_schedules","dagster_list_sensors","dagster_materialize_assets","dagster_reexecute_run","dagster_report_asset_materialization","dagster_start_schedule","dagster_start_sensor","dagster_stop_schedule","dagster_stop_sensor","dagster_terminate_run","dagster_wipe_asset","databricks_cancel_run","databricks_execute_sql","databricks_get_cluster","databricks_get_job","databricks_get_run","databricks_get_run_output","databricks_get_statement","databricks_list_clusters","databricks_list_jobs","databricks_list_runs","databricks_list_warehouses","databricks_run_job","datadog_add_incident_todo","datadog_cancel_downtime","datadog_create_dashboard","datadog_create_downtime","datadog_create_event","datadog_create_incident","datadog_create_monitor","datadog_create_slo","datadog_delete_dashboard","datadog_delete_slo","datadog_get_browser_synthetics_results","datadog_get_dashboard","datadog_get_incident","datadog_get_monitor","datadog_get_security_signal","datadog_get_slo","datadog_get_slo_history","datadog_get_synthetics_results","datadog_get_synthetics_test","datadog_list_dashboards","datadog_list_downtimes","datadog_list_incidents","datadog_list_monitors","datadog_list_security_rules","datadog_list_security_signals","datadog_list_services","datadog_list_slos","datadog_list_synthetics_tests","datadog_mute_monitor","datadog_query_logs","datadog_query_timeseries","datadog_search_spans","datadog_send_logs","datadog_submit_metrics","datadog_trigger_synthetics_tests","datadog_unmute_monitor","datadog_update_incident","datadog_update_security_signal_assignee","datadog_update_security_signal_state","datadog_update_slo","datadog_update_synthetics_status","datagma_enrich_company","datagma_enrich_person","datagma_find_email","datagma_find_phone","datagma_get_credits","daytona_create_sandbox","daytona_delete_sandbox","daytona_download_file","daytona_execute_command","daytona_get_sandbox","daytona_git_clone","daytona_list_files","daytona_list_sandboxes","daytona_run_code","daytona_start_sandbox","daytona_stop_sandbox","daytona_upload_file","deployed_block_executor","deployments_deploy","deployments_get_version","deployments_list_versions","deployments_promote","deployments_undeploy","devin_append_session_tags","devin_archive_session","devin_create_session","devin_get_session","devin_get_session_tags","devin_list_session_attachments","devin_list_session_messages","devin_list_sessions","devin_replace_session_tags","devin_send_message","devin_terminate_session","discord_add_reaction","discord_archive_thread","discord_assign_role","discord_ban_member","discord_bulk_delete_messages","discord_create_channel","discord_create_invite","discord_create_role","discord_create_thread","discord_create_webhook","discord_delete_channel","discord_delete_invite","discord_delete_message","discord_delete_role","discord_delete_webhook","discord_edit_message","discord_execute_webhook","discord_get_channel","discord_get_invite","discord_get_member","discord_get_messages","discord_get_pinned_messages","discord_get_server","discord_get_user","discord_get_webhook","discord_join_thread","discord_kick_member","discord_leave_thread","discord_list_channels","discord_list_roles","discord_pin_message","discord_remove_reaction","discord_remove_role","discord_send_message","discord_unban_member","discord_unpin_message","discord_update_channel","discord_update_member","discord_update_role","docusign_create_from_template","docusign_download_document","docusign_get_envelope","docusign_list_envelopes","docusign_list_recipients","docusign_list_templates","docusign_send_envelope","docusign_void_envelope","downdetector_get_company","downdetector_get_company_attribution","downdetector_get_company_baseline","downdetector_get_company_events","downdetector_get_company_incidents","downdetector_get_company_indicators","downdetector_get_company_last_15","downdetector_get_company_status","downdetector_get_provider","downdetector_get_reports","downdetector_get_site_companies","downdetector_list_categories","downdetector_list_incidents","downdetector_list_sites","downdetector_search_companies","dropbox_copy","dropbox_create_folder","dropbox_create_shared_link","dropbox_delete","dropbox_download","dropbox_get_metadata","dropbox_list_folder","dropbox_list_revisions","dropbox_list_shared_links","dropbox_move","dropbox_restore","dropbox_search","dropbox_upload","dropcontact_enrich_contact","dspy_chain_of_thought","dspy_predict","dspy_react","dub_bulk_create_links","dub_bulk_delete_links","dub_bulk_update_links","dub_create_link","dub_create_tag","dub_delete_link","dub_get_analytics","dub_get_events","dub_get_link","dub_get_links_count","dub_get_qr_code","dub_list_domains","dub_list_folders","dub_list_links","dub_list_tags","dub_update_link","dub_upsert_link","duckduckgo_search","dynamodb_delete","dynamodb_get","dynamodb_introspect","dynamodb_put","dynamodb_query","dynamodb_scan","dynamodb_update","dynatrace_add_problem_comment","dynatrace_add_tags","dynatrace_close_problem","dynatrace_create_settings_object","dynatrace_create_slo","dynatrace_delete_problem_comment","dynatrace_delete_settings_object","dynatrace_delete_slo","dynatrace_delete_tag","dynatrace_execute_synthetic_monitors","dynatrace_get_attack","dynatrace_get_audit_logs","dynatrace_get_entity","dynatrace_get_event","dynatrace_get_metric","dynatrace_get_problem","dynatrace_get_problem_comment","dynatrace_get_security_problem","dynatrace_get_settings_object","dynatrace_get_slo","dynatrace_get_synthetic_batch","dynatrace_ingest_event","dynatrace_ingest_logs","dynatrace_ingest_metrics","dynatrace_list_attacks","dynatrace_list_entities","dynatrace_list_entity_types","dynatrace_list_events","dynatrace_list_metrics","dynatrace_list_problem_comments","dynatrace_list_problems","dynatrace_list_remediation_items","dynatrace_list_security_problems","dynatrace_list_settings_objects","dynatrace_list_settings_schemas","dynatrace_list_slos","dynatrace_list_synthetic_monitors","dynatrace_list_tags","dynatrace_mute_security_problem","dynatrace_mute_security_problems","dynatrace_query_metrics","dynatrace_search_logs","dynatrace_unmute_security_problem","dynatrace_unmute_security_problems","dynatrace_update_problem_comment","dynatrace_update_settings_object","dynatrace_update_slo","elasticsearch_bulk","elasticsearch_cluster_health","elasticsearch_cluster_stats","elasticsearch_count","elasticsearch_create_index","elasticsearch_delete_document","elasticsearch_delete_index","elasticsearch_get_document","elasticsearch_get_index","elasticsearch_index_document","elasticsearch_list_indices","elasticsearch_search","elasticsearch_update_document","elevenlabs_audio_isolation","elevenlabs_edit_voice_settings","elevenlabs_get_user","elevenlabs_get_voice","elevenlabs_get_voice_settings","elevenlabs_list_models","elevenlabs_list_voices","elevenlabs_sound_effects","elevenlabs_speech_to_speech","elevenlabs_tts","emailbison_attach_leads_to_campaign","emailbison_attach_tags_to_leads","emailbison_create_campaign","emailbison_create_lead","emailbison_create_tag","emailbison_get_lead","emailbison_list_campaigns","emailbison_list_leads","emailbison_list_replies","emailbison_list_tags","emailbison_update_campaign","emailbison_update_campaign_status","emailbison_update_lead","embeddings_cohere","embeddings_gemini","embeddings_mistral","embeddings_ollama","embeddings_openai","embeddings_openrouter","enrich_check_credits","enrich_company_funding","enrich_company_lookup","enrich_company_revenue","enrich_disposable_email_check","enrich_email_to_ip","enrich_email_to_person_lite","enrich_email_to_phone","enrich_email_to_profile","enrich_find_email","enrich_get_post_details","enrich_ip_to_company","enrich_linkedin_profile","enrich_linkedin_to_personal_email","enrich_linkedin_to_work_email","enrich_phone_finder","enrich_reverse_hash_lookup","enrich_sales_pointer_people","enrich_search_company","enrich_search_company_activities","enrich_search_company_employees","enrich_search_jobs","enrich_search_logo","enrich_search_people","enrich_search_people_activities","enrich_search_post_comments","enrich_search_post_comments_by_url","enrich_search_post_reactions","enrich_search_post_reactions_by_url","enrich_search_posts","enrich_search_similar_companies","enrich_verify_email","enrichment_run","enrow_find_email","enrow_verify_email","exa_agent","exa_answer","exa_find_similar_links","exa_get_contents","exa_search","extend_parser","extend_parser_v2","fathom_get_summary","fathom_get_transcript","fathom_list_meeting_types","fathom_list_meetings","fathom_list_team_members","fathom_list_teams","file_append","file_compress","file_create_folder","file_decompress","file_delete_folder","file_edit","file_fetch","file_get","file_get_content","file_list","file_manage_sharing","file_move","file_parser","file_parser_v2","file_parser_v3","file_read","file_restore_folder","file_search","file_update_folder","file_write","findymail_find_email_from_linkedin","findymail_find_email_from_name","findymail_find_emails_by_domain","findymail_find_employees","findymail_find_phone","findymail_get_company","findymail_get_credits","findymail_lookup_technologies","findymail_reverse_email_lookup","findymail_search_technologies","findymail_verify_email","firecrawl_agent","firecrawl_batch_scrape","firecrawl_batch_scrape_status","firecrawl_cancel_crawl","firecrawl_crawl","firecrawl_crawl_status","firecrawl_credit_usage","firecrawl_extract","firecrawl_extract_status","firecrawl_map","firecrawl_parse","firecrawl_scrape","firecrawl_search","fireflies_add_to_live_meeting","fireflies_create_bite","fireflies_delete_transcript","fireflies_get_transcript","fireflies_get_user","fireflies_list_bites","fireflies_list_contacts","fireflies_list_transcripts","fireflies_list_users","fireflies_upload_audio","flint_create_task","flint_generate_pages","flint_get_task","function_execute","gamma_check_status","gamma_generate","gamma_generate_from_template","gamma_list_folders","gamma_list_themes","github_add_assignees","github_add_assignees_v2","github_add_labels","github_add_labels_v2","github_cancel_workflow_run","github_cancel_workflow_run_v2","github_check_star","github_check_star_v2","github_close_issue","github_close_issue_v2","github_close_pr","github_close_pr_v2","github_comment","github_comment_v2","github_compare_commits","github_compare_commits_v2","github_create_branch","github_create_branch_v2","github_create_comment_reaction","github_create_comment_reaction_v2","github_create_file","github_create_file_v2","github_create_gist","github_create_gist_v2","github_create_issue","github_create_issue_reaction","github_create_issue_reaction_v2","github_create_issue_v2","github_create_milestone","github_create_milestone_v2","github_create_pr","github_create_pr_review","github_create_pr_review_v2","github_create_pr_v2","github_create_project","github_create_project_v2","github_create_release","github_create_release_v2","github_delete_branch","github_delete_branch_v2","github_delete_comment","github_delete_comment_reaction","github_delete_comment_reaction_v2","github_delete_comment_v2","github_delete_file","github_delete_file_v2","github_delete_gist","github_delete_gist_v2","github_delete_issue_reaction","github_delete_issue_reaction_v2","github_delete_milestone","github_delete_milestone_v2","github_delete_project","github_delete_project_v2","github_delete_release","github_delete_release_v2","github_fork_gist","github_fork_gist_v2","github_fork_repo","github_fork_repo_v2","github_get_branch","github_get_branch_protection","github_get_branch_protection_v2","github_get_branch_v2","github_get_commit","github_get_commit_v2","github_get_file_content","github_get_file_content_v2","github_get_gist","github_get_gist_v2","github_get_issue","github_get_issue_v2","github_get_latest_release","github_get_latest_release_v2","github_get_milestone","github_get_milestone_v2","github_get_pr_files","github_get_pr_files_v2","github_get_project","github_get_project_v2","github_get_readme","github_get_readme_v2","github_get_release","github_get_release_v2","github_get_tree","github_get_tree_v2","github_get_workflow","github_get_workflow_run","github_get_workflow_run_v2","github_get_workflow_v2","github_issue_comment","github_issue_comment_v2","github_job_logs","github_latest_commit","github_latest_commit_v2","github_list_branches","github_list_branches_v2","github_list_commits","github_list_commits_v2","github_list_forks","github_list_forks_v2","github_list_gists","github_list_gists_v2","github_list_issue_comments","github_list_issue_comments_v2","github_list_issues","github_list_issues_v2","github_list_milestones","github_list_milestones_v2","github_list_pr_comments","github_list_pr_comments_v2","github_list_projects","github_list_projects_v2","github_list_prs","github_list_prs_v2","github_list_releases","github_list_releases_v2","github_list_review_threads","github_list_stargazers","github_list_stargazers_v2","github_list_tags","github_list_tags_v2","github_list_workflow_runs","github_list_workflow_runs_v2","github_list_workflows","github_list_workflows_v2","github_merge_pr","github_merge_pr_v2","github_pr","github_pr_v2","github_remove_label","github_remove_label_v2","github_reply_review_thread","github_repo_info","github_repo_info_v2","github_request_reviewers","github_request_reviewers_v2","github_rerun_workflow","github_rerun_workflow_v2","github_resolve_review_thread","github_search_code","github_search_code_v2","github_search_commits","github_search_commits_v2","github_search_issues","github_search_issues_v2","github_search_repos","github_search_repos_v2","github_search_users","github_search_users_v2","github_star_gist","github_star_gist_v2","github_star_repo","github_star_repo_v2","github_status_check_rollup","github_trigger_workflow","github_trigger_workflow_v2","github_unstar_gist","github_unstar_gist_v2","github_unstar_repo","github_unstar_repo_v2","github_update_branch_protection","github_update_branch_protection_v2","github_update_comment","github_update_comment_v2","github_update_file","github_update_file_v2","github_update_gist","github_update_gist_v2","github_update_issue","github_update_issue_v2","github_update_milestone","github_update_milestone_v2","github_update_pr","github_update_pr_v2","github_update_project","github_update_project_v2","github_update_release","github_update_release_v2","gitlab_activate_user","gitlab_add_member","gitlab_add_saml_group_link","gitlab_approve_access_request","gitlab_approve_merge_request","gitlab_approve_user","gitlab_ban_user","gitlab_block_user","gitlab_cancel_pipeline","gitlab_compare_branches","gitlab_create_branch","gitlab_create_file","gitlab_create_issue","gitlab_create_issue_note","gitlab_create_merge_request","gitlab_create_merge_request_note","gitlab_create_pipeline","gitlab_create_release","gitlab_create_user","gitlab_deactivate_user","gitlab_delete_branch","gitlab_delete_issue","gitlab_delete_saml_group_link","gitlab_delete_user","gitlab_delete_user_identity","gitlab_deny_access_request","gitlab_get_file","gitlab_get_group","gitlab_get_issue","gitlab_get_job_log","gitlab_get_merge_request","gitlab_get_merge_request_changes","gitlab_get_pipeline","gitlab_get_project","gitlab_invite_member","gitlab_list_access_requests","gitlab_list_branches","gitlab_list_commits","gitlab_list_groups","gitlab_list_invitations","gitlab_list_issues","gitlab_list_members","gitlab_list_merge_requests","gitlab_list_pipeline_jobs","gitlab_list_pipelines","gitlab_list_projects","gitlab_list_releases","gitlab_list_repository_tree","gitlab_list_saml_group_links","gitlab_list_user_memberships","gitlab_merge_merge_request","gitlab_play_job","gitlab_reject_user","gitlab_remove_member","gitlab_retry_pipeline","gitlab_revoke_invitation","gitlab_search_users","gitlab_unban_user","gitlab_unblock_user","gitlab_update_file","gitlab_update_invitation","gitlab_update_issue","gitlab_update_member","gitlab_update_merge_request","gitlab_update_user","gmail_add_label","gmail_add_label_v2","gmail_archive","gmail_archive_v2","gmail_create_label_v2","gmail_delete","gmail_delete_draft_v2","gmail_delete_label_v2","gmail_delete_v2","gmail_draft","gmail_draft_v2","gmail_edit_draft_v2","gmail_get_draft_v2","gmail_get_thread_v2","gmail_list_drafts_v2","gmail_list_labels_v2","gmail_list_threads_v2","gmail_mark_read","gmail_mark_read_v2","gmail_mark_unread","gmail_mark_unread_v2","gmail_move","gmail_move_v2","gmail_read","gmail_read_v2","gmail_remove_label","gmail_remove_label_v2","gmail_search","gmail_search_v2","gmail_send","gmail_send_v2","gmail_trash_thread_v2","gmail_unarchive","gmail_unarchive_v2","gmail_untrash_thread_v2","gmail_update_label_v2","gong_aggregate_activity","gong_aggregate_by_period","gong_answered_scorecards","gong_ask_anything","gong_assign_flow_prospects","gong_create_call","gong_day_by_day_activity","gong_get_brief","gong_get_call","gong_get_call_transcript","gong_get_coaching","gong_get_extensive_calls","gong_get_folder_content","gong_get_logs","gong_get_prospect_flows","gong_get_user","gong_interaction_stats","gong_list_calls","gong_list_flows","gong_list_library_folders","gong_list_scorecards","gong_list_trackers","gong_list_users","gong_list_workspaces","gong_lookup_email","gong_lookup_phone","gong_purge_email_address","gong_purge_phone_number","gong_unassign_flow_prospects","google_ads_ad_performance","google_ads_campaign_performance","google_ads_list_ad_groups","google_ads_list_campaigns","google_ads_list_customers","google_ads_search","google_appsheet_add_rows","google_appsheet_delete_rows","google_appsheet_edit_rows","google_appsheet_find_rows","google_bigquery_create_dataset","google_bigquery_create_table","google_bigquery_delete_dataset","google_bigquery_delete_table","google_bigquery_get_query_results","google_bigquery_get_table","google_bigquery_insert_rows","google_bigquery_list_datasets","google_bigquery_list_table_data","google_bigquery_list_tables","google_bigquery_query","google_books_volume_details","google_books_volume_search","google_calendar_create","google_calendar_create_calendar","google_calendar_create_calendar_v2","google_calendar_create_v2","google_calendar_delete","google_calendar_delete_calendar","google_calendar_delete_calendar_v2","google_calendar_delete_v2","google_calendar_freebusy","google_calendar_freebusy_v2","google_calendar_get","google_calendar_get_v2","google_calendar_instances","google_calendar_instances_v2","google_calendar_invite","google_calendar_invite_v2","google_calendar_list","google_calendar_list_acl","google_calendar_list_acl_v2","google_calendar_list_calendars","google_calendar_list_calendars_v2","google_calendar_list_v2","google_calendar_move","google_calendar_move_v2","google_calendar_quick_add","google_calendar_quick_add_v2","google_calendar_share_calendar","google_calendar_share_calendar_v2","google_calendar_unshare_calendar","google_calendar_unshare_calendar_v2","google_calendar_update","google_calendar_update_acl","google_calendar_update_acl_v2","google_calendar_update_calendar","google_calendar_update_calendar_v2","google_calendar_update_v2","google_contacts_create","google_contacts_delete","google_contacts_get","google_contacts_list","google_contacts_search","google_contacts_update","google_docs_create","google_docs_create_named_range","google_docs_create_paragraph_bullets","google_docs_delete_content_range","google_docs_delete_named_range","google_docs_delete_paragraph_bullets","google_docs_insert_image","google_docs_insert_page_break","google_docs_insert_table","google_docs_insert_text","google_docs_read","google_docs_replace_text","google_docs_update_paragraph_style","google_docs_update_text_style","google_docs_write","google_drive_copy","google_drive_create_comment","google_drive_create_folder","google_drive_delete","google_drive_delete_comment","google_drive_download","google_drive_export","google_drive_get_about","google_drive_get_content","google_drive_get_file","google_drive_get_revision","google_drive_list","google_drive_list_comments","google_drive_list_permissions","google_drive_list_revisions","google_drive_move","google_drive_search","google_drive_share","google_drive_trash","google_drive_unshare","google_drive_untrash","google_drive_update","google_drive_upload","google_forms_batch_update","google_forms_create_form","google_forms_create_watch","google_forms_delete_watch","google_forms_get_form","google_forms_get_responses","google_forms_list_watches","google_forms_renew_watch","google_forms_set_publish_settings","google_groups_add_alias","google_groups_add_member","google_groups_create_group","google_groups_delete_group","google_groups_get_group","google_groups_get_member","google_groups_get_settings","google_groups_has_member","google_groups_list_aliases","google_groups_list_groups","google_groups_list_members","google_groups_remove_alias","google_groups_remove_member","google_groups_update_group","google_groups_update_member","google_groups_update_settings","google_maps_air_quality","google_maps_directions","google_maps_distance_matrix","google_maps_elevation","google_maps_geocode","google_maps_geolocate","google_maps_place_details","google_maps_places_nearby","google_maps_places_search","google_maps_pollen","google_maps_reverse_geocode","google_maps_snap_to_roads","google_maps_solar","google_maps_speed_limits","google_maps_timezone","google_maps_validate_address","google_meet_create_space","google_meet_end_conference","google_meet_get_conference_record","google_meet_get_space","google_meet_list_conference_records","google_meet_list_participants","google_pagespeed_analyze","google_search","google_sheets_append","google_sheets_append_v2","google_sheets_batch_clear_v2","google_sheets_batch_get_v2","google_sheets_batch_update_v2","google_sheets_clear_v2","google_sheets_copy_sheet_v2","google_sheets_create_spreadsheet_v2","google_sheets_delete_rows_v2","google_sheets_delete_sheet_v2","google_sheets_delete_spreadsheet_v2","google_sheets_get_spreadsheet_v2","google_sheets_read","google_sheets_read_v2","google_sheets_update","google_sheets_update_v2","google_sheets_write","google_sheets_write_v2","google_slides_add_image","google_slides_add_slide","google_slides_batch_update","google_slides_copy_presentation","google_slides_create","google_slides_create_line","google_slides_create_paragraph_bullets","google_slides_create_shape","google_slides_create_sheets_chart","google_slides_create_table","google_slides_create_video","google_slides_delete_object","google_slides_delete_paragraph_bullets","google_slides_delete_table_column","google_slides_delete_table_row","google_slides_delete_text","google_slides_duplicate_object","google_slides_export_presentation","google_slides_get_page","google_slides_get_thumbnail","google_slides_group_objects","google_slides_insert_table_columns","google_slides_insert_table_rows","google_slides_insert_text","google_slides_merge_table_cells","google_slides_read","google_slides_refresh_sheets_chart","google_slides_replace_all_shapes_with_image","google_slides_replace_all_shapes_with_sheets_chart","google_slides_replace_all_text","google_slides_replace_image","google_slides_reroute_line","google_slides_ungroup_objects","google_slides_unmerge_table_cells","google_slides_update_image_properties","google_slides_update_line_category","google_slides_update_line_properties","google_slides_update_page_element_alt_text","google_slides_update_page_element_transform","google_slides_update_page_elements_z_order","google_slides_update_page_properties","google_slides_update_paragraph_style","google_slides_update_shape_properties","google_slides_update_slide_properties","google_slides_update_slides_position","google_slides_update_table_border_properties","google_slides_update_table_cell_properties","google_slides_update_table_column_properties","google_slides_update_table_row_properties","google_slides_update_text_style","google_slides_update_video_properties","google_slides_write","google_tasks_create","google_tasks_delete","google_tasks_get","google_tasks_list","google_tasks_list_task_lists","google_tasks_update","google_translate_detect","google_translate_text","google_vault_add_held_accounts","google_vault_add_matters_permissions","google_vault_close_matters","google_vault_create_matters","google_vault_create_matters_export","google_vault_create_matters_holds","google_vault_create_saved_query","google_vault_delete_matters","google_vault_delete_matters_export","google_vault_delete_matters_holds","google_vault_delete_saved_query","google_vault_download_export_file","google_vault_list_matters","google_vault_list_matters_export","google_vault_list_matters_holds","google_vault_list_saved_queries","google_vault_remove_held_accounts","google_vault_remove_matters_permissions","google_vault_reopen_matters","google_vault_undelete_matters","google_vault_update_matters","google_vault_update_matters_holds","grafana_check_data_source_health","grafana_create_alert_rule","grafana_create_annotation","grafana_create_contact_point","grafana_create_dashboard","grafana_create_folder","grafana_delete_alert_rule","grafana_delete_annotation","grafana_delete_contact_point","grafana_delete_dashboard","grafana_delete_folder","grafana_get_alert_rule","grafana_get_alert_rule_group","grafana_get_dashboard","grafana_get_data_source","grafana_get_folder","grafana_get_health","grafana_list_alert_rules","grafana_list_annotations","grafana_list_contact_points","grafana_list_dashboards","grafana_list_data_sources","grafana_list_folders","grafana_move_folder","grafana_query_data_source","grafana_update_alert_rule","grafana_update_annotation","grafana_update_contact_point","grafana_update_dashboard","grafana_update_folder","grain_create_hook","grain_create_hook_v2","grain_delete_hook","grain_delete_hook_v2","grain_get_recording","grain_get_transcript","grain_list_hooks","grain_list_hooks_v2","grain_list_meeting_types","grain_list_recordings","grain_list_teams","grain_list_views","granola_create_webhook_endpoint","granola_delete_webhook_endpoint","granola_get_note","granola_get_transcript","granola_list_audit_events","granola_list_folders","granola_list_notes","granola_list_webhook_endpoints","granola_update_webhook_endpoint","greenhouse_get_application","greenhouse_get_candidate","greenhouse_get_job","greenhouse_get_user","greenhouse_list_applications","greenhouse_list_candidates","greenhouse_list_departments","greenhouse_list_job_stages","greenhouse_list_jobs","greenhouse_list_offices","greenhouse_list_users","greptile_index_repo","greptile_query","greptile_search","greptile_status","guardrails_validate","harmonic_batch_get_people","harmonic_clear_people_saved_search_net_new_results","harmonic_enrich_person","harmonic_get_company_employees","harmonic_get_email_enrichment_job","harmonic_get_email_enrichment_usage","harmonic_get_enrichment_status","harmonic_get_people_saved_search_net_new_results","harmonic_get_people_saved_search_results","harmonic_get_person","harmonic_list_people_saved_searches","harmonic_search_people_scout","harmonic_submit_email_enrichment_job","hex_cancel_run","hex_create_collection","hex_create_group","hex_deactivate_user","hex_delete_group","hex_get_collection","hex_get_data_connection","hex_get_group","hex_get_project","hex_get_project_runs","hex_get_queried_tables","hex_get_run_status","hex_list_collections","hex_list_data_connections","hex_list_groups","hex_list_projects","hex_list_users","hex_run_project","hex_update_collection","hex_update_group","hex_update_project","http_request","hubspot_add_list_memberships","hubspot_create_appointment","hubspot_create_association","hubspot_create_company","hubspot_create_contact","hubspot_create_deal","hubspot_create_email","hubspot_create_line_item","hubspot_create_list","hubspot_create_note","hubspot_create_ticket","hubspot_delete_association","hubspot_delete_company","hubspot_delete_contact","hubspot_delete_deal","hubspot_delete_line_item","hubspot_delete_ticket","hubspot_get_appointment","hubspot_get_association_labels","hubspot_get_cart","hubspot_get_company","hubspot_get_contact","hubspot_get_deal","hubspot_get_email","hubspot_get_line_item","hubspot_get_list","hubspot_get_list_memberships","hubspot_get_marketing_event","hubspot_get_note","hubspot_get_properties","hubspot_get_quote","hubspot_get_ticket","hubspot_get_users","hubspot_list_appointments","hubspot_list_associations","hubspot_list_carts","hubspot_list_companies","hubspot_list_contacts","hubspot_list_deals","hubspot_list_emails","hubspot_list_line_items","hubspot_list_lists","hubspot_list_marketing_events","hubspot_list_notes","hubspot_list_owners","hubspot_list_quotes","hubspot_list_tickets","hubspot_remove_list_memberships","hubspot_search_companies","hubspot_search_contacts","hubspot_search_deals","hubspot_search_emails","hubspot_search_line_items","hubspot_search_notes","hubspot_search_quotes","hubspot_search_tickets","hubspot_update_appointment","hubspot_update_company","hubspot_update_contact","hubspot_update_deal","hubspot_update_line_item","hubspot_update_ticket","huggingface_chat","hunter_companies_find","hunter_discover","hunter_domain_search","hunter_email_count","hunter_email_finder","hunter_email_verifier","iam_add_user_to_group","iam_attach_role_policy","iam_attach_user_policy","iam_create_access_key","iam_create_role","iam_create_user","iam_delete_access_key","iam_delete_role","iam_delete_user","iam_detach_role_policy","iam_detach_user_policy","iam_get_policy","iam_get_role","iam_get_user","iam_list_access_keys","iam_list_attached_role_policies","iam_list_attached_user_policies","iam_list_groups","iam_list_policies","iam_list_roles","iam_list_users","iam_remove_user_from_group","iam_simulate_principal_policy","iam_update_access_key","icypeas_find_email","icypeas_verify_email","identity_center_check_assignment_deletion_status","identity_center_check_assignment_status","identity_center_create_account_assignment","identity_center_delete_account_assignment","identity_center_describe_account","identity_center_describe_group","identity_center_describe_user","identity_center_get_group","identity_center_get_user","identity_center_list_account_assignments","identity_center_list_accounts","identity_center_list_assignments_for_account","identity_center_list_group_memberships","identity_center_list_groups","identity_center_list_instances","identity_center_list_permission_sets","image_generate","incidentio_actions_create","incidentio_actions_list","incidentio_actions_show","incidentio_actions_update","incidentio_alert_events_create","incidentio_alerts_list","incidentio_alerts_resolve","incidentio_alerts_show","incidentio_catalog_entries_list","incidentio_catalog_types_list","incidentio_custom_fields_create","incidentio_custom_fields_delete","incidentio_custom_fields_list","incidentio_custom_fields_show","incidentio_custom_fields_update","incidentio_escalation_paths_create","incidentio_escalation_paths_delete","incidentio_escalation_paths_list","incidentio_escalation_paths_show","incidentio_escalation_paths_update","incidentio_escalations_cancel","incidentio_escalations_create","incidentio_escalations_list","incidentio_escalations_show","incidentio_follow_ups_create","incidentio_follow_ups_list","incidentio_follow_ups_show","incidentio_follow_ups_update","incidentio_incident_alerts_list","incidentio_incident_memberships_create","incidentio_incident_memberships_revoke","incidentio_incident_participants_list","incidentio_incident_roles_create","incidentio_incident_roles_delete","incidentio_incident_roles_list","incidentio_incident_roles_show","incidentio_incident_roles_update","incidentio_incident_statuses_list","incidentio_incident_timestamps_list","incidentio_incident_timestamps_show","incidentio_incident_types_list","incidentio_incident_updates_list","incidentio_incidents_create","incidentio_incidents_list","incidentio_incidents_show","incidentio_incidents_update","incidentio_on_call_now","incidentio_schedule_entries_list","incidentio_schedule_overrides_create","incidentio_schedule_overrides_list","incidentio_schedules_create","incidentio_schedules_delete","incidentio_schedules_list","incidentio_schedules_show","incidentio_schedules_update","incidentio_severities_list","incidentio_teams_list","incidentio_teams_show","incidentio_users_list","incidentio_users_show","incidentio_workflows_create","incidentio_workflows_delete","incidentio_workflows_list","incidentio_workflows_show","incidentio_workflows_update","infisical_create_secret","infisical_delete_secret","infisical_get_secret","infisical_list_secrets","infisical_update_secret","instagram_delete_comment","instagram_download_media","instagram_get_account_insights","instagram_get_container_status","instagram_get_conversation_messages","instagram_get_media","instagram_get_media_insights","instagram_get_message","instagram_get_profile","instagram_get_publishing_limit","instagram_hide_comment","instagram_list_comments","instagram_list_conversations","instagram_list_media","instagram_list_stories","instagram_private_reply","instagram_publish_carousel","instagram_publish_image","instagram_publish_reel","instagram_publish_story","instagram_publish_video","instagram_reply_to_comment","instagram_send_text_message","instagram_set_comments_enabled","instantly_activate_campaign","instantly_create_campaign","instantly_create_lead","instantly_create_lead_list","instantly_delete_campaign","instantly_delete_leads","instantly_get_lead","instantly_list_campaigns","instantly_list_emails","instantly_list_lead_lists","instantly_list_leads","instantly_patch_campaign","instantly_patch_lead","instantly_pause_campaign","instantly_reply_to_email","instantly_update_lead_interest_status","intercom_assign_conversation_v2","intercom_attach_contact_to_company_v2","intercom_close_conversation_v2","intercom_create_company","intercom_create_company_v2","intercom_create_contact","intercom_create_contact_v2","intercom_create_event_v2","intercom_create_message","intercom_create_message_v2","intercom_create_note_v2","intercom_create_tag_v2","intercom_create_ticket","intercom_create_ticket_v2","intercom_delete_contact","intercom_delete_contact_v2","intercom_detach_contact_from_company_v2","intercom_get_company","intercom_get_company_v2","intercom_get_contact","intercom_get_contact_v2","intercom_get_conversation","intercom_get_conversation_v2","intercom_get_ticket","intercom_get_ticket_v2","intercom_list_admins_v2","intercom_list_companies","intercom_list_companies_v2","intercom_list_contacts","intercom_list_contacts_v2","intercom_list_conversations","intercom_list_conversations_v2","intercom_list_tags_v2","intercom_open_conversation_v2","intercom_reply_conversation","intercom_reply_conversation_v2","intercom_search_contacts","intercom_search_contacts_v2","intercom_search_conversations","intercom_search_conversations_v2","intercom_snooze_conversation_v2","intercom_tag_contact_v2","intercom_tag_conversation_v2","intercom_untag_contact_v2","intercom_update_contact","intercom_update_contact_v2","intercom_update_ticket_v2","jina_read_url","jina_search","jira_add_attachment","jira_add_comment","jira_add_watcher","jira_add_worklog","jira_assign_issue","jira_bulk_read","jira_create_issue_link","jira_delete_attachment","jira_delete_comment","jira_delete_issue","jira_delete_issue_link","jira_delete_worklog","jira_get_attachments","jira_get_comments","jira_get_fields","jira_get_project","jira_get_transitions","jira_get_users","jira_get_worklogs","jira_list_issue_types","jira_list_projects","jira_remove_watcher","jira_retrieve","jira_search_issues","jira_search_users","jira_transition_issue","jira_update","jira_update_comment","jira_update_worklog","jira_write","jotform_add_label_resources","jotform_clone_form","jotform_create_form","jotform_create_label","jotform_create_question","jotform_create_questions","jotform_create_report","jotform_create_submission","jotform_create_submissions","jotform_create_webhook","jotform_delete_form","jotform_delete_label","jotform_delete_question","jotform_delete_report","jotform_delete_submission","jotform_delete_webhook","jotform_get_form","jotform_get_form_properties","jotform_get_history","jotform_get_label","jotform_get_question","jotform_get_report","jotform_get_settings","jotform_get_submission","jotform_get_usage","jotform_get_user","jotform_list_form_files","jotform_list_form_reports","jotform_list_form_submissions","jotform_list_forms","jotform_list_label_resources","jotform_list_labels","jotform_list_questions","jotform_list_reports","jotform_list_submissions","jotform_list_subusers","jotform_list_webhooks","jotform_remove_label_resources","jotform_update_form_properties","jotform_update_label","jotform_update_question","jotform_update_settings","jotform_update_submission","jsm_add_comment","jsm_add_customer","jsm_add_organization","jsm_add_participants","jsm_answer_approval","jsm_attach_form","jsm_copy_forms","jsm_create_object","jsm_create_organization","jsm_create_request","jsm_delete_form","jsm_delete_object","jsm_externalise_form","jsm_get_approvals","jsm_get_comments","jsm_get_customers","jsm_get_form","jsm_get_form_answers","jsm_get_form_structure","jsm_get_form_templates","jsm_get_issue_forms","jsm_get_object","jsm_get_object_schema","jsm_get_object_type_attributes","jsm_get_organizations","jsm_get_participants","jsm_get_queues","jsm_get_request","jsm_get_request_type_fields","jsm_get_request_types","jsm_get_requests","jsm_get_service_desks","jsm_get_sla","jsm_get_transitions","jsm_internalise_form","jsm_list_object_schemas","jsm_list_object_types","jsm_reopen_form","jsm_save_form_answers","jsm_search_objects_aql","jsm_submit_form","jsm_transition_request","jsm_update_object","jupyter_copy_content","jupyter_create_file","jupyter_create_session","jupyter_delete_content","jupyter_delete_session","jupyter_get_content","jupyter_interrupt_kernel","jupyter_list_contents","jupyter_list_kernels","jupyter_list_kernelspecs","jupyter_list_sessions","jupyter_rename_content","jupyter_restart_kernel","jupyter_start_kernel","jupyter_stop_kernel","jupyter_upload_file","kalshi_amend_order","kalshi_amend_order_v2","kalshi_cancel_order","kalshi_cancel_order_v2","kalshi_create_order","kalshi_create_order_v2","kalshi_get_balance","kalshi_get_balance_v2","kalshi_get_candlesticks","kalshi_get_candlesticks_v2","kalshi_get_event","kalshi_get_event_candlesticks","kalshi_get_event_candlesticks_v2","kalshi_get_event_v2","kalshi_get_events","kalshi_get_events_v2","kalshi_get_exchange_announcements","kalshi_get_exchange_announcements_v2","kalshi_get_exchange_schedule","kalshi_get_exchange_schedule_v2","kalshi_get_exchange_status","kalshi_get_exchange_status_v2","kalshi_get_fills","kalshi_get_fills_v2","kalshi_get_market","kalshi_get_market_v2","kalshi_get_markets","kalshi_get_markets_v2","kalshi_get_order","kalshi_get_order_v2","kalshi_get_orderbook","kalshi_get_orderbook_v2","kalshi_get_orders","kalshi_get_orders_v2","kalshi_get_positions","kalshi_get_positions_v2","kalshi_get_series_by_ticker","kalshi_get_series_by_ticker_v2","kalshi_get_series_list","kalshi_get_series_list_v2","kalshi_get_settlements","kalshi_get_settlements_v2","kalshi_get_trades","kalshi_get_trades_v2","ketch_get_consent","ketch_get_subscriptions","ketch_invoke_right","ketch_set_consent","ketch_set_subscriptions","knowledge_create_document","knowledge_delete_chunk","knowledge_delete_document","knowledge_get_connector","knowledge_get_document","knowledge_list_chunks","knowledge_list_connectors","knowledge_list_documents","knowledge_list_tags","knowledge_search","knowledge_trigger_sync","knowledge_update_chunk","knowledge_upload_chunk","knowledge_upsert_document","lambda_add_permission","lambda_create_alias","lambda_create_event_source_mapping","lambda_create_function","lambda_create_function_url_config","lambda_delete_alias","lambda_delete_event_source_mapping","lambda_delete_function","lambda_delete_function_concurrency","lambda_delete_function_event_invoke_config","lambda_delete_function_url_config","lambda_delete_provisioned_concurrency_config","lambda_get_account_settings","lambda_get_alias","lambda_get_event_source_mapping","lambda_get_function","lambda_get_function_concurrency","lambda_get_function_configuration","lambda_get_function_event_invoke_config","lambda_get_function_recursion_config","lambda_get_function_url_config","lambda_get_layer_version","lambda_get_policy","lambda_get_provisioned_concurrency_config","lambda_get_runtime_management_config","lambda_invoke","lambda_list_aliases","lambda_list_event_source_mappings","lambda_list_function_event_invoke_configs","lambda_list_function_url_configs","lambda_list_functions","lambda_list_layer_versions","lambda_list_layers","lambda_list_provisioned_concurrency_configs","lambda_list_tags","lambda_list_versions_by_function","lambda_publish_version","lambda_put_function_concurrency","lambda_put_function_event_invoke_config","lambda_put_function_recursion_config","lambda_put_provisioned_concurrency_config","lambda_put_runtime_management_config","lambda_remove_permission","lambda_tag_resource","lambda_untag_resource","lambda_update_alias","lambda_update_event_source_mapping","lambda_update_function_code","lambda_update_function_configuration","lambda_update_function_url_config","langsmith_create_feedback","langsmith_create_run","langsmith_create_runs_batch","langsmith_get_run","langsmith_update_run","latex_compile","latex_get_package","latex_list_fonts","latex_search_packages","launchdarkly_create_flag","launchdarkly_delete_flag","launchdarkly_get_audit_log","launchdarkly_get_flag","launchdarkly_get_flag_status","launchdarkly_list_environments","launchdarkly_list_flags","launchdarkly_list_members","launchdarkly_list_projects","launchdarkly_list_segments","launchdarkly_toggle_flag","launchdarkly_update_flag","leadmagic_company_search","leadmagic_email_to_profile","leadmagic_find_email","leadmagic_find_mobile","leadmagic_get_credits","leadmagic_profile_search","leadmagic_profile_to_email","leadmagic_role_finder","leadmagic_validate_email","lemlist_get_activities","lemlist_get_lead","lemlist_send_email","linear_add_label_to_issue","linear_add_label_to_project","linear_archive_issue","linear_archive_label","linear_archive_project","linear_create_attachment","linear_create_comment","linear_create_customer","linear_create_customer_request","linear_create_customer_status","linear_create_customer_tier","linear_create_cycle","linear_create_favorite","linear_create_issue","linear_create_issue_relation","linear_create_label","linear_create_project","linear_create_project_label","linear_create_project_milestone","linear_create_project_status","linear_create_project_update","linear_create_workflow_state","linear_delete_attachment","linear_delete_comment","linear_delete_customer","linear_delete_customer_status","linear_delete_customer_tier","linear_delete_issue","linear_delete_issue_relation","linear_delete_project","linear_delete_project_label","linear_delete_project_milestone","linear_delete_project_status","linear_get_active_cycle","linear_get_customer","linear_get_cycle","linear_get_issue","linear_get_project","linear_get_viewer","linear_list_attachments","linear_list_comments","linear_list_customer_requests","linear_list_customer_statuses","linear_list_customer_tiers","linear_list_customers","linear_list_cycles","linear_list_favorites","linear_list_issue_relations","linear_list_labels","linear_list_notifications","linear_list_project_labels","linear_list_project_milestones","linear_list_project_statuses","linear_list_project_updates","linear_list_projects","linear_list_teams","linear_list_users","linear_list_workflow_states","linear_merge_customers","linear_read_issues","linear_remove_label_from_issue","linear_remove_label_from_project","linear_search_issues","linear_unarchive_issue","linear_update_attachment","linear_update_comment","linear_update_customer","linear_update_customer_request","linear_update_customer_status","linear_update_customer_tier","linear_update_issue","linear_update_label","linear_update_notification","linear_update_project","linear_update_project_label","linear_update_project_milestone","linear_update_project_status","linear_update_workflow_state","linkedin_get_profile","linkedin_share_post","linkup_search","linq_add_participant","linq_check_imessage","linq_check_rcs","linq_create_attachment","linq_create_chat","linq_create_contact_card","linq_create_webhook_subscription","linq_delete_attachment","linq_delete_message","linq_delete_webhook_subscription","linq_edit_message","linq_get_attachment","linq_get_chat","linq_get_contact_card","linq_get_message","linq_get_webhook_subscription","linq_leave_chat","linq_list_chats","linq_list_messages","linq_list_phone_numbers","linq_list_thread","linq_list_webhook_events","linq_list_webhook_subscriptions","linq_mark_chat_read","linq_react_to_message","linq_remove_participant","linq_send_message","linq_send_voice_memo","linq_share_contact_card","linq_start_typing","linq_stop_typing","linq_update_chat","linq_update_contact_card","linq_update_webhook_subscription","llm_chat","logfire_get_token_info","logfire_get_trace","logfire_query","logfire_search_records","logrocket_create_release","logrocket_get_audit_logs","logrocket_get_highlights","logrocket_identify_user","logrocket_list_exported_sessions","logrocket_request_highlights","logs_get","logs_get_execution","logs_get_run_details","logs_query","logs_query_runs","loops_check_contact_suppression","loops_create_contact","loops_create_contact_property","loops_delete_contact","loops_find_contact","loops_get_transactional_email","loops_list_contact_properties","loops_list_mailing_lists","loops_list_transactional_emails","loops_remove_contact_suppression","loops_send_event","loops_send_transactional_email","loops_update_contact","luma_add_guests","luma_cancel_event","luma_create_event","luma_get_event","luma_get_guest","luma_get_guests","luma_list_events","luma_lookup_event","luma_send_invites","luma_update_event","luma_update_guest_status","mailchimp_add_member","mailchimp_add_member_tags","mailchimp_add_or_update_member","mailchimp_add_segment_member","mailchimp_add_subscriber_to_automation","mailchimp_archive_member","mailchimp_create_audience","mailchimp_create_batch_operation","mailchimp_create_campaign","mailchimp_create_interest","mailchimp_create_interest_category","mailchimp_create_landing_page","mailchimp_create_merge_field","mailchimp_create_segment","mailchimp_create_template","mailchimp_delete_audience","mailchimp_delete_batch_operation","mailchimp_delete_campaign","mailchimp_delete_interest","mailchimp_delete_interest_category","mailchimp_delete_landing_page","mailchimp_delete_member","mailchimp_delete_merge_field","mailchimp_delete_segment","mailchimp_delete_template","mailchimp_get_audience","mailchimp_get_audiences","mailchimp_get_automation","mailchimp_get_automations","mailchimp_get_batch_operation","mailchimp_get_batch_operations","mailchimp_get_campaign","mailchimp_get_campaign_content","mailchimp_get_campaign_report","mailchimp_get_campaign_reports","mailchimp_get_campaigns","mailchimp_get_interest","mailchimp_get_interest_categories","mailchimp_get_interest_category","mailchimp_get_interests","mailchimp_get_landing_page","mailchimp_get_landing_pages","mailchimp_get_member","mailchimp_get_member_tags","mailchimp_get_members","mailchimp_get_merge_field","mailchimp_get_merge_fields","mailchimp_get_segment","mailchimp_get_segment_members","mailchimp_get_segments","mailchimp_get_template","mailchimp_get_templates","mailchimp_pause_automation","mailchimp_publish_landing_page","mailchimp_remove_member_tags","mailchimp_remove_segment_member","mailchimp_replicate_campaign","mailchimp_schedule_campaign","mailchimp_send_campaign","mailchimp_set_campaign_content","mailchimp_start_automation","mailchimp_unarchive_member","mailchimp_unpublish_landing_page","mailchimp_unschedule_campaign","mailchimp_update_audience","mailchimp_update_campaign","mailchimp_update_interest","mailchimp_update_interest_category","mailchimp_update_landing_page","mailchimp_update_member","mailchimp_update_merge_field","mailchimp_update_segment","mailchimp_update_template","mailgun_add_list_member","mailgun_create_mailing_list","mailgun_get_domain","mailgun_get_mailing_list","mailgun_get_message","mailgun_list_domains","mailgun_list_messages","mailgun_send_message","managed_agent_archive_session","managed_agent_create_session","managed_agent_delete_session","managed_agent_get_session","managed_agent_interrupt_session","managed_agent_list_events","managed_agent_respond_custom_tool","managed_agent_respond_tool_confirmation","managed_agent_run_session","managed_agent_send_message","managed_agent_update_session","manageengine_sdp_add_change_note","manageengine_sdp_add_problem_note","manageengine_sdp_add_request_note","manageengine_sdp_create_asset","manageengine_sdp_create_change","manageengine_sdp_create_problem","manageengine_sdp_create_request","manageengine_sdp_create_solution","manageengine_sdp_delete_asset","manageengine_sdp_delete_change","manageengine_sdp_delete_problem","manageengine_sdp_delete_request","manageengine_sdp_delete_solution","manageengine_sdp_get_asset","manageengine_sdp_get_change","manageengine_sdp_get_problem","manageengine_sdp_get_request","manageengine_sdp_get_solution","manageengine_sdp_list_assets","manageengine_sdp_list_change_notes","manageengine_sdp_list_changes","manageengine_sdp_list_problem_notes","manageengine_sdp_list_problems","manageengine_sdp_list_request_notes","manageengine_sdp_list_requests","manageengine_sdp_list_solutions","manageengine_sdp_update_asset","manageengine_sdp_update_change","manageengine_sdp_update_problem","manageengine_sdp_update_request","manageengine_sdp_update_solution","mcp_list_operations","mcp_run_operation","mem0_add_memories","mem0_get_memories","mem0_search_memories","memory_add","memory_delete","memory_get","memory_get_all","microsoft_ad_add_directory_role_member","microsoft_ad_add_group_member","microsoft_ad_add_user_app_role_assignment","microsoft_ad_assign_license","microsoft_ad_create_group","microsoft_ad_create_user","microsoft_ad_delete_group","microsoft_ad_delete_user","microsoft_ad_get_conditional_access_policy","microsoft_ad_get_device","microsoft_ad_get_group","microsoft_ad_get_user","microsoft_ad_list_authentication_methods","microsoft_ad_list_conditional_access_policies","microsoft_ad_list_devices","microsoft_ad_list_directory_audits","microsoft_ad_list_directory_role_members","microsoft_ad_list_directory_roles","microsoft_ad_list_group_members","microsoft_ad_list_groups","microsoft_ad_list_service_principal_app_role_assignments","microsoft_ad_list_service_principals","microsoft_ad_list_sign_ins","microsoft_ad_list_subscribed_skus","microsoft_ad_list_user_app_role_assignments","microsoft_ad_list_user_devices","microsoft_ad_list_user_licenses","microsoft_ad_list_users","microsoft_ad_remove_directory_role_member","microsoft_ad_remove_group_member","microsoft_ad_remove_user_app_role_assignment","microsoft_ad_reset_password","microsoft_ad_revoke_sign_in_sessions","microsoft_ad_set_password","microsoft_ad_update_group","microsoft_ad_update_user","microsoft_dataverse_associate","microsoft_dataverse_create_multiple","microsoft_dataverse_create_record","microsoft_dataverse_delete_record","microsoft_dataverse_disassociate","microsoft_dataverse_download_file","microsoft_dataverse_execute_action","microsoft_dataverse_execute_function","microsoft_dataverse_fetchxml_query","microsoft_dataverse_get_entity_metadata","microsoft_dataverse_get_record","microsoft_dataverse_list_records","microsoft_dataverse_search","microsoft_dataverse_update_multiple","microsoft_dataverse_update_record","microsoft_dataverse_upload_file","microsoft_dataverse_upsert_record","microsoft_dataverse_whoami","microsoft_dynamics_365_close_case","microsoft_dynamics_365_close_opportunity","microsoft_dynamics_365_create_record","microsoft_dynamics_365_get_record","microsoft_dynamics_365_list_records","microsoft_dynamics_365_qualify_lead","microsoft_dynamics_365_search_records","microsoft_dynamics_365_update_record","microsoft_excel_clear_range","microsoft_excel_create_table","microsoft_excel_delete_worksheet","microsoft_excel_format_range","microsoft_excel_read","microsoft_excel_read_v2","microsoft_excel_sort_range","microsoft_excel_table_add","microsoft_excel_worksheet_add","microsoft_excel_write","microsoft_excel_write_v2","microsoft_planner_create_bucket","microsoft_planner_create_plan","microsoft_planner_create_task","microsoft_planner_delete_bucket","microsoft_planner_delete_plan","microsoft_planner_delete_task","microsoft_planner_get_plan_details","microsoft_planner_get_task_details","microsoft_planner_list_buckets","microsoft_planner_list_plans","microsoft_planner_read_bucket","microsoft_planner_read_plan","microsoft_planner_read_task","microsoft_planner_update_bucket","microsoft_planner_update_plan","microsoft_planner_update_plan_details","microsoft_planner_update_task","microsoft_planner_update_task_details","microsoft_teams_delete_channel_message","microsoft_teams_delete_chat_message","microsoft_teams_get_message","microsoft_teams_list_channel_members","microsoft_teams_list_channels","microsoft_teams_list_chat_members","microsoft_teams_list_chats","microsoft_teams_list_team_members","microsoft_teams_list_teams","microsoft_teams_read_channel","microsoft_teams_read_chat","microsoft_teams_reply_to_message","microsoft_teams_set_reaction","microsoft_teams_unset_reaction","microsoft_teams_update_channel_message","microsoft_teams_update_chat_message","microsoft_teams_write_channel","microsoft_teams_write_chat","microsoft_word_append","microsoft_word_create","microsoft_word_create_from_template","microsoft_word_export_pdf","microsoft_word_list","microsoft_word_read","microsoft_word_replace_text","microsoft_word_update","millionverifier_get_credits","millionverifier_verify_email","mintlify_create_agent_job","mintlify_create_assistant_message","mintlify_detect_ai_prose","mintlify_get_agent_job","mintlify_get_assistant_caller_stats","mintlify_get_assistant_conversations","mintlify_get_feedback","mintlify_get_feedback_by_page","mintlify_get_page_content","mintlify_get_searches","mintlify_get_update_status","mintlify_get_views","mintlify_get_visitors","mintlify_search","mintlify_send_agent_message","mintlify_trigger_automation","mintlify_trigger_preview","mintlify_trigger_update","mistral_parser","mistral_parser_v2","mistral_parser_v3","modal_call_function","modal_chat_completion","modal_list_models","monday_archive_item","monday_change_column_value","monday_create_board","monday_create_column","monday_create_group","monday_create_item","monday_create_subitem","monday_create_update","monday_delete_item","monday_duplicate_item","monday_get_board","monday_get_groups","monday_get_item","monday_get_items","monday_list_boards","monday_move_item_to_group","monday_search_items","monday_update_item","mongodb_delete","mongodb_execute","mongodb_insert","mongodb_introspect","mongodb_query","mongodb_update","mssql_delete","mssql_execute","mssql_insert","mssql_introspect","mssql_query","mssql_update","mysql_delete","mysql_execute","mysql_insert","mysql_introspect","mysql_query","mysql_update","neo4j_create","neo4j_delete","neo4j_execute","neo4j_introspect","neo4j_merge","neo4j_query","neo4j_update","netsuite_attach_record","netsuite_batch_create_records","netsuite_batch_delete_records","netsuite_batch_get_records","netsuite_batch_update_records","netsuite_batch_upsert_records","netsuite_create_record","netsuite_delete_record","netsuite_detach_record","netsuite_execute_action","netsuite_execute_dataset","netsuite_execute_suiteql","netsuite_get_async_result","netsuite_get_async_status","netsuite_get_governance_limits","netsuite_get_record","netsuite_get_record_form","netsuite_get_record_metadata","netsuite_get_select_options","netsuite_get_server_time","netsuite_get_subresource","netsuite_list_datasets","netsuite_list_record_types","netsuite_list_records","netsuite_transform_record","netsuite_update_record","netsuite_upsert_record","neverbounce_get_credits","neverbounce_verify_email","new_relic_create_deployment_event","new_relic_get_entity","new_relic_nrql_query","new_relic_search_entities","notion_add_database_row","notion_add_database_row_v2","notion_append_blocks","notion_append_blocks_v2","notion_create_comment","notion_create_comment_v2","notion_create_database","notion_create_database_v2","notion_create_page","notion_create_page_v2","notion_delete_block","notion_delete_block_v2","notion_list_comments","notion_list_comments_v2","notion_list_users","notion_list_users_v2","notion_query_database","notion_query_database_v2","notion_read","notion_read_database","notion_read_database_v2","notion_read_v2","notion_retrieve_block","notion_retrieve_block_children","notion_retrieve_block_children_v2","notion_retrieve_block_v2","notion_retrieve_user","notion_retrieve_user_v2","notion_search","notion_search_v2","notion_update_block","notion_update_block_v2","notion_update_page","notion_update_page_v2","notion_write","notion_write_v2","obsidian_append_active","obsidian_append_note","obsidian_append_periodic_note","obsidian_create_note","obsidian_delete_note","obsidian_execute_command","obsidian_get_active","obsidian_get_note","obsidian_get_periodic_note","obsidian_list_commands","obsidian_list_files","obsidian_open_file","obsidian_patch_active","obsidian_patch_note","obsidian_search","okta_activate_group_rule","okta_activate_user","okta_add_user_to_group","okta_assign_group_to_app","okta_assign_user_role","okta_assign_user_to_app","okta_clear_user_sessions","okta_create_group","okta_create_group_rule","okta_create_user","okta_deactivate_group_rule","okta_deactivate_user","okta_delete_group","okta_delete_group_rule","okta_delete_user","okta_enroll_factor","okta_get_app","okta_get_factor","okta_get_group","okta_get_group_rule","okta_get_logs","okta_get_session","okta_get_user","okta_list_app_groups","okta_list_app_users","okta_list_apps","okta_list_factors","okta_list_group_members","okta_list_group_rules","okta_list_groups","okta_list_user_roles","okta_list_users","okta_remove_group_from_app","okta_remove_user_from_app","okta_remove_user_from_group","okta_remove_user_role","okta_reset_all_factors","okta_reset_factor","okta_reset_password","okta_revoke_session","okta_suspend_user","okta_unsuspend_user","okta_update_group","okta_update_user","onedrive_copy","onedrive_create_folder","onedrive_create_share_link","onedrive_delete","onedrive_download","onedrive_get_drive_info","onedrive_get_item","onedrive_list","onedrive_move","onedrive_search","onedrive_upload","onepassword_create_item","onepassword_delete_item","onepassword_get_item","onepassword_get_item_file","onepassword_get_vault","onepassword_list_items","onepassword_list_vaults","onepassword_replace_item","onepassword_resolve_secret","onepassword_update_item","openai_embeddings","openai_image","outlook_calendar_create_event","outlook_calendar_delete_event","outlook_calendar_get_event","outlook_calendar_list_events","outlook_calendar_respond","outlook_calendar_update_event","outlook_copy","outlook_create_folder","outlook_delete","outlook_draft","outlook_forward","outlook_get_attachment","outlook_list_attachments","outlook_list_folders","outlook_mark_read","outlook_mark_unread","outlook_move","outlook_read","outlook_reply","outlook_reply_all","outlook_search","outlook_send","outlook_update_message","pagerduty_add_note","pagerduty_create_incident","pagerduty_get_incident","pagerduty_get_service","pagerduty_list_escalation_policies","pagerduty_list_incident_alerts","pagerduty_list_incidents","pagerduty_list_oncalls","pagerduty_list_schedules","pagerduty_list_services","pagerduty_list_users","pagerduty_merge_incidents","pagerduty_send_event","pagerduty_snooze_incident","pagerduty_update_incident","parallel_deep_research","parallel_extract","parallel_search","pdl_autocomplete","pdl_bulk_company_enrich","pdl_bulk_person_enrich","pdl_clean_company","pdl_clean_location","pdl_clean_school","pdl_company_enrich","pdl_company_search","pdl_person_enrich","pdl_person_identify","pdl_person_search","perplexity_chat","perplexity_search","persona_approve_inquiry","persona_create_account","persona_create_inquiry","persona_create_report","persona_decline_inquiry","persona_expire_inquiry","persona_generate_inquiry_link","persona_get_account","persona_get_case","persona_get_document","persona_get_inquiry","persona_get_report","persona_get_verification","persona_import_accounts","persona_list_accounts","persona_list_cases","persona_list_inquiries","persona_list_inquiry_templates","persona_list_reports","persona_mark_inquiry_for_review","persona_print_inquiry_pdf","persona_redact_account","persona_redact_inquiry","persona_resume_inquiry","persona_update_account","persona_update_inquiry","pinecone_delete_vectors","pinecone_describe_index","pinecone_describe_index_stats","pinecone_fetch","pinecone_generate_embeddings","pinecone_list_indexes","pinecone_list_vector_ids","pinecone_search_text","pinecone_search_vector","pinecone_update_vector","pinecone_upsert_text","pipedrive_create_activity","pipedrive_create_deal","pipedrive_create_lead","pipedrive_create_project","pipedrive_delete_lead","pipedrive_get_activities","pipedrive_get_all_deals","pipedrive_get_deal","pipedrive_get_files","pipedrive_get_leads","pipedrive_get_mail_messages","pipedrive_get_mail_thread","pipedrive_get_pipeline_deals","pipedrive_get_pipelines","pipedrive_get_projects","pipedrive_update_activity","pipedrive_update_deal","pipedrive_update_lead","pitchbook_company_active_investors","pitchbook_company_bio","pitchbook_company_deal_service_providers","pitchbook_company_deals","pitchbook_company_financials","pitchbook_company_general_service_providers","pitchbook_company_industries","pitchbook_company_investors","pitchbook_company_most_recent_debt_financing","pitchbook_company_most_recent_financials","pitchbook_company_most_recent_financing","pitchbook_company_search","pitchbook_company_similar_companies","pitchbook_company_social_analytics","pitchbook_company_updates","pitchbook_company_vc_exit_predictions","pitchbook_contracts_history","pitchbook_cost_of_calls","pitchbook_credit_history","pitchbook_credit_news","pitchbook_credit_news_bulk","pitchbook_credit_news_most_recent","pitchbook_credit_news_search","pitchbook_deal_bio","pitchbook_deal_cap_table_history","pitchbook_deal_debt_lenders","pitchbook_deal_detailed","pitchbook_deal_investors","pitchbook_deal_multiples","pitchbook_deal_search","pitchbook_deal_service_providers","pitchbook_deal_stock_info","pitchbook_deal_tranche_info","pitchbook_deal_updates","pitchbook_deal_valuation","pitchbook_entity_affiliates","pitchbook_entity_locations","pitchbook_entity_news","pitchbook_entity_people","pitchbook_entity_updates","pitchbook_fund_active_investments","pitchbook_fund_benchmark","pitchbook_fund_bio","pitchbook_fund_cash_flows","pitchbook_fund_commitments","pitchbook_fund_investment_preferences","pitchbook_fund_investments","pitchbook_fund_performance","pitchbook_fund_search","pitchbook_fund_team","pitchbook_fund_updates","pitchbook_investor_active_investments","pitchbook_investor_bio","pitchbook_investor_board_seats","pitchbook_investor_deal_service_providers","pitchbook_investor_funds","pitchbook_investor_general_service_providers","pitchbook_investor_investments","pitchbook_investor_last_closed_fund","pitchbook_investor_preferences","pitchbook_investor_search","pitchbook_investor_updates","pitchbook_limited_partner_actual_allocations","pitchbook_limited_partner_bio","pitchbook_limited_partner_commitment_aggregates","pitchbook_limited_partner_commitment_preferences","pitchbook_limited_partner_commitments_detailed","pitchbook_limited_partner_search","pitchbook_limited_partner_service_providers","pitchbook_limited_partner_target_allocations","pitchbook_limited_partner_updates","pitchbook_lookup_table_structure","pitchbook_lookup_tables","pitchbook_patent_detailed","pitchbook_patent_search","pitchbook_people_search","pitchbook_person_bio","pitchbook_person_contact","pitchbook_person_education_work","pitchbook_sandbox_entities","pitchbook_search","pitchbook_service_provider_bio","pitchbook_service_provider_search","pitchbook_service_provider_updates","pitchbook_serviced_companies","pitchbook_serviced_deals","pitchbook_serviced_funds","pitchbook_serviced_investors","pitchbook_serviced_limited_partners","pitchbook_shared_search","pitchbook_usage_report","polymarket_get_activity","polymarket_get_event","polymarket_get_events","polymarket_get_holders","polymarket_get_last_trade_price","polymarket_get_leaderboard","polymarket_get_market","polymarket_get_markets","polymarket_get_midpoint","polymarket_get_orderbook","polymarket_get_positions","polymarket_get_price","polymarket_get_price_history","polymarket_get_series","polymarket_get_series_by_id","polymarket_get_spread","polymarket_get_tags","polymarket_get_tick_size","polymarket_get_trades","polymarket_search","postgresql_delete","postgresql_execute","postgresql_insert","postgresql_introspect","postgresql_query","postgresql_update","posthog_batch_events","posthog_capture_event","posthog_create_annotation","posthog_create_cohort","posthog_create_dashboard","posthog_create_experiment","posthog_create_feature_flag","posthog_create_insight","posthog_create_survey","posthog_delete_feature_flag","posthog_delete_person","posthog_delete_survey","posthog_evaluate_flags","posthog_get_cohort","posthog_get_dashboard","posthog_get_event_definition","posthog_get_experiment","posthog_get_feature_flag","posthog_get_insight","posthog_get_organization","posthog_get_person","posthog_get_project","posthog_get_property_definition","posthog_get_session_recording","posthog_get_survey","posthog_list_actions","posthog_list_annotations","posthog_list_cohorts","posthog_list_dashboards","posthog_list_event_definitions","posthog_list_experiments","posthog_list_feature_flags","posthog_list_insights","posthog_list_organizations","posthog_list_persons","posthog_list_projects","posthog_list_property_definitions","posthog_list_recording_playlists","posthog_list_session_recordings","posthog_list_surveys","posthog_query","posthog_update_cohort","posthog_update_event_definition","posthog_update_experiment","posthog_update_feature_flag","posthog_update_insight","posthog_update_property_definition","posthog_update_survey","profound_bot_logs","profound_bots_report","profound_category_assets","profound_category_personas","profound_category_prompts","profound_category_tags","profound_category_topics","profound_citation_prompts","profound_citations_report","profound_list_assets","profound_list_categories","profound_list_domains","profound_list_models","profound_list_optimizations","profound_list_personas","profound_list_regions","profound_optimization_analysis","profound_prompt_answers","profound_prompt_volume","profound_query_fanouts","profound_raw_logs","profound_referrals_report","profound_sentiment_report","profound_visibility_report","prospeo_account_information","prospeo_bulk_enrich_company","prospeo_bulk_enrich_person","prospeo_enrich_company","prospeo_enrich_person","prospeo_search_company","prospeo_search_person","prospeo_search_suggestions","pulse_parser","pulse_parser_v2","qdrant_fetch_points","qdrant_search_vector","qdrant_upsert_points","quartr_get_audio","quartr_get_company","quartr_get_event","quartr_get_event_summary","quartr_get_report","quartr_get_slide_deck","quartr_get_transcript","quartr_list_audio","quartr_list_companies","quartr_list_document_types","quartr_list_documents","quartr_list_event_types","quartr_list_events","quartr_list_live_events","quartr_list_reports","quartr_list_slide_decks","quartr_list_transcripts","quickbooks_add_attachment","quickbooks_create_bill","quickbooks_create_bill_payment","quickbooks_create_credit_memo","quickbooks_create_customer","quickbooks_create_customer_payment","quickbooks_create_deposit","quickbooks_create_employee","quickbooks_create_estimate","quickbooks_create_invoice","quickbooks_create_item","quickbooks_create_journal_entry","quickbooks_create_purchase","quickbooks_create_purchase_order","quickbooks_create_refund_receipt","quickbooks_create_sales_receipt","quickbooks_create_vendor","quickbooks_create_vendor_credit","quickbooks_download_attachment","quickbooks_download_transaction_pdf","quickbooks_email_transaction","quickbooks_get_company_info","quickbooks_read_accounting_transactions","quickbooks_read_attachments","quickbooks_read_master_data","quickbooks_read_purchasing_transactions","quickbooks_read_sales_transactions","quickbooks_run_financial_report","quickbooks_update_bill","quickbooks_update_bill_payment","quickbooks_update_credit_memo","quickbooks_update_customer","quickbooks_update_customer_payment","quickbooks_update_deposit","quickbooks_update_employee","quickbooks_update_estimate","quickbooks_update_invoice","quickbooks_update_item","quickbooks_update_journal_entry","quickbooks_update_purchase","quickbooks_update_purchase_order","quickbooks_update_refund_receipt","quickbooks_update_sales_receipt","quickbooks_update_vendor","quickbooks_update_vendor_credit","quickbooks_void_bill_payment","quickbooks_void_customer_payment","quickbooks_void_invoice","quickbooks_void_sales_receipt","quiver_image_to_svg","quiver_list_models","quiver_text_to_svg","rabbitmq_create_binding","rabbitmq_create_exchange","rabbitmq_create_policy","rabbitmq_create_queue","rabbitmq_delete_binding","rabbitmq_delete_exchange","rabbitmq_delete_policy","rabbitmq_delete_queue","rabbitmq_get_exchange","rabbitmq_get_messages","rabbitmq_get_overview","rabbitmq_get_queue","rabbitmq_health_check","rabbitmq_list_bindings","rabbitmq_list_channels","rabbitmq_list_connections","rabbitmq_list_consumers","rabbitmq_list_exchange_bindings","rabbitmq_list_exchanges","rabbitmq_list_nodes","rabbitmq_list_policies","rabbitmq_list_queues","rabbitmq_list_vhosts","rabbitmq_publish_message","rabbitmq_purge_queue","railway_create_environment","railway_create_project","railway_create_service","railway_delete_environment","railway_delete_project","railway_delete_service","railway_delete_variable","railway_deploy_service","railway_get_deployment","railway_get_deployment_logs","railway_get_project","railway_list_deployments","railway_list_project_members","railway_list_projects","railway_list_variables","railway_restart_deployment","railway_rollback_deployment","railway_transfer_project","railway_update_project","railway_upsert_variable","rb2b_credit_check","rb2b_email_to_activity","rb2b_hem_to_best_linkedin","rb2b_hem_to_business_profile","rb2b_hem_to_linkedin","rb2b_hem_to_maid","rb2b_ip_to_company","rb2b_ip_to_hem","rb2b_ip_to_maid","rb2b_linkedin_slug_search","rb2b_linkedin_to_best_personal_email","rb2b_linkedin_to_business_profile","rb2b_linkedin_to_hashed_emails","rb2b_linkedin_to_mobile_phone","rb2b_linkedin_to_personal_email","rds_delete","rds_execute","rds_insert","rds_introspect","rds_query","rds_update","reddit_delete","reddit_edit","reddit_get_comments","reddit_get_controversial","reddit_get_info","reddit_get_me","reddit_get_messages","reddit_get_posts","reddit_get_saved","reddit_get_subreddit_info","reddit_get_subreddit_rules","reddit_get_user","reddit_get_user_comments","reddit_get_user_posts","reddit_hide","reddit_hot_posts","reddit_list_my_subreddits","reddit_lock","reddit_mark_all_read","reddit_mark_read","reddit_marknsfw","reddit_mod_approve","reddit_mod_distinguish","reddit_mod_remove","reddit_mod_sticky","reddit_reply","reddit_report","reddit_save","reddit_search","reddit_search_subreddits","reddit_send_message","reddit_submit_post","reddit_subscribe","reddit_unhide","reddit_unlock","reddit_unmarknsfw","reddit_unsave","reddit_vote","redis_command","redis_delete","redis_exists","redis_expire","redis_get","redis_hdel","redis_hget","redis_hgetall","redis_hset","redis_incr","redis_incrby","redis_keys","redis_llen","redis_lpop","redis_lpush","redis_lrange","redis_persist","redis_rpop","redis_rpush","redis_set","redis_setnx","redis_ttl","reducto_parser","reducto_parser_v2","resend_cancel_email","resend_create_audience","resend_create_broadcast","resend_create_contact","resend_delete_audience","resend_delete_contact","resend_get_audience","resend_get_broadcast","resend_get_contact","resend_get_email","resend_list_audiences","resend_list_contacts","resend_list_domains","resend_send","resend_send_broadcast","resend_update_contact","revenuecat_create_purchase","revenuecat_defer_google_subscription","revenuecat_delete_customer","revenuecat_get_customer","revenuecat_grant_entitlement","revenuecat_list_offerings","revenuecat_refund_google_subscription","revenuecat_revoke_entitlement","revenuecat_revoke_google_subscription","revenuecat_update_subscriber_attributes","rippling_bulk_create_custom_object_records","rippling_bulk_delete_custom_object_records","rippling_bulk_update_custom_object_records","rippling_create_business_partner","rippling_create_business_partner_group","rippling_create_custom_app","rippling_create_custom_object","rippling_create_custom_object_field","rippling_create_custom_object_record","rippling_create_custom_page","rippling_create_custom_setting","rippling_create_department","rippling_create_draft_hires","rippling_create_object_category","rippling_create_title","rippling_create_work_location","rippling_delete_business_partner","rippling_delete_business_partner_group","rippling_delete_custom_app","rippling_delete_custom_object","rippling_delete_custom_object_field","rippling_delete_custom_object_record","rippling_delete_custom_page","rippling_delete_custom_setting","rippling_delete_object_category","rippling_delete_title","rippling_delete_work_location","rippling_get_business_partner","rippling_get_business_partner_group","rippling_get_current_user","rippling_get_custom_app","rippling_get_custom_object","rippling_get_custom_object_field","rippling_get_custom_object_record","rippling_get_custom_object_record_by_external_id","rippling_get_custom_page","rippling_get_custom_setting","rippling_get_department","rippling_get_employment_type","rippling_get_job_function","rippling_get_object_category","rippling_get_report_run","rippling_get_supergroup","rippling_get_team","rippling_get_title","rippling_get_user","rippling_get_work_location","rippling_get_worker","rippling_list_business_partner_groups","rippling_list_business_partners","rippling_list_companies","rippling_list_custom_apps","rippling_list_custom_fields","rippling_list_custom_object_fields","rippling_list_custom_object_records","rippling_list_custom_objects","rippling_list_custom_pages","rippling_list_custom_settings","rippling_list_departments","rippling_list_employment_types","rippling_list_entitlements","rippling_list_job_functions","rippling_list_object_categories","rippling_list_supergroup_exclusion_members","rippling_list_supergroup_inclusion_members","rippling_list_supergroup_members","rippling_list_supergroups","rippling_list_teams","rippling_list_titles","rippling_list_users","rippling_list_work_locations","rippling_list_workers","rippling_query_custom_object_records","rippling_trigger_report_run","rippling_update_custom_app","rippling_update_custom_object","rippling_update_custom_object_field","rippling_update_custom_object_record","rippling_update_custom_page","rippling_update_custom_setting","rippling_update_department","rippling_update_object_category","rippling_update_supergroup_exclusion_members","rippling_update_supergroup_inclusion_members","rippling_update_title","rippling_update_work_location","rocketlane_add_field_option","rocketlane_add_project_members","rocketlane_add_task_assignees","rocketlane_add_task_dependencies","rocketlane_add_task_followers","rocketlane_archive_project","rocketlane_assign_placeholders","rocketlane_create_field","rocketlane_create_phase","rocketlane_create_project","rocketlane_create_space","rocketlane_create_space_document","rocketlane_create_task","rocketlane_create_time_entry","rocketlane_create_time_off","rocketlane_delete_field","rocketlane_delete_phase","rocketlane_delete_project","rocketlane_delete_space","rocketlane_delete_space_document","rocketlane_delete_task","rocketlane_delete_time_entry","rocketlane_delete_time_off","rocketlane_get_field","rocketlane_get_invoice","rocketlane_get_invoice_line_items","rocketlane_get_invoice_payments","rocketlane_get_phase","rocketlane_get_project","rocketlane_get_space","rocketlane_get_space_document","rocketlane_get_task","rocketlane_get_time_entry","rocketlane_get_time_off","rocketlane_get_user","rocketlane_import_template","rocketlane_list_fields","rocketlane_list_invoices","rocketlane_list_phases","rocketlane_list_placeholders","rocketlane_list_projects","rocketlane_list_resource_allocations","rocketlane_list_space_documents","rocketlane_list_spaces","rocketlane_list_tasks","rocketlane_list_time_entries","rocketlane_list_time_entry_categories","rocketlane_list_time_offs","rocketlane_list_users","rocketlane_move_task_to_phase","rocketlane_remove_project_members","rocketlane_remove_task_assignees","rocketlane_remove_task_dependencies","rocketlane_remove_task_followers","rocketlane_search_time_entries","rocketlane_unassign_placeholders","rocketlane_update_field","rocketlane_update_field_option","rocketlane_update_phase","rocketlane_update_project","rocketlane_update_space","rocketlane_update_space_document","rocketlane_update_task","rocketlane_update_time_entry","rootly_acknowledge_alert","rootly_add_incident_event","rootly_add_subscribers","rootly_assign_incident_role","rootly_create_action_item","rootly_create_alert","rootly_create_incident","rootly_create_status_page_event","rootly_delete_action_item","rootly_delete_incident","rootly_escalate_alert","rootly_get_alert","rootly_get_incident","rootly_list_action_items","rootly_list_alerts","rootly_list_causes","rootly_list_environments","rootly_list_escalation_policies","rootly_list_functionalities","rootly_list_incident_events","rootly_list_incident_roles","rootly_list_incident_types","rootly_list_incidents","rootly_list_on_calls","rootly_list_playbooks","rootly_list_retrospectives","rootly_list_schedules","rootly_list_services","rootly_list_severities","rootly_list_teams","rootly_list_users","rootly_mitigate_incident","rootly_remove_subscribers","rootly_resolve_alert","rootly_resolve_incident","rootly_run_workflow","rootly_snooze_alert","rootly_unassign_incident_role","rootly_update_action_item","rootly_update_alert","rootly_update_incident","s3_copy_object","s3_create_bucket","s3_delete_bucket","s3_delete_object","s3_delete_objects","s3_get_object","s3_head_object","s3_list_buckets","s3_list_objects","s3_presigned_url","s3_put_object","sailpoint_approve_access_request","sailpoint_cancel_access_request","sailpoint_decide_certification_review_items","sailpoint_get_access_profile","sailpoint_get_access_profile_entitlements","sailpoint_get_access_request_config","sailpoint_get_access_request_status","sailpoint_get_account","sailpoint_get_account_activity","sailpoint_get_account_entitlements","sailpoint_get_account_selections","sailpoint_get_campaign","sailpoint_get_certification","sailpoint_get_entitlement","sailpoint_get_entitlement_request_config","sailpoint_get_identity","sailpoint_get_role","sailpoint_get_role_entitlements","sailpoint_get_source","sailpoint_get_task_status","sailpoint_list_access_profiles","sailpoint_list_account_activities","sailpoint_list_accounts","sailpoint_list_campaigns","sailpoint_list_certification_review_items","sailpoint_list_certifications","sailpoint_list_entitlements","sailpoint_list_identities","sailpoint_list_identity_entitlements","sailpoint_list_pending_access_request_approvals","sailpoint_list_roles","sailpoint_list_sources","sailpoint_load_accounts","sailpoint_load_entitlements","sailpoint_reject_access_request","sailpoint_request_access","sailpoint_search","sailpoint_search_aggregate","sailpoint_search_count","sailpoint_sign_off_certification","salesforce_create_account","salesforce_create_case","salesforce_create_contact","salesforce_create_custom_field","salesforce_create_custom_object","salesforce_create_lead","salesforce_create_opportunity","salesforce_create_task","salesforce_delete_account","salesforce_delete_case","salesforce_delete_contact","salesforce_delete_custom_field","salesforce_delete_lead","salesforce_delete_opportunity","salesforce_delete_task","salesforce_describe_object","salesforce_get_accounts","salesforce_get_cases","salesforce_get_contacts","salesforce_get_dashboard","salesforce_get_leads","salesforce_get_opportunities","salesforce_get_report","salesforce_get_tasks","salesforce_list_dashboards","salesforce_list_objects","salesforce_list_report_types","salesforce_list_reports","salesforce_query","salesforce_query_more","salesforce_refresh_dashboard","salesforce_run_report","salesforce_tooling_query","salesforce_update_account","salesforce_update_case","salesforce_update_contact","salesforce_update_custom_field","salesforce_update_lead","salesforce_update_opportunity","salesforce_update_task","sap_concur_approve_expense_report","sap_concur_associate_attendees","sap_concur_create_cash_advance","sap_concur_create_expected_expense","sap_concur_create_expense_report","sap_concur_create_list_item","sap_concur_create_purchase_request","sap_concur_create_quick_expense","sap_concur_create_quick_expense_with_image","sap_concur_create_report_comment","sap_concur_create_travel_request","sap_concur_create_user","sap_concur_delete_expected_expense","sap_concur_delete_expense","sap_concur_delete_expense_report","sap_concur_delete_list_item","sap_concur_delete_travel_request","sap_concur_delete_user","sap_concur_get_allocation","sap_concur_get_budget","sap_concur_get_cash_advance","sap_concur_get_expected_expense","sap_concur_get_expense","sap_concur_get_expense_report","sap_concur_get_itemizations","sap_concur_get_itinerary","sap_concur_get_list","sap_concur_get_list_item","sap_concur_get_purchase_request","sap_concur_get_receipt","sap_concur_get_receipt_status","sap_concur_get_request_cash_advance","sap_concur_get_travel_profile","sap_concur_get_travel_request","sap_concur_get_user","sap_concur_issue_cash_advance","sap_concur_list_allocations","sap_concur_list_attendee_associations","sap_concur_list_budget_categories","sap_concur_list_budgets","sap_concur_list_exceptions","sap_concur_list_expected_expenses","sap_concur_list_expense_reports","sap_concur_list_expenses","sap_concur_list_itineraries","sap_concur_list_list_items","sap_concur_list_lists","sap_concur_list_receipts","sap_concur_list_report_comments","sap_concur_list_reports_to_approve","sap_concur_list_travel_profiles_summary","sap_concur_list_travel_request_comments","sap_concur_list_travel_requests","sap_concur_list_users","sap_concur_move_travel_request","sap_concur_recall_expense_report","sap_concur_remove_all_attendees","sap_concur_search_locations","sap_concur_search_users","sap_concur_send_back_expense_report","sap_concur_submit_expense_report","sap_concur_update_allocation","sap_concur_update_expected_expense","sap_concur_update_expense","sap_concur_update_expense_report","sap_concur_update_list_item","sap_concur_update_travel_request","sap_concur_update_user","sap_concur_upload_exchange_rates","sap_concur_upload_receipt_image","sap_s4hana_create_business_partner","sap_s4hana_create_purchase_order","sap_s4hana_create_purchase_requisition","sap_s4hana_create_sales_order","sap_s4hana_delete_sales_order","sap_s4hana_get_billing_document","sap_s4hana_get_business_partner","sap_s4hana_get_customer","sap_s4hana_get_inbound_delivery","sap_s4hana_get_material_document","sap_s4hana_get_outbound_delivery","sap_s4hana_get_product","sap_s4hana_get_purchase_order","sap_s4hana_get_purchase_requisition","sap_s4hana_get_sales_order","sap_s4hana_get_supplier","sap_s4hana_get_supplier_invoice","sap_s4hana_list_billing_documents","sap_s4hana_list_business_partners","sap_s4hana_list_customers","sap_s4hana_list_inbound_deliveries","sap_s4hana_list_material_documents","sap_s4hana_list_material_stock","sap_s4hana_list_outbound_deliveries","sap_s4hana_list_products","sap_s4hana_list_purchase_orders","sap_s4hana_list_purchase_requisitions","sap_s4hana_list_sales_orders","sap_s4hana_list_supplier_invoices","sap_s4hana_list_suppliers","sap_s4hana_odata_query","sap_s4hana_update_business_partner","sap_s4hana_update_customer","sap_s4hana_update_product","sap_s4hana_update_purchase_order","sap_s4hana_update_purchase_requisition","sap_s4hana_update_sales_order","sap_s4hana_update_supplier","search_tool","secrets_manager_create_secret","secrets_manager_delete_secret","secrets_manager_describe_secret","secrets_manager_get_secret","secrets_manager_list_secrets","secrets_manager_restore_secret","secrets_manager_rotate_secret","secrets_manager_tag_resource","secrets_manager_untag_resource","secrets_manager_update_secret","semrush_backlinks","semrush_backlinks_anchors","semrush_backlinks_competitors","semrush_backlinks_geo_distribution","semrush_backlinks_indexed_pages","semrush_backlinks_overview","semrush_backlinks_tld_distribution","semrush_batch_keyword_overview","semrush_broad_match_keywords","semrush_domain_ad_copies","semrush_domain_ad_history","semrush_domain_organic_competitors","semrush_domain_organic_keywords","semrush_domain_overview","semrush_domain_overview_all","semrush_domain_overview_history","semrush_domain_paid_competitors","semrush_domain_paid_keywords","semrush_domain_pla_copies","semrush_domain_pla_keywords","semrush_domain_vs_domain","semrush_keyword_ad_history","semrush_keyword_difficulty","semrush_keyword_overview","semrush_keyword_overview_all","semrush_keyword_questions","semrush_organic_results","semrush_paid_results","semrush_referring_domains","semrush_referring_ips","semrush_related_keywords","semrush_subdomain_ad_copies","semrush_subdomain_organic_keywords","semrush_subdomain_overview","semrush_subdomain_overview_all","semrush_subdomain_overview_history","semrush_subdomain_paid_keywords","semrush_top_domains","semrush_url_organic_keywords","semrush_url_overview","semrush_url_overview_all","semrush_url_overview_history","semrush_url_paid_keywords","semrush_winners_and_losers","sendblue_evaluate_service","sendblue_get_message","sendblue_send_group_message","sendblue_send_message","sendblue_send_typing_indicator","sendgrid_add_contact","sendgrid_add_contacts_to_list","sendgrid_create_list","sendgrid_create_template","sendgrid_create_template_version","sendgrid_delete_contacts","sendgrid_delete_list","sendgrid_delete_template","sendgrid_get_contact","sendgrid_get_list","sendgrid_get_template","sendgrid_list_all_lists","sendgrid_list_templates","sendgrid_remove_contacts_from_list","sendgrid_search_contacts","sendgrid_send_mail","sentry_events_get","sentry_events_list","sentry_issues_get","sentry_issues_list","sentry_issues_update","sentry_projects_create","sentry_projects_get","sentry_projects_list","sentry_projects_update","sentry_releases_create","sentry_releases_deploy","sentry_releases_list","sentry_teams_list","serper_search","servicenow_add_incident_comment","servicenow_aggregate","servicenow_close_incident","servicenow_create_change_request","servicenow_create_incident","servicenow_create_record","servicenow_delete_record","servicenow_download_attachment","servicenow_find_user","servicenow_get_change_next_states","servicenow_get_change_request","servicenow_get_ci","servicenow_get_incident","servicenow_get_knowledge_article","servicenow_get_requested_item","servicenow_list_approvals","servicenow_list_attachments","servicenow_list_catalog_items","servicenow_list_change_requests","servicenow_list_change_tasks","servicenow_list_ci_relationships","servicenow_list_group_members","servicenow_list_incidents","servicenow_list_requested_items","servicenow_order_catalog_item","servicenow_read_record","servicenow_resolve_incident","servicenow_search_cis","servicenow_search_knowledge","servicenow_update_approval","servicenow_update_change_request","servicenow_update_change_state","servicenow_update_incident","servicenow_update_record","servicenow_upload_attachment","ses_create_configuration_set","ses_create_email_identity","ses_create_template","ses_delete_email_identity","ses_delete_suppressed_destination","ses_delete_template","ses_get_account","ses_get_email_identity","ses_get_suppressed_destination","ses_get_template","ses_list_identities","ses_list_suppressed_destinations","ses_list_templates","ses_put_suppressed_destination","ses_send_bulk_email","ses_send_custom_verification_email","ses_send_email","ses_send_templated_email","ses_update_template","sftp_delete","sftp_download","sftp_list","sftp_mkdir","sftp_upload","sharepoint_add_list_items","sharepoint_create_list","sharepoint_create_page","sharepoint_delete_file","sharepoint_delete_list_item","sharepoint_delete_page","sharepoint_download_file","sharepoint_get_drive_item","sharepoint_get_list","sharepoint_get_list_item","sharepoint_list_sites","sharepoint_publish_page","sharepoint_read_page","sharepoint_update_list","sharepoint_update_page","sharepoint_upload_file","shopify_adjust_inventory","shopify_cancel_order","shopify_create_customer","shopify_create_fulfillment","shopify_create_product","shopify_delete_customer","shopify_delete_product","shopify_get_collection","shopify_get_customer","shopify_get_inventory_level","shopify_get_order","shopify_get_product","shopify_list_collections","shopify_list_customers","shopify_list_inventory_items","shopify_list_locations","shopify_list_orders","shopify_list_products","shopify_update_customer","shopify_update_order","shopify_update_product","similarweb_bounce_rate","similarweb_page_views","similarweb_pages_per_visit","similarweb_traffic_visits","similarweb_visit_duration","similarweb_website_overview","sixtyfour_enrich_company","sixtyfour_enrich_lead","sixtyfour_find_email","sixtyfour_find_phone","slack_add_reaction","slack_archive_conversation","slack_canvas","slack_create_channel_canvas","slack_create_conversation","slack_delete_canvas","slack_delete_message","slack_delete_scheduled_message","slack_download","slack_edit_canvas","slack_ephemeral_message","slack_get_canvas","slack_get_channel_history","slack_get_channel_info","slack_get_message","slack_get_permalink","slack_get_thread","slack_get_thread_replies","slack_get_user","slack_get_user_presence","slack_invite_to_conversation","slack_list_canvases","slack_list_channels","slack_list_members","slack_list_scheduled_messages","slack_list_users","slack_lookup_canvas_sections","slack_message","slack_message_reader","slack_open_view","slack_publish_view","slack_push_view","slack_remove_reaction","slack_rename_agent_session_v2","slack_rename_conversation","slack_schedule_message","slack_set_agent_session_status_v2","slack_set_conversation_purpose","slack_set_conversation_topic","slack_set_status","slack_set_suggested_prompts","slack_set_suggested_prompts_v2","slack_set_title","slack_update_message","slack_update_view","smartlead_add_email_accounts_to_campaign","smartlead_add_leads_to_campaign","smartlead_create_campaign","smartlead_create_lead_list","smartlead_delete_campaign","smartlead_delete_campaign_webhook","smartlead_delete_lead_from_campaign","smartlead_delete_lead_list","smartlead_duplicate_campaign","smartlead_export_campaign_leads","smartlead_get_campaign","smartlead_get_campaign_analytics","smartlead_get_campaign_analytics_by_date","smartlead_get_campaign_lead_statistics","smartlead_get_campaign_mailbox_statistics","smartlead_get_campaign_sequences","smartlead_get_campaign_statistics","smartlead_get_campaign_top_level_analytics_by_date","smartlead_get_campaign_webhook_summary","smartlead_get_lead_by_email","smartlead_get_lead_by_id","smartlead_get_lead_list","smartlead_get_lead_message_history","smartlead_list_campaign_email_accounts","smartlead_list_campaign_leads","smartlead_list_campaign_webhooks","smartlead_list_campaigns","smartlead_list_clients","smartlead_list_email_accounts","smartlead_list_inbox_replies","smartlead_list_lead_activities","smartlead_list_lead_categories","smartlead_list_lead_lists","smartlead_mark_lead_complete","smartlead_pause_lead","smartlead_remove_email_accounts_from_campaign","smartlead_resume_lead","smartlead_save_campaign_sequences","smartlead_unsubscribe_lead_from_campaign","smartlead_unsubscribe_lead_globally","smartlead_update_campaign_schedule","smartlead_update_campaign_settings","smartlead_update_campaign_status","smartlead_update_lead","smartlead_update_lead_category","smartlead_update_lead_list","smartlead_upsert_campaign_webhook","sms_send","smtp_send_mail","snowflake_alter_warehouse","snowflake_call_procedure","snowflake_cancel_statement","snowflake_cancel_task_run","snowflake_delete_rows","snowflake_execute_sql","snowflake_get_statement","snowflake_get_task","snowflake_get_task_run","snowflake_get_task_run_output","snowflake_get_warehouse","snowflake_insert_rows","snowflake_introspect_schema","snowflake_list_copy_history","snowflake_list_databases","snowflake_list_query_history","snowflake_list_schemas","snowflake_list_tables","snowflake_list_task_runs","snowflake_list_tasks","snowflake_list_warehouses","snowflake_load_data","snowflake_resume_task","snowflake_resume_warehouse","snowflake_run_task","snowflake_suspend_task","snowflake_suspend_warehouse","snowflake_unload_data","snowflake_update_rows","snowflake_upsert_rows","splunk_cancel_search_job","splunk_create_search_job","splunk_dispatch_saved_search","splunk_get_fired_alerts","splunk_get_saved_search","splunk_get_search_job","splunk_get_search_results","splunk_list_apps","splunk_list_fired_alerts","splunk_list_indexes","splunk_list_saved_searches","splunk_run_search","sportmonks_core_get_cities","sportmonks_core_get_city","sportmonks_core_get_continent","sportmonks_core_get_continents","sportmonks_core_get_countries","sportmonks_core_get_country","sportmonks_core_get_entity_filters","sportmonks_core_get_my_usage","sportmonks_core_get_region","sportmonks_core_get_regions","sportmonks_core_get_timezones","sportmonks_core_get_type","sportmonks_core_get_type_by_entity","sportmonks_core_get_types","sportmonks_core_search_cities","sportmonks_core_search_countries","sportmonks_core_search_regions","sportmonks_football_expected_by_player","sportmonks_football_expected_by_team","sportmonks_football_get_all_commentaries","sportmonks_football_get_all_fixtures","sportmonks_football_get_all_players","sportmonks_football_get_all_rivals","sportmonks_football_get_all_teams","sportmonks_football_get_all_transfer_rumours","sportmonks_football_get_all_transfers","sportmonks_football_get_brackets_by_season","sportmonks_football_get_coach","sportmonks_football_get_coaches","sportmonks_football_get_coaches_by_country","sportmonks_football_get_commentaries_by_fixture","sportmonks_football_get_current_leagues_by_team","sportmonks_football_get_expected_lineups_by_player","sportmonks_football_get_expected_lineups_by_team","sportmonks_football_get_extended_team_squad","sportmonks_football_get_fixture","sportmonks_football_get_fixtures_by_date","sportmonks_football_get_fixtures_by_date_range","sportmonks_football_get_fixtures_by_date_range_for_team","sportmonks_football_get_fixtures_by_ids","sportmonks_football_get_grouped_standings_by_round","sportmonks_football_get_head_to_head","sportmonks_football_get_inplay_livescores","sportmonks_football_get_latest_coaches","sportmonks_football_get_latest_fixtures","sportmonks_football_get_latest_livescores","sportmonks_football_get_latest_players","sportmonks_football_get_latest_totw","sportmonks_football_get_latest_transfers","sportmonks_football_get_league","sportmonks_football_get_leagues","sportmonks_football_get_leagues_by_country","sportmonks_football_get_leagues_by_date","sportmonks_football_get_leagues_by_team","sportmonks_football_get_live_leagues","sportmonks_football_get_live_probabilities","sportmonks_football_get_live_probabilities_by_fixture","sportmonks_football_get_live_standings_by_league","sportmonks_football_get_livescores","sportmonks_football_get_match_facts","sportmonks_football_get_match_facts_by_date_range","sportmonks_football_get_match_facts_by_fixture","sportmonks_football_get_match_facts_by_league","sportmonks_football_get_past_fixtures_by_tv_station","sportmonks_football_get_player","sportmonks_football_get_players_by_country","sportmonks_football_get_postmatch_news","sportmonks_football_get_postmatch_news_by_season","sportmonks_football_get_predictability_by_league","sportmonks_football_get_prematch_news","sportmonks_football_get_prematch_news_by_season","sportmonks_football_get_prematch_news_upcoming","sportmonks_football_get_probabilities","sportmonks_football_get_probabilities_by_fixture","sportmonks_football_get_referee","sportmonks_football_get_referees","sportmonks_football_get_referees_by_country","sportmonks_football_get_referees_by_season","sportmonks_football_get_rivals_by_team","sportmonks_football_get_round","sportmonks_football_get_round_statistics","sportmonks_football_get_rounds","sportmonks_football_get_rounds_by_season","sportmonks_football_get_schedules_by_season","sportmonks_football_get_schedules_by_season_and_team","sportmonks_football_get_schedules_by_team","sportmonks_football_get_season","sportmonks_football_get_seasons","sportmonks_football_get_seasons_by_team","sportmonks_football_get_stage","sportmonks_football_get_stage_statistics","sportmonks_football_get_stages","sportmonks_football_get_stages_by_season","sportmonks_football_get_standing_corrections_by_season","sportmonks_football_get_standings","sportmonks_football_get_standings_by_round","sportmonks_football_get_standings_by_season","sportmonks_football_get_state","sportmonks_football_get_states","sportmonks_football_get_team","sportmonks_football_get_team_rankings","sportmonks_football_get_team_rankings_by_date","sportmonks_football_get_team_rankings_by_team","sportmonks_football_get_team_squad","sportmonks_football_get_team_squad_by_season","sportmonks_football_get_teams_by_country","sportmonks_football_get_teams_by_season","sportmonks_football_get_topscorers_by_season","sportmonks_football_get_topscorers_by_stage","sportmonks_football_get_totw","sportmonks_football_get_totw_by_round","sportmonks_football_get_transfer","sportmonks_football_get_transfer_rumour","sportmonks_football_get_transfer_rumours_between_dates","sportmonks_football_get_transfer_rumours_by_player","sportmonks_football_get_transfer_rumours_by_team","sportmonks_football_get_transfers_between_dates","sportmonks_football_get_transfers_by_player","sportmonks_football_get_transfers_by_team","sportmonks_football_get_tv_station","sportmonks_football_get_tv_stations","sportmonks_football_get_tv_stations_by_fixture","sportmonks_football_get_upcoming_fixtures_by_market","sportmonks_football_get_upcoming_fixtures_by_tv_station","sportmonks_football_get_value_bets","sportmonks_football_get_value_bets_by_fixture","sportmonks_football_get_venue","sportmonks_football_get_venues","sportmonks_football_get_venues_by_season","sportmonks_football_search_coaches","sportmonks_football_search_fixtures","sportmonks_football_search_leagues","sportmonks_football_search_players","sportmonks_football_search_referees","sportmonks_football_search_rounds","sportmonks_football_search_seasons","sportmonks_football_search_stages","sportmonks_football_search_teams","sportmonks_football_search_venues","sportmonks_motorsport_get_all_fixtures","sportmonks_motorsport_get_current_leagues_by_team","sportmonks_motorsport_get_driver","sportmonks_motorsport_get_driver_standings","sportmonks_motorsport_get_driver_standings_by_season","sportmonks_motorsport_get_drivers","sportmonks_motorsport_get_drivers_by_country","sportmonks_motorsport_get_drivers_by_season","sportmonks_motorsport_get_fixture","sportmonks_motorsport_get_fixtures_by_date","sportmonks_motorsport_get_fixtures_by_date_range","sportmonks_motorsport_get_fixtures_by_ids","sportmonks_motorsport_get_laps_by_fixture","sportmonks_motorsport_get_laps_by_fixture_and_driver","sportmonks_motorsport_get_laps_by_fixture_and_lap","sportmonks_motorsport_get_latest_laps_by_fixture","sportmonks_motorsport_get_latest_pitstops_by_fixture","sportmonks_motorsport_get_latest_stints_by_fixture","sportmonks_motorsport_get_latest_updated_drivers","sportmonks_motorsport_get_latest_updated_fixtures","sportmonks_motorsport_get_league","sportmonks_motorsport_get_leagues","sportmonks_motorsport_get_leagues_by_country","sportmonks_motorsport_get_leagues_by_date","sportmonks_motorsport_get_leagues_by_live","sportmonks_motorsport_get_leagues_by_team","sportmonks_motorsport_get_livescores","sportmonks_motorsport_get_pitstops_by_fixture","sportmonks_motorsport_get_pitstops_by_fixture_and_driver","sportmonks_motorsport_get_pitstops_by_fixture_and_lap","sportmonks_motorsport_get_race_results_by_season_and_driver","sportmonks_motorsport_get_race_results_by_season_and_team","sportmonks_motorsport_get_schedules_by_season","sportmonks_motorsport_get_season","sportmonks_motorsport_get_seasons","sportmonks_motorsport_get_stage","sportmonks_motorsport_get_stages","sportmonks_motorsport_get_stages_by_season","sportmonks_motorsport_get_state","sportmonks_motorsport_get_states","sportmonks_motorsport_get_stints_by_fixture","sportmonks_motorsport_get_stints_by_fixture_and_driver","sportmonks_motorsport_get_stints_by_fixture_and_stint","sportmonks_motorsport_get_team","sportmonks_motorsport_get_team_standings","sportmonks_motorsport_get_team_standings_by_season","sportmonks_motorsport_get_teams","sportmonks_motorsport_get_teams_by_country","sportmonks_motorsport_get_teams_by_season","sportmonks_motorsport_get_venue","sportmonks_motorsport_get_venues","sportmonks_motorsport_get_venues_by_season","sportmonks_motorsport_search_drivers","sportmonks_motorsport_search_leagues","sportmonks_motorsport_search_stages","sportmonks_motorsport_search_teams","sportmonks_motorsport_search_venues","sportmonks_odds_get_all_historical_odds","sportmonks_odds_get_all_inplay_odds","sportmonks_odds_get_all_pre_match_odds","sportmonks_odds_get_all_premium_odds","sportmonks_odds_get_bookmaker","sportmonks_odds_get_bookmaker_event_ids_by_fixture","sportmonks_odds_get_bookmakers","sportmonks_odds_get_bookmakers_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture_and_bookmaker","sportmonks_odds_get_inplay_odds_by_fixture_and_market","sportmonks_odds_get_last_updated_inplay_odds","sportmonks_odds_get_last_updated_pre_match_odds","sportmonks_odds_get_market","sportmonks_odds_get_markets","sportmonks_odds_get_pre_match_odds_by_fixture","sportmonks_odds_get_pre_match_odds_by_fixture_and_bookmaker","sportmonks_odds_get_pre_match_odds_by_fixture_and_market","sportmonks_odds_get_premium_odds_by_fixture","sportmonks_odds_get_premium_odds_by_fixture_and_bookmaker","sportmonks_odds_get_premium_odds_by_fixture_and_market","sportmonks_odds_get_updated_historical_odds_between","sportmonks_odds_get_updated_premium_odds_between","sportmonks_odds_search_bookmakers","sportmonks_odds_search_markets","spotify_add_playlist_cover","spotify_add_to_queue","spotify_add_tracks_to_playlist","spotify_check_following","spotify_check_playlist_followers","spotify_check_saved_albums","spotify_check_saved_audiobooks","spotify_check_saved_episodes","spotify_check_saved_shows","spotify_check_saved_tracks","spotify_create_playlist","spotify_follow_artists","spotify_follow_playlist","spotify_get_album","spotify_get_album_tracks","spotify_get_albums","spotify_get_artist","spotify_get_artist_albums","spotify_get_artist_top_tracks","spotify_get_artists","spotify_get_audiobook","spotify_get_audiobook_chapters","spotify_get_audiobooks","spotify_get_categories","spotify_get_current_user","spotify_get_currently_playing","spotify_get_devices","spotify_get_episode","spotify_get_episodes","spotify_get_followed_artists","spotify_get_markets","spotify_get_new_releases","spotify_get_playback_state","spotify_get_playlist","spotify_get_playlist_cover","spotify_get_playlist_tracks","spotify_get_queue","spotify_get_recently_played","spotify_get_saved_albums","spotify_get_saved_audiobooks","spotify_get_saved_episodes","spotify_get_saved_shows","spotify_get_saved_tracks","spotify_get_show","spotify_get_show_episodes","spotify_get_shows","spotify_get_top_artists","spotify_get_top_tracks","spotify_get_track","spotify_get_tracks","spotify_get_user_playlists","spotify_get_user_profile","spotify_pause","spotify_play","spotify_remove_saved_albums","spotify_remove_saved_audiobooks","spotify_remove_saved_episodes","spotify_remove_saved_shows","spotify_remove_saved_tracks","spotify_remove_tracks_from_playlist","spotify_reorder_playlist_items","spotify_replace_playlist_items","spotify_save_albums","spotify_save_audiobooks","spotify_save_episodes","spotify_save_shows","spotify_save_tracks","spotify_search","spotify_seek","spotify_set_repeat","spotify_set_shuffle","spotify_set_volume","spotify_skip_next","spotify_skip_previous","spotify_transfer_playback","spotify_unfollow_artists","spotify_unfollow_playlist","spotify_update_playlist","sqs_cancel_message_move_task","sqs_change_message_visibility","sqs_change_message_visibility_batch","sqs_create_queue","sqs_delete_message","sqs_delete_message_batch","sqs_delete_queue","sqs_get_queue_attributes","sqs_get_queue_url","sqs_list_dead_letter_source_queues","sqs_list_message_move_tasks","sqs_list_queue_tags","sqs_list_queues","sqs_purge_queue","sqs_receive_message","sqs_send","sqs_send_message_batch","sqs_set_queue_attributes","sqs_start_message_move_task","sqs_tag_queue","sqs_untag_queue","square_batch_retrieve_inventory_counts","square_cancel_invoice","square_cancel_payment","square_complete_payment","square_create_catalog_image","square_create_customer","square_create_invoice","square_create_order","square_create_payment","square_delete_catalog_object","square_delete_customer","square_delete_invoice","square_get_catalog_object","square_get_customer","square_get_invoice","square_get_location","square_get_order","square_get_payment","square_get_refund","square_list_catalog","square_list_customers","square_list_invoices","square_list_locations","square_list_payments","square_list_refunds","square_pay_order","square_publish_invoice","square_refund_payment","square_search_catalog_objects","square_search_customers","square_search_invoices","square_search_orders","square_update_customer","square_upsert_catalog_object","ssh_check_command_exists","ssh_check_file_exists","ssh_create_directory","ssh_delete_file","ssh_download_file","ssh_execute_command","ssh_execute_script","ssh_get_system_info","ssh_list_directory","ssh_move_rename","ssh_read_file_content","ssh_upload_file","ssh_write_file_content","ssm_cancel_command","ssm_delete_parameter","ssm_describe_automation_executions","ssm_describe_instance_information","ssm_describe_instance_patch_states","ssm_describe_instance_patches","ssm_describe_parameters","ssm_get_automation_execution","ssm_get_command_invocation","ssm_get_document","ssm_get_parameter","ssm_get_parameters","ssm_get_parameters_by_path","ssm_list_command_invocations","ssm_list_commands","ssm_list_compliance_items","ssm_list_compliance_summaries","ssm_list_documents","ssm_put_parameter","ssm_send_command","ssm_start_automation_execution","ssm_stop_automation_execution","stagehand_agent","stagehand_extract","stripe_cancel_payment_intent","stripe_cancel_subscription","stripe_capture_charge","stripe_capture_payment_intent","stripe_confirm_payment_intent","stripe_create_charge","stripe_create_customer","stripe_create_invoice","stripe_create_payment_intent","stripe_create_price","stripe_create_product","stripe_create_subscription","stripe_delete_customer","stripe_delete_invoice","stripe_delete_product","stripe_finalize_invoice","stripe_list_charges","stripe_list_customers","stripe_list_events","stripe_list_invoices","stripe_list_payment_intents","stripe_list_prices","stripe_list_products","stripe_list_subscriptions","stripe_pay_invoice","stripe_resume_subscription","stripe_retrieve_charge","stripe_retrieve_customer","stripe_retrieve_event","stripe_retrieve_invoice","stripe_retrieve_payment_intent","stripe_retrieve_price","stripe_retrieve_product","stripe_retrieve_subscription","stripe_search_charges","stripe_search_customers","stripe_search_invoices","stripe_search_payment_intents","stripe_search_prices","stripe_search_products","stripe_search_subscriptions","stripe_send_invoice","stripe_update_charge","stripe_update_customer","stripe_update_invoice","stripe_update_payment_intent","stripe_update_price","stripe_update_product","stripe_update_subscription","stripe_void_invoice","sts_assume_role","sts_assume_role_with_saml","sts_assume_role_with_web_identity","sts_get_access_key_info","sts_get_caller_identity","sts_get_session_token","stt_assemblyai","stt_assemblyai_v2","stt_deepgram","stt_deepgram_v2","stt_elevenlabs","stt_elevenlabs_v2","stt_gemini","stt_gemini_v2","stt_whisper","stt_whisper_v2","supabase_count","supabase_delete","supabase_get_row","supabase_insert","supabase_introspect","supabase_invoke_function","supabase_query","supabase_rpc","supabase_storage_copy","supabase_storage_create_bucket","supabase_storage_create_signed_upload_url","supabase_storage_create_signed_url","supabase_storage_delete","supabase_storage_delete_bucket","supabase_storage_download","supabase_storage_empty_bucket","supabase_storage_get_public_url","supabase_storage_list","supabase_storage_list_buckets","supabase_storage_move","supabase_storage_update_bucket","supabase_storage_upload","supabase_text_search","supabase_update","supabase_upsert","supabase_vector_search","table_batch_insert_rows","table_create","table_delete_row","table_delete_rows_by_filter","table_get_row","table_get_schema","table_insert_row","table_list","table_query_rows","table_query_rows_v2","table_update_row","table_update_rows_by_filter","table_upsert_row","tailscale_authorize_device","tailscale_create_auth_key","tailscale_delete_auth_key","tailscale_delete_device","tailscale_delete_user","tailscale_expire_device_key","tailscale_get_acl","tailscale_get_auth_key","tailscale_get_device","tailscale_get_device_routes","tailscale_get_dns_preferences","tailscale_get_dns_searchpaths","tailscale_list_auth_keys","tailscale_list_devices","tailscale_list_dns_nameservers","tailscale_list_users","tailscale_set_acl","tailscale_set_device_routes","tailscale_set_device_tags","tailscale_set_dns_nameservers","tailscale_set_dns_preferences","tailscale_set_dns_searchpaths","tailscale_suspend_user","tailscale_update_device_key","tavily_crawl","tavily_extract","tavily_map","tavily_search","telegram_copy_message","telegram_delete_message","telegram_edit_message_text","telegram_forward_message","telegram_get_chat","telegram_get_chat_member","telegram_message","telegram_pin_message","telegram_send_animation","telegram_send_audio","telegram_send_chat_action","telegram_send_contact","telegram_send_document","telegram_send_location","telegram_send_photo","telegram_send_poll","telegram_send_video","telegram_set_message_reaction","telegram_unpin_message","temporal_cancel_workflow","temporal_count_workflows","temporal_create_schedule","temporal_delete_schedule","temporal_describe_schedule","temporal_describe_task_queue","temporal_describe_workflow","temporal_get_workflow_history","temporal_list_schedules","temporal_list_workflows","temporal_pause_schedule","temporal_query_workflow","temporal_reset_workflow","temporal_signal_with_start","temporal_signal_workflow","temporal_start_workflow","temporal_terminate_workflow","temporal_trigger_schedule","temporal_unpause_schedule","temporal_update_workflow","textract_analyze_expense","textract_analyze_id","textract_parser","textract_parser_v2","thinking_tool","thrive_add_audience_managers","thrive_add_audience_members","thrive_add_user_tags","thrive_create_assignment","thrive_create_audience","thrive_create_completion","thrive_create_user","thrive_delete_assignment","thrive_delete_audience","thrive_delete_user","thrive_get_activity","thrive_get_assignment","thrive_get_audience","thrive_get_completion","thrive_get_content","thrive_get_cpd_category","thrive_get_cpd_entry","thrive_get_cpd_requirement","thrive_get_enrolment","thrive_get_skill_levels","thrive_get_tag","thrive_get_user_by_id","thrive_get_user_by_ref","thrive_list_assignments","thrive_list_audience_managers","thrive_list_audience_members","thrive_list_audiences","thrive_list_completions","thrive_list_enrolments","thrive_list_tags","thrive_query_activities","thrive_query_content","thrive_query_cpd_categories","thrive_query_cpd_entries","thrive_query_cpd_requirements","thrive_query_cpd_user_summaries","thrive_remove_audience_manager","thrive_remove_audience_member","thrive_remove_user_tags","thrive_replace_audience_managers","thrive_replace_audience_members","thrive_search_users","thrive_suspend_user","thrive_update_assignment","thrive_update_audience","thrive_update_user","thrive_update_user_skills","tiktok_get_post_status","tiktok_get_user","tiktok_list_videos","tiktok_query_videos","tiktok_upload_video_draft","tinybird_append_datasource","tinybird_delete_datasource_rows","tinybird_events","tinybird_get_job","tinybird_query","tinybird_query_pipe","tinybird_truncate_datasource","tinyfish_cancel_run","tinyfish_fetch","tinyfish_get_run","tinyfish_list_profiles","tinyfish_list_runs","tinyfish_list_vault_items","tinyfish_run","tinyfish_run_async","tinyfish_search","trello_add_checklist","trello_add_checklist_item","trello_add_comment","trello_add_label","trello_add_member","trello_create_board","trello_create_card","trello_create_list","trello_delete_card","trello_get_actions","trello_get_board","trello_get_card","trello_list_cards","trello_list_lists","trello_list_members","trello_remove_label","trello_remove_member","trello_search","trello_update_card","trello_update_checklist_item","trello_update_list","trigger_dev_activate_schedule","trigger_dev_add_run_tags","trigger_dev_batch_trigger_task","trigger_dev_cancel_run","trigger_dev_complete_waitpoint_token","trigger_dev_create_env_var","trigger_dev_create_schedule","trigger_dev_create_waitpoint_token","trigger_dev_deactivate_schedule","trigger_dev_delete_env_var","trigger_dev_delete_schedule","trigger_dev_execute_query","trigger_dev_get_batch","trigger_dev_get_batch_results","trigger_dev_get_deployment","trigger_dev_get_env_var","trigger_dev_get_latest_deployment","trigger_dev_get_query_schema","trigger_dev_get_queue","trigger_dev_get_run","trigger_dev_get_run_events","trigger_dev_get_run_result","trigger_dev_get_run_trace","trigger_dev_get_schedule","trigger_dev_get_waitpoint_token","trigger_dev_import_env_vars","trigger_dev_list_deployments","trigger_dev_list_env_vars","trigger_dev_list_queues","trigger_dev_list_runs","trigger_dev_list_schedules","trigger_dev_list_timezones","trigger_dev_list_waitpoint_tokens","trigger_dev_override_queue_concurrency","trigger_dev_pause_queue","trigger_dev_promote_deployment","trigger_dev_replay_run","trigger_dev_reschedule_run","trigger_dev_reset_queue_concurrency","trigger_dev_resume_queue","trigger_dev_trigger_task","trigger_dev_update_env_var","trigger_dev_update_run_metadata","trigger_dev_update_schedule","tts_azure","tts_cartesia","tts_deepgram","tts_elevenlabs","tts_google","tts_openai","tts_playht","twilio_send_sms","twilio_voice_get_recording","twilio_voice_list_calls","twilio_voice_make_call","typeform_create_form","typeform_delete_form","typeform_files","typeform_get_form","typeform_insights","typeform_list_forms","typeform_responses","typeform_update_form","upstash_redis_command","upstash_redis_delete","upstash_redis_exists","upstash_redis_expire","upstash_redis_get","upstash_redis_hget","upstash_redis_hgetall","upstash_redis_hset","upstash_redis_incr","upstash_redis_incrby","upstash_redis_keys","upstash_redis_lpush","upstash_redis_lrange","upstash_redis_set","upstash_redis_setnx","upstash_redis_ttl","uptimerobot_create_alert_contact","uptimerobot_create_maintenance_window","uptimerobot_create_monitor","uptimerobot_create_psp","uptimerobot_delete_alert_contact","uptimerobot_delete_maintenance_window","uptimerobot_delete_monitor","uptimerobot_delete_psp","uptimerobot_get_account","uptimerobot_get_alert_contact","uptimerobot_get_incident","uptimerobot_get_maintenance_window","uptimerobot_get_monitor","uptimerobot_get_psp","uptimerobot_list_alert_contacts","uptimerobot_list_incidents","uptimerobot_list_maintenance_windows","uptimerobot_list_monitors","uptimerobot_list_psps","uptimerobot_pause_monitor","uptimerobot_start_monitor","uptimerobot_update_maintenance_window","uptimerobot_update_monitor","uptimerobot_update_psp","vanta_download_document_file","vanta_get_control","vanta_get_document","vanta_get_framework","vanta_get_person","vanta_get_policy","vanta_get_risk_scenario","vanta_get_test","vanta_get_vendor","vanta_get_vulnerable_asset","vanta_list_control_documents","vanta_list_control_tests","vanta_list_controls","vanta_list_document_uploads","vanta_list_documents","vanta_list_framework_controls","vanta_list_frameworks","vanta_list_monitored_computers","vanta_list_people","vanta_list_policies","vanta_list_risk_scenarios","vanta_list_test_entities","vanta_list_tests","vanta_list_vendors","vanta_list_vulnerabilities","vanta_list_vulnerability_remediations","vanta_list_vulnerable_assets","vanta_submit_document","vanta_upload_document_file","vercel_add_domain","vercel_add_project_domain","vercel_cancel_deployment","vercel_create_alias","vercel_create_check","vercel_create_deployment","vercel_create_dns_record","vercel_create_edge_config","vercel_create_env_var","vercel_create_project","vercel_create_webhook","vercel_delete_alias","vercel_delete_deployment","vercel_delete_dns_record","vercel_delete_domain","vercel_delete_edge_config","vercel_delete_env_var","vercel_delete_project","vercel_delete_webhook","vercel_get_alias","vercel_get_check","vercel_get_deployment","vercel_get_deployment_events","vercel_get_domain","vercel_get_domain_config","vercel_get_edge_config","vercel_get_edge_config_items","vercel_get_env_vars","vercel_get_project","vercel_get_team","vercel_get_user","vercel_get_webhook","vercel_list_aliases","vercel_list_checks","vercel_list_deployment_files","vercel_list_deployments","vercel_list_dns_records","vercel_list_domains","vercel_list_edge_configs","vercel_list_project_domains","vercel_list_projects","vercel_list_team_members","vercel_list_teams","vercel_list_webhooks","vercel_pause_project","vercel_promote_deployment","vercel_remove_project_domain","vercel_rerequest_check","vercel_unpause_project","vercel_update_check","vercel_update_dns_record","vercel_update_edge_config_items","vercel_update_env_var","vercel_update_project","vercel_update_project_domain","vercel_verify_project_domain","video_falai","video_luma","video_minimax","video_runway","video_veo","vision_tool","vision_tool_v2","wealthbox_read_contact","wealthbox_read_note","wealthbox_read_task","wealthbox_write_contact","wealthbox_write_note","wealthbox_write_task","webflow_create_item","webflow_delete_item","webflow_get_item","webflow_list_items","webflow_update_item","webhook_request","whatsapp_get_media","whatsapp_mark_read","whatsapp_send_interactive","whatsapp_send_media","whatsapp_send_message","whatsapp_send_reaction","whatsapp_send_template","whatsapp_upload_media","wikipedia_content","wikipedia_random","wikipedia_search","wikipedia_summary","windchill_check_in_document","windchill_check_in_documents","windchill_check_out_document","windchill_check_out_documents","windchill_create_document","windchill_create_documents","windchill_delete_document","windchill_delete_documents","windchill_download_attachment","windchill_download_primary_content","windchill_get_document","windchill_get_document_structure","windchill_get_primary_content","windchill_get_valid_state_transitions","windchill_list_attachments","windchill_list_documents","windchill_revise_document","windchill_revise_documents","windchill_set_lifecycle_state","windchill_undo_check_out_document","windchill_undo_check_out_documents","windchill_update_common_properties","windchill_update_document","windchill_update_document_security_labels","windchill_update_documents","windchill_upload_attachments","windchill_upload_primary_content","wiza_company_enrichment","wiza_get_credits","wiza_individual_reveal","wiza_prospect_search","wordpress_create_category","wordpress_create_comment","wordpress_create_page","wordpress_create_post","wordpress_create_tag","wordpress_delete_category","wordpress_delete_comment","wordpress_delete_media","wordpress_delete_page","wordpress_delete_post","wordpress_delete_tag","wordpress_get_category","wordpress_get_current_user","wordpress_get_media","wordpress_get_page","wordpress_get_post","wordpress_get_tag","wordpress_get_user","wordpress_list_categories","wordpress_list_comments","wordpress_list_media","wordpress_list_pages","wordpress_list_posts","wordpress_list_tags","wordpress_list_users","wordpress_search_content","wordpress_update_category","wordpress_update_comment","wordpress_update_page","wordpress_update_post","wordpress_update_tag","wordpress_upload_media","workday_assign_onboarding","workday_change_job","workday_create_prehire","workday_get_compensation","workday_get_organizations","workday_get_worker","workday_hire_employee","workday_list_workers","workday_terminate_worker","workday_update_worker","workflow_executor","x_create_bookmark","x_create_tweet","x_delete_bookmark","x_delete_tweet","x_get_blocking","x_get_bookmarks","x_get_followers","x_get_following","x_get_liked_tweets","x_get_liking_users","x_get_me","x_get_personalized_trends","x_get_quote_tweets","x_get_retweeted_by","x_get_trends_by_woeid","x_get_tweets_by_ids","x_get_usage","x_get_user_mentions","x_get_user_timeline","x_get_user_tweets","x_hide_reply","x_manage_block","x_manage_follow","x_manage_like","x_manage_mute","x_manage_retweet","x_read","x_search","x_search_tweets","x_search_users","x_user","x_write","youtube_channel_info","youtube_channel_playlists","youtube_channel_videos","youtube_comments","youtube_playlist_items","youtube_search","youtube_trending","youtube_video_categories","youtube_video_details","zendesk_autocomplete_organizations","zendesk_create_organization","zendesk_create_organizations_bulk","zendesk_create_ticket","zendesk_create_tickets_bulk","zendesk_create_user","zendesk_create_users_bulk","zendesk_delete_organization","zendesk_delete_ticket","zendesk_delete_user","zendesk_get_current_user","zendesk_get_organization","zendesk_get_organizations","zendesk_get_ticket","zendesk_get_tickets","zendesk_get_user","zendesk_get_users","zendesk_merge_tickets","zendesk_search","zendesk_search_count","zendesk_search_users","zendesk_update_organization","zendesk_update_ticket","zendesk_update_tickets_bulk","zendesk_update_user","zendesk_update_users_bulk","zep_add_messages","zep_add_user","zep_create_thread","zep_delete_thread","zep_get_context","zep_get_messages","zep_get_threads","zep_get_user","zep_get_user_threads","zerobounce_get_credits","zerobounce_verify_email","zoho_desk_add_comment","zoho_desk_get_attachment","zoho_desk_get_contact","zoho_desk_get_thread","zoho_desk_get_ticket","zoho_desk_list_comments","zoho_desk_list_organizations","zoho_desk_list_threads","zoho_desk_list_tickets","zoho_desk_update_ticket","zoom_create_meeting","zoom_delete_meeting","zoom_delete_recording","zoom_get_meeting","zoom_get_meeting_invitation","zoom_get_meeting_recordings","zoom_list_meetings","zoom_list_past_participants","zoom_list_recordings","zoom_update_meeting","zoominfo_enrich_companies","zoominfo_enrich_contacts","zoominfo_search_companies","zoominfo_search_contacts","zoominfo_search_intent","zoominfo_search_news"]' + '["a2a_cancel_task","a2a_get_agent_card","a2a_get_task","a2a_send_message","affinity_batch_update_entity_fields","affinity_batch_update_list_entry_fields","affinity_create_list","affinity_create_list_field_dropdown_option","affinity_create_merge","affinity_create_note","affinity_create_reminder","affinity_delete_list_field_dropdown_option","affinity_delete_note","affinity_get_company","affinity_get_current_user","affinity_get_entity_field_value","affinity_get_list","affinity_get_list_entry","affinity_get_list_entry_field","affinity_get_list_field_dropdown_option","affinity_get_merge","affinity_get_merge_task","affinity_get_note","affinity_get_opportunity","affinity_get_person","affinity_get_saved_view","affinity_get_transcript","affinity_get_user","affinity_list_calls","affinity_list_chat_messages","affinity_list_companies","affinity_list_coworker_connections","affinity_list_emails","affinity_list_entity_field_values","affinity_list_entity_list_entries","affinity_list_entity_lists","affinity_list_entity_notes","affinity_list_entity_relationships","affinity_list_field_dropdown_options","affinity_list_field_metadata","affinity_list_field_value_changes","affinity_list_investor_executive_connections","affinity_list_list_entries","affinity_list_list_entry_field_value_changes","affinity_list_list_entry_fields","affinity_list_list_field_dropdown_options","affinity_list_list_fields","affinity_list_lists","affinity_list_meetings","affinity_list_merge_tasks","affinity_list_merges","affinity_list_note_attached_companies","affinity_list_note_attached_opportunities","affinity_list_note_attached_persons","affinity_list_note_replies","affinity_list_notes","affinity_list_opportunities","affinity_list_persons","affinity_list_reminders","affinity_list_saved_view_entries","affinity_list_saved_views","affinity_list_transcript_fragments","affinity_list_transcripts","affinity_list_users","affinity_search_companies","affinity_search_files","affinity_search_list_entries","affinity_search_notes","affinity_search_persons","affinity_semantic_search","affinity_update_entity_field_value","affinity_update_list_entry_field","affinity_update_list_field_dropdown_option","affinity_update_note","agentmail_create_draft","agentmail_create_inbox","agentmail_delete_draft","agentmail_delete_inbox","agentmail_delete_thread","agentmail_forward_message","agentmail_get_draft","agentmail_get_inbox","agentmail_get_message","agentmail_get_thread","agentmail_list_drafts","agentmail_list_inboxes","agentmail_list_messages","agentmail_list_threads","agentmail_reply_message","agentmail_send_draft","agentmail_send_message","agentmail_update_draft","agentmail_update_inbox","agentmail_update_message","agentmail_update_thread","agentphone_create_call","agentphone_create_contact","agentphone_create_number","agentphone_delete_contact","agentphone_get_call","agentphone_get_call_transcript","agentphone_get_contact","agentphone_get_conversation","agentphone_get_conversation_messages","agentphone_get_number_messages","agentphone_get_usage","agentphone_get_usage_daily","agentphone_get_usage_monthly","agentphone_list_calls","agentphone_list_contacts","agentphone_list_conversations","agentphone_list_numbers","agentphone_react_to_message","agentphone_release_number","agentphone_send_message","agentphone_update_contact","agentphone_update_conversation","agiloft_async_status","agiloft_attach_file","agiloft_attachment_info","agiloft_create_record","agiloft_delete_record","agiloft_get_choice_line_id","agiloft_list_tables","agiloft_lock_record","agiloft_nlp_search","agiloft_read_record","agiloft_remove_attachment","agiloft_retrieve_attachment","agiloft_run_action_button","agiloft_saved_search","agiloft_search_records","agiloft_select_records","agiloft_update_record","agiloft_upsert_record","ahrefs_anchors","ahrefs_backlinks","ahrefs_backlinks_stats","ahrefs_batch_analysis","ahrefs_broken_backlinks","ahrefs_domain_rating","ahrefs_domain_rating_history","ahrefs_keyword_overview","ahrefs_keywords_history","ahrefs_metrics","ahrefs_metrics_history","ahrefs_organic_competitors","ahrefs_organic_keywords","ahrefs_paid_pages","ahrefs_rank_tracker_competitors_overview","ahrefs_rank_tracker_competitors_stats","ahrefs_rank_tracker_overview","ahrefs_rank_tracker_serp_overview","ahrefs_refdomains_history","ahrefs_referring_domains","ahrefs_related_terms","ahrefs_site_audit_page_explorer","ahrefs_top_pages","airtable_create_records","airtable_delete_records","airtable_get_base_schema","airtable_get_record","airtable_list_bases","airtable_list_records","airtable_list_tables","airtable_update_multiple_records","airtable_update_record","airtable_upsert_records","airweave_search","algolia_add_record","algolia_batch_operations","algolia_browse_records","algolia_clear_records","algolia_copy_move_index","algolia_delete_by_filter","algolia_delete_index","algolia_delete_record","algolia_get_record","algolia_get_records","algolia_get_settings","algolia_get_task_status","algolia_list_indices","algolia_partial_update_record","algolia_search","algolia_update_settings","amplitude_event_segmentation","amplitude_funnels","amplitude_get_active_users","amplitude_get_revenue","amplitude_group_identify","amplitude_identify_user","amplitude_list_events","amplitude_realtime_active_users","amplitude_retention","amplitude_send_event","amplitude_user_activity","amplitude_user_profile","amplitude_user_search","apify_get_dataset_items","apify_get_run","apify_run_actor_async","apify_run_actor_sync","apify_run_task","apollo_account_bulk_create","apollo_account_bulk_update","apollo_account_create","apollo_account_search","apollo_account_update","apollo_contact_bulk_create","apollo_contact_bulk_update","apollo_contact_create","apollo_contact_search","apollo_contact_update","apollo_email_accounts","apollo_opportunity_create","apollo_opportunity_get","apollo_opportunity_search","apollo_opportunity_update","apollo_organization_bulk_enrich","apollo_organization_enrich","apollo_organization_search","apollo_people_bulk_enrich","apollo_people_enrich","apollo_people_search","apollo_sequence_add_contacts","apollo_sequence_search","apollo_task_create","apollo_task_search","appconfig_create_application","appconfig_create_configuration_profile","appconfig_create_environment","appconfig_create_hosted_configuration_version","appconfig_delete_application","appconfig_delete_configuration_profile","appconfig_delete_environment","appconfig_delete_hosted_configuration_version","appconfig_get_application","appconfig_get_configuration","appconfig_get_configuration_profile","appconfig_get_deployment","appconfig_get_environment","appconfig_get_hosted_configuration_version","appconfig_list_applications","appconfig_list_configuration_profiles","appconfig_list_deployment_strategies","appconfig_list_deployments","appconfig_list_environments","appconfig_list_hosted_configuration_versions","appconfig_start_deployment","appconfig_stop_deployment","appconfig_update_application","appconfig_update_configuration_profile","appconfig_update_environment","arxiv_get_author_papers","arxiv_get_paper","arxiv_search","asana_add_comment","asana_add_followers","asana_create_project","asana_create_section","asana_create_subtask","asana_create_task","asana_delete_task","asana_get_project","asana_get_projects","asana_get_task","asana_list_sections","asana_list_workspaces","asana_search_tasks","asana_update_task","ashby_add_candidate_tag","ashby_anonymize_candidate","ashby_change_application_source","ashby_change_application_stage","ashby_create_application","ashby_create_candidate","ashby_create_note","ashby_delete_application","ashby_get_application","ashby_get_candidate","ashby_get_job","ashby_get_job_posting","ashby_get_offer","ashby_get_opening","ashby_list_application_feedback","ashby_list_application_history","ashby_list_applications","ashby_list_archive_reasons","ashby_list_candidate_tags","ashby_list_candidates","ashby_list_custom_fields","ashby_list_departments","ashby_list_interview_plans","ashby_list_interview_stages","ashby_list_interviews","ashby_list_job_postings","ashby_list_jobs","ashby_list_locations","ashby_list_notes","ashby_list_offers","ashby_list_openings","ashby_list_sources","ashby_list_users","ashby_remove_candidate_tag","ashby_search_candidates","ashby_search_jobs","ashby_search_openings","ashby_search_users","ashby_set_custom_field_value","ashby_set_custom_field_values","ashby_transfer_application","ashby_update_candidate","ashby_upload_candidate_file","ashby_upload_resume","athena_batch_get_named_query","athena_batch_get_prepared_statement","athena_batch_get_query_execution","athena_create_named_query","athena_create_prepared_statement","athena_delete_named_query","athena_delete_prepared_statement","athena_get_data_catalog","athena_get_database","athena_get_named_query","athena_get_prepared_statement","athena_get_query_execution","athena_get_query_results","athena_get_query_runtime_statistics","athena_get_table_metadata","athena_get_work_group","athena_list_data_catalogs","athena_list_databases","athena_list_named_queries","athena_list_prepared_statements","athena_list_query_executions","athena_list_table_metadata","athena_list_work_groups","athena_start_query","athena_stop_query","athena_update_named_query","athena_update_prepared_statement","attio_assert_record","attio_create_attribute","attio_create_comment","attio_create_list","attio_create_list_entry","attio_create_note","attio_create_object","attio_create_record","attio_create_task","attio_create_webhook","attio_delete_comment","attio_delete_list_entry","attio_delete_note","attio_delete_record","attio_delete_task","attio_delete_webhook","attio_get_attribute","attio_get_comment","attio_get_list","attio_get_list_entry","attio_get_member","attio_get_note","attio_get_object","attio_get_record","attio_get_task","attio_get_thread","attio_get_webhook","attio_list_attributes","attio_list_lists","attio_list_members","attio_list_notes","attio_list_objects","attio_list_records","attio_list_tasks","attio_list_threads","attio_list_webhooks","attio_query_list_entries","attio_search_records","attio_update_attribute","attio_update_list","attio_update_list_entry","attio_update_object","attio_update_record","attio_update_task","attio_update_webhook","azure_data_explorer_create_table","azure_data_explorer_drop_table","azure_data_explorer_ingest_from_query","azure_data_explorer_ingest_inline","azure_data_explorer_list_databases","azure_data_explorer_list_functions","azure_data_explorer_list_tables","azure_data_explorer_management","azure_data_explorer_query","azure_data_explorer_show_database_schema","azure_data_explorer_show_ingestion_failures","azure_data_explorer_show_operations","azure_data_explorer_show_table_details","azure_data_explorer_show_table_schema","azure_devops_add_comment","azure_devops_create_work_item","azure_devops_get_build_log","azure_devops_get_build_timeline","azure_devops_get_comments","azure_devops_get_pipeline","azure_devops_get_pipeline_run","azure_devops_get_work_item","azure_devops_get_work_items_batch","azure_devops_get_work_items_between_builds","azure_devops_list_build_logs","azure_devops_list_builds","azure_devops_list_pipeline_runs","azure_devops_list_pipelines","azure_devops_query_work_items","azure_devops_update_work_item","bitbucket_approve_pull_request","bitbucket_create_branch","bitbucket_create_pull_request","bitbucket_create_pull_request_comment","bitbucket_decline_pull_request","bitbucket_delete_branch","bitbucket_get_commit","bitbucket_get_file","bitbucket_get_file_metadata","bitbucket_get_pipeline","bitbucket_get_pipeline_step_log","bitbucket_get_pull_request","bitbucket_get_pull_request_diff","bitbucket_get_pull_request_diffstat","bitbucket_get_pull_request_merge_task_status","bitbucket_get_repository","bitbucket_list_branches","bitbucket_list_commits","bitbucket_list_directory","bitbucket_list_pipeline_steps","bitbucket_list_pipelines","bitbucket_list_pull_request_comments","bitbucket_list_pull_request_commit_statuses","bitbucket_list_pull_requests","bitbucket_list_repositories","bitbucket_list_workspaces","bitbucket_merge_pull_request","bitbucket_request_pull_request_changes","bitbucket_stop_pipeline","bitbucket_trigger_pipeline","box_copy_file","box_create_folder","box_delete_file","box_delete_folder","box_download_file","box_download_file_v2","box_get_file_info","box_list_folder_items","box_search","box_sign_cancel_request","box_sign_create_request","box_sign_get_request","box_sign_list_requests","box_sign_resend_request","box_update_file","box_upload_file","brandfetch_get_brand","brandfetch_search","brex_archive_budget","brex_create_budget","brex_create_spend_limit","brex_create_transfer","brex_create_vendor","brex_get_budget","brex_get_cash_account","brex_get_company","brex_get_current_user","brex_get_expense","brex_get_spend_limit","brex_get_transfer","brex_get_user","brex_get_vendor","brex_list_budgets","brex_list_card_accounts","brex_list_card_statements","brex_list_card_transactions","brex_list_cards","brex_list_cash_accounts","brex_list_cash_statements","brex_list_cash_transactions","brex_list_departments","brex_list_expenses","brex_list_locations","brex_list_spend_limits","brex_list_titles","brex_list_transfers","brex_list_users","brex_list_vendors","brex_match_receipt","brex_update_expense","brex_update_vendor","brex_upload_receipt","brightdata_cancel_snapshot","brightdata_discover","brightdata_download_snapshot","brightdata_scrape_dataset","brightdata_scrape_url","brightdata_serp_search","brightdata_snapshot_status","brightdata_sync_scrape","browser_use_run_task","buffer_create_idea","buffer_create_post","buffer_delete_post","buffer_edit_post","buffer_get_account","buffer_get_channels","buffer_get_idea_groups","buffer_get_ideas","buffer_get_post","buffer_get_posts","calcom_cancel_booking","calcom_confirm_booking","calcom_create_booking","calcom_create_event_type","calcom_create_schedule","calcom_decline_booking","calcom_delete_event_type","calcom_delete_schedule","calcom_get_booking","calcom_get_default_schedule","calcom_get_event_type","calcom_get_schedule","calcom_get_slots","calcom_list_bookings","calcom_list_event_types","calcom_list_schedules","calcom_reschedule_booking","calcom_update_event_type","calcom_update_schedule","calendly_cancel_event","calendly_create_event_invitee","calendly_create_invitee_no_show","calendly_create_scheduling_link","calendly_create_webhook","calendly_delete_invitee_no_show","calendly_delete_webhook","calendly_get_current_user","calendly_get_event_invitee","calendly_get_event_type","calendly_get_scheduled_event","calendly_get_user","calendly_list_event_invitees","calendly_list_event_type_available_times","calendly_list_event_types","calendly_list_organization_memberships","calendly_list_routing_form_submissions","calendly_list_routing_forms","calendly_list_scheduled_events","calendly_list_user_availability_schedules","calendly_list_user_busy_times","calendly_list_webhooks","cbinsights_chat","cbinsights_get_commercial_maturity_history","cbinsights_get_exit_probability_history","cbinsights_get_mosaic_history","cbinsights_get_org_business_relationships","cbinsights_get_org_funding_window","cbinsights_get_org_fundings","cbinsights_get_org_investments","cbinsights_get_org_management_and_board","cbinsights_get_org_outlook","cbinsights_get_org_portfolio_exits","cbinsights_get_org_revenue","cbinsights_get_scouting_report","cbinsights_get_strategy_map","cbinsights_list_business_relationships","cbinsights_list_funding_window","cbinsights_list_fundings","cbinsights_list_investments","cbinsights_list_management_and_board","cbinsights_list_outlook","cbinsights_list_portfolio_exits","cbinsights_list_revenue","cbinsights_lookup_organizations","cbinsights_rag","cbinsights_search_firmographics","circleback_add_tag_to_meetings","circleback_create_tag","circleback_delete_action_item","circleback_delete_meeting","circleback_delete_tag","circleback_get_company","circleback_get_meeting","circleback_get_person","circleback_get_transcript","circleback_list_action_items","circleback_list_calendar_events","circleback_list_companies","circleback_list_meetings","circleback_list_people","circleback_list_tags","circleback_remove_tag_from_meetings","circleback_search_meetings","circleback_update_action_item","circleback_update_meeting","circleback_update_tag","clay_populate","clerk_add_organization_member","clerk_ban_user","clerk_create_actor_token","clerk_create_allowlist_identifier","clerk_create_blocklist_identifier","clerk_create_organization","clerk_create_organization_invitation","clerk_create_user","clerk_delete_allowlist_identifier","clerk_delete_blocklist_identifier","clerk_delete_organization","clerk_delete_user","clerk_get_jwt_template","clerk_get_organization","clerk_get_session","clerk_get_user","clerk_get_user_oauth_token","clerk_list_allowlist_identifiers","clerk_list_blocklist_identifiers","clerk_list_jwt_templates","clerk_list_organization_invitations","clerk_list_organization_memberships","clerk_list_organizations","clerk_list_sessions","clerk_list_users","clerk_lock_user","clerk_remove_organization_member","clerk_revoke_actor_token","clerk_revoke_session","clerk_unban_user","clerk_unlock_user","clerk_update_organization","clerk_update_organization_membership","clerk_update_user","clickhouse_count_rows","clickhouse_create_database","clickhouse_create_table","clickhouse_delete","clickhouse_describe_table","clickhouse_drop_database","clickhouse_drop_partition","clickhouse_drop_table","clickhouse_execute","clickhouse_insert","clickhouse_insert_rows","clickhouse_introspect","clickhouse_kill_query","clickhouse_list_clusters","clickhouse_list_databases","clickhouse_list_mutations","clickhouse_list_partitions","clickhouse_list_running_queries","clickhouse_list_tables","clickhouse_optimize_table","clickhouse_query","clickhouse_rename_table","clickhouse_show_create_table","clickhouse_table_stats","clickhouse_truncate_table","clickhouse_update","clickup_add_tag_to_task","clickup_create_checklist","clickup_create_checklist_item","clickup_create_comment","clickup_create_folder","clickup_create_list","clickup_create_task","clickup_create_time_entry","clickup_delete_checklist","clickup_delete_checklist_item","clickup_delete_comment","clickup_delete_task","clickup_delete_time_entry","clickup_get_comments","clickup_get_custom_fields","clickup_get_folders","clickup_get_list_members","clickup_get_lists","clickup_get_running_timer","clickup_get_space_tags","clickup_get_spaces","clickup_get_task","clickup_get_task_members","clickup_get_tasks","clickup_get_time_entries","clickup_get_workspaces","clickup_remove_custom_field_value","clickup_remove_tag_from_task","clickup_search_tasks","clickup_set_custom_field_value","clickup_start_timer","clickup_stop_timer","clickup_update_checklist","clickup_update_checklist_item","clickup_update_comment","clickup_update_task","clickup_update_time_entry","clickup_upload_attachment","cloudflare_create_access_application","cloudflare_create_access_policy","cloudflare_create_access_service_token","cloudflare_create_dns_record","cloudflare_create_r2_bucket","cloudflare_create_rate_limit_rule","cloudflare_create_ruleset","cloudflare_create_ruleset_rule","cloudflare_create_zone","cloudflare_delete_access_application","cloudflare_delete_access_policy","cloudflare_delete_dns_record","cloudflare_delete_r2_bucket","cloudflare_delete_ruleset_rule","cloudflare_delete_zone","cloudflare_dns_analytics","cloudflare_get_access_application","cloudflare_get_r2_bucket","cloudflare_get_ruleset","cloudflare_get_ruleset_entrypoint","cloudflare_get_tunnel","cloudflare_get_tunnel_configuration","cloudflare_get_worker_script_settings","cloudflare_get_zone","cloudflare_get_zone_settings","cloudflare_list_access_applications","cloudflare_list_access_groups","cloudflare_list_access_identity_providers","cloudflare_list_access_policies","cloudflare_list_access_service_tokens","cloudflare_list_certificates","cloudflare_list_dns_records","cloudflare_list_managed_ruleset_overrides","cloudflare_list_r2_buckets","cloudflare_list_rate_limit_rules","cloudflare_list_rulesets","cloudflare_list_tunnels","cloudflare_list_worker_routes","cloudflare_list_worker_scripts","cloudflare_list_zones","cloudflare_purge_cache","cloudflare_revoke_access_service_token","cloudflare_update_access_application","cloudflare_update_access_policy","cloudflare_update_dns_record","cloudflare_update_rate_limit_rule","cloudflare_update_ruleset_rule","cloudflare_update_zone_setting","cloudformation_cancel_update_stack","cloudformation_create_change_set","cloudformation_create_stack","cloudformation_delete_stack","cloudformation_describe_change_set","cloudformation_describe_stack_drift_detection_status","cloudformation_describe_stack_events","cloudformation_describe_stacks","cloudformation_detect_stack_drift","cloudformation_execute_change_set","cloudformation_get_template","cloudformation_get_template_summary","cloudformation_list_stack_resources","cloudformation_update_stack","cloudformation_validate_template","cloudtrail_cancel_query","cloudtrail_describe_query","cloudtrail_describe_trails","cloudtrail_get_event_data_store","cloudtrail_get_event_selectors","cloudtrail_get_insight_selectors","cloudtrail_get_query_results","cloudtrail_get_trail","cloudtrail_get_trail_status","cloudtrail_list_event_data_stores","cloudtrail_list_tags","cloudtrail_list_trails","cloudtrail_lookup_events","cloudtrail_start_query","cloudwatch_describe_alarm_history","cloudwatch_describe_alarms","cloudwatch_describe_log_groups","cloudwatch_describe_log_streams","cloudwatch_filter_log_events","cloudwatch_get_log_events","cloudwatch_get_metric_statistics","cloudwatch_list_metrics","cloudwatch_mute_alarm","cloudwatch_put_log_group_retention","cloudwatch_put_metric_data","cloudwatch_query_logs","cloudwatch_unmute_alarm","codepipeline_disable_stage_transition","codepipeline_enable_stage_transition","codepipeline_get_pipeline","codepipeline_get_pipeline_execution","codepipeline_get_pipeline_state","codepipeline_list_action_executions","codepipeline_list_pipeline_executions","codepipeline_list_pipelines","codepipeline_put_approval_result","codepipeline_retry_stage_execution","codepipeline_start_execution","codepipeline_stop_execution","confluence_add_label","confluence_create_blogpost","confluence_create_comment","confluence_create_page","confluence_create_page_property","confluence_create_space","confluence_create_space_property","confluence_delete_attachment","confluence_delete_blogpost","confluence_delete_comment","confluence_delete_label","confluence_delete_page","confluence_delete_page_property","confluence_delete_space","confluence_delete_space_property","confluence_get_blogpost","confluence_get_page_ancestors","confluence_get_page_children","confluence_get_page_descendants","confluence_get_page_version","confluence_get_pages_by_label","confluence_get_space","confluence_get_task","confluence_get_user","confluence_list_attachments","confluence_list_blogposts","confluence_list_blogposts_in_space","confluence_list_comments","confluence_list_labels","confluence_list_page_properties","confluence_list_page_versions","confluence_list_pages_in_space","confluence_list_space_labels","confluence_list_space_permissions","confluence_list_space_properties","confluence_list_spaces","confluence_list_tasks","confluence_retrieve","confluence_search","confluence_search_in_space","confluence_update","confluence_update_blogpost","confluence_update_comment","confluence_update_space","confluence_update_task","confluence_upload_attachment","context_dev_classify_naics","context_dev_classify_sic","context_dev_crawl","context_dev_extract","context_dev_extract_product","context_dev_extract_products","context_dev_get_brand","context_dev_get_brand_by_email","context_dev_get_brand_by_name","context_dev_get_brand_by_ticker","context_dev_identify_transaction","context_dev_map","context_dev_scrape_fonts","context_dev_scrape_html","context_dev_scrape_images","context_dev_scrape_markdown","context_dev_scrape_styleguide","context_dev_screenshot","context_dev_search","convex_action","convex_document_deltas","convex_list_documents","convex_list_tables","convex_mutation","convex_query","convex_run_function","crowdstrike_create_indicators","crowdstrike_delete_indicators","crowdstrike_delete_rtr_session","crowdstrike_execute_rtr_command","crowdstrike_get_alert_details","crowdstrike_get_case_details","crowdstrike_get_host_group_details","crowdstrike_get_indicator_details","crowdstrike_get_rtr_command_status","crowdstrike_get_sensor_aggregates","crowdstrike_get_sensor_details","crowdstrike_get_vulnerability_details","crowdstrike_init_rtr_session","crowdstrike_perform_host_action","crowdstrike_perform_host_group_action","crowdstrike_query_alerts","crowdstrike_query_cases","crowdstrike_query_host_groups","crowdstrike_query_indicators","crowdstrike_query_sensors","crowdstrike_query_vulnerabilities","crowdstrike_update_alerts","crowdstrike_update_indicators","crunchbase_autocomplete","crunchbase_get_acquisition","crunchbase_get_entity","crunchbase_get_entity_card","crunchbase_get_fields_metadata","crunchbase_get_funding_round","crunchbase_get_organization","crunchbase_get_person","crunchbase_list_deleted_entities","crunchbase_search_acquisitions","crunchbase_search_entities","crunchbase_search_funding_rounds","crunchbase_search_organizations","crunchbase_search_people","cursor_add_followup","cursor_add_followup_v2","cursor_delete_agent","cursor_delete_agent_v2","cursor_download_artifact","cursor_download_artifact_v2","cursor_get_agent","cursor_get_agent_v2","cursor_get_api_key_info","cursor_get_api_key_info_v2","cursor_get_conversation","cursor_get_conversation_v2","cursor_launch_agent","cursor_launch_agent_v2","cursor_list_agents","cursor_list_agents_v2","cursor_list_artifacts","cursor_list_artifacts_v2","cursor_list_models","cursor_list_models_v2","cursor_list_repositories","cursor_list_repositories_v2","cursor_stop_agent","cursor_stop_agent_v2","dagster_delete_run","dagster_get_asset","dagster_get_run","dagster_get_run_logs","dagster_launch_run","dagster_list_assets","dagster_list_jobs","dagster_list_runs","dagster_list_schedules","dagster_list_sensors","dagster_materialize_assets","dagster_reexecute_run","dagster_report_asset_materialization","dagster_start_schedule","dagster_start_sensor","dagster_stop_schedule","dagster_stop_sensor","dagster_terminate_run","dagster_wipe_asset","databricks_cancel_run","databricks_execute_sql","databricks_get_cluster","databricks_get_job","databricks_get_run","databricks_get_run_output","databricks_get_statement","databricks_list_clusters","databricks_list_jobs","databricks_list_runs","databricks_list_warehouses","databricks_run_job","datadog_add_incident_todo","datadog_cancel_downtime","datadog_create_dashboard","datadog_create_downtime","datadog_create_event","datadog_create_incident","datadog_create_monitor","datadog_create_slo","datadog_delete_dashboard","datadog_delete_slo","datadog_get_browser_synthetics_results","datadog_get_dashboard","datadog_get_incident","datadog_get_monitor","datadog_get_security_signal","datadog_get_slo","datadog_get_slo_history","datadog_get_synthetics_results","datadog_get_synthetics_test","datadog_list_dashboards","datadog_list_downtimes","datadog_list_incidents","datadog_list_monitors","datadog_list_security_rules","datadog_list_security_signals","datadog_list_services","datadog_list_slos","datadog_list_synthetics_tests","datadog_mute_monitor","datadog_query_logs","datadog_query_timeseries","datadog_search_spans","datadog_send_logs","datadog_submit_metrics","datadog_trigger_synthetics_tests","datadog_unmute_monitor","datadog_update_incident","datadog_update_security_signal_assignee","datadog_update_security_signal_state","datadog_update_slo","datadog_update_synthetics_status","datagma_enrich_company","datagma_enrich_person","datagma_find_email","datagma_find_phone","datagma_get_credits","daytona_create_sandbox","daytona_delete_sandbox","daytona_download_file","daytona_execute_command","daytona_get_sandbox","daytona_git_clone","daytona_list_files","daytona_list_sandboxes","daytona_run_code","daytona_start_sandbox","daytona_stop_sandbox","daytona_upload_file","deployed_block_executor","deployments_deploy","deployments_get_version","deployments_list_versions","deployments_promote","deployments_undeploy","devin_append_session_tags","devin_archive_session","devin_create_session","devin_get_session","devin_get_session_tags","devin_list_session_attachments","devin_list_session_messages","devin_list_sessions","devin_replace_session_tags","devin_send_message","devin_terminate_session","discord_add_reaction","discord_archive_thread","discord_assign_role","discord_ban_member","discord_bulk_delete_messages","discord_create_channel","discord_create_invite","discord_create_role","discord_create_thread","discord_create_webhook","discord_delete_channel","discord_delete_invite","discord_delete_message","discord_delete_role","discord_delete_webhook","discord_edit_message","discord_execute_webhook","discord_get_channel","discord_get_invite","discord_get_member","discord_get_messages","discord_get_pinned_messages","discord_get_server","discord_get_user","discord_get_webhook","discord_join_thread","discord_kick_member","discord_leave_thread","discord_list_channels","discord_list_roles","discord_pin_message","discord_remove_reaction","discord_remove_role","discord_send_message","discord_unban_member","discord_unpin_message","discord_update_channel","discord_update_member","discord_update_role","docusign_create_from_template","docusign_download_document","docusign_get_envelope","docusign_list_envelopes","docusign_list_recipients","docusign_list_templates","docusign_send_envelope","docusign_void_envelope","downdetector_get_company","downdetector_get_company_attribution","downdetector_get_company_baseline","downdetector_get_company_events","downdetector_get_company_incidents","downdetector_get_company_indicators","downdetector_get_company_last_15","downdetector_get_company_status","downdetector_get_provider","downdetector_get_reports","downdetector_get_site_companies","downdetector_list_categories","downdetector_list_incidents","downdetector_list_sites","downdetector_search_companies","dropbox_copy","dropbox_create_folder","dropbox_create_shared_link","dropbox_delete","dropbox_download","dropbox_download_v2","dropbox_get_metadata","dropbox_list_folder","dropbox_list_revisions","dropbox_list_shared_links","dropbox_move","dropbox_restore","dropbox_search","dropbox_upload","dropcontact_enrich_contact","dspy_chain_of_thought","dspy_predict","dspy_react","dub_bulk_create_links","dub_bulk_delete_links","dub_bulk_update_links","dub_create_link","dub_create_tag","dub_delete_link","dub_get_analytics","dub_get_events","dub_get_link","dub_get_links_count","dub_get_qr_code","dub_get_qr_code_v2","dub_list_domains","dub_list_folders","dub_list_links","dub_list_tags","dub_update_link","dub_upsert_link","duckduckgo_search","dynamodb_delete","dynamodb_get","dynamodb_introspect","dynamodb_put","dynamodb_query","dynamodb_scan","dynamodb_update","dynatrace_add_problem_comment","dynatrace_add_tags","dynatrace_close_problem","dynatrace_create_settings_object","dynatrace_create_slo","dynatrace_delete_problem_comment","dynatrace_delete_settings_object","dynatrace_delete_slo","dynatrace_delete_tag","dynatrace_execute_synthetic_monitors","dynatrace_get_attack","dynatrace_get_audit_logs","dynatrace_get_entity","dynatrace_get_event","dynatrace_get_metric","dynatrace_get_problem","dynatrace_get_problem_comment","dynatrace_get_security_problem","dynatrace_get_settings_object","dynatrace_get_slo","dynatrace_get_synthetic_batch","dynatrace_ingest_event","dynatrace_ingest_logs","dynatrace_ingest_metrics","dynatrace_list_attacks","dynatrace_list_entities","dynatrace_list_entity_types","dynatrace_list_events","dynatrace_list_metrics","dynatrace_list_problem_comments","dynatrace_list_problems","dynatrace_list_remediation_items","dynatrace_list_security_problems","dynatrace_list_settings_objects","dynatrace_list_settings_schemas","dynatrace_list_slos","dynatrace_list_synthetic_monitors","dynatrace_list_tags","dynatrace_mute_security_problem","dynatrace_mute_security_problems","dynatrace_query_metrics","dynatrace_search_logs","dynatrace_unmute_security_problem","dynatrace_unmute_security_problems","dynatrace_update_problem_comment","dynatrace_update_settings_object","dynatrace_update_slo","elasticsearch_bulk","elasticsearch_cluster_health","elasticsearch_cluster_stats","elasticsearch_count","elasticsearch_create_index","elasticsearch_delete_document","elasticsearch_delete_index","elasticsearch_get_document","elasticsearch_get_index","elasticsearch_index_document","elasticsearch_list_indices","elasticsearch_search","elasticsearch_update_document","elevenlabs_audio_isolation","elevenlabs_edit_voice_settings","elevenlabs_get_user","elevenlabs_get_voice","elevenlabs_get_voice_settings","elevenlabs_list_models","elevenlabs_list_voices","elevenlabs_sound_effects","elevenlabs_speech_to_speech","elevenlabs_tts","emailbison_attach_leads_to_campaign","emailbison_attach_tags_to_leads","emailbison_create_campaign","emailbison_create_lead","emailbison_create_tag","emailbison_get_lead","emailbison_list_campaigns","emailbison_list_leads","emailbison_list_replies","emailbison_list_tags","emailbison_update_campaign","emailbison_update_campaign_status","emailbison_update_lead","embeddings_cohere","embeddings_gemini","embeddings_mistral","embeddings_ollama","embeddings_openai","embeddings_openrouter","enrich_check_credits","enrich_company_funding","enrich_company_lookup","enrich_company_revenue","enrich_disposable_email_check","enrich_email_to_ip","enrich_email_to_person_lite","enrich_email_to_phone","enrich_email_to_profile","enrich_find_email","enrich_get_post_details","enrich_ip_to_company","enrich_linkedin_profile","enrich_linkedin_to_personal_email","enrich_linkedin_to_work_email","enrich_phone_finder","enrich_reverse_hash_lookup","enrich_sales_pointer_people","enrich_search_company","enrich_search_company_activities","enrich_search_company_employees","enrich_search_jobs","enrich_search_logo","enrich_search_people","enrich_search_people_activities","enrich_search_post_comments","enrich_search_post_comments_by_url","enrich_search_post_reactions","enrich_search_post_reactions_by_url","enrich_search_posts","enrich_search_similar_companies","enrich_verify_email","enrichment_run","enrow_find_email","enrow_verify_email","exa_agent","exa_answer","exa_find_similar_links","exa_get_contents","exa_search","extend_parser","extend_parser_v2","fathom_get_summary","fathom_get_transcript","fathom_list_meeting_types","fathom_list_meetings","fathom_list_team_members","fathom_list_teams","file_append","file_compress","file_create_folder","file_decompress","file_delete_folder","file_edit","file_fetch","file_get","file_get_content","file_list","file_manage_sharing","file_move","file_parser","file_parser_v2","file_parser_v3","file_read","file_restore_folder","file_search","file_update_folder","file_write","findymail_find_email_from_linkedin","findymail_find_email_from_name","findymail_find_emails_by_domain","findymail_find_employees","findymail_find_phone","findymail_get_company","findymail_get_credits","findymail_lookup_technologies","findymail_reverse_email_lookup","findymail_search_technologies","findymail_verify_email","firecrawl_agent","firecrawl_batch_scrape","firecrawl_batch_scrape_status","firecrawl_cancel_crawl","firecrawl_crawl","firecrawl_crawl_status","firecrawl_credit_usage","firecrawl_extract","firecrawl_extract_status","firecrawl_map","firecrawl_parse","firecrawl_scrape","firecrawl_search","fireflies_add_to_live_meeting","fireflies_create_bite","fireflies_delete_transcript","fireflies_get_transcript","fireflies_get_user","fireflies_list_bites","fireflies_list_contacts","fireflies_list_transcripts","fireflies_list_users","fireflies_upload_audio","flint_create_task","flint_generate_pages","flint_get_task","function_execute","gamma_check_status","gamma_generate","gamma_generate_from_template","gamma_list_folders","gamma_list_themes","github_add_assignees","github_add_assignees_v2","github_add_labels","github_add_labels_v2","github_cancel_workflow_run","github_cancel_workflow_run_v2","github_check_star","github_check_star_v2","github_close_issue","github_close_issue_v2","github_close_pr","github_close_pr_v2","github_comment","github_comment_v2","github_compare_commits","github_compare_commits_v2","github_create_branch","github_create_branch_v2","github_create_comment_reaction","github_create_comment_reaction_v2","github_create_file","github_create_file_v2","github_create_gist","github_create_gist_v2","github_create_issue","github_create_issue_reaction","github_create_issue_reaction_v2","github_create_issue_v2","github_create_milestone","github_create_milestone_v2","github_create_pr","github_create_pr_review","github_create_pr_review_v2","github_create_pr_v2","github_create_project","github_create_project_v2","github_create_release","github_create_release_v2","github_delete_branch","github_delete_branch_v2","github_delete_comment","github_delete_comment_reaction","github_delete_comment_reaction_v2","github_delete_comment_v2","github_delete_file","github_delete_file_v2","github_delete_gist","github_delete_gist_v2","github_delete_issue_reaction","github_delete_issue_reaction_v2","github_delete_milestone","github_delete_milestone_v2","github_delete_project","github_delete_project_v2","github_delete_release","github_delete_release_v2","github_fork_gist","github_fork_gist_v2","github_fork_repo","github_fork_repo_v2","github_get_branch","github_get_branch_protection","github_get_branch_protection_v2","github_get_branch_v2","github_get_commit","github_get_commit_v2","github_get_file_content","github_get_file_content_v2","github_get_gist","github_get_gist_v2","github_get_issue","github_get_issue_v2","github_get_latest_release","github_get_latest_release_v2","github_get_milestone","github_get_milestone_v2","github_get_pr_files","github_get_pr_files_v2","github_get_project","github_get_project_v2","github_get_readme","github_get_readme_v2","github_get_release","github_get_release_v2","github_get_tree","github_get_tree_v2","github_get_workflow","github_get_workflow_run","github_get_workflow_run_v2","github_get_workflow_v2","github_issue_comment","github_issue_comment_v2","github_job_logs","github_latest_commit","github_latest_commit_v2","github_list_branches","github_list_branches_v2","github_list_commits","github_list_commits_v2","github_list_forks","github_list_forks_v2","github_list_gists","github_list_gists_v2","github_list_issue_comments","github_list_issue_comments_v2","github_list_issues","github_list_issues_v2","github_list_milestones","github_list_milestones_v2","github_list_pr_comments","github_list_pr_comments_v2","github_list_projects","github_list_projects_v2","github_list_prs","github_list_prs_v2","github_list_releases","github_list_releases_v2","github_list_review_threads","github_list_stargazers","github_list_stargazers_v2","github_list_tags","github_list_tags_v2","github_list_workflow_runs","github_list_workflow_runs_v2","github_list_workflows","github_list_workflows_v2","github_merge_pr","github_merge_pr_v2","github_pr","github_pr_v2","github_remove_label","github_remove_label_v2","github_reply_review_thread","github_repo_info","github_repo_info_v2","github_request_reviewers","github_request_reviewers_v2","github_rerun_workflow","github_rerun_workflow_v2","github_resolve_review_thread","github_search_code","github_search_code_v2","github_search_commits","github_search_commits_v2","github_search_issues","github_search_issues_v2","github_search_repos","github_search_repos_v2","github_search_users","github_search_users_v2","github_star_gist","github_star_gist_v2","github_star_repo","github_star_repo_v2","github_status_check_rollup","github_trigger_workflow","github_trigger_workflow_v2","github_unstar_gist","github_unstar_gist_v2","github_unstar_repo","github_unstar_repo_v2","github_update_branch_protection","github_update_branch_protection_v2","github_update_comment","github_update_comment_v2","github_update_file","github_update_file_v2","github_update_gist","github_update_gist_v2","github_update_issue","github_update_issue_v2","github_update_milestone","github_update_milestone_v2","github_update_pr","github_update_pr_v2","github_update_project","github_update_project_v2","github_update_release","github_update_release_v2","gitlab_activate_user","gitlab_add_member","gitlab_add_saml_group_link","gitlab_approve_access_request","gitlab_approve_merge_request","gitlab_approve_user","gitlab_ban_user","gitlab_block_user","gitlab_cancel_pipeline","gitlab_compare_branches","gitlab_create_branch","gitlab_create_file","gitlab_create_issue","gitlab_create_issue_note","gitlab_create_merge_request","gitlab_create_merge_request_note","gitlab_create_pipeline","gitlab_create_release","gitlab_create_user","gitlab_deactivate_user","gitlab_delete_branch","gitlab_delete_issue","gitlab_delete_saml_group_link","gitlab_delete_user","gitlab_delete_user_identity","gitlab_deny_access_request","gitlab_get_file","gitlab_get_group","gitlab_get_issue","gitlab_get_job_log","gitlab_get_merge_request","gitlab_get_merge_request_changes","gitlab_get_pipeline","gitlab_get_project","gitlab_invite_member","gitlab_list_access_requests","gitlab_list_branches","gitlab_list_commits","gitlab_list_groups","gitlab_list_invitations","gitlab_list_issues","gitlab_list_members","gitlab_list_merge_requests","gitlab_list_pipeline_jobs","gitlab_list_pipelines","gitlab_list_projects","gitlab_list_releases","gitlab_list_repository_tree","gitlab_list_saml_group_links","gitlab_list_user_memberships","gitlab_merge_merge_request","gitlab_play_job","gitlab_reject_user","gitlab_remove_member","gitlab_retry_pipeline","gitlab_revoke_invitation","gitlab_search_users","gitlab_unban_user","gitlab_unblock_user","gitlab_update_file","gitlab_update_invitation","gitlab_update_issue","gitlab_update_member","gitlab_update_merge_request","gitlab_update_user","gmail_add_label","gmail_add_label_v2","gmail_archive","gmail_archive_v2","gmail_create_label_v2","gmail_delete","gmail_delete_draft_v2","gmail_delete_label_v2","gmail_delete_v2","gmail_draft","gmail_draft_v2","gmail_edit_draft_v2","gmail_get_draft_v2","gmail_get_thread_v2","gmail_list_drafts_v2","gmail_list_labels_v2","gmail_list_threads_v2","gmail_mark_read","gmail_mark_read_v2","gmail_mark_unread","gmail_mark_unread_v2","gmail_move","gmail_move_v2","gmail_read","gmail_read_v2","gmail_remove_label","gmail_remove_label_v2","gmail_search","gmail_search_v2","gmail_send","gmail_send_v2","gmail_trash_thread_v2","gmail_unarchive","gmail_unarchive_v2","gmail_untrash_thread_v2","gmail_update_label_v2","gong_aggregate_activity","gong_aggregate_by_period","gong_answered_scorecards","gong_ask_anything","gong_assign_flow_prospects","gong_create_call","gong_day_by_day_activity","gong_get_brief","gong_get_call","gong_get_call_transcript","gong_get_coaching","gong_get_extensive_calls","gong_get_folder_content","gong_get_logs","gong_get_prospect_flows","gong_get_user","gong_interaction_stats","gong_list_calls","gong_list_flows","gong_list_library_folders","gong_list_scorecards","gong_list_trackers","gong_list_users","gong_list_workspaces","gong_lookup_email","gong_lookup_phone","gong_purge_email_address","gong_purge_phone_number","gong_unassign_flow_prospects","google_ads_ad_performance","google_ads_campaign_performance","google_ads_list_ad_groups","google_ads_list_campaigns","google_ads_list_customers","google_ads_search","google_appsheet_add_rows","google_appsheet_delete_rows","google_appsheet_edit_rows","google_appsheet_find_rows","google_bigquery_create_dataset","google_bigquery_create_table","google_bigquery_delete_dataset","google_bigquery_delete_table","google_bigquery_get_query_results","google_bigquery_get_table","google_bigquery_insert_rows","google_bigquery_list_datasets","google_bigquery_list_table_data","google_bigquery_list_tables","google_bigquery_query","google_books_volume_details","google_books_volume_search","google_calendar_create","google_calendar_create_calendar","google_calendar_create_calendar_v2","google_calendar_create_v2","google_calendar_delete","google_calendar_delete_calendar","google_calendar_delete_calendar_v2","google_calendar_delete_v2","google_calendar_freebusy","google_calendar_freebusy_v2","google_calendar_get","google_calendar_get_v2","google_calendar_instances","google_calendar_instances_v2","google_calendar_invite","google_calendar_invite_v2","google_calendar_list","google_calendar_list_acl","google_calendar_list_acl_v2","google_calendar_list_calendars","google_calendar_list_calendars_v2","google_calendar_list_v2","google_calendar_move","google_calendar_move_v2","google_calendar_quick_add","google_calendar_quick_add_v2","google_calendar_share_calendar","google_calendar_share_calendar_v2","google_calendar_unshare_calendar","google_calendar_unshare_calendar_v2","google_calendar_update","google_calendar_update_acl","google_calendar_update_acl_v2","google_calendar_update_calendar","google_calendar_update_calendar_v2","google_calendar_update_v2","google_contacts_create","google_contacts_delete","google_contacts_get","google_contacts_list","google_contacts_search","google_contacts_update","google_docs_create","google_docs_create_named_range","google_docs_create_paragraph_bullets","google_docs_delete_content_range","google_docs_delete_named_range","google_docs_delete_paragraph_bullets","google_docs_insert_image","google_docs_insert_page_break","google_docs_insert_table","google_docs_insert_text","google_docs_read","google_docs_replace_text","google_docs_update_paragraph_style","google_docs_update_text_style","google_docs_write","google_drive_copy","google_drive_create_comment","google_drive_create_folder","google_drive_delete","google_drive_delete_comment","google_drive_download","google_drive_export","google_drive_get_about","google_drive_get_content","google_drive_get_file","google_drive_get_revision","google_drive_list","google_drive_list_comments","google_drive_list_permissions","google_drive_list_revisions","google_drive_move","google_drive_search","google_drive_share","google_drive_trash","google_drive_unshare","google_drive_untrash","google_drive_update","google_drive_upload","google_forms_batch_update","google_forms_create_form","google_forms_create_watch","google_forms_delete_watch","google_forms_get_form","google_forms_get_responses","google_forms_list_watches","google_forms_renew_watch","google_forms_set_publish_settings","google_groups_add_alias","google_groups_add_member","google_groups_create_group","google_groups_delete_group","google_groups_get_group","google_groups_get_member","google_groups_get_settings","google_groups_has_member","google_groups_list_aliases","google_groups_list_groups","google_groups_list_members","google_groups_remove_alias","google_groups_remove_member","google_groups_update_group","google_groups_update_member","google_groups_update_settings","google_maps_air_quality","google_maps_directions","google_maps_distance_matrix","google_maps_elevation","google_maps_geocode","google_maps_geolocate","google_maps_place_details","google_maps_places_nearby","google_maps_places_search","google_maps_pollen","google_maps_reverse_geocode","google_maps_snap_to_roads","google_maps_solar","google_maps_speed_limits","google_maps_timezone","google_maps_validate_address","google_meet_create_space","google_meet_end_conference","google_meet_get_conference_record","google_meet_get_space","google_meet_list_conference_records","google_meet_list_participants","google_pagespeed_analyze","google_search","google_sheets_append","google_sheets_append_v2","google_sheets_batch_clear_v2","google_sheets_batch_get_v2","google_sheets_batch_update_v2","google_sheets_clear_v2","google_sheets_copy_sheet_v2","google_sheets_create_spreadsheet_v2","google_sheets_delete_rows_v2","google_sheets_delete_sheet_v2","google_sheets_delete_spreadsheet_v2","google_sheets_get_spreadsheet_v2","google_sheets_read","google_sheets_read_v2","google_sheets_update","google_sheets_update_v2","google_sheets_write","google_sheets_write_v2","google_slides_add_image","google_slides_add_slide","google_slides_batch_update","google_slides_copy_presentation","google_slides_create","google_slides_create_line","google_slides_create_paragraph_bullets","google_slides_create_shape","google_slides_create_sheets_chart","google_slides_create_table","google_slides_create_video","google_slides_delete_object","google_slides_delete_paragraph_bullets","google_slides_delete_table_column","google_slides_delete_table_row","google_slides_delete_text","google_slides_duplicate_object","google_slides_export_presentation","google_slides_get_page","google_slides_get_thumbnail","google_slides_group_objects","google_slides_insert_table_columns","google_slides_insert_table_rows","google_slides_insert_text","google_slides_merge_table_cells","google_slides_read","google_slides_refresh_sheets_chart","google_slides_replace_all_shapes_with_image","google_slides_replace_all_shapes_with_sheets_chart","google_slides_replace_all_text","google_slides_replace_image","google_slides_reroute_line","google_slides_ungroup_objects","google_slides_unmerge_table_cells","google_slides_update_image_properties","google_slides_update_line_category","google_slides_update_line_properties","google_slides_update_page_element_alt_text","google_slides_update_page_element_transform","google_slides_update_page_elements_z_order","google_slides_update_page_properties","google_slides_update_paragraph_style","google_slides_update_shape_properties","google_slides_update_slide_properties","google_slides_update_slides_position","google_slides_update_table_border_properties","google_slides_update_table_cell_properties","google_slides_update_table_column_properties","google_slides_update_table_row_properties","google_slides_update_text_style","google_slides_update_video_properties","google_slides_write","google_tasks_create","google_tasks_delete","google_tasks_get","google_tasks_list","google_tasks_list_task_lists","google_tasks_update","google_translate_detect","google_translate_text","google_vault_add_held_accounts","google_vault_add_matters_permissions","google_vault_close_matters","google_vault_create_matters","google_vault_create_matters_export","google_vault_create_matters_holds","google_vault_create_saved_query","google_vault_delete_matters","google_vault_delete_matters_export","google_vault_delete_matters_holds","google_vault_delete_saved_query","google_vault_download_export_file","google_vault_list_matters","google_vault_list_matters_export","google_vault_list_matters_holds","google_vault_list_saved_queries","google_vault_remove_held_accounts","google_vault_remove_matters_permissions","google_vault_reopen_matters","google_vault_undelete_matters","google_vault_update_matters","google_vault_update_matters_holds","grafana_check_data_source_health","grafana_create_alert_rule","grafana_create_annotation","grafana_create_contact_point","grafana_create_dashboard","grafana_create_folder","grafana_delete_alert_rule","grafana_delete_annotation","grafana_delete_contact_point","grafana_delete_dashboard","grafana_delete_folder","grafana_get_alert_rule","grafana_get_alert_rule_group","grafana_get_dashboard","grafana_get_data_source","grafana_get_folder","grafana_get_health","grafana_list_alert_rules","grafana_list_annotations","grafana_list_contact_points","grafana_list_dashboards","grafana_list_data_sources","grafana_list_folders","grafana_move_folder","grafana_query_data_source","grafana_update_alert_rule","grafana_update_annotation","grafana_update_contact_point","grafana_update_dashboard","grafana_update_folder","grain_create_hook","grain_create_hook_v2","grain_delete_hook","grain_delete_hook_v2","grain_get_recording","grain_get_transcript","grain_list_hooks","grain_list_hooks_v2","grain_list_meeting_types","grain_list_recordings","grain_list_teams","grain_list_views","granola_create_webhook_endpoint","granola_delete_webhook_endpoint","granola_get_note","granola_get_transcript","granola_list_audit_events","granola_list_folders","granola_list_notes","granola_list_webhook_endpoints","granola_update_webhook_endpoint","greenhouse_get_application","greenhouse_get_candidate","greenhouse_get_job","greenhouse_get_user","greenhouse_list_applications","greenhouse_list_candidates","greenhouse_list_departments","greenhouse_list_job_stages","greenhouse_list_jobs","greenhouse_list_offices","greenhouse_list_users","greptile_index_repo","greptile_query","greptile_search","greptile_status","guardrails_validate","harmonic_batch_get_people","harmonic_clear_people_saved_search_net_new_results","harmonic_enrich_person","harmonic_get_company_employees","harmonic_get_email_enrichment_job","harmonic_get_email_enrichment_usage","harmonic_get_enrichment_status","harmonic_get_people_saved_search_net_new_results","harmonic_get_people_saved_search_results","harmonic_get_person","harmonic_list_people_saved_searches","harmonic_search_people_scout","harmonic_submit_email_enrichment_job","hex_cancel_run","hex_create_collection","hex_create_group","hex_deactivate_user","hex_delete_group","hex_get_collection","hex_get_data_connection","hex_get_group","hex_get_project","hex_get_project_runs","hex_get_queried_tables","hex_get_run_status","hex_list_collections","hex_list_data_connections","hex_list_groups","hex_list_projects","hex_list_users","hex_run_project","hex_update_collection","hex_update_group","hex_update_project","http_request","hubspot_add_list_memberships","hubspot_create_appointment","hubspot_create_association","hubspot_create_company","hubspot_create_contact","hubspot_create_deal","hubspot_create_email","hubspot_create_line_item","hubspot_create_list","hubspot_create_note","hubspot_create_ticket","hubspot_delete_association","hubspot_delete_company","hubspot_delete_contact","hubspot_delete_deal","hubspot_delete_line_item","hubspot_delete_ticket","hubspot_get_appointment","hubspot_get_association_labels","hubspot_get_cart","hubspot_get_company","hubspot_get_contact","hubspot_get_deal","hubspot_get_email","hubspot_get_line_item","hubspot_get_list","hubspot_get_list_memberships","hubspot_get_marketing_event","hubspot_get_note","hubspot_get_properties","hubspot_get_quote","hubspot_get_ticket","hubspot_get_users","hubspot_list_appointments","hubspot_list_associations","hubspot_list_carts","hubspot_list_companies","hubspot_list_contacts","hubspot_list_deals","hubspot_list_emails","hubspot_list_line_items","hubspot_list_lists","hubspot_list_marketing_events","hubspot_list_notes","hubspot_list_owners","hubspot_list_quotes","hubspot_list_tickets","hubspot_remove_list_memberships","hubspot_search_companies","hubspot_search_contacts","hubspot_search_deals","hubspot_search_emails","hubspot_search_line_items","hubspot_search_notes","hubspot_search_quotes","hubspot_search_tickets","hubspot_update_appointment","hubspot_update_company","hubspot_update_contact","hubspot_update_deal","hubspot_update_line_item","hubspot_update_ticket","huggingface_chat","hunter_companies_find","hunter_discover","hunter_domain_search","hunter_email_count","hunter_email_finder","hunter_email_verifier","iam_add_user_to_group","iam_attach_role_policy","iam_attach_user_policy","iam_create_access_key","iam_create_role","iam_create_user","iam_delete_access_key","iam_delete_role","iam_delete_user","iam_detach_role_policy","iam_detach_user_policy","iam_get_policy","iam_get_role","iam_get_user","iam_list_access_keys","iam_list_attached_role_policies","iam_list_attached_user_policies","iam_list_groups","iam_list_policies","iam_list_roles","iam_list_users","iam_remove_user_from_group","iam_simulate_principal_policy","iam_update_access_key","icypeas_find_email","icypeas_verify_email","identity_center_check_assignment_deletion_status","identity_center_check_assignment_status","identity_center_create_account_assignment","identity_center_delete_account_assignment","identity_center_describe_account","identity_center_describe_group","identity_center_describe_user","identity_center_get_group","identity_center_get_user","identity_center_list_account_assignments","identity_center_list_accounts","identity_center_list_assignments_for_account","identity_center_list_group_memberships","identity_center_list_groups","identity_center_list_instances","identity_center_list_permission_sets","image_generate","incidentio_actions_create","incidentio_actions_list","incidentio_actions_show","incidentio_actions_update","incidentio_alert_events_create","incidentio_alerts_list","incidentio_alerts_resolve","incidentio_alerts_show","incidentio_catalog_entries_list","incidentio_catalog_types_list","incidentio_custom_fields_create","incidentio_custom_fields_delete","incidentio_custom_fields_list","incidentio_custom_fields_show","incidentio_custom_fields_update","incidentio_escalation_paths_create","incidentio_escalation_paths_delete","incidentio_escalation_paths_list","incidentio_escalation_paths_show","incidentio_escalation_paths_update","incidentio_escalations_cancel","incidentio_escalations_create","incidentio_escalations_list","incidentio_escalations_show","incidentio_follow_ups_create","incidentio_follow_ups_list","incidentio_follow_ups_show","incidentio_follow_ups_update","incidentio_incident_alerts_list","incidentio_incident_memberships_create","incidentio_incident_memberships_revoke","incidentio_incident_participants_list","incidentio_incident_roles_create","incidentio_incident_roles_delete","incidentio_incident_roles_list","incidentio_incident_roles_show","incidentio_incident_roles_update","incidentio_incident_statuses_list","incidentio_incident_timestamps_list","incidentio_incident_timestamps_show","incidentio_incident_types_list","incidentio_incident_updates_list","incidentio_incidents_create","incidentio_incidents_list","incidentio_incidents_show","incidentio_incidents_update","incidentio_on_call_now","incidentio_schedule_entries_list","incidentio_schedule_overrides_create","incidentio_schedule_overrides_list","incidentio_schedules_create","incidentio_schedules_delete","incidentio_schedules_list","incidentio_schedules_show","incidentio_schedules_update","incidentio_severities_list","incidentio_teams_list","incidentio_teams_show","incidentio_users_list","incidentio_users_show","incidentio_workflows_create","incidentio_workflows_delete","incidentio_workflows_list","incidentio_workflows_show","incidentio_workflows_update","infisical_create_secret","infisical_delete_secret","infisical_get_secret","infisical_list_secrets","infisical_update_secret","instagram_delete_comment","instagram_download_media","instagram_get_account_insights","instagram_get_container_status","instagram_get_conversation_messages","instagram_get_media","instagram_get_media_insights","instagram_get_message","instagram_get_profile","instagram_get_publishing_limit","instagram_hide_comment","instagram_list_comments","instagram_list_conversations","instagram_list_media","instagram_list_stories","instagram_private_reply","instagram_publish_carousel","instagram_publish_image","instagram_publish_reel","instagram_publish_story","instagram_publish_video","instagram_reply_to_comment","instagram_send_text_message","instagram_set_comments_enabled","instantly_activate_campaign","instantly_create_campaign","instantly_create_lead","instantly_create_lead_list","instantly_delete_campaign","instantly_delete_leads","instantly_get_lead","instantly_list_campaigns","instantly_list_emails","instantly_list_lead_lists","instantly_list_leads","instantly_patch_campaign","instantly_patch_lead","instantly_pause_campaign","instantly_reply_to_email","instantly_update_lead_interest_status","intercom_assign_conversation_v2","intercom_attach_contact_to_company_v2","intercom_close_conversation_v2","intercom_create_company","intercom_create_company_v2","intercom_create_contact","intercom_create_contact_v2","intercom_create_event_v2","intercom_create_message","intercom_create_message_v2","intercom_create_note_v2","intercom_create_tag_v2","intercom_create_ticket","intercom_create_ticket_v2","intercom_delete_contact","intercom_delete_contact_v2","intercom_detach_contact_from_company_v2","intercom_get_company","intercom_get_company_v2","intercom_get_contact","intercom_get_contact_v2","intercom_get_conversation","intercom_get_conversation_v2","intercom_get_ticket","intercom_get_ticket_v2","intercom_list_admins_v2","intercom_list_companies","intercom_list_companies_v2","intercom_list_contacts","intercom_list_contacts_v2","intercom_list_conversations","intercom_list_conversations_v2","intercom_list_tags_v2","intercom_open_conversation_v2","intercom_reply_conversation","intercom_reply_conversation_v2","intercom_search_contacts","intercom_search_contacts_v2","intercom_search_conversations","intercom_search_conversations_v2","intercom_snooze_conversation_v2","intercom_tag_contact_v2","intercom_tag_conversation_v2","intercom_untag_contact_v2","intercom_update_contact","intercom_update_contact_v2","intercom_update_ticket_v2","jina_read_url","jina_search","jira_add_attachment","jira_add_comment","jira_add_watcher","jira_add_worklog","jira_assign_issue","jira_bulk_read","jira_create_issue_link","jira_delete_attachment","jira_delete_comment","jira_delete_issue","jira_delete_issue_link","jira_delete_worklog","jira_get_attachments","jira_get_comments","jira_get_fields","jira_get_project","jira_get_transitions","jira_get_users","jira_get_worklogs","jira_list_issue_types","jira_list_projects","jira_remove_watcher","jira_retrieve","jira_search_issues","jira_search_users","jira_transition_issue","jira_update","jira_update_comment","jira_update_worklog","jira_write","jotform_add_label_resources","jotform_clone_form","jotform_create_form","jotform_create_label","jotform_create_question","jotform_create_questions","jotform_create_report","jotform_create_submission","jotform_create_submissions","jotform_create_webhook","jotform_delete_form","jotform_delete_label","jotform_delete_question","jotform_delete_report","jotform_delete_submission","jotform_delete_webhook","jotform_get_form","jotform_get_form_properties","jotform_get_history","jotform_get_label","jotform_get_question","jotform_get_report","jotform_get_settings","jotform_get_submission","jotform_get_usage","jotform_get_user","jotform_list_form_files","jotform_list_form_reports","jotform_list_form_submissions","jotform_list_forms","jotform_list_label_resources","jotform_list_labels","jotform_list_questions","jotform_list_reports","jotform_list_submissions","jotform_list_subusers","jotform_list_webhooks","jotform_remove_label_resources","jotform_update_form_properties","jotform_update_label","jotform_update_question","jotform_update_settings","jotform_update_submission","jsm_add_comment","jsm_add_customer","jsm_add_organization","jsm_add_participants","jsm_answer_approval","jsm_attach_form","jsm_copy_forms","jsm_create_object","jsm_create_organization","jsm_create_request","jsm_delete_form","jsm_delete_object","jsm_externalise_form","jsm_get_approvals","jsm_get_comments","jsm_get_customers","jsm_get_form","jsm_get_form_answers","jsm_get_form_structure","jsm_get_form_templates","jsm_get_issue_forms","jsm_get_object","jsm_get_object_schema","jsm_get_object_type_attributes","jsm_get_organizations","jsm_get_participants","jsm_get_queues","jsm_get_request","jsm_get_request_type_fields","jsm_get_request_types","jsm_get_requests","jsm_get_service_desks","jsm_get_sla","jsm_get_transitions","jsm_internalise_form","jsm_list_object_schemas","jsm_list_object_types","jsm_reopen_form","jsm_save_form_answers","jsm_search_objects_aql","jsm_submit_form","jsm_transition_request","jsm_update_object","jupyter_copy_content","jupyter_create_file","jupyter_create_session","jupyter_delete_content","jupyter_delete_session","jupyter_get_content","jupyter_get_content_v2","jupyter_interrupt_kernel","jupyter_list_contents","jupyter_list_kernels","jupyter_list_kernelspecs","jupyter_list_sessions","jupyter_rename_content","jupyter_restart_kernel","jupyter_start_kernel","jupyter_stop_kernel","jupyter_upload_file","kalshi_amend_order","kalshi_amend_order_v2","kalshi_cancel_order","kalshi_cancel_order_v2","kalshi_create_order","kalshi_create_order_v2","kalshi_get_balance","kalshi_get_balance_v2","kalshi_get_candlesticks","kalshi_get_candlesticks_v2","kalshi_get_event","kalshi_get_event_candlesticks","kalshi_get_event_candlesticks_v2","kalshi_get_event_v2","kalshi_get_events","kalshi_get_events_v2","kalshi_get_exchange_announcements","kalshi_get_exchange_announcements_v2","kalshi_get_exchange_schedule","kalshi_get_exchange_schedule_v2","kalshi_get_exchange_status","kalshi_get_exchange_status_v2","kalshi_get_fills","kalshi_get_fills_v2","kalshi_get_market","kalshi_get_market_v2","kalshi_get_markets","kalshi_get_markets_v2","kalshi_get_order","kalshi_get_order_v2","kalshi_get_orderbook","kalshi_get_orderbook_v2","kalshi_get_orders","kalshi_get_orders_v2","kalshi_get_positions","kalshi_get_positions_v2","kalshi_get_series_by_ticker","kalshi_get_series_by_ticker_v2","kalshi_get_series_list","kalshi_get_series_list_v2","kalshi_get_settlements","kalshi_get_settlements_v2","kalshi_get_trades","kalshi_get_trades_v2","ketch_get_consent","ketch_get_subscriptions","ketch_invoke_right","ketch_set_consent","ketch_set_subscriptions","knowledge_create_document","knowledge_delete_chunk","knowledge_delete_document","knowledge_get_connector","knowledge_get_document","knowledge_list_chunks","knowledge_list_connectors","knowledge_list_documents","knowledge_list_tags","knowledge_search","knowledge_trigger_sync","knowledge_update_chunk","knowledge_upload_chunk","knowledge_upsert_document","lambda_add_permission","lambda_create_alias","lambda_create_event_source_mapping","lambda_create_function","lambda_create_function_url_config","lambda_delete_alias","lambda_delete_event_source_mapping","lambda_delete_function","lambda_delete_function_concurrency","lambda_delete_function_event_invoke_config","lambda_delete_function_url_config","lambda_delete_provisioned_concurrency_config","lambda_get_account_settings","lambda_get_alias","lambda_get_event_source_mapping","lambda_get_function","lambda_get_function_concurrency","lambda_get_function_configuration","lambda_get_function_event_invoke_config","lambda_get_function_recursion_config","lambda_get_function_url_config","lambda_get_layer_version","lambda_get_policy","lambda_get_provisioned_concurrency_config","lambda_get_runtime_management_config","lambda_invoke","lambda_list_aliases","lambda_list_event_source_mappings","lambda_list_function_event_invoke_configs","lambda_list_function_url_configs","lambda_list_functions","lambda_list_layer_versions","lambda_list_layers","lambda_list_provisioned_concurrency_configs","lambda_list_tags","lambda_list_versions_by_function","lambda_publish_version","lambda_put_function_concurrency","lambda_put_function_event_invoke_config","lambda_put_function_recursion_config","lambda_put_provisioned_concurrency_config","lambda_put_runtime_management_config","lambda_remove_permission","lambda_tag_resource","lambda_untag_resource","lambda_update_alias","lambda_update_event_source_mapping","lambda_update_function_code","lambda_update_function_configuration","lambda_update_function_url_config","langsmith_create_feedback","langsmith_create_run","langsmith_create_runs_batch","langsmith_get_run","langsmith_update_run","latex_compile","latex_get_package","latex_list_fonts","latex_search_packages","launchdarkly_create_flag","launchdarkly_delete_flag","launchdarkly_get_audit_log","launchdarkly_get_flag","launchdarkly_get_flag_status","launchdarkly_list_environments","launchdarkly_list_flags","launchdarkly_list_members","launchdarkly_list_projects","launchdarkly_list_segments","launchdarkly_toggle_flag","launchdarkly_update_flag","leadmagic_company_search","leadmagic_email_to_profile","leadmagic_find_email","leadmagic_find_mobile","leadmagic_get_credits","leadmagic_profile_search","leadmagic_profile_to_email","leadmagic_role_finder","leadmagic_validate_email","lemlist_get_activities","lemlist_get_lead","lemlist_send_email","linear_add_label_to_issue","linear_add_label_to_project","linear_archive_issue","linear_archive_label","linear_archive_project","linear_create_attachment","linear_create_comment","linear_create_customer","linear_create_customer_request","linear_create_customer_status","linear_create_customer_tier","linear_create_cycle","linear_create_favorite","linear_create_issue","linear_create_issue_relation","linear_create_label","linear_create_project","linear_create_project_label","linear_create_project_milestone","linear_create_project_status","linear_create_project_update","linear_create_workflow_state","linear_delete_attachment","linear_delete_comment","linear_delete_customer","linear_delete_customer_status","linear_delete_customer_tier","linear_delete_issue","linear_delete_issue_relation","linear_delete_project","linear_delete_project_label","linear_delete_project_milestone","linear_delete_project_status","linear_get_active_cycle","linear_get_customer","linear_get_cycle","linear_get_issue","linear_get_project","linear_get_viewer","linear_list_attachments","linear_list_comments","linear_list_customer_requests","linear_list_customer_statuses","linear_list_customer_tiers","linear_list_customers","linear_list_cycles","linear_list_favorites","linear_list_issue_relations","linear_list_labels","linear_list_notifications","linear_list_project_labels","linear_list_project_milestones","linear_list_project_statuses","linear_list_project_updates","linear_list_projects","linear_list_teams","linear_list_users","linear_list_workflow_states","linear_merge_customers","linear_read_issues","linear_remove_label_from_issue","linear_remove_label_from_project","linear_search_issues","linear_unarchive_issue","linear_update_attachment","linear_update_comment","linear_update_customer","linear_update_customer_request","linear_update_customer_status","linear_update_customer_tier","linear_update_issue","linear_update_label","linear_update_notification","linear_update_project","linear_update_project_label","linear_update_project_milestone","linear_update_project_status","linear_update_workflow_state","linkedin_get_profile","linkedin_share_post","linkup_search","linq_add_participant","linq_check_imessage","linq_check_rcs","linq_create_attachment","linq_create_chat","linq_create_contact_card","linq_create_webhook_subscription","linq_delete_attachment","linq_delete_message","linq_delete_webhook_subscription","linq_edit_message","linq_get_attachment","linq_get_chat","linq_get_contact_card","linq_get_message","linq_get_webhook_subscription","linq_leave_chat","linq_list_chats","linq_list_messages","linq_list_phone_numbers","linq_list_thread","linq_list_webhook_events","linq_list_webhook_subscriptions","linq_mark_chat_read","linq_react_to_message","linq_remove_participant","linq_send_message","linq_send_voice_memo","linq_share_contact_card","linq_start_typing","linq_stop_typing","linq_update_chat","linq_update_contact_card","linq_update_webhook_subscription","llm_chat","logfire_get_token_info","logfire_get_trace","logfire_query","logfire_search_records","logrocket_create_release","logrocket_get_audit_logs","logrocket_get_highlights","logrocket_identify_user","logrocket_list_exported_sessions","logrocket_request_highlights","logs_get","logs_get_execution","logs_get_run_details","logs_query","logs_query_runs","loops_check_contact_suppression","loops_create_contact","loops_create_contact_property","loops_delete_contact","loops_find_contact","loops_get_transactional_email","loops_list_contact_properties","loops_list_mailing_lists","loops_list_transactional_emails","loops_remove_contact_suppression","loops_send_event","loops_send_transactional_email","loops_update_contact","luma_add_guests","luma_cancel_event","luma_create_event","luma_get_event","luma_get_guest","luma_get_guests","luma_list_events","luma_lookup_event","luma_send_invites","luma_update_event","luma_update_guest_status","mailchimp_add_member","mailchimp_add_member_tags","mailchimp_add_or_update_member","mailchimp_add_segment_member","mailchimp_add_subscriber_to_automation","mailchimp_archive_member","mailchimp_create_audience","mailchimp_create_batch_operation","mailchimp_create_campaign","mailchimp_create_interest","mailchimp_create_interest_category","mailchimp_create_landing_page","mailchimp_create_merge_field","mailchimp_create_segment","mailchimp_create_template","mailchimp_delete_audience","mailchimp_delete_batch_operation","mailchimp_delete_campaign","mailchimp_delete_interest","mailchimp_delete_interest_category","mailchimp_delete_landing_page","mailchimp_delete_member","mailchimp_delete_merge_field","mailchimp_delete_segment","mailchimp_delete_template","mailchimp_get_audience","mailchimp_get_audiences","mailchimp_get_automation","mailchimp_get_automations","mailchimp_get_batch_operation","mailchimp_get_batch_operations","mailchimp_get_campaign","mailchimp_get_campaign_content","mailchimp_get_campaign_report","mailchimp_get_campaign_reports","mailchimp_get_campaigns","mailchimp_get_interest","mailchimp_get_interest_categories","mailchimp_get_interest_category","mailchimp_get_interests","mailchimp_get_landing_page","mailchimp_get_landing_pages","mailchimp_get_member","mailchimp_get_member_tags","mailchimp_get_members","mailchimp_get_merge_field","mailchimp_get_merge_fields","mailchimp_get_segment","mailchimp_get_segment_members","mailchimp_get_segments","mailchimp_get_template","mailchimp_get_templates","mailchimp_pause_automation","mailchimp_publish_landing_page","mailchimp_remove_member_tags","mailchimp_remove_segment_member","mailchimp_replicate_campaign","mailchimp_schedule_campaign","mailchimp_send_campaign","mailchimp_set_campaign_content","mailchimp_start_automation","mailchimp_unarchive_member","mailchimp_unpublish_landing_page","mailchimp_unschedule_campaign","mailchimp_update_audience","mailchimp_update_campaign","mailchimp_update_interest","mailchimp_update_interest_category","mailchimp_update_landing_page","mailchimp_update_member","mailchimp_update_merge_field","mailchimp_update_segment","mailchimp_update_template","mailgun_add_list_member","mailgun_create_mailing_list","mailgun_get_domain","mailgun_get_mailing_list","mailgun_get_message","mailgun_list_domains","mailgun_list_messages","mailgun_send_message","managed_agent_archive_session","managed_agent_create_session","managed_agent_delete_session","managed_agent_get_session","managed_agent_interrupt_session","managed_agent_list_events","managed_agent_respond_custom_tool","managed_agent_respond_tool_confirmation","managed_agent_run_session","managed_agent_send_message","managed_agent_update_session","manageengine_sdp_add_change_note","manageengine_sdp_add_problem_note","manageengine_sdp_add_request_note","manageengine_sdp_create_asset","manageengine_sdp_create_change","manageengine_sdp_create_problem","manageengine_sdp_create_request","manageengine_sdp_create_solution","manageengine_sdp_delete_asset","manageengine_sdp_delete_change","manageengine_sdp_delete_problem","manageengine_sdp_delete_request","manageengine_sdp_delete_solution","manageengine_sdp_get_asset","manageengine_sdp_get_change","manageengine_sdp_get_problem","manageengine_sdp_get_request","manageengine_sdp_get_solution","manageengine_sdp_list_assets","manageengine_sdp_list_change_notes","manageengine_sdp_list_changes","manageengine_sdp_list_problem_notes","manageengine_sdp_list_problems","manageengine_sdp_list_request_notes","manageengine_sdp_list_requests","manageengine_sdp_list_solutions","manageengine_sdp_update_asset","manageengine_sdp_update_change","manageengine_sdp_update_problem","manageengine_sdp_update_request","manageengine_sdp_update_solution","mcp_list_operations","mcp_run_operation","mem0_add_memories","mem0_get_memories","mem0_search_memories","memory_add","memory_delete","memory_get","memory_get_all","microsoft_ad_add_directory_role_member","microsoft_ad_add_group_member","microsoft_ad_add_user_app_role_assignment","microsoft_ad_assign_license","microsoft_ad_create_group","microsoft_ad_create_user","microsoft_ad_delete_group","microsoft_ad_delete_user","microsoft_ad_get_conditional_access_policy","microsoft_ad_get_device","microsoft_ad_get_group","microsoft_ad_get_user","microsoft_ad_list_authentication_methods","microsoft_ad_list_conditional_access_policies","microsoft_ad_list_devices","microsoft_ad_list_directory_audits","microsoft_ad_list_directory_role_members","microsoft_ad_list_directory_roles","microsoft_ad_list_group_members","microsoft_ad_list_groups","microsoft_ad_list_service_principal_app_role_assignments","microsoft_ad_list_service_principals","microsoft_ad_list_sign_ins","microsoft_ad_list_subscribed_skus","microsoft_ad_list_user_app_role_assignments","microsoft_ad_list_user_devices","microsoft_ad_list_user_licenses","microsoft_ad_list_users","microsoft_ad_remove_directory_role_member","microsoft_ad_remove_group_member","microsoft_ad_remove_user_app_role_assignment","microsoft_ad_reset_password","microsoft_ad_revoke_sign_in_sessions","microsoft_ad_set_password","microsoft_ad_update_group","microsoft_ad_update_user","microsoft_dataverse_associate","microsoft_dataverse_create_multiple","microsoft_dataverse_create_record","microsoft_dataverse_delete_record","microsoft_dataverse_disassociate","microsoft_dataverse_download_file","microsoft_dataverse_download_file_v2","microsoft_dataverse_execute_action","microsoft_dataverse_execute_function","microsoft_dataverse_fetchxml_query","microsoft_dataverse_get_entity_metadata","microsoft_dataverse_get_record","microsoft_dataverse_list_records","microsoft_dataverse_search","microsoft_dataverse_update_multiple","microsoft_dataverse_update_record","microsoft_dataverse_upload_file","microsoft_dataverse_upsert_record","microsoft_dataverse_whoami","microsoft_dynamics_365_close_case","microsoft_dynamics_365_close_opportunity","microsoft_dynamics_365_create_record","microsoft_dynamics_365_get_record","microsoft_dynamics_365_list_records","microsoft_dynamics_365_qualify_lead","microsoft_dynamics_365_search_records","microsoft_dynamics_365_update_record","microsoft_excel_clear_range","microsoft_excel_create_table","microsoft_excel_delete_worksheet","microsoft_excel_format_range","microsoft_excel_read","microsoft_excel_read_v2","microsoft_excel_sort_range","microsoft_excel_table_add","microsoft_excel_worksheet_add","microsoft_excel_write","microsoft_excel_write_v2","microsoft_planner_create_bucket","microsoft_planner_create_plan","microsoft_planner_create_task","microsoft_planner_delete_bucket","microsoft_planner_delete_plan","microsoft_planner_delete_task","microsoft_planner_get_plan_details","microsoft_planner_get_task_details","microsoft_planner_list_buckets","microsoft_planner_list_plans","microsoft_planner_read_bucket","microsoft_planner_read_plan","microsoft_planner_read_task","microsoft_planner_update_bucket","microsoft_planner_update_plan","microsoft_planner_update_plan_details","microsoft_planner_update_task","microsoft_planner_update_task_details","microsoft_teams_delete_channel_message","microsoft_teams_delete_chat_message","microsoft_teams_get_message","microsoft_teams_list_channel_members","microsoft_teams_list_channels","microsoft_teams_list_chat_members","microsoft_teams_list_chats","microsoft_teams_list_team_members","microsoft_teams_list_teams","microsoft_teams_read_channel","microsoft_teams_read_chat","microsoft_teams_reply_to_message","microsoft_teams_set_reaction","microsoft_teams_unset_reaction","microsoft_teams_update_channel_message","microsoft_teams_update_chat_message","microsoft_teams_write_channel","microsoft_teams_write_chat","microsoft_word_append","microsoft_word_create","microsoft_word_create_from_template","microsoft_word_export_pdf","microsoft_word_list","microsoft_word_read","microsoft_word_replace_text","microsoft_word_update","millionverifier_get_credits","millionverifier_verify_email","mintlify_create_agent_job","mintlify_create_assistant_message","mintlify_detect_ai_prose","mintlify_get_agent_job","mintlify_get_assistant_caller_stats","mintlify_get_assistant_conversations","mintlify_get_feedback","mintlify_get_feedback_by_page","mintlify_get_page_content","mintlify_get_searches","mintlify_get_update_status","mintlify_get_views","mintlify_get_visitors","mintlify_search","mintlify_send_agent_message","mintlify_trigger_automation","mintlify_trigger_preview","mintlify_trigger_update","mistral_parser","mistral_parser_v2","mistral_parser_v3","modal_call_function","modal_chat_completion","modal_list_models","monday_archive_item","monday_change_column_value","monday_create_board","monday_create_column","monday_create_group","monday_create_item","monday_create_subitem","monday_create_update","monday_delete_item","monday_duplicate_item","monday_get_board","monday_get_groups","monday_get_item","monday_get_items","monday_list_boards","monday_move_item_to_group","monday_search_items","monday_update_item","mongodb_delete","mongodb_execute","mongodb_insert","mongodb_introspect","mongodb_query","mongodb_update","mssql_delete","mssql_execute","mssql_insert","mssql_introspect","mssql_query","mssql_update","mysql_delete","mysql_execute","mysql_insert","mysql_introspect","mysql_query","mysql_update","neo4j_create","neo4j_delete","neo4j_execute","neo4j_introspect","neo4j_merge","neo4j_query","neo4j_update","netsuite_attach_record","netsuite_batch_create_records","netsuite_batch_delete_records","netsuite_batch_get_records","netsuite_batch_update_records","netsuite_batch_upsert_records","netsuite_create_record","netsuite_delete_record","netsuite_detach_record","netsuite_execute_action","netsuite_execute_dataset","netsuite_execute_suiteql","netsuite_get_async_result","netsuite_get_async_status","netsuite_get_governance_limits","netsuite_get_record","netsuite_get_record_form","netsuite_get_record_metadata","netsuite_get_select_options","netsuite_get_server_time","netsuite_get_subresource","netsuite_list_datasets","netsuite_list_record_types","netsuite_list_records","netsuite_transform_record","netsuite_update_record","netsuite_upsert_record","neverbounce_get_credits","neverbounce_verify_email","new_relic_create_deployment_event","new_relic_get_entity","new_relic_nrql_query","new_relic_search_entities","notion_add_database_row","notion_add_database_row_v2","notion_append_blocks","notion_append_blocks_v2","notion_create_comment","notion_create_comment_v2","notion_create_database","notion_create_database_v2","notion_create_page","notion_create_page_v2","notion_delete_block","notion_delete_block_v2","notion_list_comments","notion_list_comments_v2","notion_list_users","notion_list_users_v2","notion_query_database","notion_query_database_v2","notion_read","notion_read_database","notion_read_database_v2","notion_read_v2","notion_retrieve_block","notion_retrieve_block_children","notion_retrieve_block_children_v2","notion_retrieve_block_v2","notion_retrieve_user","notion_retrieve_user_v2","notion_search","notion_search_v2","notion_update_block","notion_update_block_v2","notion_update_page","notion_update_page_v2","notion_write","notion_write_v2","obsidian_append_active","obsidian_append_note","obsidian_append_periodic_note","obsidian_create_note","obsidian_delete_note","obsidian_execute_command","obsidian_get_active","obsidian_get_note","obsidian_get_periodic_note","obsidian_list_commands","obsidian_list_files","obsidian_open_file","obsidian_patch_active","obsidian_patch_note","obsidian_search","okta_activate_group_rule","okta_activate_user","okta_add_user_to_group","okta_assign_group_to_app","okta_assign_user_role","okta_assign_user_to_app","okta_clear_user_sessions","okta_create_group","okta_create_group_rule","okta_create_user","okta_deactivate_group_rule","okta_deactivate_user","okta_delete_group","okta_delete_group_rule","okta_delete_user","okta_enroll_factor","okta_get_app","okta_get_factor","okta_get_group","okta_get_group_rule","okta_get_logs","okta_get_session","okta_get_user","okta_list_app_groups","okta_list_app_users","okta_list_apps","okta_list_factors","okta_list_group_members","okta_list_group_rules","okta_list_groups","okta_list_user_roles","okta_list_users","okta_remove_group_from_app","okta_remove_user_from_app","okta_remove_user_from_group","okta_remove_user_role","okta_reset_all_factors","okta_reset_factor","okta_reset_password","okta_revoke_session","okta_suspend_user","okta_unsuspend_user","okta_update_group","okta_update_user","onedrive_copy","onedrive_create_folder","onedrive_create_share_link","onedrive_delete","onedrive_download","onedrive_get_drive_info","onedrive_get_item","onedrive_list","onedrive_move","onedrive_search","onedrive_upload","onepassword_create_item","onepassword_delete_item","onepassword_get_item","onepassword_get_item_file","onepassword_get_vault","onepassword_list_items","onepassword_list_vaults","onepassword_replace_item","onepassword_resolve_secret","onepassword_update_item","openai_embeddings","openai_image","outlook_calendar_create_event","outlook_calendar_delete_event","outlook_calendar_get_event","outlook_calendar_list_events","outlook_calendar_respond","outlook_calendar_update_event","outlook_copy","outlook_create_folder","outlook_delete","outlook_draft","outlook_forward","outlook_get_attachment","outlook_list_attachments","outlook_list_folders","outlook_mark_read","outlook_mark_unread","outlook_move","outlook_read","outlook_reply","outlook_reply_all","outlook_search","outlook_send","outlook_update_message","pagerduty_add_note","pagerduty_create_incident","pagerduty_get_incident","pagerduty_get_service","pagerduty_list_escalation_policies","pagerduty_list_incident_alerts","pagerduty_list_incidents","pagerduty_list_oncalls","pagerduty_list_schedules","pagerduty_list_services","pagerduty_list_users","pagerduty_merge_incidents","pagerduty_send_event","pagerduty_snooze_incident","pagerduty_update_incident","parallel_deep_research","parallel_extract","parallel_search","pdl_autocomplete","pdl_bulk_company_enrich","pdl_bulk_person_enrich","pdl_clean_company","pdl_clean_location","pdl_clean_school","pdl_company_enrich","pdl_company_search","pdl_person_enrich","pdl_person_identify","pdl_person_search","perplexity_chat","perplexity_search","persona_approve_inquiry","persona_create_account","persona_create_inquiry","persona_create_report","persona_decline_inquiry","persona_expire_inquiry","persona_generate_inquiry_link","persona_get_account","persona_get_case","persona_get_document","persona_get_inquiry","persona_get_report","persona_get_verification","persona_import_accounts","persona_list_accounts","persona_list_cases","persona_list_inquiries","persona_list_inquiry_templates","persona_list_reports","persona_mark_inquiry_for_review","persona_print_inquiry_pdf","persona_redact_account","persona_redact_inquiry","persona_resume_inquiry","persona_update_account","persona_update_inquiry","pinecone_delete_vectors","pinecone_describe_index","pinecone_describe_index_stats","pinecone_fetch","pinecone_generate_embeddings","pinecone_list_indexes","pinecone_list_vector_ids","pinecone_search_text","pinecone_search_vector","pinecone_update_vector","pinecone_upsert_text","pipedrive_create_activity","pipedrive_create_deal","pipedrive_create_lead","pipedrive_create_project","pipedrive_delete_lead","pipedrive_get_activities","pipedrive_get_all_deals","pipedrive_get_deal","pipedrive_get_files","pipedrive_get_leads","pipedrive_get_mail_messages","pipedrive_get_mail_thread","pipedrive_get_pipeline_deals","pipedrive_get_pipelines","pipedrive_get_projects","pipedrive_update_activity","pipedrive_update_deal","pipedrive_update_lead","pitchbook_company_active_investors","pitchbook_company_bio","pitchbook_company_deal_service_providers","pitchbook_company_deals","pitchbook_company_financials","pitchbook_company_general_service_providers","pitchbook_company_industries","pitchbook_company_investors","pitchbook_company_most_recent_debt_financing","pitchbook_company_most_recent_financials","pitchbook_company_most_recent_financing","pitchbook_company_search","pitchbook_company_similar_companies","pitchbook_company_social_analytics","pitchbook_company_updates","pitchbook_company_vc_exit_predictions","pitchbook_contracts_history","pitchbook_cost_of_calls","pitchbook_credit_history","pitchbook_credit_news","pitchbook_credit_news_bulk","pitchbook_credit_news_most_recent","pitchbook_credit_news_search","pitchbook_deal_bio","pitchbook_deal_cap_table_history","pitchbook_deal_debt_lenders","pitchbook_deal_detailed","pitchbook_deal_investors","pitchbook_deal_multiples","pitchbook_deal_search","pitchbook_deal_service_providers","pitchbook_deal_stock_info","pitchbook_deal_tranche_info","pitchbook_deal_updates","pitchbook_deal_valuation","pitchbook_entity_affiliates","pitchbook_entity_locations","pitchbook_entity_news","pitchbook_entity_people","pitchbook_entity_updates","pitchbook_fund_active_investments","pitchbook_fund_benchmark","pitchbook_fund_bio","pitchbook_fund_cash_flows","pitchbook_fund_commitments","pitchbook_fund_investment_preferences","pitchbook_fund_investments","pitchbook_fund_performance","pitchbook_fund_search","pitchbook_fund_team","pitchbook_fund_updates","pitchbook_investor_active_investments","pitchbook_investor_bio","pitchbook_investor_board_seats","pitchbook_investor_deal_service_providers","pitchbook_investor_funds","pitchbook_investor_general_service_providers","pitchbook_investor_investments","pitchbook_investor_last_closed_fund","pitchbook_investor_preferences","pitchbook_investor_search","pitchbook_investor_updates","pitchbook_limited_partner_actual_allocations","pitchbook_limited_partner_bio","pitchbook_limited_partner_commitment_aggregates","pitchbook_limited_partner_commitment_preferences","pitchbook_limited_partner_commitments_detailed","pitchbook_limited_partner_search","pitchbook_limited_partner_service_providers","pitchbook_limited_partner_target_allocations","pitchbook_limited_partner_updates","pitchbook_lookup_table_structure","pitchbook_lookup_tables","pitchbook_patent_detailed","pitchbook_patent_search","pitchbook_people_search","pitchbook_person_bio","pitchbook_person_contact","pitchbook_person_education_work","pitchbook_sandbox_entities","pitchbook_search","pitchbook_service_provider_bio","pitchbook_service_provider_search","pitchbook_service_provider_updates","pitchbook_serviced_companies","pitchbook_serviced_deals","pitchbook_serviced_funds","pitchbook_serviced_investors","pitchbook_serviced_limited_partners","pitchbook_shared_search","pitchbook_usage_report","polymarket_get_activity","polymarket_get_event","polymarket_get_events","polymarket_get_holders","polymarket_get_last_trade_price","polymarket_get_leaderboard","polymarket_get_market","polymarket_get_markets","polymarket_get_midpoint","polymarket_get_orderbook","polymarket_get_positions","polymarket_get_price","polymarket_get_price_history","polymarket_get_series","polymarket_get_series_by_id","polymarket_get_spread","polymarket_get_tags","polymarket_get_tick_size","polymarket_get_trades","polymarket_search","postgresql_delete","postgresql_execute","postgresql_insert","postgresql_introspect","postgresql_query","postgresql_update","posthog_batch_events","posthog_capture_event","posthog_create_annotation","posthog_create_cohort","posthog_create_dashboard","posthog_create_experiment","posthog_create_feature_flag","posthog_create_insight","posthog_create_survey","posthog_delete_feature_flag","posthog_delete_person","posthog_delete_survey","posthog_evaluate_flags","posthog_get_cohort","posthog_get_dashboard","posthog_get_event_definition","posthog_get_experiment","posthog_get_feature_flag","posthog_get_insight","posthog_get_organization","posthog_get_person","posthog_get_project","posthog_get_property_definition","posthog_get_session_recording","posthog_get_survey","posthog_list_actions","posthog_list_annotations","posthog_list_cohorts","posthog_list_dashboards","posthog_list_event_definitions","posthog_list_experiments","posthog_list_feature_flags","posthog_list_insights","posthog_list_organizations","posthog_list_persons","posthog_list_projects","posthog_list_property_definitions","posthog_list_recording_playlists","posthog_list_session_recordings","posthog_list_surveys","posthog_query","posthog_update_cohort","posthog_update_event_definition","posthog_update_experiment","posthog_update_feature_flag","posthog_update_insight","posthog_update_property_definition","posthog_update_survey","profound_bot_logs","profound_bots_report","profound_category_assets","profound_category_personas","profound_category_prompts","profound_category_tags","profound_category_topics","profound_citation_prompts","profound_citations_report","profound_list_assets","profound_list_categories","profound_list_domains","profound_list_models","profound_list_optimizations","profound_list_personas","profound_list_regions","profound_optimization_analysis","profound_prompt_answers","profound_prompt_volume","profound_query_fanouts","profound_raw_logs","profound_referrals_report","profound_sentiment_report","profound_visibility_report","prospeo_account_information","prospeo_bulk_enrich_company","prospeo_bulk_enrich_person","prospeo_enrich_company","prospeo_enrich_person","prospeo_search_company","prospeo_search_person","prospeo_search_suggestions","pulse_parser","pulse_parser_v2","qdrant_fetch_points","qdrant_search_vector","qdrant_upsert_points","quartr_get_audio","quartr_get_company","quartr_get_event","quartr_get_event_summary","quartr_get_report","quartr_get_slide_deck","quartr_get_transcript","quartr_list_audio","quartr_list_companies","quartr_list_document_types","quartr_list_documents","quartr_list_event_types","quartr_list_events","quartr_list_live_events","quartr_list_reports","quartr_list_slide_decks","quartr_list_transcripts","quickbooks_add_attachment","quickbooks_create_bill","quickbooks_create_bill_payment","quickbooks_create_credit_memo","quickbooks_create_customer","quickbooks_create_customer_payment","quickbooks_create_deposit","quickbooks_create_employee","quickbooks_create_estimate","quickbooks_create_invoice","quickbooks_create_item","quickbooks_create_journal_entry","quickbooks_create_purchase","quickbooks_create_purchase_order","quickbooks_create_refund_receipt","quickbooks_create_sales_receipt","quickbooks_create_vendor","quickbooks_create_vendor_credit","quickbooks_download_attachment","quickbooks_download_transaction_pdf","quickbooks_email_transaction","quickbooks_get_company_info","quickbooks_read_accounting_transactions","quickbooks_read_attachments","quickbooks_read_master_data","quickbooks_read_purchasing_transactions","quickbooks_read_sales_transactions","quickbooks_run_financial_report","quickbooks_update_bill","quickbooks_update_bill_payment","quickbooks_update_credit_memo","quickbooks_update_customer","quickbooks_update_customer_payment","quickbooks_update_deposit","quickbooks_update_employee","quickbooks_update_estimate","quickbooks_update_invoice","quickbooks_update_item","quickbooks_update_journal_entry","quickbooks_update_purchase","quickbooks_update_purchase_order","quickbooks_update_refund_receipt","quickbooks_update_sales_receipt","quickbooks_update_vendor","quickbooks_update_vendor_credit","quickbooks_void_bill_payment","quickbooks_void_customer_payment","quickbooks_void_invoice","quickbooks_void_sales_receipt","quiver_image_to_svg","quiver_image_to_svg_v2","quiver_list_models","quiver_text_to_svg","quiver_text_to_svg_v2","rabbitmq_create_binding","rabbitmq_create_exchange","rabbitmq_create_policy","rabbitmq_create_queue","rabbitmq_delete_binding","rabbitmq_delete_exchange","rabbitmq_delete_policy","rabbitmq_delete_queue","rabbitmq_get_exchange","rabbitmq_get_messages","rabbitmq_get_overview","rabbitmq_get_queue","rabbitmq_health_check","rabbitmq_list_bindings","rabbitmq_list_channels","rabbitmq_list_connections","rabbitmq_list_consumers","rabbitmq_list_exchange_bindings","rabbitmq_list_exchanges","rabbitmq_list_nodes","rabbitmq_list_policies","rabbitmq_list_queues","rabbitmq_list_vhosts","rabbitmq_publish_message","rabbitmq_purge_queue","railway_create_environment","railway_create_project","railway_create_service","railway_delete_environment","railway_delete_project","railway_delete_service","railway_delete_variable","railway_deploy_service","railway_get_deployment","railway_get_deployment_logs","railway_get_project","railway_list_deployments","railway_list_project_members","railway_list_projects","railway_list_variables","railway_restart_deployment","railway_rollback_deployment","railway_transfer_project","railway_update_project","railway_upsert_variable","rb2b_credit_check","rb2b_email_to_activity","rb2b_hem_to_best_linkedin","rb2b_hem_to_business_profile","rb2b_hem_to_linkedin","rb2b_hem_to_maid","rb2b_ip_to_company","rb2b_ip_to_hem","rb2b_ip_to_maid","rb2b_linkedin_slug_search","rb2b_linkedin_to_best_personal_email","rb2b_linkedin_to_business_profile","rb2b_linkedin_to_hashed_emails","rb2b_linkedin_to_mobile_phone","rb2b_linkedin_to_personal_email","rds_delete","rds_execute","rds_insert","rds_introspect","rds_query","rds_update","reddit_delete","reddit_edit","reddit_get_comments","reddit_get_controversial","reddit_get_info","reddit_get_me","reddit_get_messages","reddit_get_posts","reddit_get_saved","reddit_get_subreddit_info","reddit_get_subreddit_rules","reddit_get_user","reddit_get_user_comments","reddit_get_user_posts","reddit_hide","reddit_hot_posts","reddit_list_my_subreddits","reddit_lock","reddit_mark_all_read","reddit_mark_read","reddit_marknsfw","reddit_mod_approve","reddit_mod_distinguish","reddit_mod_remove","reddit_mod_sticky","reddit_reply","reddit_report","reddit_save","reddit_search","reddit_search_subreddits","reddit_send_message","reddit_submit_post","reddit_subscribe","reddit_unhide","reddit_unlock","reddit_unmarknsfw","reddit_unsave","reddit_vote","redis_command","redis_delete","redis_exists","redis_expire","redis_get","redis_hdel","redis_hget","redis_hgetall","redis_hset","redis_incr","redis_incrby","redis_keys","redis_llen","redis_lpop","redis_lpush","redis_lrange","redis_persist","redis_rpop","redis_rpush","redis_set","redis_setnx","redis_ttl","reducto_parser","reducto_parser_v2","resend_cancel_email","resend_create_audience","resend_create_broadcast","resend_create_contact","resend_delete_audience","resend_delete_contact","resend_get_audience","resend_get_broadcast","resend_get_contact","resend_get_email","resend_list_audiences","resend_list_contacts","resend_list_domains","resend_send","resend_send_broadcast","resend_update_contact","revenuecat_create_purchase","revenuecat_defer_google_subscription","revenuecat_delete_customer","revenuecat_get_customer","revenuecat_grant_entitlement","revenuecat_list_offerings","revenuecat_refund_google_subscription","revenuecat_revoke_entitlement","revenuecat_revoke_google_subscription","revenuecat_update_subscriber_attributes","rippling_bulk_create_custom_object_records","rippling_bulk_delete_custom_object_records","rippling_bulk_update_custom_object_records","rippling_create_business_partner","rippling_create_business_partner_group","rippling_create_custom_app","rippling_create_custom_object","rippling_create_custom_object_field","rippling_create_custom_object_record","rippling_create_custom_page","rippling_create_custom_setting","rippling_create_department","rippling_create_draft_hires","rippling_create_object_category","rippling_create_title","rippling_create_work_location","rippling_delete_business_partner","rippling_delete_business_partner_group","rippling_delete_custom_app","rippling_delete_custom_object","rippling_delete_custom_object_field","rippling_delete_custom_object_record","rippling_delete_custom_page","rippling_delete_custom_setting","rippling_delete_object_category","rippling_delete_title","rippling_delete_work_location","rippling_get_business_partner","rippling_get_business_partner_group","rippling_get_current_user","rippling_get_custom_app","rippling_get_custom_object","rippling_get_custom_object_field","rippling_get_custom_object_record","rippling_get_custom_object_record_by_external_id","rippling_get_custom_page","rippling_get_custom_setting","rippling_get_department","rippling_get_employment_type","rippling_get_job_function","rippling_get_object_category","rippling_get_report_run","rippling_get_supergroup","rippling_get_team","rippling_get_title","rippling_get_user","rippling_get_work_location","rippling_get_worker","rippling_list_business_partner_groups","rippling_list_business_partners","rippling_list_companies","rippling_list_custom_apps","rippling_list_custom_fields","rippling_list_custom_object_fields","rippling_list_custom_object_records","rippling_list_custom_objects","rippling_list_custom_pages","rippling_list_custom_settings","rippling_list_departments","rippling_list_employment_types","rippling_list_entitlements","rippling_list_job_functions","rippling_list_object_categories","rippling_list_supergroup_exclusion_members","rippling_list_supergroup_inclusion_members","rippling_list_supergroup_members","rippling_list_supergroups","rippling_list_teams","rippling_list_titles","rippling_list_users","rippling_list_work_locations","rippling_list_workers","rippling_query_custom_object_records","rippling_trigger_report_run","rippling_update_custom_app","rippling_update_custom_object","rippling_update_custom_object_field","rippling_update_custom_object_record","rippling_update_custom_page","rippling_update_custom_setting","rippling_update_department","rippling_update_object_category","rippling_update_supergroup_exclusion_members","rippling_update_supergroup_inclusion_members","rippling_update_title","rippling_update_work_location","rocketlane_add_field_option","rocketlane_add_project_members","rocketlane_add_task_assignees","rocketlane_add_task_dependencies","rocketlane_add_task_followers","rocketlane_archive_project","rocketlane_assign_placeholders","rocketlane_create_field","rocketlane_create_phase","rocketlane_create_project","rocketlane_create_space","rocketlane_create_space_document","rocketlane_create_task","rocketlane_create_time_entry","rocketlane_create_time_off","rocketlane_delete_field","rocketlane_delete_phase","rocketlane_delete_project","rocketlane_delete_space","rocketlane_delete_space_document","rocketlane_delete_task","rocketlane_delete_time_entry","rocketlane_delete_time_off","rocketlane_get_field","rocketlane_get_invoice","rocketlane_get_invoice_line_items","rocketlane_get_invoice_payments","rocketlane_get_phase","rocketlane_get_project","rocketlane_get_space","rocketlane_get_space_document","rocketlane_get_task","rocketlane_get_time_entry","rocketlane_get_time_off","rocketlane_get_user","rocketlane_import_template","rocketlane_list_fields","rocketlane_list_invoices","rocketlane_list_phases","rocketlane_list_placeholders","rocketlane_list_projects","rocketlane_list_resource_allocations","rocketlane_list_space_documents","rocketlane_list_spaces","rocketlane_list_tasks","rocketlane_list_time_entries","rocketlane_list_time_entry_categories","rocketlane_list_time_offs","rocketlane_list_users","rocketlane_move_task_to_phase","rocketlane_remove_project_members","rocketlane_remove_task_assignees","rocketlane_remove_task_dependencies","rocketlane_remove_task_followers","rocketlane_search_time_entries","rocketlane_unassign_placeholders","rocketlane_update_field","rocketlane_update_field_option","rocketlane_update_phase","rocketlane_update_project","rocketlane_update_space","rocketlane_update_space_document","rocketlane_update_task","rocketlane_update_time_entry","rootly_acknowledge_alert","rootly_add_incident_event","rootly_add_subscribers","rootly_assign_incident_role","rootly_create_action_item","rootly_create_alert","rootly_create_incident","rootly_create_status_page_event","rootly_delete_action_item","rootly_delete_incident","rootly_escalate_alert","rootly_get_alert","rootly_get_incident","rootly_list_action_items","rootly_list_alerts","rootly_list_causes","rootly_list_environments","rootly_list_escalation_policies","rootly_list_functionalities","rootly_list_incident_events","rootly_list_incident_roles","rootly_list_incident_types","rootly_list_incidents","rootly_list_on_calls","rootly_list_playbooks","rootly_list_retrospectives","rootly_list_schedules","rootly_list_services","rootly_list_severities","rootly_list_teams","rootly_list_users","rootly_mitigate_incident","rootly_remove_subscribers","rootly_resolve_alert","rootly_resolve_incident","rootly_run_workflow","rootly_snooze_alert","rootly_unassign_incident_role","rootly_update_action_item","rootly_update_alert","rootly_update_incident","s3_copy_object","s3_create_bucket","s3_delete_bucket","s3_delete_object","s3_delete_objects","s3_get_object","s3_head_object","s3_list_buckets","s3_list_objects","s3_presigned_url","s3_put_object","sailpoint_approve_access_request","sailpoint_cancel_access_request","sailpoint_decide_certification_review_items","sailpoint_get_access_profile","sailpoint_get_access_profile_entitlements","sailpoint_get_access_request_config","sailpoint_get_access_request_status","sailpoint_get_account","sailpoint_get_account_activity","sailpoint_get_account_entitlements","sailpoint_get_account_selections","sailpoint_get_campaign","sailpoint_get_certification","sailpoint_get_entitlement","sailpoint_get_entitlement_request_config","sailpoint_get_identity","sailpoint_get_role","sailpoint_get_role_entitlements","sailpoint_get_source","sailpoint_get_task_status","sailpoint_list_access_profiles","sailpoint_list_account_activities","sailpoint_list_accounts","sailpoint_list_campaigns","sailpoint_list_certification_review_items","sailpoint_list_certifications","sailpoint_list_entitlements","sailpoint_list_identities","sailpoint_list_identity_entitlements","sailpoint_list_pending_access_request_approvals","sailpoint_list_roles","sailpoint_list_sources","sailpoint_load_accounts","sailpoint_load_entitlements","sailpoint_reject_access_request","sailpoint_request_access","sailpoint_search","sailpoint_search_aggregate","sailpoint_search_count","sailpoint_sign_off_certification","salesforce_create_account","salesforce_create_case","salesforce_create_contact","salesforce_create_custom_field","salesforce_create_custom_object","salesforce_create_lead","salesforce_create_opportunity","salesforce_create_task","salesforce_delete_account","salesforce_delete_case","salesforce_delete_contact","salesforce_delete_custom_field","salesforce_delete_lead","salesforce_delete_opportunity","salesforce_delete_task","salesforce_describe_object","salesforce_get_accounts","salesforce_get_cases","salesforce_get_contacts","salesforce_get_dashboard","salesforce_get_leads","salesforce_get_opportunities","salesforce_get_report","salesforce_get_tasks","salesforce_list_dashboards","salesforce_list_objects","salesforce_list_report_types","salesforce_list_reports","salesforce_query","salesforce_query_more","salesforce_refresh_dashboard","salesforce_run_report","salesforce_tooling_query","salesforce_update_account","salesforce_update_case","salesforce_update_contact","salesforce_update_custom_field","salesforce_update_lead","salesforce_update_opportunity","salesforce_update_task","sap_concur_approve_expense_report","sap_concur_associate_attendees","sap_concur_create_cash_advance","sap_concur_create_expected_expense","sap_concur_create_expense_report","sap_concur_create_list_item","sap_concur_create_purchase_request","sap_concur_create_quick_expense","sap_concur_create_quick_expense_with_image","sap_concur_create_report_comment","sap_concur_create_travel_request","sap_concur_create_user","sap_concur_delete_expected_expense","sap_concur_delete_expense","sap_concur_delete_expense_report","sap_concur_delete_list_item","sap_concur_delete_travel_request","sap_concur_delete_user","sap_concur_get_allocation","sap_concur_get_budget","sap_concur_get_cash_advance","sap_concur_get_expected_expense","sap_concur_get_expense","sap_concur_get_expense_report","sap_concur_get_itemizations","sap_concur_get_itinerary","sap_concur_get_list","sap_concur_get_list_item","sap_concur_get_purchase_request","sap_concur_get_receipt","sap_concur_get_receipt_status","sap_concur_get_request_cash_advance","sap_concur_get_travel_profile","sap_concur_get_travel_request","sap_concur_get_user","sap_concur_issue_cash_advance","sap_concur_list_allocations","sap_concur_list_attendee_associations","sap_concur_list_budget_categories","sap_concur_list_budgets","sap_concur_list_exceptions","sap_concur_list_expected_expenses","sap_concur_list_expense_reports","sap_concur_list_expenses","sap_concur_list_itineraries","sap_concur_list_list_items","sap_concur_list_lists","sap_concur_list_receipts","sap_concur_list_report_comments","sap_concur_list_reports_to_approve","sap_concur_list_travel_profiles_summary","sap_concur_list_travel_request_comments","sap_concur_list_travel_requests","sap_concur_list_users","sap_concur_move_travel_request","sap_concur_recall_expense_report","sap_concur_remove_all_attendees","sap_concur_search_locations","sap_concur_search_users","sap_concur_send_back_expense_report","sap_concur_submit_expense_report","sap_concur_update_allocation","sap_concur_update_expected_expense","sap_concur_update_expense","sap_concur_update_expense_report","sap_concur_update_list_item","sap_concur_update_travel_request","sap_concur_update_user","sap_concur_upload_exchange_rates","sap_concur_upload_receipt_image","sap_s4hana_create_business_partner","sap_s4hana_create_purchase_order","sap_s4hana_create_purchase_requisition","sap_s4hana_create_sales_order","sap_s4hana_delete_sales_order","sap_s4hana_get_billing_document","sap_s4hana_get_business_partner","sap_s4hana_get_customer","sap_s4hana_get_inbound_delivery","sap_s4hana_get_material_document","sap_s4hana_get_outbound_delivery","sap_s4hana_get_product","sap_s4hana_get_purchase_order","sap_s4hana_get_purchase_requisition","sap_s4hana_get_sales_order","sap_s4hana_get_supplier","sap_s4hana_get_supplier_invoice","sap_s4hana_list_billing_documents","sap_s4hana_list_business_partners","sap_s4hana_list_customers","sap_s4hana_list_inbound_deliveries","sap_s4hana_list_material_documents","sap_s4hana_list_material_stock","sap_s4hana_list_outbound_deliveries","sap_s4hana_list_products","sap_s4hana_list_purchase_orders","sap_s4hana_list_purchase_requisitions","sap_s4hana_list_sales_orders","sap_s4hana_list_supplier_invoices","sap_s4hana_list_suppliers","sap_s4hana_odata_query","sap_s4hana_update_business_partner","sap_s4hana_update_customer","sap_s4hana_update_product","sap_s4hana_update_purchase_order","sap_s4hana_update_purchase_requisition","sap_s4hana_update_sales_order","sap_s4hana_update_supplier","search_tool","secrets_manager_create_secret","secrets_manager_delete_secret","secrets_manager_describe_secret","secrets_manager_get_secret","secrets_manager_list_secrets","secrets_manager_restore_secret","secrets_manager_rotate_secret","secrets_manager_tag_resource","secrets_manager_untag_resource","secrets_manager_update_secret","semrush_backlinks","semrush_backlinks_anchors","semrush_backlinks_competitors","semrush_backlinks_geo_distribution","semrush_backlinks_indexed_pages","semrush_backlinks_overview","semrush_backlinks_tld_distribution","semrush_batch_keyword_overview","semrush_broad_match_keywords","semrush_domain_ad_copies","semrush_domain_ad_history","semrush_domain_organic_competitors","semrush_domain_organic_keywords","semrush_domain_overview","semrush_domain_overview_all","semrush_domain_overview_history","semrush_domain_paid_competitors","semrush_domain_paid_keywords","semrush_domain_pla_copies","semrush_domain_pla_keywords","semrush_domain_vs_domain","semrush_keyword_ad_history","semrush_keyword_difficulty","semrush_keyword_overview","semrush_keyword_overview_all","semrush_keyword_questions","semrush_organic_results","semrush_paid_results","semrush_referring_domains","semrush_referring_ips","semrush_related_keywords","semrush_subdomain_ad_copies","semrush_subdomain_organic_keywords","semrush_subdomain_overview","semrush_subdomain_overview_all","semrush_subdomain_overview_history","semrush_subdomain_paid_keywords","semrush_top_domains","semrush_url_organic_keywords","semrush_url_overview","semrush_url_overview_all","semrush_url_overview_history","semrush_url_paid_keywords","semrush_winners_and_losers","sendblue_evaluate_service","sendblue_get_message","sendblue_send_group_message","sendblue_send_message","sendblue_send_typing_indicator","sendgrid_add_contact","sendgrid_add_contacts_to_list","sendgrid_create_list","sendgrid_create_template","sendgrid_create_template_version","sendgrid_delete_contacts","sendgrid_delete_list","sendgrid_delete_template","sendgrid_get_contact","sendgrid_get_list","sendgrid_get_template","sendgrid_list_all_lists","sendgrid_list_templates","sendgrid_remove_contacts_from_list","sendgrid_search_contacts","sendgrid_send_mail","sentry_events_get","sentry_events_list","sentry_issues_get","sentry_issues_list","sentry_issues_update","sentry_projects_create","sentry_projects_get","sentry_projects_list","sentry_projects_update","sentry_releases_create","sentry_releases_deploy","sentry_releases_list","sentry_teams_list","serper_search","servicenow_add_incident_comment","servicenow_aggregate","servicenow_close_incident","servicenow_create_change_request","servicenow_create_incident","servicenow_create_record","servicenow_delete_record","servicenow_download_attachment","servicenow_download_attachment_v2","servicenow_find_user","servicenow_get_change_next_states","servicenow_get_change_request","servicenow_get_ci","servicenow_get_incident","servicenow_get_knowledge_article","servicenow_get_requested_item","servicenow_list_approvals","servicenow_list_attachments","servicenow_list_catalog_items","servicenow_list_change_requests","servicenow_list_change_tasks","servicenow_list_ci_relationships","servicenow_list_group_members","servicenow_list_incidents","servicenow_list_requested_items","servicenow_order_catalog_item","servicenow_read_record","servicenow_resolve_incident","servicenow_search_cis","servicenow_search_knowledge","servicenow_update_approval","servicenow_update_change_request","servicenow_update_change_state","servicenow_update_incident","servicenow_update_record","servicenow_upload_attachment","ses_create_configuration_set","ses_create_email_identity","ses_create_template","ses_delete_email_identity","ses_delete_suppressed_destination","ses_delete_template","ses_get_account","ses_get_email_identity","ses_get_suppressed_destination","ses_get_template","ses_list_identities","ses_list_suppressed_destinations","ses_list_templates","ses_put_suppressed_destination","ses_send_bulk_email","ses_send_custom_verification_email","ses_send_email","ses_send_templated_email","ses_update_template","sftp_delete","sftp_download","sftp_download_v2","sftp_list","sftp_mkdir","sftp_upload","sharepoint_add_list_items","sharepoint_create_list","sharepoint_create_page","sharepoint_delete_file","sharepoint_delete_list_item","sharepoint_delete_page","sharepoint_download_file","sharepoint_get_drive_item","sharepoint_get_list","sharepoint_get_list_item","sharepoint_list_sites","sharepoint_publish_page","sharepoint_read_page","sharepoint_update_list","sharepoint_update_page","sharepoint_upload_file","shopify_adjust_inventory","shopify_cancel_order","shopify_create_customer","shopify_create_fulfillment","shopify_create_product","shopify_delete_customer","shopify_delete_product","shopify_get_collection","shopify_get_customer","shopify_get_inventory_level","shopify_get_order","shopify_get_product","shopify_list_collections","shopify_list_customers","shopify_list_inventory_items","shopify_list_locations","shopify_list_orders","shopify_list_products","shopify_update_customer","shopify_update_order","shopify_update_product","similarweb_bounce_rate","similarweb_page_views","similarweb_pages_per_visit","similarweb_traffic_visits","similarweb_visit_duration","similarweb_website_overview","sixtyfour_enrich_company","sixtyfour_enrich_lead","sixtyfour_find_email","sixtyfour_find_phone","slack_add_reaction","slack_archive_conversation","slack_canvas","slack_create_channel_canvas","slack_create_conversation","slack_delete_canvas","slack_delete_message","slack_delete_scheduled_message","slack_download","slack_edit_canvas","slack_ephemeral_message","slack_get_canvas","slack_get_channel_history","slack_get_channel_info","slack_get_message","slack_get_permalink","slack_get_thread","slack_get_thread_replies","slack_get_user","slack_get_user_presence","slack_invite_to_conversation","slack_list_canvases","slack_list_channels","slack_list_members","slack_list_scheduled_messages","slack_list_users","slack_lookup_canvas_sections","slack_message","slack_message_reader","slack_open_view","slack_publish_view","slack_push_view","slack_remove_reaction","slack_rename_agent_session_v2","slack_rename_conversation","slack_schedule_message","slack_set_agent_session_status_v2","slack_set_conversation_purpose","slack_set_conversation_topic","slack_set_status","slack_set_suggested_prompts","slack_set_suggested_prompts_v2","slack_set_title","slack_update_message","slack_update_view","smartlead_add_email_accounts_to_campaign","smartlead_add_leads_to_campaign","smartlead_create_campaign","smartlead_create_lead_list","smartlead_delete_campaign","smartlead_delete_campaign_webhook","smartlead_delete_lead_from_campaign","smartlead_delete_lead_list","smartlead_duplicate_campaign","smartlead_export_campaign_leads","smartlead_get_campaign","smartlead_get_campaign_analytics","smartlead_get_campaign_analytics_by_date","smartlead_get_campaign_lead_statistics","smartlead_get_campaign_mailbox_statistics","smartlead_get_campaign_sequences","smartlead_get_campaign_statistics","smartlead_get_campaign_top_level_analytics_by_date","smartlead_get_campaign_webhook_summary","smartlead_get_lead_by_email","smartlead_get_lead_by_id","smartlead_get_lead_list","smartlead_get_lead_message_history","smartlead_list_campaign_email_accounts","smartlead_list_campaign_leads","smartlead_list_campaign_webhooks","smartlead_list_campaigns","smartlead_list_clients","smartlead_list_email_accounts","smartlead_list_inbox_replies","smartlead_list_lead_activities","smartlead_list_lead_categories","smartlead_list_lead_lists","smartlead_mark_lead_complete","smartlead_pause_lead","smartlead_remove_email_accounts_from_campaign","smartlead_resume_lead","smartlead_save_campaign_sequences","smartlead_unsubscribe_lead_from_campaign","smartlead_unsubscribe_lead_globally","smartlead_update_campaign_schedule","smartlead_update_campaign_settings","smartlead_update_campaign_status","smartlead_update_lead","smartlead_update_lead_category","smartlead_update_lead_list","smartlead_upsert_campaign_webhook","sms_send","smtp_send_mail","snowflake_alter_warehouse","snowflake_call_procedure","snowflake_cancel_statement","snowflake_cancel_task_run","snowflake_delete_rows","snowflake_execute_sql","snowflake_get_statement","snowflake_get_task","snowflake_get_task_run","snowflake_get_task_run_output","snowflake_get_warehouse","snowflake_insert_rows","snowflake_introspect_schema","snowflake_list_copy_history","snowflake_list_databases","snowflake_list_query_history","snowflake_list_schemas","snowflake_list_tables","snowflake_list_task_runs","snowflake_list_tasks","snowflake_list_warehouses","snowflake_load_data","snowflake_resume_task","snowflake_resume_warehouse","snowflake_run_task","snowflake_suspend_task","snowflake_suspend_warehouse","snowflake_unload_data","snowflake_update_rows","snowflake_upsert_rows","splunk_cancel_search_job","splunk_create_search_job","splunk_dispatch_saved_search","splunk_get_fired_alerts","splunk_get_saved_search","splunk_get_search_job","splunk_get_search_results","splunk_list_apps","splunk_list_fired_alerts","splunk_list_indexes","splunk_list_saved_searches","splunk_run_search","sportmonks_core_get_cities","sportmonks_core_get_city","sportmonks_core_get_continent","sportmonks_core_get_continents","sportmonks_core_get_countries","sportmonks_core_get_country","sportmonks_core_get_entity_filters","sportmonks_core_get_my_usage","sportmonks_core_get_region","sportmonks_core_get_regions","sportmonks_core_get_timezones","sportmonks_core_get_type","sportmonks_core_get_type_by_entity","sportmonks_core_get_types","sportmonks_core_search_cities","sportmonks_core_search_countries","sportmonks_core_search_regions","sportmonks_football_expected_by_player","sportmonks_football_expected_by_team","sportmonks_football_get_all_commentaries","sportmonks_football_get_all_fixtures","sportmonks_football_get_all_players","sportmonks_football_get_all_rivals","sportmonks_football_get_all_teams","sportmonks_football_get_all_transfer_rumours","sportmonks_football_get_all_transfers","sportmonks_football_get_brackets_by_season","sportmonks_football_get_coach","sportmonks_football_get_coaches","sportmonks_football_get_coaches_by_country","sportmonks_football_get_commentaries_by_fixture","sportmonks_football_get_current_leagues_by_team","sportmonks_football_get_expected_lineups_by_player","sportmonks_football_get_expected_lineups_by_team","sportmonks_football_get_extended_team_squad","sportmonks_football_get_fixture","sportmonks_football_get_fixtures_by_date","sportmonks_football_get_fixtures_by_date_range","sportmonks_football_get_fixtures_by_date_range_for_team","sportmonks_football_get_fixtures_by_ids","sportmonks_football_get_grouped_standings_by_round","sportmonks_football_get_head_to_head","sportmonks_football_get_inplay_livescores","sportmonks_football_get_latest_coaches","sportmonks_football_get_latest_fixtures","sportmonks_football_get_latest_livescores","sportmonks_football_get_latest_players","sportmonks_football_get_latest_totw","sportmonks_football_get_latest_transfers","sportmonks_football_get_league","sportmonks_football_get_leagues","sportmonks_football_get_leagues_by_country","sportmonks_football_get_leagues_by_date","sportmonks_football_get_leagues_by_team","sportmonks_football_get_live_leagues","sportmonks_football_get_live_probabilities","sportmonks_football_get_live_probabilities_by_fixture","sportmonks_football_get_live_standings_by_league","sportmonks_football_get_livescores","sportmonks_football_get_match_facts","sportmonks_football_get_match_facts_by_date_range","sportmonks_football_get_match_facts_by_fixture","sportmonks_football_get_match_facts_by_league","sportmonks_football_get_past_fixtures_by_tv_station","sportmonks_football_get_player","sportmonks_football_get_players_by_country","sportmonks_football_get_postmatch_news","sportmonks_football_get_postmatch_news_by_season","sportmonks_football_get_predictability_by_league","sportmonks_football_get_prematch_news","sportmonks_football_get_prematch_news_by_season","sportmonks_football_get_prematch_news_upcoming","sportmonks_football_get_probabilities","sportmonks_football_get_probabilities_by_fixture","sportmonks_football_get_referee","sportmonks_football_get_referees","sportmonks_football_get_referees_by_country","sportmonks_football_get_referees_by_season","sportmonks_football_get_rivals_by_team","sportmonks_football_get_round","sportmonks_football_get_round_statistics","sportmonks_football_get_rounds","sportmonks_football_get_rounds_by_season","sportmonks_football_get_schedules_by_season","sportmonks_football_get_schedules_by_season_and_team","sportmonks_football_get_schedules_by_team","sportmonks_football_get_season","sportmonks_football_get_seasons","sportmonks_football_get_seasons_by_team","sportmonks_football_get_stage","sportmonks_football_get_stage_statistics","sportmonks_football_get_stages","sportmonks_football_get_stages_by_season","sportmonks_football_get_standing_corrections_by_season","sportmonks_football_get_standings","sportmonks_football_get_standings_by_round","sportmonks_football_get_standings_by_season","sportmonks_football_get_state","sportmonks_football_get_states","sportmonks_football_get_team","sportmonks_football_get_team_rankings","sportmonks_football_get_team_rankings_by_date","sportmonks_football_get_team_rankings_by_team","sportmonks_football_get_team_squad","sportmonks_football_get_team_squad_by_season","sportmonks_football_get_teams_by_country","sportmonks_football_get_teams_by_season","sportmonks_football_get_topscorers_by_season","sportmonks_football_get_topscorers_by_stage","sportmonks_football_get_totw","sportmonks_football_get_totw_by_round","sportmonks_football_get_transfer","sportmonks_football_get_transfer_rumour","sportmonks_football_get_transfer_rumours_between_dates","sportmonks_football_get_transfer_rumours_by_player","sportmonks_football_get_transfer_rumours_by_team","sportmonks_football_get_transfers_between_dates","sportmonks_football_get_transfers_by_player","sportmonks_football_get_transfers_by_team","sportmonks_football_get_tv_station","sportmonks_football_get_tv_stations","sportmonks_football_get_tv_stations_by_fixture","sportmonks_football_get_upcoming_fixtures_by_market","sportmonks_football_get_upcoming_fixtures_by_tv_station","sportmonks_football_get_value_bets","sportmonks_football_get_value_bets_by_fixture","sportmonks_football_get_venue","sportmonks_football_get_venues","sportmonks_football_get_venues_by_season","sportmonks_football_search_coaches","sportmonks_football_search_fixtures","sportmonks_football_search_leagues","sportmonks_football_search_players","sportmonks_football_search_referees","sportmonks_football_search_rounds","sportmonks_football_search_seasons","sportmonks_football_search_stages","sportmonks_football_search_teams","sportmonks_football_search_venues","sportmonks_motorsport_get_all_fixtures","sportmonks_motorsport_get_current_leagues_by_team","sportmonks_motorsport_get_driver","sportmonks_motorsport_get_driver_standings","sportmonks_motorsport_get_driver_standings_by_season","sportmonks_motorsport_get_drivers","sportmonks_motorsport_get_drivers_by_country","sportmonks_motorsport_get_drivers_by_season","sportmonks_motorsport_get_fixture","sportmonks_motorsport_get_fixtures_by_date","sportmonks_motorsport_get_fixtures_by_date_range","sportmonks_motorsport_get_fixtures_by_ids","sportmonks_motorsport_get_laps_by_fixture","sportmonks_motorsport_get_laps_by_fixture_and_driver","sportmonks_motorsport_get_laps_by_fixture_and_lap","sportmonks_motorsport_get_latest_laps_by_fixture","sportmonks_motorsport_get_latest_pitstops_by_fixture","sportmonks_motorsport_get_latest_stints_by_fixture","sportmonks_motorsport_get_latest_updated_drivers","sportmonks_motorsport_get_latest_updated_fixtures","sportmonks_motorsport_get_league","sportmonks_motorsport_get_leagues","sportmonks_motorsport_get_leagues_by_country","sportmonks_motorsport_get_leagues_by_date","sportmonks_motorsport_get_leagues_by_live","sportmonks_motorsport_get_leagues_by_team","sportmonks_motorsport_get_livescores","sportmonks_motorsport_get_pitstops_by_fixture","sportmonks_motorsport_get_pitstops_by_fixture_and_driver","sportmonks_motorsport_get_pitstops_by_fixture_and_lap","sportmonks_motorsport_get_race_results_by_season_and_driver","sportmonks_motorsport_get_race_results_by_season_and_team","sportmonks_motorsport_get_schedules_by_season","sportmonks_motorsport_get_season","sportmonks_motorsport_get_seasons","sportmonks_motorsport_get_stage","sportmonks_motorsport_get_stages","sportmonks_motorsport_get_stages_by_season","sportmonks_motorsport_get_state","sportmonks_motorsport_get_states","sportmonks_motorsport_get_stints_by_fixture","sportmonks_motorsport_get_stints_by_fixture_and_driver","sportmonks_motorsport_get_stints_by_fixture_and_stint","sportmonks_motorsport_get_team","sportmonks_motorsport_get_team_standings","sportmonks_motorsport_get_team_standings_by_season","sportmonks_motorsport_get_teams","sportmonks_motorsport_get_teams_by_country","sportmonks_motorsport_get_teams_by_season","sportmonks_motorsport_get_venue","sportmonks_motorsport_get_venues","sportmonks_motorsport_get_venues_by_season","sportmonks_motorsport_search_drivers","sportmonks_motorsport_search_leagues","sportmonks_motorsport_search_stages","sportmonks_motorsport_search_teams","sportmonks_motorsport_search_venues","sportmonks_odds_get_all_historical_odds","sportmonks_odds_get_all_inplay_odds","sportmonks_odds_get_all_pre_match_odds","sportmonks_odds_get_all_premium_odds","sportmonks_odds_get_bookmaker","sportmonks_odds_get_bookmaker_event_ids_by_fixture","sportmonks_odds_get_bookmakers","sportmonks_odds_get_bookmakers_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture","sportmonks_odds_get_inplay_odds_by_fixture_and_bookmaker","sportmonks_odds_get_inplay_odds_by_fixture_and_market","sportmonks_odds_get_last_updated_inplay_odds","sportmonks_odds_get_last_updated_pre_match_odds","sportmonks_odds_get_market","sportmonks_odds_get_markets","sportmonks_odds_get_pre_match_odds_by_fixture","sportmonks_odds_get_pre_match_odds_by_fixture_and_bookmaker","sportmonks_odds_get_pre_match_odds_by_fixture_and_market","sportmonks_odds_get_premium_odds_by_fixture","sportmonks_odds_get_premium_odds_by_fixture_and_bookmaker","sportmonks_odds_get_premium_odds_by_fixture_and_market","sportmonks_odds_get_updated_historical_odds_between","sportmonks_odds_get_updated_premium_odds_between","sportmonks_odds_search_bookmakers","sportmonks_odds_search_markets","spotify_add_playlist_cover","spotify_add_to_queue","spotify_add_tracks_to_playlist","spotify_check_following","spotify_check_playlist_followers","spotify_check_saved_albums","spotify_check_saved_audiobooks","spotify_check_saved_episodes","spotify_check_saved_shows","spotify_check_saved_tracks","spotify_create_playlist","spotify_follow_artists","spotify_follow_playlist","spotify_get_album","spotify_get_album_tracks","spotify_get_albums","spotify_get_artist","spotify_get_artist_albums","spotify_get_artist_top_tracks","spotify_get_artists","spotify_get_audiobook","spotify_get_audiobook_chapters","spotify_get_audiobooks","spotify_get_categories","spotify_get_current_user","spotify_get_currently_playing","spotify_get_devices","spotify_get_episode","spotify_get_episodes","spotify_get_followed_artists","spotify_get_markets","spotify_get_new_releases","spotify_get_playback_state","spotify_get_playlist","spotify_get_playlist_cover","spotify_get_playlist_tracks","spotify_get_queue","spotify_get_recently_played","spotify_get_saved_albums","spotify_get_saved_audiobooks","spotify_get_saved_episodes","spotify_get_saved_shows","spotify_get_saved_tracks","spotify_get_show","spotify_get_show_episodes","spotify_get_shows","spotify_get_top_artists","spotify_get_top_tracks","spotify_get_track","spotify_get_tracks","spotify_get_user_playlists","spotify_get_user_profile","spotify_pause","spotify_play","spotify_remove_saved_albums","spotify_remove_saved_audiobooks","spotify_remove_saved_episodes","spotify_remove_saved_shows","spotify_remove_saved_tracks","spotify_remove_tracks_from_playlist","spotify_reorder_playlist_items","spotify_replace_playlist_items","spotify_save_albums","spotify_save_audiobooks","spotify_save_episodes","spotify_save_shows","spotify_save_tracks","spotify_search","spotify_seek","spotify_set_repeat","spotify_set_shuffle","spotify_set_volume","spotify_skip_next","spotify_skip_previous","spotify_transfer_playback","spotify_unfollow_artists","spotify_unfollow_playlist","spotify_update_playlist","sqs_cancel_message_move_task","sqs_change_message_visibility","sqs_change_message_visibility_batch","sqs_create_queue","sqs_delete_message","sqs_delete_message_batch","sqs_delete_queue","sqs_get_queue_attributes","sqs_get_queue_url","sqs_list_dead_letter_source_queues","sqs_list_message_move_tasks","sqs_list_queue_tags","sqs_list_queues","sqs_purge_queue","sqs_receive_message","sqs_send","sqs_send_message_batch","sqs_set_queue_attributes","sqs_start_message_move_task","sqs_tag_queue","sqs_untag_queue","square_batch_retrieve_inventory_counts","square_cancel_invoice","square_cancel_payment","square_complete_payment","square_create_catalog_image","square_create_customer","square_create_invoice","square_create_order","square_create_payment","square_delete_catalog_object","square_delete_customer","square_delete_invoice","square_get_catalog_object","square_get_customer","square_get_invoice","square_get_location","square_get_order","square_get_payment","square_get_refund","square_list_catalog","square_list_customers","square_list_invoices","square_list_locations","square_list_payments","square_list_refunds","square_pay_order","square_publish_invoice","square_refund_payment","square_search_catalog_objects","square_search_customers","square_search_invoices","square_search_orders","square_update_customer","square_upsert_catalog_object","ssh_check_command_exists","ssh_check_file_exists","ssh_create_directory","ssh_delete_file","ssh_download_file","ssh_download_file_v2","ssh_execute_command","ssh_execute_script","ssh_get_system_info","ssh_list_directory","ssh_move_rename","ssh_read_file_content","ssh_upload_file","ssh_write_file_content","ssm_cancel_command","ssm_delete_parameter","ssm_describe_automation_executions","ssm_describe_instance_information","ssm_describe_instance_patch_states","ssm_describe_instance_patches","ssm_describe_parameters","ssm_get_automation_execution","ssm_get_command_invocation","ssm_get_document","ssm_get_parameter","ssm_get_parameters","ssm_get_parameters_by_path","ssm_list_command_invocations","ssm_list_commands","ssm_list_compliance_items","ssm_list_compliance_summaries","ssm_list_documents","ssm_put_parameter","ssm_send_command","ssm_start_automation_execution","ssm_stop_automation_execution","stagehand_agent","stagehand_extract","stripe_cancel_payment_intent","stripe_cancel_subscription","stripe_capture_charge","stripe_capture_payment_intent","stripe_confirm_payment_intent","stripe_create_charge","stripe_create_customer","stripe_create_invoice","stripe_create_payment_intent","stripe_create_price","stripe_create_product","stripe_create_subscription","stripe_delete_customer","stripe_delete_invoice","stripe_delete_product","stripe_finalize_invoice","stripe_list_charges","stripe_list_customers","stripe_list_events","stripe_list_invoices","stripe_list_payment_intents","stripe_list_prices","stripe_list_products","stripe_list_subscriptions","stripe_pay_invoice","stripe_resume_subscription","stripe_retrieve_charge","stripe_retrieve_customer","stripe_retrieve_event","stripe_retrieve_invoice","stripe_retrieve_payment_intent","stripe_retrieve_price","stripe_retrieve_product","stripe_retrieve_subscription","stripe_search_charges","stripe_search_customers","stripe_search_invoices","stripe_search_payment_intents","stripe_search_prices","stripe_search_products","stripe_search_subscriptions","stripe_send_invoice","stripe_update_charge","stripe_update_customer","stripe_update_invoice","stripe_update_payment_intent","stripe_update_price","stripe_update_product","stripe_update_subscription","stripe_void_invoice","sts_assume_role","sts_assume_role_with_saml","sts_assume_role_with_web_identity","sts_get_access_key_info","sts_get_caller_identity","sts_get_session_token","stt_assemblyai","stt_assemblyai_v2","stt_deepgram","stt_deepgram_v2","stt_elevenlabs","stt_elevenlabs_v2","stt_gemini","stt_gemini_v2","stt_whisper","stt_whisper_v2","supabase_count","supabase_delete","supabase_get_row","supabase_insert","supabase_introspect","supabase_invoke_function","supabase_query","supabase_rpc","supabase_storage_copy","supabase_storage_create_bucket","supabase_storage_create_signed_upload_url","supabase_storage_create_signed_url","supabase_storage_delete","supabase_storage_delete_bucket","supabase_storage_download","supabase_storage_empty_bucket","supabase_storage_get_public_url","supabase_storage_list","supabase_storage_list_buckets","supabase_storage_move","supabase_storage_update_bucket","supabase_storage_upload","supabase_text_search","supabase_update","supabase_upsert","supabase_vector_search","table_batch_insert_rows","table_create","table_delete_row","table_delete_rows_by_filter","table_get_row","table_get_schema","table_insert_row","table_list","table_query_rows","table_query_rows_v2","table_update_row","table_update_rows_by_filter","table_upsert_row","tailscale_authorize_device","tailscale_create_auth_key","tailscale_delete_auth_key","tailscale_delete_device","tailscale_delete_user","tailscale_expire_device_key","tailscale_get_acl","tailscale_get_auth_key","tailscale_get_device","tailscale_get_device_routes","tailscale_get_dns_preferences","tailscale_get_dns_searchpaths","tailscale_list_auth_keys","tailscale_list_devices","tailscale_list_dns_nameservers","tailscale_list_users","tailscale_set_acl","tailscale_set_device_routes","tailscale_set_device_tags","tailscale_set_dns_nameservers","tailscale_set_dns_preferences","tailscale_set_dns_searchpaths","tailscale_suspend_user","tailscale_update_device_key","tavily_crawl","tavily_extract","tavily_map","tavily_search","telegram_copy_message","telegram_delete_message","telegram_edit_message_text","telegram_forward_message","telegram_get_chat","telegram_get_chat_member","telegram_message","telegram_pin_message","telegram_send_animation","telegram_send_audio","telegram_send_chat_action","telegram_send_contact","telegram_send_document","telegram_send_location","telegram_send_photo","telegram_send_poll","telegram_send_video","telegram_set_message_reaction","telegram_unpin_message","temporal_cancel_workflow","temporal_count_workflows","temporal_create_schedule","temporal_delete_schedule","temporal_describe_schedule","temporal_describe_task_queue","temporal_describe_workflow","temporal_get_workflow_history","temporal_list_schedules","temporal_list_workflows","temporal_pause_schedule","temporal_query_workflow","temporal_reset_workflow","temporal_signal_with_start","temporal_signal_workflow","temporal_start_workflow","temporal_terminate_workflow","temporal_trigger_schedule","temporal_unpause_schedule","temporal_update_workflow","textract_analyze_expense","textract_analyze_id","textract_parser","textract_parser_v2","thinking_tool","thrive_add_audience_managers","thrive_add_audience_members","thrive_add_user_tags","thrive_create_assignment","thrive_create_audience","thrive_create_completion","thrive_create_user","thrive_delete_assignment","thrive_delete_audience","thrive_delete_user","thrive_get_activity","thrive_get_assignment","thrive_get_audience","thrive_get_completion","thrive_get_content","thrive_get_cpd_category","thrive_get_cpd_entry","thrive_get_cpd_requirement","thrive_get_enrolment","thrive_get_skill_levels","thrive_get_tag","thrive_get_user_by_id","thrive_get_user_by_ref","thrive_list_assignments","thrive_list_audience_managers","thrive_list_audience_members","thrive_list_audiences","thrive_list_completions","thrive_list_enrolments","thrive_list_tags","thrive_query_activities","thrive_query_content","thrive_query_cpd_categories","thrive_query_cpd_entries","thrive_query_cpd_requirements","thrive_query_cpd_user_summaries","thrive_remove_audience_manager","thrive_remove_audience_member","thrive_remove_user_tags","thrive_replace_audience_managers","thrive_replace_audience_members","thrive_search_users","thrive_suspend_user","thrive_update_assignment","thrive_update_audience","thrive_update_user","thrive_update_user_skills","tiktok_get_post_status","tiktok_get_user","tiktok_list_videos","tiktok_query_videos","tiktok_upload_video_draft","tinybird_append_datasource","tinybird_delete_datasource_rows","tinybird_events","tinybird_get_job","tinybird_query","tinybird_query_pipe","tinybird_truncate_datasource","tinyfish_cancel_run","tinyfish_fetch","tinyfish_get_run","tinyfish_list_profiles","tinyfish_list_runs","tinyfish_list_vault_items","tinyfish_run","tinyfish_run_async","tinyfish_search","trello_add_checklist","trello_add_checklist_item","trello_add_comment","trello_add_label","trello_add_member","trello_create_board","trello_create_card","trello_create_list","trello_delete_card","trello_get_actions","trello_get_board","trello_get_card","trello_list_cards","trello_list_lists","trello_list_members","trello_remove_label","trello_remove_member","trello_search","trello_update_card","trello_update_checklist_item","trello_update_list","trigger_dev_activate_schedule","trigger_dev_add_run_tags","trigger_dev_batch_trigger_task","trigger_dev_cancel_run","trigger_dev_complete_waitpoint_token","trigger_dev_create_env_var","trigger_dev_create_schedule","trigger_dev_create_waitpoint_token","trigger_dev_deactivate_schedule","trigger_dev_delete_env_var","trigger_dev_delete_schedule","trigger_dev_execute_query","trigger_dev_get_batch","trigger_dev_get_batch_results","trigger_dev_get_deployment","trigger_dev_get_env_var","trigger_dev_get_latest_deployment","trigger_dev_get_query_schema","trigger_dev_get_queue","trigger_dev_get_run","trigger_dev_get_run_events","trigger_dev_get_run_result","trigger_dev_get_run_trace","trigger_dev_get_schedule","trigger_dev_get_waitpoint_token","trigger_dev_import_env_vars","trigger_dev_list_deployments","trigger_dev_list_env_vars","trigger_dev_list_queues","trigger_dev_list_runs","trigger_dev_list_schedules","trigger_dev_list_timezones","trigger_dev_list_waitpoint_tokens","trigger_dev_override_queue_concurrency","trigger_dev_pause_queue","trigger_dev_promote_deployment","trigger_dev_replay_run","trigger_dev_reschedule_run","trigger_dev_reset_queue_concurrency","trigger_dev_resume_queue","trigger_dev_trigger_task","trigger_dev_update_env_var","trigger_dev_update_run_metadata","trigger_dev_update_schedule","tts_azure","tts_cartesia","tts_deepgram","tts_elevenlabs","tts_google","tts_openai","tts_playht","twilio_send_sms","twilio_voice_get_recording","twilio_voice_list_calls","twilio_voice_make_call","typeform_create_form","typeform_delete_form","typeform_files","typeform_get_form","typeform_insights","typeform_list_forms","typeform_responses","typeform_update_form","upstash_redis_command","upstash_redis_delete","upstash_redis_exists","upstash_redis_expire","upstash_redis_get","upstash_redis_hget","upstash_redis_hgetall","upstash_redis_hset","upstash_redis_incr","upstash_redis_incrby","upstash_redis_keys","upstash_redis_lpush","upstash_redis_lrange","upstash_redis_set","upstash_redis_setnx","upstash_redis_ttl","uptimerobot_create_alert_contact","uptimerobot_create_maintenance_window","uptimerobot_create_monitor","uptimerobot_create_psp","uptimerobot_delete_alert_contact","uptimerobot_delete_maintenance_window","uptimerobot_delete_monitor","uptimerobot_delete_psp","uptimerobot_get_account","uptimerobot_get_alert_contact","uptimerobot_get_incident","uptimerobot_get_maintenance_window","uptimerobot_get_monitor","uptimerobot_get_psp","uptimerobot_list_alert_contacts","uptimerobot_list_incidents","uptimerobot_list_maintenance_windows","uptimerobot_list_monitors","uptimerobot_list_psps","uptimerobot_pause_monitor","uptimerobot_start_monitor","uptimerobot_update_maintenance_window","uptimerobot_update_monitor","uptimerobot_update_psp","vanta_download_document_file","vanta_get_control","vanta_get_document","vanta_get_framework","vanta_get_person","vanta_get_policy","vanta_get_risk_scenario","vanta_get_test","vanta_get_vendor","vanta_get_vulnerable_asset","vanta_list_control_documents","vanta_list_control_tests","vanta_list_controls","vanta_list_document_uploads","vanta_list_documents","vanta_list_framework_controls","vanta_list_frameworks","vanta_list_monitored_computers","vanta_list_people","vanta_list_policies","vanta_list_risk_scenarios","vanta_list_test_entities","vanta_list_tests","vanta_list_vendors","vanta_list_vulnerabilities","vanta_list_vulnerability_remediations","vanta_list_vulnerable_assets","vanta_submit_document","vanta_upload_document_file","vercel_add_domain","vercel_add_project_domain","vercel_cancel_deployment","vercel_create_alias","vercel_create_check","vercel_create_deployment","vercel_create_dns_record","vercel_create_edge_config","vercel_create_env_var","vercel_create_project","vercel_create_webhook","vercel_delete_alias","vercel_delete_deployment","vercel_delete_dns_record","vercel_delete_domain","vercel_delete_edge_config","vercel_delete_env_var","vercel_delete_project","vercel_delete_webhook","vercel_get_alias","vercel_get_check","vercel_get_deployment","vercel_get_deployment_events","vercel_get_domain","vercel_get_domain_config","vercel_get_edge_config","vercel_get_edge_config_items","vercel_get_env_vars","vercel_get_project","vercel_get_team","vercel_get_user","vercel_get_webhook","vercel_list_aliases","vercel_list_checks","vercel_list_deployment_files","vercel_list_deployments","vercel_list_dns_records","vercel_list_domains","vercel_list_edge_configs","vercel_list_project_domains","vercel_list_projects","vercel_list_team_members","vercel_list_teams","vercel_list_webhooks","vercel_pause_project","vercel_promote_deployment","vercel_remove_project_domain","vercel_rerequest_check","vercel_unpause_project","vercel_update_check","vercel_update_dns_record","vercel_update_edge_config_items","vercel_update_env_var","vercel_update_project","vercel_update_project_domain","vercel_verify_project_domain","video_falai","video_luma","video_minimax","video_runway","video_veo","vision_tool","vision_tool_v2","wealthbox_read_contact","wealthbox_read_note","wealthbox_read_task","wealthbox_write_contact","wealthbox_write_note","wealthbox_write_task","webflow_create_item","webflow_delete_item","webflow_get_item","webflow_list_items","webflow_update_item","webhook_request","whatsapp_get_media","whatsapp_mark_read","whatsapp_send_interactive","whatsapp_send_media","whatsapp_send_message","whatsapp_send_reaction","whatsapp_send_template","whatsapp_upload_media","wikipedia_content","wikipedia_random","wikipedia_search","wikipedia_summary","windchill_check_in_document","windchill_check_in_documents","windchill_check_out_document","windchill_check_out_documents","windchill_create_document","windchill_create_documents","windchill_delete_document","windchill_delete_documents","windchill_download_attachment","windchill_download_primary_content","windchill_get_document","windchill_get_document_structure","windchill_get_primary_content","windchill_get_valid_state_transitions","windchill_list_attachments","windchill_list_documents","windchill_revise_document","windchill_revise_documents","windchill_set_lifecycle_state","windchill_undo_check_out_document","windchill_undo_check_out_documents","windchill_update_common_properties","windchill_update_document","windchill_update_document_security_labels","windchill_update_documents","windchill_upload_attachments","windchill_upload_primary_content","wiza_company_enrichment","wiza_get_credits","wiza_individual_reveal","wiza_prospect_search","wordpress_create_category","wordpress_create_comment","wordpress_create_page","wordpress_create_post","wordpress_create_tag","wordpress_delete_category","wordpress_delete_comment","wordpress_delete_media","wordpress_delete_page","wordpress_delete_post","wordpress_delete_tag","wordpress_get_category","wordpress_get_current_user","wordpress_get_media","wordpress_get_page","wordpress_get_post","wordpress_get_tag","wordpress_get_user","wordpress_list_categories","wordpress_list_comments","wordpress_list_media","wordpress_list_pages","wordpress_list_posts","wordpress_list_tags","wordpress_list_users","wordpress_search_content","wordpress_update_category","wordpress_update_comment","wordpress_update_page","wordpress_update_post","wordpress_update_tag","wordpress_upload_media","workday_assign_onboarding","workday_change_job","workday_create_prehire","workday_get_compensation","workday_get_organizations","workday_get_worker","workday_hire_employee","workday_list_workers","workday_terminate_worker","workday_update_worker","workflow_executor","x_create_bookmark","x_create_tweet","x_delete_bookmark","x_delete_tweet","x_get_blocking","x_get_bookmarks","x_get_followers","x_get_following","x_get_liked_tweets","x_get_liking_users","x_get_me","x_get_personalized_trends","x_get_quote_tweets","x_get_retweeted_by","x_get_trends_by_woeid","x_get_tweets_by_ids","x_get_usage","x_get_user_mentions","x_get_user_timeline","x_get_user_tweets","x_hide_reply","x_manage_block","x_manage_follow","x_manage_like","x_manage_mute","x_manage_retweet","x_read","x_search","x_search_tweets","x_search_users","x_user","x_write","youtube_channel_info","youtube_channel_playlists","youtube_channel_videos","youtube_comments","youtube_playlist_items","youtube_search","youtube_trending","youtube_video_categories","youtube_video_details","zendesk_autocomplete_organizations","zendesk_create_organization","zendesk_create_organizations_bulk","zendesk_create_ticket","zendesk_create_tickets_bulk","zendesk_create_user","zendesk_create_users_bulk","zendesk_delete_organization","zendesk_delete_ticket","zendesk_delete_user","zendesk_get_current_user","zendesk_get_organization","zendesk_get_organizations","zendesk_get_ticket","zendesk_get_tickets","zendesk_get_user","zendesk_get_users","zendesk_merge_tickets","zendesk_search","zendesk_search_count","zendesk_search_users","zendesk_update_organization","zendesk_update_ticket","zendesk_update_tickets_bulk","zendesk_update_user","zendesk_update_users_bulk","zep_add_messages","zep_add_user","zep_create_thread","zep_delete_thread","zep_get_context","zep_get_messages","zep_get_threads","zep_get_user","zep_get_user_threads","zerobounce_get_credits","zerobounce_verify_email","zoho_desk_add_comment","zoho_desk_get_attachment","zoho_desk_get_contact","zoho_desk_get_thread","zoho_desk_get_ticket","zoho_desk_list_comments","zoho_desk_list_organizations","zoho_desk_list_threads","zoho_desk_list_tickets","zoho_desk_update_ticket","zoom_create_meeting","zoom_delete_meeting","zoom_delete_recording","zoom_get_meeting","zoom_get_meeting_invitation","zoom_get_meeting_recordings","zoom_list_meetings","zoom_list_past_participants","zoom_list_recordings","zoom_update_meeting","zoominfo_enrich_companies","zoominfo_enrich_contacts","zoominfo_search_companies","zoominfo_search_contacts","zoominfo_search_intent","zoominfo_search_news"]' ) export default toolIds diff --git a/apps/sim/tools/generated/tool-metadata.ts b/apps/sim/tools/generated/tool-metadata.ts index 275e8594c99..dacfa485f2a 100644 --- a/apps/sim/tools/generated/tool-metadata.ts +++ b/apps/sim/tools/generated/tool-metadata.ts @@ -3,7 +3,7 @@ /** Serializable metadata for every built-in tool, keyed by tool id. */ const toolMetadata: Record = JSON.parse( - '{"a2a_cancel_task":{"id":"a2a_cancel_task","name":"A2A Cancel Task","description":"Request cancellation of an in-progress A2A task.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The task ID to cancel"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_get_agent_card":{"id":"a2a_get_agent_card","name":"A2A Get Agent Card","description":"Fetch the Agent Card (discovery document) for an external A2A agent.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_get_task":{"id":"a2a_get_task","name":"A2A Get Task","description":"Retrieve the current state and result of an A2A task.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The task ID to retrieve"},"historyLength":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of history messages to include"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"a2a_send_message":{"id":"a2a_send_message","name":"A2A Send Message","description":"Send a message to an external A2A agent and return its response.","version":"1.0.0","params":{"agentUrl":{"type":"string","required":true,"visibility":"user-only","description":"The A2A agent endpoint URL"},"message":{"type":"string","required":true,"visibility":"user-or-llm","description":"The message text to send"},"data":{"type":"json","required":false,"visibility":"user-or-llm","description":"Optional structured JSON data to attach"},"files":{"type":"json","required":false,"visibility":"user-or-llm","description":"Optional files to attach"},"taskId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Existing task ID to continue"},"contextId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Conversation context ID to continue"},"apiKey":{"type":"string","required":false,"visibility":"user-only","description":"API key for authentication (if required)"}},"hostedApiKey":"none"},"affinity_batch_update_entity_fields":{"id":"affinity_batch_update_entity_fields","name":"Affinity Batch Update Entity Fields","description":"Write up to 100 non-list field values on one company or person in a single request.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to write the fields on: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"updates":{"type":"json","required":true,"visibility":"user-or-llm","description":"Up to 100 field updates as [{\\"id\\":\\"\\",\\"value\\":{\\"type\\":\\"…\\",\\"data\\":…}}], using the same value shapes as a single field update"}},"hostedApiKey":"none"},"affinity_batch_update_list_entry_fields":{"id":"affinity_batch_update_list_entry_fields","name":"Affinity Batch Update List Entry Fields","description":"Write up to 100 field values on one list row in a single request. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"updates":{"type":"json","required":true,"visibility":"user-or-llm","description":"Up to 100 field updates as [{\\"id\\":\\"\\",\\"value\\":{\\"type\\":\\"…\\",\\"data\\":…}}], using the same value shapes as a single field update"}},"hostedApiKey":"none"},"affinity_create_list":{"id":"affinity_create_list","name":"Affinity Create List","description":"Create a list. Its type fixes which entities it can hold, and the API key holder becomes its creator and owner.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the new list"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Entity kind the list holds: company, opportunity, or person"},"isPublic":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether everyone in the organization can see the list"}},"hostedApiKey":"none"},"affinity_create_list_field_dropdown_option":{"id":"affinity_create_list_field_dropdown_option","name":"Affinity Create List Field Dropdown Option","description":"Add a selectable option to a dropdown field on a list. A ranked or status option also needs a rank and a color.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Kind of option to create, matching the field. dropdown takes only a label; ranked-dropdown also requires rank and color; status-dropdown additionally requires a status category. Sending a field the kind does not accept is rejected"},"text":{"type":"string","required":true,"visibility":"user-or-llm","description":"The option label"},"rank":{"type":"number","required":false,"visibility":"user-or-llm","description":"Sort order. Required on a ranked-dropdown or status-dropdown option"},"color":{"type":"string","required":false,"visibility":"user-or-llm","description":"Option color: white, gray, blue, green, purple, orange, or red. Required on a ranked-dropdown or status-dropdown option"},"statusCategory":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pipeline meaning of the option: open, won, lost, or on-hold. Status-dropdown options only"},"winRate":{"type":"number","required":false,"visibility":"user-or-llm","description":"Expected win rate of the status. Status-dropdown options only"}},"hostedApiKey":"none"},"affinity_create_merge":{"id":"affinity_create_merge","name":"Affinity Create Merge","description":"Fold a duplicate company or person into the record you are keeping. The merge runs asynchronously — poll the returned task to see it finish. Requires the \\"Manage duplicates\\" permission and an admin role.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to merge: companies or persons"},"primaryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to keep"},"duplicateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the duplicate record to fold in"}},"hostedApiKey":"none"},"affinity_create_note":{"id":"affinity_create_note","name":"Affinity Create Note","description":"Write a note — attached to companies, persons, and opportunities, anchored to a meeting, call, or chat message, or posted as a reply to an existing note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Note shape: entities to attach it to records, interaction to anchor it to a meeting, call, or chat message, or user-reply to reply to a note"},"html":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note body as HTML"},"companyIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Companies to attach the note to, e.g. [1, 2]. Not used on a reply"},"personIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Persons to attach the note to, e.g. [1, 2]. Not used on a reply"},"opportunityIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Opportunities to attach the note to, e.g. [1, 2]. Not used on a reply"},"interactionId":{"type":"string","required":false,"visibility":"user-or-llm","description":"The interaction to anchor the note to. Required for an interaction note"},"interactionType":{"type":"string","required":false,"visibility":"user-or-llm","description":"Kind of the anchoring interaction: meeting, call, or chat-message. Required for an interaction note"},"parentId":{"type":"string","required":false,"visibility":"user-or-llm","description":"The note being replied to. Required for a user-reply note"},"creatorId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Attribute the note to another internal person. Defaults to the API key holder"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"Backdate the note to this ISO 8601 timestamp"}},"hostedApiKey":"none"},"affinity_create_reminder":{"id":"affinity_create_reminder","name":"Affinity Create Reminder","description":"Create a reminder on one company, person, or opportunity. A recurring reminder resets whenever the chosen signal happens instead of firing once.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"one-time to fire once, or recurring to reset on a signal"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"What the reminder is about: company, person, or opportunity"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company, person, or opportunity"},"dueDate":{"type":"string","required":false,"visibility":"user-or-llm","description":"When the reminder is due, as an ISO 8601 timestamp. Required for a one-time reminder; on a recurring one Affinity computes it from the period when omitted"},"content":{"type":"string","required":false,"visibility":"user-or-llm","description":"What the reminder says"},"ownerId":{"type":"string","required":true,"visibility":"user-or-llm","description":"User the reminder is assigned to. Must be an internal user. The API key holder is recorded as the creator, which is a separate field"},"resetTrigger":{"type":"string","required":false,"visibility":"user-or-llm","description":"What restarts a recurring reminder: interaction, email, or event. Required when the type is recurring"},"periodDays":{"type":"number","required":false,"visibility":"user-or-llm","description":"Days between firings of a recurring reminder. Required when the type is recurring"}},"hostedApiKey":"none"},"affinity_delete_list_field_dropdown_option":{"id":"affinity_delete_list_field_dropdown_option","name":"Affinity Delete List Field Dropdown Option","description":"Permanently delete a dropdown option on a list field. Every list entry currently set to it is cleared, and those values cannot be recovered.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID to delete"}},"hostedApiKey":"none"},"affinity_delete_note":{"id":"affinity_delete_note","name":"Affinity Delete Note","description":"Delete a note you created. Deleting a root note also deletes its replies; deleting a reply removes only that reply.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID to delete"}},"hostedApiKey":"none"},"affinity_get_company":{"id":"affinity_get_company","name":"Affinity Get Company","description":"Look up one company by ID. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"companyId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The company ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_current_user":{"id":"affinity_get_current_user","name":"Affinity Get Current User","description":"Verify an Affinity API key and return the tenant, the user behind the key, and the scopes the grant carries.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"}},"hostedApiKey":"none"},"affinity_get_entity_field_value":{"id":"affinity_get_entity_field_value","name":"Affinity Get Entity Field Value","description":"Read one non-list field value from a company or person.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to read the field from: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to read"}},"hostedApiKey":"none"},"affinity_get_list":{"id":"affinity_get_list","name":"Affinity Get List","description":"Read one list — its name, type, owner, and privacy setting.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"}},"hostedApiKey":"none"},"affinity_get_list_entry":{"id":"affinity_get_list_entry","name":"Affinity Get List Entry","description":"Read one row of a list with its entity. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_list_entry_field":{"id":"affinity_get_list_entry_field","name":"Affinity Get List Entry Field","description":"Read one field value on a list row.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to read"}},"hostedApiKey":"none"},"affinity_get_list_field_dropdown_option":{"id":"affinity_get_list_field_dropdown_option","name":"Affinity Get List Field Dropdown Option","description":"Read one dropdown option on a list field.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID"}},"hostedApiKey":"none"},"affinity_get_merge":{"id":"affinity_get_merge","name":"Affinity Get Merge","description":"Read the status of one company or person merge, including why it failed if it did.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge to read: companies or persons"},"mergeId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The merge ID"}},"hostedApiKey":"none"},"affinity_get_merge_task":{"id":"affinity_get_merge_task","name":"Affinity Get Merge Task","description":"Read one merge task and how its merges are progressing. Poll this after starting a merge.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge task to read: companies or persons"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The merge task ID"}},"hostedApiKey":"none"},"affinity_get_note":{"id":"affinity_get_note","name":"Affinity Get Note","description":"Read one note with its body, author, mentions, and attached records.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return, e.g. [\\"repliesCount\\",\\"personsPreview\\",\\"companiesPreview\\",\\"opportunitiesPreview\\"]. Those four fields are omitted unless requested here"}},"hostedApiKey":"none"},"affinity_get_opportunity":{"id":"affinity_get_opportunity","name":"Affinity Get Opportunity","description":"Read one opportunity and the list it belongs to. Its field data lives on the list entry.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"opportunityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The opportunity ID"}},"hostedApiKey":"none"},"affinity_get_person":{"id":"affinity_get_person","name":"Affinity Get Person","description":"Look up one person by ID. Field data is returned only for the Field IDs or Field Types asked for.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"personId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The person ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"}},"hostedApiKey":"none"},"affinity_get_saved_view":{"id":"affinity_get_saved_view","name":"Affinity Get Saved View","description":"Read one saved view — its name, kind, and creation date.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"viewId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The saved view ID"}},"hostedApiKey":"none"},"affinity_get_transcript":{"id":"affinity_get_transcript","name":"Affinity Get Transcript","description":"Read one transcript with its first 100 fragments. Page the fragments endpoint for a longer meeting.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"transcriptId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The transcript ID"}},"hostedApiKey":"none"},"affinity_get_user":{"id":"affinity_get_user","name":"Affinity Get User","description":"Read one internal user. A user and their person record share the same numeric ID, so a person ID works here.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"userId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The user ID, which is also their person ID"}},"hostedApiKey":"none"},"affinity_list_calls":{"id":"affinity_list_calls","name":"Affinity List Calls","description":"Page through logged calls and their participants. Only calls the API key holder can see are returned.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_chat_messages":{"id":"affinity_list_chat_messages","name":"Affinity List Chat Messages","description":"Page through logged chat messages and their participants. Only messages the API key holder can see are returned.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_companies":{"id":"affinity_list_companies","name":"Affinity List Companies","description":"Page through companies. Companies come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these company IDs, e.g. [1, 2, 3]"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_coworker_connections":{"id":"affinity_list_coworker_connections","name":"Affinity List Coworker Connections","description":"Find warm paths into a company through shared work history: who in your Affinity data once worked alongside the people you want to reach. Grouped by target, strongest first.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":true,"visibility":"user-or-llm","description":"Required scope. The only supported filter is target.currentCompany.id, e.g. \\"target.currentCompany.id=123\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of targets to return per page, 1-50. Defaults to 20"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_emails":{"id":"affinity_list_emails","name":"Affinity List Emails","description":"Page through email metadata — subject, participants, and timestamps. Affinity never exposes email bodies through the API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_field_values":{"id":"affinity_list_entity_field_values","name":"Affinity List Entity Field Values","description":"Page through a company\'s or person\'s non-list field values. List fields are not returned here — read those through the list entry.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to read field values from: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field IDs. Mutually exclusive with Field Types"},"types":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field categories: enriched, global, relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_list_entity_list_entries":{"id":"affinity_list_entity_list_entries","name":"Affinity List Entity List Entries","description":"Page through a company\'s or person\'s rows across every list, each carrying that list\'s field values and when the entity was added.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up the rows of: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_lists":{"id":"affinity_list_entity_lists","name":"Affinity List Entity Lists","description":"List every list a company or person appears on that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up the lists of: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_entity_notes":{"id":"affinity_list_entity_notes","name":"Affinity List Entity Notes","description":"List the notes relevant to one company, person, or opportunity — directly attached notes plus notes reaching it through its people and meetings.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity the notes hang off: companies, persons, or opportunities"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company, person, or opportunity"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_entity_relationships":{"id":"affinity_list_entity_relationships","name":"Affinity List Entity Relationships","description":"List who knows a company or person, scored 0.0 to 1.0 by how much the two actually interact. Strongest first by default.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to look up relationships for: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on interactionScore only, e.g. \\"interactionScore>=0.5\\""},"orderBy":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order: [\\"interactionScore\\"] for weakest first, [\\"-interactionScore\\"] for strongest first (the default)"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_field_dropdown_options":{"id":"affinity_list_field_dropdown_options","name":"Affinity List Field Dropdown Options","description":"List the selectable options on a dropdown or ranked-dropdown company or person field. Writing such a field needs the option ID, not its text.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which field family the field belongs to: companies or persons"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown or ranked-dropdown field ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_field_metadata":{"id":"affinity_list_field_metadata","name":"Affinity List Field Metadata","description":"List the non-list company or person fields, with the value type, filter operators, and sort support of each. Start here to find the Field IDs the read and write tools take.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which fields to describe: companies or persons"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return: [\\"filterability\\",\\"sortability\\"]. Both are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on name only, e.g. \\"name=~Status\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_field_value_changes":{"id":"affinity_list_field_value_changes","name":"Affinity List Field Value Changes","description":"Page through field value changes across the whole workspace. Built for delta sync: follow nextCursor to the end of a run, then resume from the last cursor next time.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over field.id, listEntry.id, changer.id, changedAt, or actionType. Resume a sync with e.g. \\"changedAt>2026-06-01T12:00:00Z\\""},"orderBy":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order: [\\"changedAt\\"] for oldest first (the default), [\\"-changedAt\\"] for newest first"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_investor_executive_connections":{"id":"affinity_list_investor_executive_connections","name":"Affinity List Investor Executive Connections","description":"Find warm paths into a company through investment history: which investors in your Affinity data backed a company the people you want to reach once led. Grouped by target, strongest first.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":true,"visibility":"user-or-llm","description":"Required scope. The only supported filter is target.currentCompany.id, e.g. \\"target.currentCompany.id=123\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of targets to return per page, 1-50. Defaults to 20"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_list_entries":{"id":"affinity_list_list_entries","name":"Affinity List List Entries","description":"Page through the rows of a list. Rows come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_entry_field_value_changes":{"id":"affinity_list_list_entry_field_value_changes","name":"Affinity List List Entry Field Value Changes","description":"Page through the history of one list row — who changed which field, when, and to what.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over field.id, changer.id, changedAt, or actionType, e.g. \\"field.id=field-1234\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_entry_fields":{"id":"affinity_list_list_entry_fields","name":"Affinity List List Entry Fields","description":"Page through every field value on one list row, including the list-specific columns. All fields are returned unless narrowed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field IDs. Mutually exclusive with Field Types"},"types":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict to these field categories: enriched, global, list, relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_list_list_field_dropdown_options":{"id":"affinity_list_list_field_dropdown_options","name":"Affinity List List Field Dropdown Options","description":"List the selectable options on a dropdown, ranked-dropdown, or status-dropdown field of a list.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_list_fields":{"id":"affinity_list_list_fields","name":"Affinity List List Fields","description":"List the fields available on one list, including its list-specific columns. Use these Field IDs when reading or writing list entries.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return: [\\"filterability\\",\\"sortability\\"]. Both are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on name only, e.g. \\"name=~Stage\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_lists":{"id":"affinity_list_lists","name":"Affinity List Lists","description":"Page through the lists in the organization that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"term":{"type":"string","required":false,"visibility":"user-or-llm","description":"Case-insensitive substring match on the list name"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_meetings":{"id":"affinity_list_meetings","name":"Affinity List Meetings","description":"Page through past and upcoming meetings with their organizer and attendees.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_merge_tasks":{"id":"affinity_list_merge_tasks","name":"Affinity List Merge Tasks","description":"Page through merge tasks, each summarizing how many of its merges are in progress, succeeded, or failed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merge tasks to list: companies or persons"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression. This endpoint filters on status only, e.g. \\"status=in-progress\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_merges":{"id":"affinity_list_merges","name":"Affinity List Merges","description":"Page through the company or person merges the organization has run, with the status and the records involved in each.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which merges to list: companies or persons"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over status or taskId, e.g. \\"status=failed\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_note_attached_companies":{"id":"affinity_list_note_attached_companies","name":"Affinity List Note Attached Companies","description":"List the companies directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_attached_opportunities":{"id":"affinity_list_note_attached_opportunities","name":"Affinity List Note Attached Opportunities","description":"List the opportunities directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_attached_persons":{"id":"affinity_list_note_attached_persons","name":"Affinity List Note Attached Persons","description":"List the persons directly attached to one note.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_note_replies":{"id":"affinity_list_note_replies","name":"Affinity List Note Replies","description":"Page through the replies on one note, including AI Notetaker replies.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID whose replies to read"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_notes":{"id":"affinity_list_notes","name":"Affinity List Notes","description":"Page through every note the caller can see. Replies are excluded.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"includes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Extra properties to return, e.g. [\\"repliesCount\\",\\"personsPreview\\",\\"companiesPreview\\",\\"opportunitiesPreview\\"]. Those four fields are omitted unless requested here"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_opportunities":{"id":"affinity_list_opportunities","name":"Affinity List Opportunities","description":"Page through opportunities. Field data lives on the list entry, not here — read it through the list or saved view the opportunity belongs to.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these opportunity IDs, e.g. [1, 2, 3]"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_persons":{"id":"affinity_list_persons","name":"Affinity List Persons","description":"Page through persons. Persons come back without field data unless Field IDs or Field Types asks for it.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the page to these person IDs, e.g. [1, 2, 3]"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_reminders":{"id":"affinity_list_reminders","name":"Affinity List Reminders","description":"Page through the reminders the caller can see. Filter by status to surface what is overdue.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_saved_view_entries":{"id":"affinity_list_saved_view_entries","name":"Affinity List Saved View Entries","description":"Page through the rows of a saved view. The view\'s own filters and columns decide which rows and which field data come back.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"viewId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The saved view ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_saved_views":{"id":"affinity_list_saved_views","name":"Affinity List Saved Views","description":"List the saved views on a list that the caller can view.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_list_transcript_fragments":{"id":"affinity_list_transcript_fragments","name":"Affinity List Transcript Fragments","description":"Page through everything said in a meeting, segment by segment with the speaker.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"transcriptId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The transcript ID"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_transcripts":{"id":"affinity_list_transcripts","name":"Affinity List Transcripts","description":"Page through meeting transcript metadata. Read one transcript to get what was actually said.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression, e.g. \\"createdAt>=2026-01-01\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_list_users":{"id":"affinity_list_users","name":"Affinity List Users","description":"Page through the internal users in the organization. Email addresses and roles are returned only to callers with the \\"Manage Users\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"term":{"type":"string","required":false,"visibility":"user-or-llm","description":"Case-insensitive match across first name, last name, and primary email"},"filter":{"type":"string","required":false,"visibility":"user-or-llm","description":"Affinity Filtering Language expression over id or status, e.g. \\"status=active\\""},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_search_companies":{"id":"affinity_search_companies","name":"Affinity Search Companies","description":"Search companies by filters, sorts, and a free-text term. Requires the \\"Export All Organizations directory\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_search_files":{"id":"affinity_search_files","name":"Affinity Search Files","description":"Search files by keyword, ordered by relevance. Narrow to specific files or to one company, or leave both unset to search the whole account.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to search for. Between 3 and 500 characters"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to these file IDs. Cannot be combined with Company ID"},"companyId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Restrict the search to one company\'s files. Cannot be combined with file IDs"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of files to return, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_search_list_entries":{"id":"affinity_search_list_entries","name":"Affinity Search List Entries","description":"Search the rows of one list by filters, sorts, and a free-text term. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID to search"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, list, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_search_notes":{"id":"affinity_search_notes","name":"Affinity Search Notes","description":"Search notes by keyword, ordered by relevance. Narrow to specific notes or to one company, or leave both unset to search the whole account.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to search for. Between 3 and 500 characters"},"ids":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to these note IDs. Cannot be combined with Company ID"},"companyId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Restrict the search to one company\'s notes. Cannot be combined with note IDs"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of notes to return, 1-100. Defaults to 20"}},"hostedApiKey":"none"},"affinity_search_persons":{"id":"affinity_search_persons","name":"Affinity Search Persons","description":"Search persons by filters, sorts, and a free-text term. Requires the \\"Export All People directory\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"filters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Filter group as {operator: \\"and\\"|\\"or\\", filters: [...]}, at most 50 leaves. Each leaf is {valueType, fieldId, operator, value}, and a leaf may itself be a nested group"},"searchTerm":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-text term matched against the searchable fields. At least 3 characters"},"searchFieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs the search term is matched against. Defaults to the searchable fields"},"sorts":{"type":"json","required":false,"visibility":"user-or-llm","description":"Sort order as [{fieldId, direction: \\"asc\\"|\\"desc\\", attributeId?}], up to 5, applied in order"},"fieldIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field IDs to return values for, e.g. [\\"affinity-data-location\\"]. Mutually exclusive with Field Types"},"fieldTypes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Field categories to return values for: enriched, global, or relationship-intelligence. Mutually exclusive with Field IDs"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous page, returned as nextCursor or prevCursor"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to return per page, 1-100. Defaults to 100"},"totalCount":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the total size of the collection. Costs an extra query"}},"hostedApiKey":"none"},"affinity_semantic_search":{"id":"affinity_semantic_search","name":"Affinity Semantic Search","description":"Find companies from a description in plain language — industry, technology, stage, or business model. Currently searches companies only.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"prompt":{"type":"string","required":true,"visibility":"user-or-llm","description":"What to look for, in plain language, e.g. \\"climate tech companies in our pipeline\\". Up to 500 characters"},"listIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Restrict the search to companies on these lists, e.g. [1, 2]"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of companies to return, 1-100. Defaults to 100"}},"hostedApiKey":"none"},"affinity_update_entity_field_value":{"id":"affinity_update_entity_field_value","name":"Affinity Update Entity Field Value","description":"Write one non-list field value on a company or person. The value type must match how the field is defined.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"entityType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Which entity to write the field on: companies or persons"},"entityId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of that company or person"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to write"},"value":{"type":"json","required":true,"visibility":"user-or-llm","description":"The new value as {type, data}, where type matches the field\'s value type. Examples: {\\"type\\":\\"text\\",\\"data\\":\\"Series B\\"}, {\\"type\\":\\"number\\",\\"data\\":42}, {\\"type\\":\\"dropdown\\",\\"data\\":{\\"dropdownOptionId\\":7}}, {\\"type\\":\\"person\\",\\"data\\":{\\"id\\":123}}, {\\"type\\":\\"person-multi\\",\\"data\\":[{\\"id\\":123}]}. Pass data as null to clear the field"}},"hostedApiKey":"none"},"affinity_update_list_entry_field":{"id":"affinity_update_list_entry_field","name":"Affinity Update List Entry Field","description":"Write one field value on a list row. Requires the \\"Export data from Lists\\" permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"listEntryId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list entry ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The field ID to write"},"value":{"type":"json","required":true,"visibility":"user-or-llm","description":"The new value as {type, data}, where type matches the field\'s value type. Examples: {\\"type\\":\\"text\\",\\"data\\":\\"Series B\\"}, {\\"type\\":\\"number\\",\\"data\\":42}, {\\"type\\":\\"dropdown\\",\\"data\\":{\\"dropdownOptionId\\":7}}, {\\"type\\":\\"person\\",\\"data\\":{\\"id\\":123}}, {\\"type\\":\\"person-multi\\",\\"data\\":[{\\"id\\":123}]}. Pass data as null to clear the field"}},"hostedApiKey":"none"},"affinity_update_list_field_dropdown_option":{"id":"affinity_update_list_field_dropdown_option","name":"Affinity Update List Field Dropdown Option","description":"Change a dropdown option on a list field. Every field is optional — supply only what should change, and only fields the option\'s kind actually has.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"listId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The list ID"},"fieldId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown field ID on that list"},"dropdownOptionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The dropdown option ID to update"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Replacement option label. Supply at least one field to change"},"rank":{"type":"number","required":false,"visibility":"user-or-llm","description":"Sort order. Required on a ranked-dropdown or status-dropdown option"},"color":{"type":"string","required":false,"visibility":"user-or-llm","description":"Option color: white, gray, blue, green, purple, orange, or red. Required on a ranked-dropdown or status-dropdown option"},"statusCategory":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pipeline meaning of the option: open, won, lost, or on-hold. Status-dropdown options only"},"winRate":{"type":"number","required":false,"visibility":"user-or-llm","description":"Expected win rate of the status. Status-dropdown options only"}},"hostedApiKey":"none"},"affinity_update_note":{"id":"affinity_update_note","name":"Affinity Update Note","description":"Rewrite a note\'s body or replace which records it is attached to. Each list of IDs replaces that association wholesale, an empty list clears it, and omitting one leaves it untouched. A note\'s type cannot be changed.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Affinity API key, sent as a bearer token"},"noteId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note ID to update"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"Replacement note body as HTML"},"companyIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached companies, e.g. [1, 2]. Send [] to detach every company; omit to leave them unchanged"},"personIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached persons, e.g. [1, 2]. Send [] to detach every person; omit to leave them unchanged"},"opportunityIds":{"type":"json","required":false,"visibility":"user-or-llm","description":"Replacement set of attached opportunities, e.g. [1, 2]. Send [] to detach every opportunity; omit to leave them unchanged"}},"hostedApiKey":"none"},"agentmail_create_draft":{"id":"agentmail_create_draft","name":"Create Draft","description":"Create a new email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to create the draft in"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Draft subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text draft body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML draft body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"},"inReplyTo":{"type":"string","required":false,"visibility":"user-or-llm","description":"ID of message being replied to"},"sendAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to schedule sending"}},"hostedApiKey":"none"},"agentmail_create_inbox":{"id":"agentmail_create_inbox","name":"Create Inbox","description":"Create a new email inbox with AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"username":{"type":"string","required":false,"visibility":"user-or-llm","description":"Username for the inbox email address"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Domain for the inbox email address"},"displayName":{"type":"string","required":false,"visibility":"user-or-llm","description":"Display name for the inbox"}},"hostedApiKey":"none"},"agentmail_delete_draft":{"id":"agentmail_delete_draft","name":"Delete Draft","description":"Delete an email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to delete"}},"hostedApiKey":"none"},"agentmail_delete_inbox":{"id":"agentmail_delete_inbox","name":"Delete Inbox","description":"Delete an email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to delete"}},"hostedApiKey":"none"},"agentmail_delete_thread":{"id":"agentmail_delete_thread","name":"Delete Thread","description":"Delete an email thread in AgentMail (moves to trash, or permanently deletes if already in trash)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to delete"},"permanent":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Force permanent deletion instead of moving to trash"}},"hostedApiKey":"none"},"agentmail_forward_message":{"id":"agentmail_forward_message","name":"Forward Message","description":"Forward an email message to new recipients in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to forward"},"to":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Override subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Additional plain text to prepend"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"Additional HTML to prepend"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"}},"hostedApiKey":"none"},"agentmail_get_draft":{"id":"agentmail_get_draft","name":"Get Draft","description":"Get details of a specific email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox the draft belongs to"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to retrieve"}},"hostedApiKey":"none"},"agentmail_get_inbox":{"id":"agentmail_get_inbox","name":"Get Inbox","description":"Get details of a specific email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to retrieve"}},"hostedApiKey":"none"},"agentmail_get_message":{"id":"agentmail_get_message","name":"Get Message","description":"Get details of a specific email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to retrieve"}},"hostedApiKey":"none"},"agentmail_get_thread":{"id":"agentmail_get_thread","name":"Get Thread","description":"Get details of a specific email thread including messages in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to retrieve"}},"hostedApiKey":"none"},"agentmail_list_drafts":{"id":"agentmail_list_drafts","name":"List Drafts","description":"List email drafts in an inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list drafts from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of drafts to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_inboxes":{"id":"agentmail_list_inboxes","name":"List Inboxes","description":"List all email inboxes in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of inboxes to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_messages":{"id":"agentmail_list_messages","name":"List Messages","description":"List messages in an inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list messages from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of messages to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"}},"hostedApiKey":"none"},"agentmail_list_threads":{"id":"agentmail_list_threads","name":"List Threads","description":"List email threads in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to list threads from"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of threads to return"},"pageToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token for next page of results"},"labels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to filter threads by"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter threads before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter threads after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentmail_reply_message":{"id":"agentmail_reply_message","name":"Reply to Message","description":"Reply to an existing email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to reply from"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to reply to"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text reply body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML reply body"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Override recipient email addresses (comma-separated)"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC email addresses (comma-separated)"},"replyAll":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Reply to all recipients of the original message"}},"hostedApiKey":"none"},"agentmail_send_draft":{"id":"agentmail_send_draft","name":"Send Draft","description":"Send an existing email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to send"}},"hostedApiKey":"none"},"agentmail_send_message":{"id":"agentmail_send_message","name":"Send Message","description":"Send an email message from an AgentMail inbox","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to send from"},"to":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient email address (comma-separated for multiple)"},"subject":{"type":"string","required":true,"visibility":"user-or-llm","description":"Email subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text email body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML email body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"}},"hostedApiKey":"none"},"agentmail_update_draft":{"id":"agentmail_update_draft","name":"Update Draft","description":"Update an existing email draft in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the draft"},"draftId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the draft to update"},"to":{"type":"string","required":false,"visibility":"user-or-llm","description":"Recipient email addresses (comma-separated)"},"subject":{"type":"string","required":false,"visibility":"user-or-llm","description":"Draft subject line"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Plain text draft body"},"html":{"type":"string","required":false,"visibility":"user-or-llm","description":"HTML draft body"},"cc":{"type":"string","required":false,"visibility":"user-or-llm","description":"CC recipient email addresses (comma-separated)"},"bcc":{"type":"string","required":false,"visibility":"user-or-llm","description":"BCC recipient email addresses (comma-separated)"},"sendAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to schedule sending"}},"hostedApiKey":"none"},"agentmail_update_inbox":{"id":"agentmail_update_inbox","name":"Update Inbox","description":"Update the display name of an email inbox in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox to update"},"displayName":{"type":"string","required":true,"visibility":"user-or-llm","description":"New display name for the inbox"}},"hostedApiKey":"none"},"agentmail_update_message":{"id":"agentmail_update_message","name":"Update Message","description":"Add or remove labels on an email message in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the message"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to update"},"addLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to add to the message"},"removeLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to remove from the message"}},"hostedApiKey":"none"},"agentmail_update_thread":{"id":"agentmail_update_thread","name":"Update Thread Labels","description":"Add or remove labels on an email thread in AgentMail","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentMail API key"},"inboxId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the inbox containing the thread"},"threadId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the thread to update"},"addLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to add to the thread"},"removeLabels":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated labels to remove from the thread"}},"hostedApiKey":"none"},"agentphone_create_call":{"id":"agentphone_create_call","name":"Create Outbound Call","description":"Initiate an outbound voice call from an AgentPhone agent","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"agentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Agent that will handle the call"},"toNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Phone number to call in E.164 format (e.g. +14155551234)"},"fromNumberId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Phone number ID to use as caller ID. Must belong to the agent. If omitted, the agent\'s first assigned number is used."},"initialGreeting":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optional greeting spoken when the recipient answers"},"voice":{"type":"string","required":false,"visibility":"user-or-llm","description":"Voice ID override for this call (defaults to the agent\'s configured voice)"},"systemPrompt":{"type":"string","required":false,"visibility":"user-or-llm","description":"When provided, uses a built-in LLM for the conversation instead of forwarding to your webhook"}},"hostedApiKey":"none"},"agentphone_create_contact":{"id":"agentphone_create_contact","name":"Create Contact","description":"Create a new contact in AgentPhone","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"phoneNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Phone number in E.164 format (e.g. +14155551234)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact\'s full name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Contact\'s email address"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Freeform notes stored on the contact"}},"hostedApiKey":"none"},"agentphone_create_number":{"id":"agentphone_create_number","name":"Create Phone Number","description":"Provision a new SMS- and voice-enabled phone number","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Two-letter country code (e.g. US, CA). Defaults to US."},"areaCode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Preferred area code (US/CA only, e.g. \\"415\\"). Best-effort — may be ignored if unavailable."},"agentId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optionally attach the number to an agent immediately"}},"hostedApiKey":"none"},"agentphone_delete_contact":{"id":"agentphone_delete_contact","name":"Delete Contact","description":"Delete a contact by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"}},"hostedApiKey":"none"},"agentphone_get_call":{"id":"agentphone_get_call","name":"Get Call","description":"Fetch a call and its full transcript","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"callId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the call to retrieve"}},"hostedApiKey":"none"},"agentphone_get_call_transcript":{"id":"agentphone_get_call_transcript","name":"Get Call Transcript","description":"Get the full ordered transcript for a call","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"callId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the call to retrieve the transcript for"}},"hostedApiKey":"none"},"agentphone_get_contact":{"id":"agentphone_get_contact","name":"Get Contact","description":"Fetch a single contact by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"}},"hostedApiKey":"none"},"agentphone_get_conversation":{"id":"agentphone_get_conversation","name":"Get Conversation","description":"Get a conversation along with its recent messages","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"messageLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of recent messages to include (default 50, max 100)"}},"hostedApiKey":"none"},"agentphone_get_conversation_messages":{"id":"agentphone_get_conversation_messages","name":"Get Conversation Messages","description":"Get paginated messages for a conversation","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of messages to return (default 50, max 200)"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentphone_get_number_messages":{"id":"agentphone_get_number_messages","name":"Get Phone Number Messages","description":"Fetch messages received on a specific phone number","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"numberId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the phone number"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of messages to return (default 50, max 200)"},"before":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received before this ISO 8601 timestamp"},"after":{"type":"string","required":false,"visibility":"user-or-llm","description":"Return messages received after this ISO 8601 timestamp"}},"hostedApiKey":"none"},"agentphone_get_usage":{"id":"agentphone_get_usage","name":"Get Usage","description":"Retrieve current usage statistics for the AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"}},"hostedApiKey":"none"},"agentphone_get_usage_daily":{"id":"agentphone_get_usage_daily","name":"Get Daily Usage","description":"Get a daily breakdown of usage (messages, calls, webhooks) for the last N days","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"days":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of days to return (1-365, default 30)"}},"hostedApiKey":"none"},"agentphone_get_usage_monthly":{"id":"agentphone_get_usage_monthly","name":"Get Monthly Usage","description":"Get monthly usage aggregation (messages, calls, webhooks) for the last N months","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"months":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of months to return (1-24, default 6)"}},"hostedApiKey":"none"},"agentphone_list_calls":{"id":"agentphone_list_calls","name":"List Calls","description":"List voice calls for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"},"status":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by status (completed, in-progress, failed)"},"direction":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by direction (inbound, outbound)"},"type":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by call type (pstn, web)"},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search by phone number (matches fromNumber or toNumber)"}},"hostedApiKey":"none"},"agentphone_list_contacts":{"id":"agentphone_list_contacts","name":"List Contacts","description":"List contacts for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter by name or phone number (case-insensitive contains)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 50, max 200)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_list_conversations":{"id":"agentphone_list_conversations","name":"List Conversations","description":"List conversations (message threads) for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_list_numbers":{"id":"agentphone_list_numbers","name":"List Phone Numbers","description":"List all phone numbers provisioned for this AgentPhone account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to return (default 20, max 100)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip (min 0)"}},"hostedApiKey":"none"},"agentphone_react_to_message":{"id":"agentphone_react_to_message","name":"React to Message","description":"Send an iMessage tapback reaction to a message (iMessage only)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"messageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the message to react to"},"reaction":{"type":"string","required":true,"visibility":"user-or-llm","description":"Reaction type: love, like, dislike, laugh, emphasize, or question"}},"hostedApiKey":"none"},"agentphone_release_number":{"id":"agentphone_release_number","name":"Release Phone Number","description":"Release (delete) a phone number. This action is irreversible.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"numberId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the phone number to release"}},"hostedApiKey":"none"},"agentphone_send_message":{"id":"agentphone_send_message","name":"Send Message","description":"Send an outbound SMS or iMessage from an AgentPhone agent","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"agentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Agent sending the message"},"toNumber":{"type":"string","required":true,"visibility":"user-or-llm","description":"Recipient phone number in E.164 format (e.g. +14155551234)"},"body":{"type":"string","required":true,"visibility":"user-or-llm","description":"Message text to send"},"mediaUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"Optional URL of an image, video, or file to attach"},"numberId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Phone number ID to send from. If omitted, the agent\'s first assigned number is used."}},"hostedApiKey":"none"},"agentphone_update_contact":{"id":"agentphone_update_contact","name":"Update Contact","description":"Update a contact\'s fields","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"contactId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Contact ID"},"phoneNumber":{"type":"string","required":false,"visibility":"user-or-llm","description":"New phone number in E.164 format"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New contact name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"New email address"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"New freeform notes"}},"hostedApiKey":"none"},"agentphone_update_conversation":{"id":"agentphone_update_conversation","name":"Update Conversation","description":"Update conversation metadata (stored state). Pass null to clear existing metadata.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"AgentPhone API key"},"conversationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Conversation ID"},"metadata":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom key-value metadata to store on the conversation. Pass null to clear existing metadata."}},"hostedApiKey":"none"},"agiloft_async_status":{"id":"agiloft_async_status","name":"Agiloft Async Status","description":"Check whether an asynchronous Agiloft call, such as a run action button, has completed.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table the asynchronous call was made against"},"callbackId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Callback ID returned by the asynchronous call, e.g. from Run Action Button"}},"hostedApiKey":"none"},"agiloft_attach_file":{"id":"agiloft_attach_file","name":"Agiloft Attach File","description":"Attach a file to a field in an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to attach the file to"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"file":{"type":"file","required":true,"visibility":"user-or-llm","description":"File to attach"},"fileName":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name to assign to the file (defaults to original file name)"},"overwrite":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Replace the contents of the field instead of adding another file to it"}},"hostedApiKey":"none"},"agiloft_attachment_info":{"id":"agiloft_attachment_info","name":"Agiloft Attachment Info","description":"Get information about file attachments on a record field.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to check attachments on"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field to inspect"}},"hostedApiKey":"none"},"agiloft_create_record":{"id":"agiloft_create_record","name":"Agiloft Create Record","description":"Create a new record in an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record field values as a JSON object (e.g., {\\"first_name\\": \\"John\\", \\"status\\": \\"Active\\"})"}},"hostedApiKey":"none"},"agiloft_delete_record":{"id":"agiloft_delete_record","name":"Agiloft Delete Record","description":"Delete a record from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to delete"},"substituteIds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated IDs of records that adopt the dependants of the deleted record. Read only when the delete rule is REPLACE_WITH_ANOTHER."},"deleteRule":{"type":"string","required":false,"visibility":"user-or-llm","description":"How to treat records that depend on this one: ERROR_IF_DEPENDANTS (default — fails rather than cascading), APPLY_DELETE_WHERE_POSSIBLE, DELETE_WHERE_POSSIBLE_OTHERWISE_UNLINK, APPLY_UNLINK, UNLINK_WHERE_POSSIBLE_OTHERWISE_DELETE, or REPLACE_WITH_ANOTHER"}},"hostedApiKey":"none"},"agiloft_get_choice_line_id":{"id":"agiloft_get_choice_line_id","name":"Agiloft Get Choice Line ID","description":"Resolve the internal numeric ID of a choice-list value, for use in EWSelect WHERE clauses against choice fields.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"case\\", \\"contracts\\")"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Choice field name (e.g., \\"priority\\", \\"status\\")"},"value":{"type":"string","required":true,"visibility":"user-or-llm","description":"Choice display value to resolve (e.g., \\"High\\", \\"Active\\")"}},"hostedApiKey":"none"},"agiloft_list_tables":{"id":"agiloft_list_tables","name":"Agiloft List Tables","description":"List the tables and fields in an Agiloft knowledge base, to discover the logical names other operations need.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":false,"visibility":"user-or-llm","description":"Logical name of a single table to describe (e.g., \\"contacts\\"). Leave empty to list every table in the knowledge base."},"includeLinkedInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Include the source table and column behind each linked field"},"skipColumnsInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Return table names only, omitting field details, for a much smaller response"}},"hostedApiKey":"none"},"agiloft_lock_record":{"id":"agiloft_lock_record","name":"Agiloft Lock Record","description":"Lock, unlock, or check the lock status of an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to lock, unlock, or check"},"lockAction":{"type":"string","required":true,"visibility":"user-or-llm","description":"Action to perform: \\"lock\\", \\"unlock\\", or \\"check\\""},"force":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Unlock only: release a lock held by another user."}},"hostedApiKey":"none"},"agiloft_nlp_search":{"id":"agiloft_nlp_search","name":"Agiloft Natural Language Search","description":"Search Agiloft records by describing what you want in plain language, such as \\"active NDAs submitted last month\\".","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"nlpQuery":{"type":"string","required":true,"visibility":"user-or-llm","description":"The request in plain language, e.g. \\"Show me open, high-priority contracts\\". Structured field filters are not accepted — use Search Records for those."},"fields":{"type":"string","required":true,"visibility":"user-or-llm","description":"Comma-separated field names to return, e.g. \\"id, contract_title1, company_name\\""},"page":{"type":"string","required":false,"visibility":"user-or-llm","description":"Page number, starting from 0"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Records per page"}},"hostedApiKey":"none"},"agiloft_read_record":{"id":"agiloft_read_record","name":"Agiloft Read Record","description":"Read a record by ID from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to read"},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of field names to include in the response"}},"hostedApiKey":"none"},"agiloft_remove_attachment":{"id":"agiloft_remove_attachment","name":"Agiloft Remove Attachment","description":"Remove an attached file from a field in an Agiloft record.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record containing the attachment"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"position":{"type":"string","required":true,"visibility":"user-or-llm","description":"Position index of the file to remove (starting from 0)"}},"hostedApiKey":"none"},"agiloft_retrieve_attachment":{"id":"agiloft_retrieve_attachment","name":"Agiloft Retrieve Attachment","description":"Download an attached file from an Agiloft record field.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record containing the attachment"},"fieldName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the attachment field"},"position":{"type":"string","required":true,"visibility":"user-or-llm","description":"Position index of the file in the field (starting from 0)"}},"hostedApiKey":"none"},"agiloft_run_action_button":{"id":"agiloft_run_action_button","name":"Agiloft Run Action Button","description":"Run an action button on an Agiloft record, such as an approval or send-for-signature step.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"case\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to run the action button on"},"actionButtonField":{"type":"string","required":true,"visibility":"user-or-llm","description":"Logical name of the field holding the action button (e.g., \\"ab_field\\")"}},"hostedApiKey":"none"},"agiloft_saved_search":{"id":"agiloft_saved_search","name":"Agiloft Saved Search","description":"List the saved searches defined for an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Logical table name to list saved searches for (e.g., \\"contract\\")"}},"hostedApiKey":"none"},"agiloft_search_records":{"id":"agiloft_search_records","name":"Agiloft Search Records","description":"Search for records in an Agiloft table using a query.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name to search in (e.g., \\"contracts\\", \\"contacts.employees\\")"},"query":{"type":"string","required":false,"visibility":"user-or-llm","description":"Ad hoc EWSearch query. Combine conditions with && (and) or || (or) and quote every value — e.g. \\"summary~=\'test\'&&priority=\'High\'\\". Required unless a saved search is given."},"search":{"type":"string","required":false,"visibility":"user-or-llm","description":"Label of a saved search defined on the table (e.g., \\"C: Status is Closed\\"). Can be combined with a query to narrow it further."},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of field names to include in the results"},"page":{"type":"string","required":false,"visibility":"user-or-llm","description":"Page number for paginated results (starting from 0)"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of records to return per page. Agiloft treats 0 as \\"all records\\", so leave it unset or use a positive value to keep result sizes bounded."}},"hostedApiKey":"none"},"agiloft_select_records":{"id":"agiloft_select_records","name":"Agiloft Select Records","description":"Select record IDs matching a SQL WHERE clause from an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"where":{"type":"string","required":true,"visibility":"user-or-llm","description":"SQL WHERE clause using database column names (e.g., \\"summary like \'%new%\'\\" or \\"assigned_person=\'John Doe\'\\"). EWSelect has no page size and returns every matching ID, so append a database limit such as \\"limit 0,200\\" to bound the result."}},"hostedApiKey":"none"},"agiloft_update_record":{"id":"agiloft_update_record","name":"Agiloft Update Record","description":"Update an existing record in an Agiloft table.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the record to update"},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Updated field values as a JSON object (e.g., {\\"status\\": \\"Active\\", \\"priority\\": \\"High\\"})"}},"hostedApiKey":"none"},"agiloft_upsert_record":{"id":"agiloft_upsert_record","name":"Agiloft Upsert Record","description":"Create an Agiloft record, or update it when a record already matches the given fields.","version":"1.0.0","params":{"instanceUrl":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft instance URL (e.g., https://mycompany.agiloft.com)"},"knowledgeBase":{"type":"string","required":true,"visibility":"user-only","description":"Knowledge base name"},"login":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft username"},"password":{"type":"string","required":true,"visibility":"user-only","description":"Agiloft password"},"table":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table name (e.g., \\"contracts\\", \\"contacts.employees\\")"},"match":{"type":"string","required":true,"visibility":"user-or-llm","description":"Field used to find an existing record (e.g., \\"ext_id\\"). Pick something that identifies a record uniquely — if more than one record matches, Agiloft writes nothing and returns a conflict."},"async":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Queue the write instead of waiting for it. Returns a callback ID instead of a record ID; pass that to Async Status to poll the result."},"data":{"type":"string","required":true,"visibility":"user-or-llm","description":"Field values as a JSON object. On create these populate the new record; on update only the supplied fields change."}},"hostedApiKey":"none"},"ahrefs_anchors":{"id":"ahrefs_anchors","name":"Ahrefs Anchors","description":"Get the anchor text distribution for a target domain or URL\'s backlinks, showing how many links and referring domains use each anchor text.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live), \\"all_time\\" (default, includes lost backlinks), or \\"since:YYYY-MM-DD\\" (backlinks found since a date)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_backlinks":{"id":"ahrefs_backlinks","name":"Ahrefs Backlinks","description":"Get a list of backlinks pointing to a target domain or URL. Returns details about each backlink including source URL, anchor text, and domain rating.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live backlinks), \\"all_time\\" (default, includes lost backlinks), or \\"since:YYYY-MM-DD\\" (backlinks found since a date)."},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_backlinks_stats":{"id":"ahrefs_backlinks_stats","name":"Ahrefs Backlinks Stats","description":"Get backlink and referring domain totals for a target domain or URL, both currently live and across all time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_batch_analysis":{"id":"ahrefs_batch_analysis","name":"Ahrefs Batch Analysis","description":"Get bulk SEO metrics (Domain Rating, backlinks, referring domains, organic traffic, and more) for multiple domains or URLs in a single request. Useful for comparing many competitors at once.","version":"1.0.0","params":{"targets":{"type":"string","required":true,"visibility":"user-or-llm","description":"Comma-separated list of domains or URLs to analyze. Example: \\"example.com,competitor.com\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode applied to every target: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"protocol":{"type":"string","required":false,"visibility":"user-or-llm","description":"Protocol applied to every target: \\"both\\" (default), \\"http\\", or \\"https\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_broken_backlinks":{"id":"ahrefs_broken_backlinks","name":"Ahrefs Broken Backlinks","description":"Get a list of broken backlinks pointing to a target domain or URL. Useful for identifying link reclamation opportunities.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_domain_rating":{"id":"ahrefs_domain_rating","name":"Ahrefs Domain Rating","description":"Get the Domain Rating (DR) and Ahrefs Rank for a target domain. Domain Rating shows the strength of a website\'s backlink profile on a scale from 0 to 100.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain to analyze (e.g., example.com)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date for historical data in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_domain_rating_history":{"id":"ahrefs_domain_rating_history","name":"Ahrefs Domain Rating History","description":"Get the historical Domain Rating (DR) trend for a target domain or URL over a date range, grouped daily, weekly, or monthly.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_keyword_overview":{"id":"ahrefs_keyword_overview","name":"Ahrefs Keyword Overview","description":"Get detailed metrics for a keyword including search volume, keyword difficulty, CPC, clicks, and traffic potential.","version":"1.0.0","params":{"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The keyword to analyze"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for keyword data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_keywords_history":{"id":"ahrefs_keywords_history","name":"Ahrefs Keywords History","description":"Get the historical organic keyword ranking distribution for a target domain or URL over a date range: how many keywords rank in each position bucket at each point in time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_metrics":{"id":"ahrefs_metrics","name":"Ahrefs Metrics","description":"Get a one-call organic and paid search overview for a target domain or URL: organic traffic, organic keywords, paid traffic, paid keywords, and estimated traffic cost.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_metrics_history":{"id":"ahrefs_metrics_history","name":"Ahrefs Metrics History","description":"Get the historical organic and paid traffic trend for a target domain or URL over a date range: organic traffic/cost and paid traffic/cost at each point in time.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_organic_competitors":{"id":"ahrefs_organic_competitors","name":"Ahrefs Organic Competitors","description":"Get domains that compete with a target domain or URL for the same organic keywords, ranked by keyword overlap.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_organic_keywords":{"id":"ahrefs_organic_keywords","name":"Ahrefs Organic Keywords","description":"Get organic keywords that a target domain or URL ranks for in Google search results. Returns keyword details including search volume, ranking position, and estimated traffic.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for search results. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_paid_pages":{"id":"ahrefs_paid_pages","name":"Ahrefs Paid Pages","description":"Get a target domain\'s pages that receive paid search traffic, sorted by estimated paid traffic. Returns page URLs with their paid traffic, keyword counts, and estimated spend.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_competitors_overview":{"id":"ahrefs_rank_tracker_competitors_overview","name":"Ahrefs Rank Tracker Competitors Overview","description":"Get competitor rankings for the keywords tracked in an Ahrefs Rank Tracker project: each tracked keyword\'s volume and difficulty alongside every competitor\'s position, traffic, and traffic value. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report rankings for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"dateCompared":{"type":"string","required":false,"visibility":"user-only","description":"Comparison date in YYYY-MM-DD format, to compute position/traffic deltas"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_competitors_stats":{"id":"ahrefs_rank_tracker_competitors_stats","name":"Ahrefs Rank Tracker Competitors Stats","description":"Get aggregate competitor stats for an Ahrefs Rank Tracker project: each competitor\'s traffic, traffic value, average position, and share of voice across all tracked keywords. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report metrics for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_overview":{"id":"ahrefs_rank_tracker_overview","name":"Ahrefs Rank Tracker Overview","description":"Get ranking overview metrics for the keywords tracked in an Ahrefs Rank Tracker project: position, search volume, keyword difficulty, and estimated traffic. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":true,"visibility":"user-only","description":"Date to report rankings for, in YYYY-MM-DD format"},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"dateCompared":{"type":"string","required":false,"visibility":"user-only","description":"Comparison date in YYYY-MM-DD format, to compute position/traffic deltas"},"volumeMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search volume calculation: \\"monthly\\" or \\"average\\" (default: \\"monthly\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_rank_tracker_serp_overview":{"id":"ahrefs_rank_tracker_serp_overview","name":"Ahrefs Rank Tracker SERP Overview","description":"Get the full SERP (search engine results page) for a keyword tracked in an Ahrefs Rank Tracker project, including every ranking URL with its position, title, and authority metrics. This endpoint is free and does not consume API units.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Rank Tracker project ID (found in the project URL in Ahrefs)"},"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The tracked keyword to retrieve SERP data for"},"country":{"type":"string","required":true,"visibility":"user-or-llm","description":"Country code for the tracked keyword. Example: \\"us\\", \\"gb\\", \\"de\\""},"device":{"type":"string","required":true,"visibility":"user-or-llm","description":"Rankings device type: \\"desktop\\" or \\"mobile\\""},"topPositions":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of top organic positions to return (defaults to all available)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Timestamp to return the last available SERP Overview at, in YYYY-MM-DDThh:mm:ss format"},"locationId":{"type":"number","required":false,"visibility":"user-or-llm","description":"Location ID of the tracked keyword, if tracked at a specific location"},"languageCode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Language code of the tracked keyword"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_refdomains_history":{"id":"ahrefs_refdomains_history","name":"Ahrefs Referring Domains History","description":"Get the historical referring domains trend for a target domain or URL over a date range, grouped daily, weekly, or monthly.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\""},"dateFrom":{"type":"string","required":true,"visibility":"user-only","description":"Start date of the historical period, in YYYY-MM-DD format"},"dateTo":{"type":"string","required":false,"visibility":"user-only","description":"End date of the historical period, in YYYY-MM-DD format (defaults to today)"},"historyGrouping":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval for grouping data points: \\"daily\\", \\"weekly\\", or \\"monthly\\" (default: \\"monthly\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_referring_domains":{"id":"ahrefs_referring_domains","name":"Ahrefs Referring Domains","description":"Get a list of domains that link to a target domain or URL. Returns unique referring domains with their domain rating, backlink counts, and discovery dates.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain or URL to analyze. Example: \\"example.com\\" or \\"https://example.com/page\\""},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"history":{"type":"string","required":false,"visibility":"user-or-llm","description":"Historical scope: \\"live\\" (currently live), \\"all_time\\" (default, includes lost domains), or \\"since:YYYY-MM-DD\\" (domains found since a date)."},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_related_terms":{"id":"ahrefs_related_terms","name":"Ahrefs Related Terms","description":"Get keyword ideas related to a seed keyword: terms the same top-ranking pages also rank for (\\"also rank for\\") or also discuss (\\"also talk about\\"), with volume, difficulty, and CPC.","version":"1.0.0","params":{"keyword":{"type":"string","required":true,"visibility":"user-or-llm","description":"The seed keyword to find related terms for"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for keyword data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"terms":{"type":"string","required":false,"visibility":"user-or-llm","description":"Type of related keywords to return: \\"also_rank_for\\", \\"also_talk_about\\", or \\"all\\" (default: \\"all\\")"},"viewFor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Whether to derive related terms from the top 10 or top 100 ranking pages (default: \\"top_10\\")"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_site_audit_page_explorer":{"id":"ahrefs_site_audit_page_explorer","name":"Ahrefs Site Audit Page Explorer","description":"Get crawled pages from an Ahrefs Site Audit project with health and SEO metrics: HTTP status, title, link counts, backlinks, indexability, and traffic. Optionally filter to pages affected by a specific issue.","version":"1.0.0","params":{"projectId":{"type":"number","required":true,"visibility":"user-or-llm","description":"The Site Audit project ID (found in the project URL in Ahrefs)"},"date":{"type":"string","required":false,"visibility":"user-only","description":"Crawl date in YYYY-MM-DDThh:mm:ss format (defaults to the most recent crawl)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of results to skip, for pagination"},"issueId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Only return pages affected by this issue ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"ahrefs_top_pages":{"id":"ahrefs_top_pages","name":"Ahrefs Top Pages","description":"Get the top pages of a target domain sorted by organic traffic. Returns page URLs with their traffic, keyword counts, and estimated traffic value.","version":"1.0.0","params":{"target":{"type":"string","required":true,"visibility":"user-or-llm","description":"The target domain to analyze. Example: \\"example.com\\""},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Country code for traffic data. Example: \\"us\\", \\"gb\\", \\"de\\" (default: \\"us\\")"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Analysis mode: domain (entire domain), prefix (URL prefix), subdomains (include all subdomains, default), exact (exact URL match). Example: \\"domain\\""},"date":{"type":"string","required":false,"visibility":"user-only","description":"Date to report metrics on, in YYYY-MM-DD format (defaults to today)"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of results to return. Example: 50 (default: 1000)"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ahrefs API Key"}},"hostedApiKey":"none"},"airtable_create_records":{"id":"airtable_create_records","name":"Airtable Create Records","description":"Write new records to an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to create, each with a `fields` object"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_delete_records":{"id":"airtable_delete_records","name":"Airtable Delete Records","description":"Delete one or more records from an Airtable table by ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordIds":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of record IDs to delete (each starts with \\"rec\\", e.g., [\\"recXXXXXXXXXXXXXX\\"]). Pass a single-element array to delete one record."}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_get_base_schema":{"id":"airtable_get_base_schema","name":"Airtable Get Base Schema","description":"Get the schema of all tables, fields, and views in an Airtable base","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_get_record":{"id":"airtable_get_record","name":"Airtable Get Record","description":"Retrieve a single record from an Airtable table by its ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record ID to retrieve (starts with \\"rec\\", e.g., \\"recXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_bases":{"id":"airtable_list_bases","name":"Airtable List Bases","description":"List all bases the authenticated user has access to","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"offset":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination offset for retrieving additional bases"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_records":{"id":"airtable_list_records","name":"Airtable List Records","description":"Read records from an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"maxRecords":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of records to return (default: all records)"},"filterFormula":{"type":"string","required":false,"visibility":"user-or-llm","description":"Formula to filter records (e.g., \\"({Field Name} = \'Value\')\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_list_tables":{"id":"airtable_list_tables","name":"Airtable List Tables","description":"List all tables and their schema in an Airtable base","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_update_multiple_records":{"id":"airtable_update_multiple_records","name":"Airtable Update Multiple Records","description":"Update multiple existing records in an Airtable table","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to update, each with an `id` and a `fields` object"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_update_record":{"id":"airtable_update_record","name":"Airtable Update Record","description":"Update an existing record in an Airtable table by ID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"recordId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Record ID to update (starts with \\"rec\\", e.g., \\"recXXXXXXXXXXXXXX\\")"},"fields":{"type":"json","required":true,"visibility":"user-or-llm","description":"An object containing the field names and their new values"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airtable_upsert_records":{"id":"airtable_upsert_records","name":"Airtable Upsert Records","description":"Update existing records or create new ones in an Airtable table, matching on the specified merge fields","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token"},"baseId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Airtable base ID (starts with \\"app\\", e.g., \\"appXXXXXXXXXXXXXX\\")"},"tableId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Table ID (starts with \\"tbl\\") or table name"},"records":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of records to upsert, each with a `fields` object"},"fieldsToMergeOn":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of field names used to match existing records (max 3). A record is updated when all merge fields match, otherwise it is created. Example: [\\"Name\\"]"},"typecast":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, Airtable automatically converts string values to the field type"}},"oauth":{"required":true,"provider":"airtable"},"hostedApiKey":"none"},"airweave_search":{"id":"airweave_search","name":"Airweave Search","description":"Search your synced data collections using Airweave. Supports semantic search with hybrid, neural, or keyword retrieval strategies. Optionally generate AI-powered answers from search results.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Airweave API Key for authentication"},"collectionId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The readable ID of the collection to search"},"query":{"type":"string","required":true,"visibility":"user-or-llm","description":"The search query text"},"limit":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 100)"},"retrievalStrategy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Retrieval strategy: hybrid (default), neural, or keyword"},"expandQuery":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Generate query variations to improve recall"},"rerank":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Reorder results for improved relevance using LLM"},"generateAnswer":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Generate a natural-language answer to the query"}},"hostedApiKey":"none"},"algolia_add_record":{"id":"algolia_add_record","name":"Algolia Add Record","description":"Add or replace a record in an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":false,"visibility":"user-or-llm","description":"Object ID for the record (auto-generated if not provided)"},"record":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object representing the record to add"}},"hostedApiKey":"none"},"algolia_batch_operations":{"id":"algolia_batch_operations","name":"Algolia Batch Operations","description":"Perform batch add, update, partial update, or delete operations on records in an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"requests":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of batch operations. Each item has \\"action\\" (addObject, updateObject, partialUpdateObject, partialUpdateObjectNoCreate, deleteObject, delete, clear) and \\"body\\" (the record data; must include objectID for update/delete; use an empty object {} for the index-level delete/clear actions)"}},"hostedApiKey":"none"},"algolia_browse_records":{"id":"algolia_browse_records","name":"Algolia Browse Records","description":"Browse and iterate over all records in an Algolia index using cursor pagination","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key (must have browse ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to browse"},"query":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search query to filter browsed records"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter string to narrow down results"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of hits per page (default: 1000, max: 1000)"},"cursor":{"type":"string","required":false,"visibility":"user-or-llm","description":"Cursor from a previous browse response for pagination"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search"}},"hostedApiKey":"none"},"algolia_clear_records":{"id":"algolia_clear_records","name":"Algolia Clear Records","description":"Clear all records from an Algolia index while keeping settings, synonyms, and rules","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to clear"}},"hostedApiKey":"none"},"algolia_copy_move_index":{"id":"algolia_copy_move_index","name":"Algolia Copy/Move Index","description":"Copy or move an Algolia index to a new destination","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the source index"},"operation":{"type":"string","required":true,"visibility":"user-or-llm","description":"Operation to perform: \\"copy\\" or \\"move\\""},"destination":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the destination index"},"scope":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of scopes to copy (only for \\"copy\\" operation): [\\"settings\\", \\"synonyms\\", \\"rules\\"]. Omit to copy everything including records."}},"hostedApiKey":"none"},"algolia_delete_by_filter":{"id":"algolia_delete_by_filter","name":"Algolia Delete By Filter","description":"Delete all records matching a filter from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter expression to match records for deletion (e.g., \\"category:outdated\\")"},"facetFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of facet filters (e.g., [\\"brand:Acme\\"])"},"numericFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of numeric filters (e.g., [\\"price > 100\\"])"},"tagFilters":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of tag filters using the _tags attribute (e.g., [\\"published\\"])"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search filter (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search filter"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search filter"}},"hostedApiKey":"none"},"algolia_delete_index":{"id":"algolia_delete_index","name":"Algolia Delete Index","description":"Delete an entire Algolia index and all its records","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have deleteIndex ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to delete"}},"hostedApiKey":"none"},"algolia_delete_record":{"id":"algolia_delete_record","name":"Algolia Delete Record","description":"Delete a record by objectID from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to delete"}},"hostedApiKey":"none"},"algolia_get_record":{"id":"algolia_get_record","name":"Algolia Get Record","description":"Get a record by objectID from an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to retrieve"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"}},"hostedApiKey":"none"},"algolia_get_records":{"id":"algolia_get_records","name":"Algolia Get Records","description":"Retrieve multiple records by objectID from one or more Algolia indices","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Default index name for all requests"},"requests":{"type":"json","required":true,"visibility":"user-or-llm","description":"Array of objects specifying records to retrieve. Each must have \\"objectID\\" and optionally \\"indexName\\" and \\"attributesToRetrieve\\"."}},"hostedApiKey":"none"},"algolia_get_settings":{"id":"algolia_get_settings","name":"Algolia Get Settings","description":"Retrieve the settings of an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"}},"hostedApiKey":"none"},"algolia_get_task_status":{"id":"algolia_get_task_status","name":"Algolia Get Task Status","description":"Check whether an Algolia indexing task has finished publishing","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index the task ran against"},"taskID":{"type":"number","required":true,"visibility":"user-or-llm","description":"The taskID returned by a previous write operation"}},"hostedApiKey":"none"},"algolia_list_indices":{"id":"algolia_list_indices","name":"Algolia List Indices","description":"List all indices in an Algolia application","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for paginating indices (default: not paginated)"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of indices per page (default: 100)"}},"hostedApiKey":"none"},"algolia_partial_update_record":{"id":"algolia_partial_update_record","name":"Algolia Partial Update Record","description":"Partially update a record in an Algolia index without replacing it entirely","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"objectID":{"type":"string","required":true,"visibility":"user-or-llm","description":"The objectID of the record to update"},"attributes":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object with attributes to update. Supports built-in operations like {\\"stock\\": {\\"_operation\\": \\"Decrement\\", \\"value\\": 1}}"},"createIfNotExists":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to create the record if it does not exist (default: true)"}},"hostedApiKey":"none"},"algolia_search":{"id":"algolia_search","name":"Algolia Search","description":"Search an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia API Key"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index to search"},"query":{"type":"string","required":true,"visibility":"user-or-llm","description":"Search query text"},"hitsPerPage":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of hits per page (default: 20)"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number to retrieve (default: 0)"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter string (e.g., \\"category:electronics AND price < 100\\")"},"attributesToRetrieve":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of attributes to retrieve"},"facets":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of facet attribute names to retrieve counts for (use \\"*\\" for all)"},"getRankingInfo":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to include detailed ranking information in each hit"},"aroundLatLng":{"type":"string","required":false,"visibility":"user-or-llm","description":"Coordinates for geo-search (e.g., \\"40.71,-74.01\\")"},"aroundRadius":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum radius in meters for geo-search, or \\"all\\" for unlimited"},"insideBoundingBox":{"type":"json","required":false,"visibility":"user-or-llm","description":"Bounding box coordinates as [[lat1, lng1, lat2, lng2]] for geo-search"},"insidePolygon":{"type":"json","required":false,"visibility":"user-or-llm","description":"Polygon coordinates as [[lat1, lng1, lat2, lng2, lat3, lng3, ...]] for geo-search"}},"hostedApiKey":"none"},"algolia_update_settings":{"id":"algolia_update_settings","name":"Algolia Update Settings","description":"Update the settings of an Algolia index","version":"1.0","params":{"applicationId":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Application ID"},"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Algolia Admin API Key (must have editSettings ACL)"},"indexName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the Algolia index"},"settings":{"type":"json","required":true,"visibility":"user-or-llm","description":"JSON object with settings to update (e.g., {\\"searchableAttributes\\": [\\"name\\", \\"description\\"], \\"customRanking\\": [\\"desc(popularity)\\"]})"},"forwardToReplicas":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to apply changes to replica indices (default: false)"}},"hostedApiKey":"none"},"amplitude_event_segmentation":{"id":"amplitude_event_segmentation","name":"Amplitude Event Segmentation","description":"Query event analytics data with segmentation. Get event counts, uniques, averages, and more.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"eventType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Event type name to analyze"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric type: uniques, totals, pct_dau, average, histogram, sums, value_avg, or formula (default: uniques)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by (prefix custom user properties with \\"gp:\\")"},"groupBy2":{"type":"string","required":false,"visibility":"user-or-llm","description":"Second property name to group by (prefix custom user properties with \\"gp:\\")"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of group-by values (max 1000)"},"filters":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON array of filter objects applied to the event, e.g. [{\\"subprop_type\\":\\"event\\",\\"subprop_key\\":\\"city\\",\\"subprop_op\\":\\"is\\",\\"subprop_value\\":[\\"San Francisco\\"]}]"},"formula":{"type":"string","required":false,"visibility":"user-or-llm","description":"Required when metric is \\"formula\\", e.g. \\"UNIQUES(A)/UNIQUES(B)\\""},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_funnels":{"id":"amplitude_funnels","name":"Amplitude Funnels","description":"Analyze conversion rates and drop-off between a sequence of events.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"events":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON array of event objects, one per funnel step in order, e.g. [{\\"event_type\\":\\"signup\\"},{\\"event_type\\":\\"purchase\\"}]"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"mode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Funnel ordering: \\"ordered\\", \\"unordered\\", or \\"sequential\\" (default: ordered)"},"userType":{"type":"string","required":false,"visibility":"user-or-llm","description":"User type: \\"new\\" or \\"active\\" (default: active)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: -300000 (real-time), -3600000 (hourly), 1 (daily), 7 (weekly), or 30 (monthly)"},"conversionWindowSeconds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Conversion window in seconds (default: 2592000, i.e. 30 days)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property to group by (limit: one; prefix custom properties with \\"gp:\\")"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of group-by values (default: 100, max: 1000)"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_get_active_users":{"id":"amplitude_get_active_users","name":"Amplitude Get Active Users","description":"Get active or new user counts over a date range from the Dashboard REST API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric type: \\"active\\" or \\"new\\" (default: active)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_get_revenue":{"id":"amplitude_get_revenue","name":"Amplitude Get Revenue","description":"Get revenue LTV data including ARPU, ARPPU, total revenue, and paying user counts.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"metric":{"type":"string","required":false,"visibility":"user-or-llm","description":"Metric: 0 (ARPU), 1 (ARPPU), 2 (Total Revenue), 3 (Paying Users)"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property name to group by (limit: one)"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_group_identify":{"id":"amplitude_group_identify","name":"Amplitude Group Identify","description":"Set group-level properties in Amplitude. Supports $set, $setOnce, $add, $append, $unset operations.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"groupType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Group classification (e.g., \\"company\\", \\"org_id\\")"},"groupValue":{"type":"string","required":true,"visibility":"user-or-llm","description":"Specific group identifier (e.g., \\"Acme Corp\\")"},"groupProperties":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON object of group properties. Use operations like $set, $setOnce, $add, $append, $unset."},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_identify_user":{"id":"amplitude_identify_user","name":"Amplitude Identify User","description":"Set user properties in Amplitude using the Identify API. Supports $set, $setOnce, $add, $append, $unset operations.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"User ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"userProperties":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON object of user properties. Use operations like $set, $setOnce, $add, $append, $unset."},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_list_events":{"id":"amplitude_list_events","name":"Amplitude List Events","description":"List all event types in the Amplitude project with their weekly totals and unique counts.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_realtime_active_users":{"id":"amplitude_realtime_active_users","name":"Amplitude Real-time Active Users","description":"Get real-time active user counts at 5-minute granularity for the last 2 days.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_retention":{"id":"amplitude_retention","name":"Amplitude Retention","description":"Measure how many users return to perform an action after a starting action.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"startEvent":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON starting event object, e.g. {\\"event_type\\":\\"_new\\"} or {\\"event_type\\":\\"_active\\"}"},"returnEvent":{"type":"string","required":true,"visibility":"user-or-llm","description":"JSON returning event object, e.g. {\\"event_type\\":\\"_all\\"} or {\\"event_type\\":\\"_active\\"}"},"start":{"type":"string","required":true,"visibility":"user-or-llm","description":"Start date in YYYYMMDD format"},"end":{"type":"string","required":true,"visibility":"user-or-llm","description":"End date in YYYYMMDD format"},"retentionMode":{"type":"string","required":false,"visibility":"user-or-llm","description":"Retention type: \\"bracket\\", \\"rolling\\", or \\"n-day\\" (default: n-day)"},"retentionBrackets":{"type":"string","required":false,"visibility":"user-or-llm","description":"Required when Retention Mode is \\"bracket\\". Day ranges, e.g. [[0,4]]"},"interval":{"type":"string","required":false,"visibility":"user-or-llm","description":"Time interval: 1 (daily), 7 (weekly), or 30 (monthly)"},"groupBy":{"type":"string","required":false,"visibility":"user-or-llm","description":"Property to group by (limit: one; prefix custom properties with \\"gp:\\")"},"segment":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON segment definition(s) applied to the query"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_send_event":{"id":"amplitude_send_event","name":"Amplitude Send Event","description":"Track an event in Amplitude using the HTTP V2 API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"User ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"eventType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the event (e.g., \\"page_view\\", \\"purchase\\")"},"eventProperties":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON object of custom event properties"},"userProperties":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON object of user properties to set (supports $set, $setOnce, $add, $append, $unset)"},"time":{"type":"string","required":false,"visibility":"user-or-llm","description":"Event timestamp in milliseconds since epoch"},"sessionId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Session start time in milliseconds since epoch"},"insertId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Unique ID for deduplication (within 7-day window)"},"appVersion":{"type":"string","required":false,"visibility":"user-or-llm","description":"Application version string"},"platform":{"type":"string","required":false,"visibility":"user-or-llm","description":"Platform (e.g., \\"Web\\", \\"iOS\\", \\"Android\\")"},"country":{"type":"string","required":false,"visibility":"user-or-llm","description":"Two-letter country code"},"language":{"type":"string","required":false,"visibility":"user-or-llm","description":"Language code (e.g., \\"en\\")"},"ip":{"type":"string","required":false,"visibility":"user-or-llm","description":"IP address for geo-location"},"price":{"type":"string","required":false,"visibility":"user-or-llm","description":"Price of the item purchased"},"quantity":{"type":"string","required":false,"visibility":"user-or-llm","description":"Quantity of items purchased"},"revenue":{"type":"string","required":false,"visibility":"user-or-llm","description":"Revenue amount"},"productId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Product identifier"},"revenueType":{"type":"string","required":false,"visibility":"user-or-llm","description":"Revenue type (e.g., \\"purchase\\", \\"refund\\")"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_user_activity":{"id":"amplitude_user_activity","name":"Amplitude User Activity","description":"Get the event stream for a specific user by their Amplitude ID.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"amplitudeId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Amplitude internal user ID"},"offset":{"type":"string","required":false,"visibility":"user-or-llm","description":"Offset for pagination (default 0)"},"limit":{"type":"string","required":false,"visibility":"user-or-llm","description":"Maximum number of events to return (default 1000, max 1000)"},"direction":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort direction: \\"latest\\" or \\"earliest\\" (default: latest)"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"amplitude_user_profile":{"id":"amplitude_user_profile","name":"Amplitude User Profile","description":"Get a user profile including properties, cohort memberships, and computed properties. Not available for EU data-residency projects.","version":"1.0.0","params":{"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"userId":{"type":"string","required":false,"visibility":"user-or-llm","description":"External user ID (required if no device_id)"},"deviceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Device ID (required if no user_id)"},"getAmpProps":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include Amplitude user properties (true/false, default: false)"},"getCohortIds":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include cohort IDs the user belongs to (true/false, default: false)"},"getComputations":{"type":"string","required":false,"visibility":"user-or-llm","description":"Include computed user properties (true/false, default: false)"}},"hostedApiKey":"none"},"amplitude_user_search":{"id":"amplitude_user_search","name":"Amplitude User Search","description":"Search for a user by User ID, Device ID, or Amplitude ID using the Dashboard REST API.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude API Key"},"secretKey":{"type":"string","required":true,"visibility":"user-only","description":"Amplitude Secret Key"},"user":{"type":"string","required":true,"visibility":"user-or-llm","description":"User ID, Device ID, or Amplitude ID to search for"},"dataResidency":{"type":"string","required":false,"visibility":"user-or-llm","description":"Data residency region: \\"us\\" (default) or \\"eu\\""}},"hostedApiKey":"none"},"apify_get_dataset_items":{"id":"apify_get_dataset_items","name":"APIFY Get Dataset Items","description":"Retrieve items stored in an APIFY dataset","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"datasetId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Dataset ID to read items from. Example: \\"9RnD3Pql2vGZkc5H5\\""},"itemLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Max items to return (1-250000). Default: all items. Example: 500"},"offset":{"type":"number","required":false,"visibility":"user-or-llm","description":"Number of items to skip at the start. Default: 0"},"fields":{"type":"string","required":false,"visibility":"user-or-llm","description":"Comma-separated list of fields to include. Example: \\"title,url,price\\""}},"hostedApiKey":"none"},"apify_get_run":{"id":"apify_get_run","name":"APIFY Get Run","description":"Get the status and details of an APIFY actor run","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"runId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor run ID to fetch. Example: \\"HG7ML7M8z78YcAPEB\\""}},"hostedApiKey":"none"},"apify_run_actor_async":{"id":"apify_run_actor_async","name":"APIFY Run Actor (Async)","description":"Run an APIFY actor asynchronously with polling for long-running tasks","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"actorId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor ID or username/actor-name. Examples: \\"apify/web-scraper\\", \\"janedoe/my-actor\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor input as JSON string. Example: { \\"startUrls\\": [ { \\"url\\": \\"https://example.com\\" } ], \\"maxPages\\": 10 }"},"waitForFinish":{"type":"number","required":false,"visibility":"user-or-llm","description":"Initial wait time in seconds (0-60) before polling starts. Example: 30"},"itemLimit":{"type":"number","required":false,"default":100,"visibility":"user-or-llm","description":"Max dataset items to fetch (1-250000). Default: 100. Example: 500"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the actor run (128-32768). Example: 1024 for 1GB, 2048 for 2GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the actor run. Example: 300 for 5 minutes, 3600 for 1 hour"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\", \\"build-tag-name\\""}},"hostedApiKey":"none"},"apify_run_actor_sync":{"id":"apify_run_actor_sync","name":"APIFY Run Actor (Sync)","description":"Run an APIFY actor synchronously and get results (max 5 minutes)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"actorId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Actor ID or username/actor-name. Examples: \\"apify/web-scraper\\", \\"janedoe/my-actor\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor input as JSON string. Example: { \\"startUrls\\": [ { \\"url\\": \\"https://example.com\\" } ], \\"maxPages\\": 10 }"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the actor run (128-32768). Example: 1024 for 1GB, 2048 for 2GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the actor run. Example: 300 for 5 minutes, 3600 for 1 hour"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\", \\"build-tag-name\\""}},"hostedApiKey":"none"},"apify_run_task":{"id":"apify_run_task","name":"APIFY Run Task","description":"Run a saved APIFY actor task synchronously and get dataset items (max 5 minutes)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"APIFY API token from console.apify.com/account#/integrations"},"taskId":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task ID or username/task-name. Examples: \\"janedoe/my-task\\", \\"moJRLRc85AitArpNN\\""},"input":{"type":"string","required":false,"visibility":"user-or-llm","description":"JSON string that overrides the task\'s saved input. Example: { \\"startUrls\\": [ { \\"url\\": \\"https://example.com\\" } ] }"},"itemLimit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Max dataset items to return (1-250000). Example: 500"},"memory":{"type":"number","required":false,"visibility":"user-or-llm","description":"Memory in megabytes allocated for the run (128-32768). Example: 1024 for 1GB"},"timeout":{"type":"number","required":false,"visibility":"user-or-llm","description":"Timeout in seconds for the run. Example: 300 for 5 minutes"},"build":{"type":"string","required":false,"visibility":"user-or-llm","description":"Actor build to run. Examples: \\"latest\\", \\"beta\\", \\"1.2.3\\""}},"hostedApiKey":"none"},"apollo_account_bulk_create":{"id":"apollo_account_bulk_create","name":"Apollo Bulk Create Accounts","description":"Create up to 100 accounts at once in your Apollo database. Set run_dedupe=true to deduplicate by domain, organization_id, and name. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"accounts":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of accounts to create (max 100). Each account should include a name, and may optionally include domain, phone, phone_status_cd, raw_address, owner_id, linkedin_url, facebook_url, twitter_url, salesforce_id, and hubspot_id."},"append_label_names":{"type":"array","required":false,"visibility":"user-only","description":"Array of label names to add to ALL accounts in this request"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"When true, performs aggressive deduplication by domain, organization_id, and name (defaults to false)"}},"hostedApiKey":"none"},"apollo_account_bulk_update":{"id":"apollo_account_bulk_update","name":"Apollo Bulk Update Accounts","description":"Update up to 1000 existing accounts at once in your Apollo database (higher limit than contacts!). Each account must include an id field. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"account_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of account IDs to update with the same values (max 1000). Use with name/owner_id for uniform updates. Use either this OR account_attributes."},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this name to all accounts"},"owner_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this owner to all accounts"},"account_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"When using account_ids, apply this account stage to all accounts"},"account_attributes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of account objects with individual updates (each must include id). Example: [{\\"id\\": \\"acc1\\", \\"name\\": \\"Acme\\", \\"owner_id\\": \\"u1\\", \\"account_stage_id\\": \\"s1\\", \\"typed_custom_fields\\": {\\"field_id\\": \\"value\\"}}]"},"async":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, processes the update asynchronously. Only supported when using account_ids; returns 422 if used with account_attributes."}},"hostedApiKey":"none"},"apollo_account_create":{"id":"apollo_account_create","name":"Apollo Create Account","description":"Create a new account (company) in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Company name (e.g., \\"Acme Corporation\\")"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain without www. prefix (e.g., \\"acme.com\\")"},"phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number for the account"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo user ID of the account owner"},"account_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo ID for the account stage to assign this account to"},"raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate location (e.g., \\"San Francisco, CA, USA\\")"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_account_search":{"id":"apollo_account_search","name":"Apollo Search Accounts","description":"Search your team\'s accounts in Apollo. Display limit: 50,000 records (100 records per page, 500 pages max). Use filters to narrow results. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"q_organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter accounts by organization name (partial-match search)"},"account_stage_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by account stage IDs"},"account_label_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by account label IDs"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"account_last_activity_date\\", \\"account_created_at\\", or \\"account_updated_at\\""},"sort_ascending":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Sort ascending when true. Defaults to descending."},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_account_update":{"id":"apollo_account_update","name":"Apollo Update Account","description":"Update an existing account in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"account_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the account to update (e.g., \\"acc_abc123\\")"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company name (e.g., \\"Acme Corporation\\")"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain (e.g., \\"acme.com\\")"},"phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company phone number"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo user ID of the account owner"},"account_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"Apollo ID for the account stage to assign this account to"},"raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate location (e.g., \\"San Francisco, CA, USA\\")"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_contact_bulk_create":{"id":"apollo_contact_bulk_create","name":"Apollo Bulk Create Contacts","description":"Create up to 100 contacts at once in your Apollo database. Supports deduplication to prevent creating duplicate contacts. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"contacts":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of contacts to create (max 100). Each contact may include first_name, last_name, email, title, organization_name, account_id, owner_id, contact_stage_id, linkedin_url, phone (single string) or phone_numbers (array of {raw_number, position}), contact_emails, typed_custom_fields, and CRM IDs (salesforce_contact_id, hubspot_id, team_id) for cross-system matching"},"append_label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Label names to add to all contacts in this request (e.g., [\\"Hot Lead\\"])"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-only","description":"Enable deduplication to prevent creating duplicate contacts. When true, existing contacts are returned without modification"}},"hostedApiKey":"none"},"apollo_contact_bulk_update":{"id":"apollo_contact_bulk_update","name":"Apollo Bulk Update Contacts","description":"Update up to 100 existing contacts at once in your Apollo database. Each contact must include an id field. Master key required.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"contact_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of contact IDs to update. Must be paired with an object-form contact_attributes specifying the fields to apply uniformly to all listed contacts."},"contact_attributes":{"type":"json","required":false,"visibility":"user-or-llm","description":"Required. Either an array of per-contact updates (each with id) — used standalone — or a single object of attributes to apply to all contact_ids. Supported fields: owner_id, email, organization_name, title, first_name, last_name, account_id, present_raw_address, linkedin_url, typed_custom_fields"},"async":{"type":"boolean","required":false,"visibility":"user-only","description":"Force asynchronous processing. Automatically enabled for >100 contacts"}},"hostedApiKey":"none"},"apollo_contact_create":{"id":"apollo_contact_create","name":"Apollo Create Contact","description":"Create a new contact in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"first_name":{"type":"string","required":true,"visibility":"user-or-llm","description":"First name of the contact"},"last_name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Last name of the contact"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address of the contact"},"title":{"type":"string","required":false,"visibility":"user-or-llm","description":"Job title (e.g., \\"VP of Sales\\", \\"Software Engineer\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo account ID to associate with (e.g., \\"acc_abc123\\")"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the contact owner (accepted by Apollo but not officially documented for POST /contacts)"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the contact\'s employer (e.g., \\"Apollo\\")"},"website_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate website URL (e.g., \\"https://www.apollo.io/\\")"},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Lists/labels to add the contact to (e.g., [\\"Prospects\\"])"},"contact_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the contact stage"},"present_raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal location for the contact (e.g., \\"Atlanta, United States\\")"},"direct_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number"},"corporate_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Work/office phone number"},"mobile_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Mobile phone number"},"home_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Home phone number"},"other_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Alternative phone number"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom field values keyed by custom field ID"},"run_dedupe":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, Apollo deduplicates against existing contacts"}},"hostedApiKey":"none"},"apollo_contact_search":{"id":"apollo_contact_search","name":"Apollo Search Contacts","description":"Search your team\'s contacts in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"q_keywords":{"type":"string","required":false,"visibility":"user-or-llm","description":"Keywords to search for"},"contact_stage_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by contact stage IDs"},"contact_label_ids":{"type":"array","required":false,"visibility":"user-only","description":"Filter by Apollo label IDs (lists)"},"sort_by_field":{"type":"string","required":false,"visibility":"user-only","description":"Sort field: contact_last_activity_date, contact_email_last_opened_at, contact_email_last_clicked_at, contact_created_at, or contact_updated_at"},"sort_ascending":{"type":"boolean","required":false,"visibility":"user-only","description":"When true, sort ascending. Must be used together with sort_by_field"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_contact_update":{"id":"apollo_contact_update","name":"Apollo Update Contact","description":"Update an existing contact in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"contact_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the contact to update (e.g., \\"con_abc123\\")"},"first_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"First name of the contact"},"last_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Last name of the contact"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address"},"title":{"type":"string","required":false,"visibility":"user-or-llm","description":"Job title (e.g., \\"VP of Sales\\", \\"Software Engineer\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo account ID (e.g., \\"acc_abc123\\")"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the contact owner (accepted by Apollo but not officially documented for PATCH /contacts/{id})"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the contact\'s employer (e.g., \\"Apollo\\")"},"website_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"Corporate website URL (e.g., \\"https://www.apollo.io/\\")"},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Lists/labels to add the contact to (e.g., [\\"Prospects\\"])"},"contact_stage_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the contact stage"},"present_raw_address":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal location for the contact (e.g., \\"Atlanta, United States\\")"},"direct_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number"},"corporate_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Work/office phone number"},"mobile_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Mobile phone number"},"home_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Home phone number"},"other_phone":{"type":"string","required":false,"visibility":"user-or-llm","description":"Alternative phone number"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-or-llm","description":"Custom field values keyed by custom field ID"}},"hostedApiKey":"none"},"apollo_email_accounts":{"id":"apollo_email_accounts","name":"Apollo Get Email Accounts","description":"Get list of team\'s linked email accounts in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"}},"hostedApiKey":"none"},"apollo_opportunity_create":{"id":"apollo_opportunity_create","name":"Apollo Create Opportunity","description":"Create a new deal for an account in your Apollo database (master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the opportunity/deal (e.g., \\"Enterprise License - Q1\\")"},"account_id":{"type":"string","required":false,"visibility":"user-or-llm","description":"ID of the account this opportunity belongs to (e.g., \\"acc_abc123\\")"},"amount":{"type":"string","required":false,"visibility":"user-or-llm","description":"Monetary value as a plain number string with no commas or currency symbols"},"opportunity_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the opportunity stage"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the opportunity owner"},"closed_date":{"type":"string","required":false,"visibility":"user-or-llm","description":"Expected close date in YYYY-MM-DD format"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_opportunity_get":{"id":"apollo_opportunity_get","name":"Apollo Get Opportunity","description":"Retrieve complete details of a specific deal/opportunity by ID","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"opportunity_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the opportunity to retrieve (e.g., \\"opp_abc123\\")"}},"hostedApiKey":"none"},"apollo_opportunity_search":{"id":"apollo_opportunity_search","name":"Apollo Search Opportunities","description":"Search and list all deals/opportunities in your team\'s Apollo account","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"amount\\", \\"is_closed\\", or \\"is_won\\""},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_opportunity_update":{"id":"apollo_opportunity_update","name":"Apollo Update Opportunity","description":"Update an existing deal/opportunity in your Apollo database","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"opportunity_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the opportunity to update (e.g., \\"opp_abc123\\")"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Name of the opportunity/deal (e.g., \\"Enterprise License - Q1\\")"},"amount":{"type":"string","required":false,"visibility":"user-or-llm","description":"Monetary value as a plain number string with no commas or currency symbols"},"opportunity_stage_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the opportunity stage"},"owner_id":{"type":"string","required":false,"visibility":"user-only","description":"User ID of the opportunity owner"},"closed_date":{"type":"string","required":false,"visibility":"user-or-llm","description":"Expected close date in YYYY-MM-DD format"},"typed_custom_fields":{"type":"json","required":false,"visibility":"user-only","description":"Custom field values as { custom_field_id: value } map"}},"hostedApiKey":"none"},"apollo_organization_bulk_enrich":{"id":"apollo_organization_bulk_enrich","name":"Apollo Bulk Organization Enrichment","description":"Enrich data for up to 10 organizations at once using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"domains":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of company domains to enrich (max 10, no www. or @, e.g., [\\"apollo.io\\", \\"stripe.com\\"])"}},"hostedApiKey":"none"},"apollo_organization_enrich":{"id":"apollo_organization_enrich","name":"Apollo Organization Enrichment","description":"Enrich data for a single organization using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"domain":{"type":"string","required":true,"visibility":"user-or-llm","description":"Company domain (e.g., \\"apollo.io\\", \\"acme.com\\")"}},"hostedApiKey":"none"},"apollo_organization_search":{"id":"apollo_organization_search","name":"Apollo Organization Search","description":"Search Apollo\'s database for companies using filters","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"organization_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Company HQ locations (cities, US states, or countries)"},"organization_not_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Exclude companies whose HQ is in these locations"},"organization_num_employees_ranges":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employee count ranges as \\"min,max\\" strings (e.g., [\\"1,10\\", \\"250,500\\", \\"10000,20000\\"])"},"q_organization_keyword_tags":{"type":"array","required":false,"visibility":"user-or-llm","description":"Industry or keyword tags"},"q_organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Organization name to search for (e.g., \\"Acme\\", \\"TechCorp\\")"},"organization_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Apollo organization IDs to include (e.g., [\\"5e66b6381e05b4008c8331b8\\"])"},"q_organization_domains_list":{"type":"array","required":false,"visibility":"user-or-llm","description":"Domain names to filter by (no www. or @, up to 1,000)"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_people_bulk_enrich":{"id":"apollo_people_bulk_enrich","name":"Apollo Bulk People Enrichment","description":"Enrich data for up to 10 people at once using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"people":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of people to enrich (max 10)"},"reveal_personal_emails":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal personal email addresses (uses credits)"},"reveal_phone_number":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal phone numbers (uses credits, requires webhook_url)"},"webhook_url":{"type":"string","required":false,"visibility":"user-only","description":"Webhook URL for async phone number delivery (required when reveal_phone_number is true)"}},"hostedApiKey":"none"},"apollo_people_enrich":{"id":"apollo_people_enrich","name":"Apollo People Enrichment","description":"Enrich data for a single person using Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"first_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"First name of the person"},"last_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Last name of the person"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Full name of the person (alternative to first_name/last_name)"},"id":{"type":"string","required":false,"visibility":"user-or-llm","description":"Apollo ID for the person"},"hashed_email":{"type":"string","required":false,"visibility":"user-or-llm","description":"MD5 or SHA-256 hashed email"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Email address of the person"},"organization_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company name where the person works"},"domain":{"type":"string","required":false,"visibility":"user-or-llm","description":"Company domain (e.g., \\"apollo.io\\", \\"acme.com\\")"},"linkedin_url":{"type":"string","required":false,"visibility":"user-or-llm","description":"LinkedIn profile URL"},"reveal_personal_emails":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal personal email addresses (uses credits)"},"reveal_phone_number":{"type":"boolean","required":false,"visibility":"user-only","description":"Reveal phone numbers (uses credits, requires webhook_url)"},"webhook_url":{"type":"string","required":false,"visibility":"user-only","description":"Webhook URL for async phone number delivery (required when reveal_phone_number is true)"}},"hostedApiKey":"none"},"apollo_people_search":{"id":"apollo_people_search","name":"Apollo People Search","description":"Search Apollo\'s database for people using demographic filters","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key"},"person_titles":{"type":"array","required":false,"visibility":"user-or-llm","description":"Job titles to search for (e.g., [\\"CEO\\", \\"VP of Sales\\"])"},"include_similar_titles":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Whether to return people with job titles similar to person_titles"},"person_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Locations to search in (e.g., [\\"San Francisco, CA\\", \\"New York, NY\\"])"},"person_seniorities":{"type":"array","required":false,"visibility":"user-or-llm","description":"Seniority levels (one of: owner, founder, c_suite, partner, vp, head, director, manager, senior, entry, intern)"},"organization_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Apollo organization IDs to filter by (e.g., [\\"5e66b6381e05b4008c8331b8\\"])"},"organization_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Company names to search within (legacy filter)"},"organization_locations":{"type":"array","required":false,"visibility":"user-or-llm","description":"Headquarters locations of the people\'s current employer (e.g., [\'texas\', \'tokyo\', \'spain\'])"},"q_organization_domains_list":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employer domain names (e.g., [\\"apollo.io\\", \\"microsoft.com\\"]) — up to 1,000, no www. or @"},"organization_num_employees_ranges":{"type":"array","required":false,"visibility":"user-or-llm","description":"Employee count ranges for the person\'s current employer. Each entry is \\"min,max\\" (e.g., [\\"1,10\\", \\"250,500\\", \\"10000,20000\\"])"},"contact_email_status":{"type":"array","required":false,"visibility":"user-or-llm","description":"Email statuses to filter by: \\"verified\\", \\"unverified\\", \\"likely to engage\\", \\"unavailable\\""},"q_keywords":{"type":"string","required":false,"visibility":"user-or-llm","description":"Keywords to search for"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination, default 1 (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, default 25, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_sequence_add_contacts":{"id":"apollo_sequence_add_contacts","name":"Apollo Add Contacts to Sequence","description":"Add contacts to an Apollo sequence","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"sequence_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the sequence to add contacts to (e.g., \\"seq_abc123\\")"},"contact_ids":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of contact IDs to add to the sequence (e.g., [\\"con_abc123\\", \\"con_def456\\"]). Either contact_ids or label_names must be provided."},"label_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of label names to identify contacts to add to the sequence. Either contact_ids or label_names must be provided."},"send_email_from_email_account_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the email account to send from. Use the Get Email Accounts operation to look this up."},"send_email_from_email_address":{"type":"string","required":false,"visibility":"user-only","description":"Specific email address to send from within the email account."},"sequence_no_email":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if they have no email address"},"sequence_unverified_email":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts with unverified email addresses"},"sequence_job_change":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts who recently changed jobs"},"sequence_active_in_other_campaigns":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts active in other campaigns"},"sequence_finished_in_other_campaigns":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts who finished other campaigns"},"sequence_same_company_in_same_campaign":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if others from the same company are in the sequence"},"contacts_without_ownership_permission":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts without ownership permission"},"add_if_in_queue":{"type":"boolean","required":false,"visibility":"user-only","description":"Add contacts even if they are in the queue"},"contact_verification_skipped":{"type":"boolean","required":false,"visibility":"user-only","description":"Skip contact verification when adding"},"user_id":{"type":"string","required":false,"visibility":"user-only","description":"ID of the user performing the action"},"status":{"type":"string","required":false,"visibility":"user-only","description":"Initial status for added contacts: \\"active\\" or \\"paused\\""},"auto_unpause_at":{"type":"string","required":false,"visibility":"user-only","description":"ISO 8601 datetime to automatically unpause contacts"}},"hostedApiKey":"none"},"apollo_sequence_search":{"id":"apollo_sequence_search","name":"Apollo Search Sequences","description":"Search for sequences/campaigns in your team\'s Apollo account (master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"q_name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Search sequences by name (e.g., \\"Outbound Q1\\", \\"Follow-up\\")"},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"apollo_task_create":{"id":"apollo_task_create","name":"Apollo Create Task","description":"Create one or more tasks in Apollo (one task per contact_id, master key required)","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"user_id":{"type":"string","required":true,"visibility":"user-or-llm","description":"ID of the Apollo user the task is assigned to"},"contact_ids":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of contact IDs. One task is created per contact."},"priority":{"type":"string","required":false,"visibility":"user-or-llm","description":"Task priority: \\"high\\", \\"medium\\", or \\"low\\" (defaults to \\"medium\\")"},"due_at":{"type":"string","required":true,"visibility":"user-or-llm","description":"Due date/time in ISO 8601 format (e.g., \\"2024-12-31T23:59:59Z\\")"},"type":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task type: \\"call\\", \\"outreach_manual_email\\", \\"linkedin_step_connect\\", \\"linkedin_step_message\\", \\"linkedin_step_view_profile\\", \\"linkedin_step_interact_post\\", or \\"action_item\\""},"status":{"type":"string","required":true,"visibility":"user-or-llm","description":"Task status: \\"scheduled\\", \\"completed\\", or \\"skipped\\""},"note":{"type":"string","required":false,"visibility":"user-or-llm","description":"Free-form note providing context for the task"}},"hostedApiKey":"none"},"apollo_task_search":{"id":"apollo_task_search","name":"Apollo Search Tasks","description":"Search for tasks in Apollo","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Apollo API key (master key required)"},"sort_by_field":{"type":"string","required":false,"visibility":"user-or-llm","description":"Sort field: \\"task_due_at\\" or \\"task_priority\\""},"open_factor_names":{"type":"array","required":false,"visibility":"user-or-llm","description":"Filter by status. Common values: [\\"task_types\\"] for open tasks, [\\"task_completed_at\\"] for completed tasks."},"page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Page number for pagination (e.g., 1, 2, 3)"},"per_page":{"type":"number","required":false,"visibility":"user-or-llm","description":"Results per page, max 100 (e.g., 25, 50, 100)"}},"hostedApiKey":"none"},"appconfig_create_application":{"id":"appconfig_create_application","name":"AppConfig Create Application","description":"Create an application in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the application to create"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the application"}},"hostedApiKey":"none"},"appconfig_create_configuration_profile":{"id":"appconfig_create_configuration_profile","name":"AppConfig Create Configuration Profile","description":"Create a configuration profile in an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to create the configuration profile in"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the configuration profile"},"locationUri":{"type":"string","required":true,"visibility":"user-or-llm","description":"Where the configuration is stored. Use \\"hosted\\" for AppConfig-hosted configurations, or an SSM/S3 URI"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the configuration profile"},"retrievalRoleArn":{"type":"string","required":false,"visibility":"user-or-llm","description":"ARN of an IAM role to retrieve the configuration (required for non-hosted URIs)"},"type":{"type":"string","required":false,"visibility":"user-or-llm","description":"Profile type: AWS.Freeform (default) or AWS.AppConfig.FeatureFlags"}},"hostedApiKey":"none"},"appconfig_create_environment":{"id":"appconfig_create_environment","name":"AppConfig Create Environment","description":"Create an environment for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to create the environment in"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the environment to create"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the environment"}},"hostedApiKey":"none"},"appconfig_create_hosted_configuration_version":{"id":"appconfig_create_hosted_configuration_version","name":"AppConfig Create Hosted Configuration Version","description":"Create a new hosted configuration version for an AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to add the version to"},"content":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration content (e.g., a JSON or YAML document)"},"contentType":{"type":"string","required":true,"visibility":"user-or-llm","description":"Content type of the configuration (e.g., application/json, text/plain)"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the configuration version"},"latestVersionNumber":{"type":"number","required":false,"visibility":"user-or-llm","description":"The version number of the latest version, used for optimistic concurrency"},"versionLabel":{"type":"string","required":false,"visibility":"user-or-llm","description":"A user-defined label for the configuration version"}},"hostedApiKey":"none"},"appconfig_delete_application":{"id":"appconfig_delete_application","name":"AppConfig Delete Application","description":"Delete an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_configuration_profile":{"id":"appconfig_delete_configuration_profile","name":"AppConfig Delete Configuration Profile","description":"Delete an AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_environment":{"id":"appconfig_delete_environment","name":"AppConfig Delete Environment","description":"Delete an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to delete"}},"hostedApiKey":"none"},"appconfig_delete_hosted_configuration_version":{"id":"appconfig_delete_hosted_configuration_version","name":"AppConfig Delete Hosted Configuration Version","description":"Delete a specific hosted configuration version from an AppConfig profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID that owns the version"},"versionNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The version number to delete"}},"hostedApiKey":"none"},"appconfig_get_application":{"id":"appconfig_get_application","name":"AppConfig Get Application","description":"Get details about a single AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_configuration":{"id":"appconfig_get_configuration","name":"AppConfig Get Configuration","description":"Retrieve the latest deployed configuration for an AppConfig application, environment, and profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID or name to retrieve configuration for"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID or name to retrieve configuration for"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID or name to retrieve"}},"hostedApiKey":"none"},"appconfig_get_configuration_profile":{"id":"appconfig_get_configuration_profile","name":"AppConfig Get Configuration Profile","description":"Get details about a single AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_deployment":{"id":"appconfig_get_deployment","name":"AppConfig Get Deployment","description":"Get details about a specific AWS AppConfig deployment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployment"},"deploymentNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The sequence number of the deployment"}},"hostedApiKey":"none"},"appconfig_get_environment":{"id":"appconfig_get_environment","name":"AppConfig Get Environment","description":"Get details about a single AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to retrieve"}},"hostedApiKey":"none"},"appconfig_get_hosted_configuration_version":{"id":"appconfig_get_hosted_configuration_version","name":"AppConfig Get Hosted Configuration Version","description":"Retrieve a specific hosted configuration version from an AppConfig profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to read the version from"},"versionNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The version number to retrieve"}},"hostedApiKey":"none"},"appconfig_list_applications":{"id":"appconfig_list_applications","name":"AppConfig List Applications","description":"List applications in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of applications to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_configuration_profiles":{"id":"appconfig_list_configuration_profiles","name":"AppConfig List Configuration Profiles","description":"List configuration profiles for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profiles"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of configuration profiles to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_deployment_strategies":{"id":"appconfig_list_deployment_strategies","name":"AppConfig List Deployment Strategies","description":"List deployment strategies available in AWS AppConfig","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of deployment strategies to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_deployments":{"id":"appconfig_list_deployments","name":"AppConfig List Deployments","description":"List deployments for an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployments"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployments"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of deployments to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_environments":{"id":"appconfig_list_environments","name":"AppConfig List Environments","description":"List environments for an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environments"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of environments to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_list_hosted_configuration_versions":{"id":"appconfig_list_hosted_configuration_versions","name":"AppConfig List Hosted Configuration Versions","description":"List hosted configuration versions for an AWS AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to list versions for"},"maxResults":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of versions to return (1-50)"},"nextToken":{"type":"string","required":false,"visibility":"user-or-llm","description":"Pagination token from a previous response"}},"hostedApiKey":"none"},"appconfig_start_deployment":{"id":"appconfig_start_deployment","name":"AppConfig Start Deployment","description":"Start deploying a configuration version to an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to deploy in"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to deploy to"},"deploymentStrategyId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The deployment strategy ID to use"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to deploy"},"configurationVersion":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration version to deploy"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"Description of the deployment"}},"hostedApiKey":"none"},"appconfig_stop_deployment":{"id":"appconfig_stop_deployment","name":"AppConfig Stop Deployment","description":"Stop an in-progress AWS AppConfig deployment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID of the deployment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID of the deployment"},"deploymentNumber":{"type":"number","required":true,"visibility":"user-or-llm","description":"The sequence number of the deployment to stop"}},"hostedApiKey":"none"},"appconfig_update_application":{"id":"appconfig_update_application","name":"AppConfig Update Application","description":"Update the name or description of an AWS AppConfig application","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the application"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the application"}},"hostedApiKey":"none"},"appconfig_update_configuration_profile":{"id":"appconfig_update_configuration_profile","name":"AppConfig Update Configuration Profile","description":"Update the name, description, or retrieval role of an AppConfig configuration profile","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the configuration profile"},"configurationProfileId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The configuration profile ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the configuration profile"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the configuration profile"},"retrievalRoleArn":{"type":"string","required":false,"visibility":"user-or-llm","description":"New ARN of the IAM role used to retrieve the configuration"}},"hostedApiKey":"none"},"appconfig_update_environment":{"id":"appconfig_update_environment","name":"AppConfig Update Environment","description":"Update the name or description of an AWS AppConfig environment","version":"1.0","params":{"region":{"type":"string","required":true,"visibility":"user-only","description":"AWS region (e.g., us-east-1)"},"accessKeyId":{"type":"string","required":true,"visibility":"user-only","description":"AWS access key ID"},"secretAccessKey":{"type":"string","required":true,"visibility":"user-only","description":"AWS secret access key"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The application ID that owns the environment"},"environmentId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The environment ID to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"New name for the environment"},"description":{"type":"string","required":false,"visibility":"user-or-llm","description":"New description for the environment"}},"hostedApiKey":"none"},"arxiv_get_author_papers":{"id":"arxiv_get_author_papers","name":"ArXiv Get Author Papers","description":"Search for papers by a specific author on ArXiv.","version":"1.0.0","params":{"authorName":{"type":"string","required":true,"visibility":"user-or-llm","description":"Author name to search for"},"maxResults":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 10, max: 2000)"}},"hostedApiKey":"none"},"arxiv_get_paper":{"id":"arxiv_get_paper","name":"ArXiv Get Paper","description":"Get detailed information about a specific ArXiv paper by its ID.","version":"1.0.0","params":{"paperId":{"type":"string","required":true,"visibility":"user-or-llm","description":"ArXiv paper ID (e.g., \\"1706.03762\\")"}},"hostedApiKey":"none"},"arxiv_search":{"id":"arxiv_search","name":"ArXiv Search","description":"Search for academic papers on ArXiv by keywords, authors, titles, or other fields.","version":"1.0.0","params":{"searchQuery":{"type":"string","required":true,"visibility":"user-or-llm","description":"The search query to execute"},"searchField":{"type":"string","required":false,"visibility":"user-only","description":"Field to search in: all, ti (title), au (author), abs (abstract), co (comment), jr (journal), cat (category), rn (report number)"},"maxResults":{"type":"number","required":false,"visibility":"user-only","description":"Maximum number of results to return (default: 10, max: 2000)"},"sortBy":{"type":"string","required":false,"visibility":"user-only","description":"Sort by: relevance, lastUpdatedDate, submittedDate (default: relevance)"},"sortOrder":{"type":"string","required":false,"visibility":"user-only","description":"Sort order: ascending, descending (default: descending)"}},"hostedApiKey":"none"},"asana_add_comment":{"id":"asana_add_comment","name":"Asana Add Comment","description":"Add a comment (story) to an Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana task GID (numeric string)"},"text":{"type":"string","required":true,"visibility":"user-or-llm","description":"The text content of the comment"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_add_followers":{"id":"asana_add_followers","name":"Asana Add Followers","description":"Add one or more followers to an Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana task (numeric string)"},"followers":{"type":"array","required":true,"visibility":"user-or-llm","description":"Array of user GIDs to add as followers to the task"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_project":{"id":"asana_create_project","name":"Asana Create Project","description":"Create a new project in an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) where the project will be created"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the project"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the project"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_section":{"id":"asana_create_section","name":"Asana Create Section","description":"Create a new section in an Asana project","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana project (numeric string) to add the section to"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the section"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_subtask":{"id":"asana_create_subtask","name":"Asana Create Subtask","description":"Create a subtask under an existing Asana task","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the parent Asana task (numeric string)"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the subtask"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the subtask"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"User GID to assign the subtask to"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_create_task":{"id":"asana_create_task","name":"Asana Create Task","description":"Create a new task in Asana","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) where the task will be created"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"Name of the task"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Notes or description for the task"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"User GID to assign the task to"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_delete_task":{"id":"asana_delete_task","name":"Asana Delete Task","description":"Delete an Asana task by its GID (moves it to the trash)","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana task to delete (numeric string)"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_project":{"id":"asana_get_project","name":"Asana Get Project","description":"Retrieve a single Asana project by its GID","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana project GID (numeric string) to retrieve"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_projects":{"id":"asana_get_projects","name":"Asana Get Projects","description":"Retrieve all projects from an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to retrieve projects from"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_get_task":{"id":"asana_get_task","name":"Asana Get Task","description":"Retrieve a single task by GID or get multiple tasks with filters","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":false,"visibility":"user-or-llm","description":"The globally unique identifier (GID) of the task. If not provided, will get multiple tasks."},"workspace":{"type":"string","required":false,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to filter tasks (required when not using taskGid)"},"project":{"type":"string","required":false,"visibility":"user-or-llm","description":"Asana project GID (numeric string) to filter tasks"},"limit":{"type":"number","required":false,"visibility":"user-or-llm","description":"Maximum number of tasks to return (default: 50)"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_list_sections":{"id":"asana_list_sections","name":"Asana List Sections","description":"List all sections in an Asana project","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"projectGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"GID of the Asana project (numeric string) to list sections from"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_list_workspaces":{"id":"asana_list_workspaces","name":"Asana List Workspaces","description":"List all Asana workspaces and organizations the authenticated user belongs to","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_search_tasks":{"id":"asana_search_tasks","name":"Asana Search Tasks","description":"Search for tasks in an Asana workspace","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"workspace":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana workspace GID (numeric string) to search tasks in"},"text":{"type":"string","required":false,"visibility":"user-or-llm","description":"Text to search for in task names"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"Filter tasks by assignee user GID"},"projects":{"type":"array","required":false,"visibility":"user-or-llm","description":"Array of Asana project GIDs (numeric strings) to filter tasks by"},"completed":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Filter by completion status"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"asana_update_task":{"id":"asana_update_task","name":"Asana Update Task","description":"Update an existing task in Asana","version":"1.0.0","params":{"accessToken":{"type":"string","required":true,"visibility":"hidden","description":"OAuth access token for Asana"},"taskGid":{"type":"string","required":true,"visibility":"user-or-llm","description":"Asana task GID (numeric string) of the task to update"},"name":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated name for the task"},"notes":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated notes or description for the task"},"assignee":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated assignee user GID"},"completed":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Mark task as completed or not completed"},"due_on":{"type":"string","required":false,"visibility":"user-or-llm","description":"Updated due date in YYYY-MM-DD format"}},"oauth":{"required":true,"provider":"asana"},"hostedApiKey":"none"},"ashby_add_candidate_tag":{"id":"ashby_add_candidate_tag","name":"Ashby Add Candidate Tag","description":"Adds a tag to a candidate in Ashby and returns the updated candidate.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to add the tag to"},"tagId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the tag to add"}},"hostedApiKey":"none"},"ashby_anonymize_candidate":{"id":"ashby_anonymize_candidate","name":"Ashby Anonymize Candidate","description":"Strips personally identifiable information from a candidate in Ashby. This does not delete the candidate - the record and its applications remain, with the PII removed. Ashby exposes no candidate deletion endpoint; true deletion is UI-only, restricted by role, and limited to a 10-day window. Requires the candidatesWrite permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"UUID of the candidate to anonymize"}},"hostedApiKey":"none"},"ashby_change_application_source":{"id":"ashby_change_application_source","name":"Ashby Change Application Source","description":"Changes the source attributed to an existing application, so programmatically created applications report correctly on the recruiting side. Requires the candidatesWrite permission.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"UUID of the application whose source should change"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to attribute the application to, as returned by List Sources. Omit only when unsetSource is true."},"unsetSource":{"type":"boolean","required":false,"visibility":"user-or-llm","description":"Set true to deliberately clear the application source. Required to unset, so that a missing or empty sourceId cannot wipe attribution by accident."}},"hostedApiKey":"none"},"ashby_change_application_stage":{"id":"ashby_change_application_stage","name":"Ashby Change Application Stage","description":"Moves an application to a different interview stage. Requires an archive reason when moving to an Archived stage.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"applicationId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the application to update the stage of"},"interviewStageId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the interview stage to move the application to"},"archiveReasonId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Archive reason UUID. Required when moving to an Archived stage, ignored otherwise"},"archiveEmail":{"type":"json","required":false,"visibility":"user-or-llm","description":"Archive email configuration with communicationTemplateId and optional sendAt ISO 8601 timestamp. Pass null or omit to send no archive email."}},"hostedApiKey":"none"},"ashby_create_application":{"id":"ashby_create_application","name":"Ashby Create Application","description":"Creates a new application for a candidate on a job. Optionally specify interview plan, stage, source, and credited user.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to consider for the job"},"jobId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the job to consider the candidate for"},"interviewPlanId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the interview plan to use (defaults to the job default plan)"},"interviewStageId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the interview stage to place the application in, or FirstPreInterviewScreen (defaults to the first Lead stage)"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to set on the application"},"creditedToUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the user the application is credited to"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"ISO 8601 timestamp to set as the application creation date (defaults to now)"},"applicationHistory":{"type":"json","required":false,"visibility":"user-or-llm","description":"Optional documented application history entries to create with the application"}},"hostedApiKey":"none"},"ashby_create_candidate":{"id":"ashby_create_candidate","name":"Ashby Create Candidate","description":"Creates a new candidate record in Ashby.","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"name":{"type":"string","required":true,"visibility":"user-or-llm","description":"The candidate full name"},"email":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary email address for the candidate"},"phoneNumber":{"type":"string","required":false,"visibility":"user-or-llm","description":"Primary phone number for the candidate"},"linkedInUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"LinkedIn profile URL"},"githubUrl":{"type":"string","required":false,"visibility":"user-or-llm","description":"GitHub profile URL"},"website":{"type":"string","required":false,"visibility":"user-or-llm","description":"Personal website URL"},"sourceId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the source to attribute the candidate to"},"creditedToUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"UUID of the Ashby user to credit with sourcing this candidate"},"createdAt":{"type":"string","required":false,"visibility":"user-or-llm","description":"Backdated creation timestamp in ISO 8601 (e.g. 2024-01-01T00:00:00Z). Defaults to now."},"alternateEmailAddresses":{"type":"json","required":false,"visibility":"user-or-llm","description":"Array of additional email address strings to add to the candidate, e.g. [\\"a@x.com\\",\\"b@y.com\\"]"},"location":{"type":"json","required":false,"visibility":"user-or-llm","description":"Candidate location object with optional city, region, and country"}},"hostedApiKey":"none"},"ashby_create_note":{"id":"ashby_create_note","name":"Ashby Create Note","description":"Creates a note on a candidate in Ashby. Supports plain text and HTML content (bold, italic, underline, links, lists, code).","version":"1.0.0","params":{"apiKey":{"type":"string","required":true,"visibility":"user-only","description":"Ashby API Key"},"onBehalfOfUserId":{"type":"string","required":false,"visibility":"user-or-llm","description":"Active Ashby user UUID to attribute this mutation to; the API key must permit on-behalf-of calls"},"candidateId":{"type":"string","required":true,"visibility":"user-or-llm","description":"The UUID of the candidate to add the note to"},"note":{"type":"string","required":true,"visibility":"user-or-llm","description":"The note content. If noteType is text/html, supports: , , , ,