[pull] main from LibreChat-AI:main

README.md

@@ -24,6 +24,40 @@ Code Interpreter (internally `codeapi`, the prefix used by its env vars, images,
 4. Files are persisted/retrieved via the **File Server** (backed by S3)
 5. Tool calls from within sandboxes are routed through the **Tool Call Server**
 
+## Execution profiles
+
+Code API can run two isolated deployments at the same time:
+
+- `default`: the AWS-free HTTP/libkrun path, with stateless executions.
+- `stateful`: the AWS Lambda MicroVM path, with runtime-session affinity.
+
+Set `CODEAPI_EXECUTION_PROFILE` consistently on an API deployment and its
+workers. The default profile keeps the existing `python-queue` and
+`other-queue`; the stateful profile uses `stateful-python-queue` and
+`stateful-other-queue`. This allows both deployments to share Redis without
+cross-consuming jobs.
+
+An existing Lambda MicroVM deployment upgraded from a pre-profile release may
+leave `CODEAPI_EXECUTION_PROFILE` unset for its first binary rollout. An
+affinity/strict deployment still identifies itself as `stateful`; a stateless
+Lambda deployment identifies itself as `default`. Both temporarily keep the
+legacy queue names so separately deployed APIs and workers remain compatible
+with old binaries.
+Move that deployment to the isolated stateful queues with a blue/green cutover:
+start replacement API and worker pools with the profile explicitly set to
+`stateful`, verify them together, switch the stateful endpoint, and drain the
+legacy pool. For rollback, switch the endpoint back before stopping the
+replacement pool. Do not run the inferred stateful compatibility mode beside a
+default deployment on the same Redis because both use the legacy queues.
+
+Trusted callers should send `X-CodeAPI-Expected-Profile: default|stateful` on
+every Code API request. A request that reaches the wrong deployment fails
+before enqueue with HTTP 409 and `error=execution_profile_mismatch`; every
+response advertises the actual deployment in `X-CodeAPI-Execution-Profile`.
+Omitting the expected-profile header remains supported for older clients, but
+provides no wrong-endpoint protection. There is deliberately no silent
+fallback between profiles and no automatic workspace or file migration.
+
 ## Sandbox Isolation
 
 Two modes are supported:

docs/lambda-microvm/README.md

@@ -246,6 +246,7 @@ builds.
 
 ```bash
 CODEAPI_SANDBOX_BACKEND=lambda-microvm
+CODEAPI_EXECUTION_PROFILE=stateful
 CODEAPI_RUNTIME_SESSION_MODE=affinity          # warm sessions + checkpoints
 LAMBDA_MICROVM_IMAGE_ARN=<from step 4>
 LAMBDA_MICROVM_IMAGE_VERSION=<exact version from step 4>  # required for affinity/strict
@@ -310,6 +311,7 @@ appear in `api/src/config.ts`.
 | Env | Default | Meaning |
 |---|---|---|
 | `CODEAPI_SANDBOX_BACKEND` | `http` | `http` (byte-identical to today) or `lambda-microvm`. |
+| `CODEAPI_EXECUTION_PROFILE` | inferred | `default` for the HTTP/stateless deployment or `stateful` for the Lambda affinity/strict deployment. An explicit `stateful` value selects isolated BullMQ queues. Inferred affinity/strict and legacy Lambda/stateless deployments keep the legacy queues only for a pre-profile binary rollout and must not share Redis with the default deployment. |
 | `CODEAPI_RUNTIME_SESSION_MODE` | `stateless` | `stateless` \| `affinity` \| `strict`. `affinity` and `strict` require the `lambda-microvm` backend. See [Operating modes](#operating-modes). |
 | `CODEAPI_RUNTIME_SESSION_LOCK_WAIT_MS` | `15000` | How long a stateful execution waits for the session lock before returning `RUNTIME_SESSION_BUSY` (HTTP 409). |
 
@@ -408,9 +410,11 @@ You do not have to adopt the whole stack at once. The knobs compose:
 **No AWS at all.** Leave `CODEAPI_SANDBOX_BACKEND` unset (`http`). Today's
 behavior, no MicroVMs, no changes needed anywhere.
 
-**MicroVM isolation without sessions.** `lambda-microvm` + `stateless`. Every
-execution gets a fresh, strongly-isolated Firecracker VM. No registry, no
-checkpoints, no session workspace. Simplest way to get the isolation boundary.
+**MicroVM isolation without sessions.** `lambda-microvm` + `stateless`, with
+`CODEAPI_EXECUTION_PROFILE` unset for compatibility. Every execution gets a
+fresh, strongly-isolated Firecracker VM. No registry, no checkpoints, no
+session workspace. This legacy profile uses the shared queue names and must
+not share Redis with a separate default deployment.
 
 **Base container image and snapshot boundary.** The default runner uses a stock
 `oven/bun` base and is **hookless** — session mode arrives per request via the

helm/codeapi/README.md

@@ -53,6 +53,54 @@ platform rather than templated here: external ingress/service mesh, KEDA-style
 queue-depth autoscaling, and cloud-IAM secret delivery (the env hooks below
 cover all of them).
 
+**Execution profile.** By default this chart leaves
+`CODEAPI_EXECUTION_PROFILE` unset. Its bundled HTTP/stateless configuration is
+inferred as the AWS-free `default` profile and retains the existing
+`python-queue` / `other-queue` BullMQ names. Set `executionProfile: default`
+explicitly when deploying it beside a stateful stack. A separate stateful
+Lambda MicroVM deployment must use `executionProfile: stateful`; it then
+consumes `stateful-python-queue` / `stateful-other-queue`, so both stacks may
+safely share Redis without consuming each other's jobs. Do not mix API and
+worker profile values within one deployment.
+
+The chart does not provision Lambda MicroVM infrastructure. Supply its
+runtime settings to both the API and worker (and AWS credentials or workload
+identity to the worker) through the existing environment hooks, for example:
+
+```yaml
+executionProfile: stateful
+api:
+  extraEnv:
+    - name: CODEAPI_RUNTIME_SESSION_MODE
+      value: affinity
+workerSandbox:
+  extraEnv:
+    - name: CODEAPI_SANDBOX_BACKEND
+      value: lambda-microvm
+    - name: CODEAPI_RUNTIME_SESSION_MODE
+      value: affinity
+    - name: LAMBDA_MICROVM_IMAGE_ARN
+      value: arn:aws:lambda:REGION:ACCOUNT:microvm-image:NAME
+    - name: LAMBDA_MICROVM_IMAGE_VERSION
+      value: "VERSION"
+```
+
+The worker also needs the remaining Lambda networking, checkpoint-store, and
+hardening variables documented in `docs/lambda-microvm/README.md`. This chart
+still renders its bundled sandbox-runner, though a Lambda worker does not call
+it; a platform-specific stateful deployment may omit that component.
+
+For an existing affinity/strict deployment from before execution profiles,
+first roll the new binary to API and worker pods with
+`CODEAPI_EXECUTION_PROFILE` still unset. The inferred stateful compatibility
+mode deliberately retains the legacy queues, so old and new binaries can
+overlap. Then create a replacement deployment with the profile explicitly set
+to `stateful`, verify its API and workers together, switch the stateful ingress,
+and drain the legacy deployment. Roll back by switching ingress to the legacy
+deployment before removing the replacement. Never share Redis between the
+inferred compatibility deployment and a default deployment: both consume the
+legacy queues.
+
 **Authentication.** Outside local mode the API verifies JWTs. Configure the
 verifier through environment variables on the api component, e.g.:
 

helm/codeapi/templates/api-deployment.yaml

@@ -41,6 +41,10 @@ spec:
 {{ include "codeapi.otel.env" (dict "root" . "serviceName" "aiml-codeapi-api") | nindent 12 }}
             - name: CODEAPI_HARDENED_SANDBOX_MODE
               value: {{ .Values.hardenedSandboxMode | quote }}
+            {{- with .Values.executionProfile }}
+            - name: CODEAPI_EXECUTION_PROFILE
+              value: {{ . | quote }}
+            {{- end }}
             # Redis connection
             - name: REDIS_HOST
               value: {{ include "codeapi.redis.host" . }}

helm/codeapi/templates/worker-sandbox-deployment.yaml

@@ -147,6 +147,10 @@ spec:
 {{ include "codeapi.otel.env" (dict "root" . "serviceName" "aiml-codeapi-service-worker") | nindent 12 }}
             - name: CODEAPI_HARDENED_SANDBOX_MODE
               value: {{ .Values.hardenedSandboxMode | quote }}
+            {{- with .Values.executionProfile }}
+            - name: CODEAPI_EXECUTION_PROFILE
+              value: {{ . | quote }}
+            {{- end }}
             - name: SANDBOX_ENDPOINT
               value: "http://{{ include "codeapi.fullname" . }}-sandbox-runner:{{ .Values.workerSandbox.sandbox.port }}/api/v2"
             - name: EGRESS_GATEWAY_URL

helm/codeapi/values.yaml

@@ -22,6 +22,12 @@ internalServiceAuth:
 
 hardenedSandboxMode: true
 
+# Stable identity advertised by this API/worker deployment. Leave empty for a
+# backwards-compatible inferred profile. Set explicitly to `default` or
+# `stateful` when deploying both stacks against shared Redis; explicit
+# `stateful` selects isolated BullMQ queues.
+executionProfile: ""
+
 otel:
   enabled: false
   # OTLP/HTTP collector endpoint, e.g. "http://opentelemetry-collector.observability:4318".

service/openapi.yml

@@ -2,7 +2,10 @@ openapi: '3.0.0'
 info:
   title: LibreChat Code Interpreter API
   version: '1.0.0'
-  description: API for sandbox code execution and file management
+  description: >-
+    API for sandbox code execution and file management. Trusted callers should
+    assert the intended deployment with X-CodeAPI-Expected-Profile on every
+    request; responses advertise the actual profile.
 servers:
   - url: https://api.librechat.ai/v1
     description: LibreChat API server
@@ -17,6 +20,49 @@ components:
       scheme: bearer
       bearerFormat: JWT
 
+  parameters:
+    ExpectedExecutionProfile:
+      name: X-CodeAPI-Expected-Profile
+      in: header
+      required: false
+      description: >-
+        Trusted routing assertion. A mismatched endpoint returns HTTP 409
+        before any work is enqueued. Optional only for backwards compatibility.
+      schema:
+        type: string
+        enum: [default, stateful]
+
+  headers:
+    ExecutionProfile:
+      description: Execution profile served by this deployment.
+      schema:
+        type: string
+        enum: [default, stateful]
+
+  responses:
+    BadRequest:
+      description: Invalid request or invalid expected execution profile
+      headers:
+        X-CodeAPI-Execution-Profile:
+          $ref: '#/components/headers/ExecutionProfile'
+      content:
+        application/json:
+          schema:
+            anyOf:
+              - $ref: '#/components/schemas/Error'
+              - $ref: '#/components/schemas/ExecutionProfileError'
+    Conflict:
+      description: Request conflict or execution-profile mismatch
+      headers:
+        X-CodeAPI-Execution-Profile:
+          $ref: '#/components/headers/ExecutionProfile'
+      content:
+        application/json:
+          schema:
+            anyOf:
+              - $ref: '#/components/schemas/Error'
+              - $ref: '#/components/schemas/ExecutionProfileError'
+
   schemas:
     FileRef:
       type: object
@@ -108,6 +154,14 @@ components:
           type: array
           items:
             $ref: '#/components/schemas/RequestFile'
+        runtime_session_hint:
+          type: string
+          maxLength: 128
+          pattern: '^[A-Za-z0-9._:-]+$'
+          description: >-
+            Stable opaque hint for stateful runtime reuse. The server binds it
+            to the authenticated tenant and user. Required in strict runtime
+            session mode and ignored by the default stateless profile.
 
     FileObject:
       type: object
@@ -155,13 +209,32 @@ components:
           type: string
         details:
           type: string
+        message:
+          type: string
+
+    ExecutionProfileError:
+      type: object
+      required: [error, message, actual_profile]
+      properties:
+        error:
+          type: string
+          enum: [invalid_execution_profile, execution_profile_mismatch]
+        message:
+          type: string
+        expected_profile:
+          type: string
+        actual_profile:
+          type: string
+          enum: [default, stateful]
 
 paths:
   /exec:
     post:
       summary: Execute code
       description: Execute code with specified language and parameters
       operationId: executeCode
+      parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
       requestBody:
         required: true
         content:
@@ -171,6 +244,9 @@ paths:
       responses:
         '200':
           description: Successful execution
+          headers:
+            X-CodeAPI-Execution-Profile:
+              $ref: '#/components/headers/ExecutionProfile'
           content:
             application/json:
               schema:
@@ -181,6 +257,10 @@ paths:
             application/json:
               schema:
                 $ref: '#/components/schemas/Error'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'
         '503':
           description: Service unavailable
           content:
@@ -192,6 +272,7 @@ paths:
     get:
       summary: Download a file
       parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
         - name: session_id
           in: path
           required: true
@@ -205,6 +286,9 @@ paths:
       responses:
         '200':
           description: File content
+          headers:
+            X-CodeAPI-Execution-Profile:
+              $ref: '#/components/headers/ExecutionProfile'
           content:
             application/octet-stream:
               schema:
@@ -216,10 +300,16 @@ paths:
             application/json:
               schema:
                 $ref: '#/components/schemas/Error'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'
 
   /upload:
     post:
       summary: Upload files
+      parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
       requestBody:
         required: true
         content:
@@ -237,6 +327,9 @@ paths:
       responses:
         '200':
           description: Successful upload
+          headers:
+            X-CodeAPI-Execution-Profile:
+              $ref: '#/components/headers/ExecutionProfile'
           content:
             application/json:
               schema:
@@ -247,11 +340,16 @@ paths:
             application/json:
               schema:
                 $ref: '#/components/schemas/Error'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'
 
   /files/{session_id}:
     get:
       summary: Get files information
       parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
         - name: session_id
           in: path
           required: true
@@ -265,17 +363,25 @@ paths:
       responses:
         '200':
           description: Files information
+          headers:
+            X-CodeAPI-Execution-Profile:
+              $ref: '#/components/headers/ExecutionProfile'
           content:
             application/json:
               schema:
                 type: array
                 items:
                   $ref: '#/components/schemas/FileObject'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'
 
   /files/{session_id}/{fileId}:
     delete:
       summary: Delete a file
       parameters:
+        - $ref: '#/components/parameters/ExpectedExecutionProfile'
         - name: session_id
           in: path
           required: true
@@ -289,9 +395,16 @@ paths:
       responses:
         '200':
           description: File deleted successfully
+          headers:
+            X-CodeAPI-Execution-Profile:
+              $ref: '#/components/headers/ExecutionProfile'
         '500':
           description: Error deleting file
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/Error'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '409':
+          $ref: '#/components/responses/Conflict'