feat(jobs): add interceptors, HTTP routes, validation, and streaming for jobs plugin

Signed-off-by: Atila Fassina <atila@fassina.eu>

Author: atilafassina

.claude/skills/install-appkit-tarball.md

@@ -0,0 +1,126 @@
+---
+name: install-appkit-tarball
+description: "Install appkit tgz builds into a project. Downloads from the prepare-release CI workflow (default) or builds locally. Use when testing appkit release candidates, installing pre-release appkit, or when user says 'install appkit tarball', 'install local appkit', 'use local appkit', 'link appkit tgz', 'install appkit from CI'."
+user-invocable: true
+allowed-tools: Read, Edit, Bash, Glob
+---
+
+# Install AppKit Tarball
+
+Installs AppKit tgz packages into a project from either the CI prepare-release workflow or a local build.
+
+## Arguments
+
+Parse the user's input to determine mode and options:
+
+- **No args** → CI mode, latest successful prepare-release run, install in CWD
+- **GitHub Actions URL** (contains `actions/runs/`) → CI mode, extract run ID from URL
+- **Numeric run ID** → CI mode, that specific run
+- **`--local <path>`** → Local build mode from the given appkit repo path
+- **`--dir <path>`** (combinable with any above) → Override install target directory (default: CWD)
+
+## Workflow
+
+Follow these steps exactly:
+
+### Step 1: Determine target directory
+
+If `--dir` was provided, use that path. Otherwise use the current working directory.
+Verify `package.json` exists in the target directory.
+
+### Step 2: Get tgz files
+
+#### Option A: CI mode (default)
+
+**2a. Find the workflow run:**
+
+If no run ID was provided, get the latest successful run:
+
+```bash
+gh run list --repo databricks/appkit --workflow prepare-release.yml --status success --limit 1 --json databaseId,number
+```
+
+If a GitHub Actions URL was provided, extract the run ID from it (the number after `/runs/`).
+
+**2b. Find the artifact name:**
+
+```bash
+gh api "repos/databricks/appkit/actions/runs/{RUN_ID}/artifacts" --jq '.artifacts[] | select(.name | test("^appkit-release-[0-9]")) | .name'
+```
+
+**2c. Download the artifact:**
+
+```bash
+gh run download {RUN_ID} --repo databricks/appkit --name "{ARTIFACT_NAME}" --dir /tmp/appkit-release-artifacts
+```
+
+**2d. Verify checksums:**
+
+```bash
+cd /tmp/appkit-release-artifacts && shasum -a 256 -c SHA256SUMS
+```
+
+Note: Use `shasum -a 256` (macOS) not `sha256sum` (Linux).
+
+**2e. Print the downloaded version:**
+
+```bash
+cat /tmp/appkit-release-artifacts/VERSION
+```
+
+The tgz files are now at `/tmp/appkit-release-artifacts/databricks-appkit-*.tgz` and `/tmp/appkit-release-artifacts/databricks-appkit-ui-*.tgz`.
+
+#### Option B: Local build mode
+
+**2a. Build tgz files:**
+
+```bash
+cd {LOCAL_APPKIT_PATH} && pnpm pack:sdk
+```
+
+**2b. Find the tgz files:**
+
+Glob for `*.tgz` in:
+- `{LOCAL_APPKIT_PATH}/packages/appkit/tmp/*.tgz`
+- `{LOCAL_APPKIT_PATH}/packages/appkit-ui/tmp/*.tgz`
+
+There should be exactly one tgz in each directory.
+
+### Step 3: Copy tgz files to target directory
+
+Copy both `databricks-appkit-*.tgz` and `databricks-appkit-ui-*.tgz` to the target directory.
+
+### Step 4: Update package.json
+
+Read `package.json` in the target directory. Edit `@databricks/appkit` and `@databricks/appkit-ui` dependency values to use `file:./` prefix pointing to the tgz filenames.
+
+For example, if the tgz file is `databricks-appkit-0.22.0.tgz`:
+```json
+"@databricks/appkit": "file:./databricks-appkit-0.22.0.tgz"
+```
+
+Same pattern for `@databricks/appkit-ui`.
+
+### Step 5: Install dependencies
+
+Run in the target directory:
+
+```bash
+npm install --force ./databricks-appkit-{VERSION}.tgz ./databricks-appkit-ui-{VERSION}.tgz
+```
+
+### Step 6: Clean up
+
+If CI mode was used, remove the temp directory:
+
+```bash
+rm -rf /tmp/appkit-release-artifacts
+```
+
+### Step 7: Report
+
+Print a summary:
+- Source: CI run #{number} or local build from {path}
+- Version installed
+- Target directory
+- Installed packages

.github/actions/setup-jfrog-npm/action.yml

@@ -0,0 +1,42 @@
+name: 'Setup JFrog npm registry'
+description: 'Obtains a JFrog OIDC token and configures npm to use the JFrog Artifactory proxy'
+inputs:
+  npmrc-path:
+    description: 'Path to write .npmrc file (use .npmrc for project-level override)'
+    required: false
+    default: '~/.npmrc'
+runs:
+  using: 'composite'
+  steps:
+    - name: Get JFrog OIDC token
+      shell: bash
+      run: |
+        set -euo pipefail
+        ID_TOKEN=$(curl -sLS \
+          -H "User-Agent: actions/oidc-client" \
+          -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
+          "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=jfrog-github" | jq .value | tr -d '"')
+        echo "::add-mask::${ID_TOKEN}"
+        ACCESS_TOKEN=$(curl -sLS -XPOST -H "Content-Type: application/json" \
+          "https://databricks.jfrog.io/access/api/v1/oidc/token" \
+          -d "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"${ID_TOKEN}\", \"provider_name\": \"github-actions\"}" | jq .access_token | tr -d '"')
+        echo "::add-mask::${ACCESS_TOKEN}"
+        if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
+          echo "FAIL: Could not extract JFrog access token"
+          exit 1
+        fi
+        echo "JFROG_ACCESS_TOKEN=${ACCESS_TOKEN}" >> "$GITHUB_ENV"
+        echo "JFrog OIDC token obtained successfully"
+
+    - name: Configure npm
+      shell: bash
+      run: |
+        set -euo pipefail
+        NPMRC_PATH="${{ inputs.npmrc-path }}"
+        NPMRC_PATH="${NPMRC_PATH/#\~/$HOME}"
+        cat > "$NPMRC_PATH" << EOF
+        registry=https://databricks.jfrog.io/artifactory/api/npm/db-npm/
+        //databricks.jfrog.io/artifactory/api/npm/db-npm/:_authToken=${JFROG_ACCESS_TOKEN}
+        always-auth=true
+        EOF
+        echo "npm configured to use JFrog registry (wrote to $NPMRC_PATH)"

.github/workflows/ci.yml

@@ -11,6 +11,7 @@ concurrency:
 permissions:
   contents: read
   pull-requests: read
+  id-token: write
 
 jobs:
   detect-changes:
@@ -42,6 +43,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:
@@ -69,6 +72,8 @@ jobs:
         run: pnpm run knip
       - name: Run License Check
         run: pnpm run check:licenses
+      - name: Check template deps are pinned
+        run: pnpm exec tsx tools/check-template-deps.ts
 
   test:
     name: Unit Tests
@@ -80,6 +85,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:
@@ -100,6 +107,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:
@@ -128,6 +137,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:
@@ -177,6 +188,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:

.github/workflows/docs-deploy.yml

@@ -25,6 +25,8 @@ jobs:
     name: Build Docs
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
       - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
         with:

.github/workflows/pr-title.yml

@@ -18,7 +18,19 @@ jobs:
     name: Conventional Commit Title
     steps:
       - name: Validate PR title
-        uses: ytanikin/pr-conventional-commits@fda730cb152c05a849d6d84325e50c6182d9d1e9 # 1.5.1
+        uses: amannn/action-semantic-pull-request@48f256284bd46cdaab1048c3721360e808335d50 # v6.1.1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         with:
-          task_types: '["feat","fix","docs","test","ci","refactor","perf","chore","revert","style","build"]'
-          add_label: 'false'
+          types: |
+            feat
+            fix
+            docs
+            test
+            ci
+            refactor
+            perf
+            chore
+            revert
+            style
+            build

.github/workflows/prepare-release-lakebase.yml

@@ -0,0 +1,105 @@
+name: Prepare Release Lakebase
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'packages/lakebase/**'
+
+concurrency:
+  group: prepare-release-lakebase
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+  id-token: write
+
+jobs:
+  prepare:
+    runs-on:
+      group: databricks-protected-runner-group
+      labels: linux-ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+        with:
+          fetch-depth: 0
+
+      - name: Setup JFrog npm
+        uses: ./.github/actions/setup-jfrog-npm
+
+      - name: Setup pnpm
+        uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
+
+      - name: Setup Node.js
+        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version: 24
+          cache: "pnpm"
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Check for releasable commits
+        id: version
+        working-directory: packages/lakebase
+        run: |
+          VERSION=$(pnpm exec release-it --release-version --ci) || true
+          if [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+            echo "version=$VERSION" >> "$GITHUB_OUTPUT"
+            echo "Next version: $VERSION"
+          else
+            echo "No releasable commits — skipping release preparation"
+            echo "version=" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Generate changelog
+        if: steps.version.outputs.version != ''
+        working-directory: packages/lakebase
+        run: |
+          pnpm exec release-it ${{ steps.version.outputs.version }} --ci
+      - name: Build
+        if: steps.version.outputs.version != ''
+        run: pnpm --filter=@databricks/lakebase build:package
+
+      - name: Dist
+        if: steps.version.outputs.version != ''
+        run: pnpm --filter=@databricks/lakebase dist
+
+      - name: SBOM
+        if: steps.version.outputs.version != ''
+        run: pnpm --filter=@databricks/lakebase release:sbom
+
+      - name: Pack
+        if: steps.version.outputs.version != ''
+        run: npm pack packages/lakebase/tmp
+
+      - name: Generate SHA256
+        if: steps.version.outputs.version != ''
+        run: sha256sum *.tgz > SHA256SUMS
+
+      - name: Write version file
+        if: steps.version.outputs.version != ''
+        run: echo "${{ steps.version.outputs.version }}" > VERSION
+
+      - name: Upload release metadata
+        if: steps.version.outputs.version != ''
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+        with:
+          name: lakebase-release-meta-${{ github.run_number }}
+          retention-days: 7
+          path: VERSION
+
+      - name: Upload release artifacts
+        if: steps.version.outputs.version != ''
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+        with:
+          name: lakebase-release-${{ github.run_number }}
+          retention-days: 7
+          path: |
+            *.tgz
+            packages/lakebase/changelog-diff.md
+            VERSION
+            SHA256SUMS