[FR] Support file-based and Azure Key Vault token sources alongside env and googleCloudSecret #1704

Describe the feature you'd like

Add two new token sources alongside env and googleCloudSecret: a file source and an
Azure Key Vault source.

{ "token": { "file": "/var/run/secrets/sourcebot/github-token" } }
{ "token": { "azureKeyVaultSecret": "https://<vault>.vault.azure.net/secrets/<name>[/<version>]" } }

Motivation

A Token in the config can only come from an environment variable or from Google Cloud
Secret Manager. schemas/v3/shared.json defines exactly two shapes, { "env": ... } and
{ "googleCloudSecret": ... }, and getTokenFromConfig throws Invalid token configuration
for anything else (packages/shared/src/crypto.ts:124-149).

That makes short-lived credentials impractical outside GCP. GitHub App installation tokens
(ghs_, accepted as connection tokens once #1662 lands) expire after one hour. Environment
variables are fixed when the container starts, so the only way to hand Sourcebot a fresh
token is to restart the whole service every hour. That restart takes the web app down and
interrupts in-flight indexing.

The rest of the code already supports rotation. Nothing caches the resolved value:
getGitHubReposFromConfig calls getTokenFromConfig on every sync
(packages/backend/src/github.ts:172), and getRepoAuth calls it on every clone and fetch
(packages/backend/src/utils.ts:167). A source whose value can change at runtime would be
picked up on the next sync with no other changes. Only the source types are missing.

Note - this is not specific to GitHub. Token is shared, so connection tokens, SSO client
secrets, LLM API keys and environmentOverrides values (DATABASE_URL, REDIS_URL) have the
same limit. On Azure there is currently no supported way to keep any of them in Key Vault.

Proposed design

Both are additive anyOf branches in Token, so existing configs are unaffected.

1. file (no new dependencies)

{ "token": { "file": "/var/run/secrets/sourcebot/github-token" } }

• Read the file each time the token is resolved and trim whitespace, matching env and
  googleCloudSecret. Fail with a clear error if it is missing or empty.
• Works on any platform: Kubernetes Secret volumes (refreshed in place), Docker/Compose
  secrets, the Secrets Store CSI driver for Azure Key Vault/AWS/Vault, and sidecars that mint
  tokens, such as a GitHub App token refresher writing to a shared emptyDir.
• On its own, this solves the rotation problem above.

2. azureKeyVaultSecret

{ "token": { "azureKeyVaultSecret": "https://<vault>.vault.azure.net/secrets/<name>" } }
{ "token": { "azureKeyVaultSecret": "https://<vault>.vault.azure.net/secrets/<name>/<version>" } }

• Uses @azure/keyvault-secrets with DefaultAzureCredential (@azure/identity), covering
  AKS Workload Identity, managed identity and AZURE_CLIENT_* environment variables. This
  mirrors how the GCP source relies on Application Default Credentials.
• Leaving out the version fetches the latest one, so rotating the secret in Key Vault is
  enough.

Example use case

1. Register a GitHub App, install it on an organisation, grant Contents: read and
   Metadata: read.
2. Run a sidecar (or CronJob) that mints an installation token every ~50 minutes via
   POST /app/installations/{installation_id}/access_tokens and writes it to a shared volume.
3. Point the connector at that file:

   { "type": "github", "token": { "file": "/var/run/secrets/github-token" }, "orgs": ["my-org"] }

4. Sourcebot reads the current token on every sync, clone and fetch. No restarts.

Today, step 3 is rejected by schema validation at startup, and the env equivalent stops
working with 401 Bad credentials after an hour.

Alternatives considered

• Restarting the container every hour: causes downtime and interrupts indexing.
• The native apps GitHub App integration: requires a license key, and its privateKey
  is itself a Token (schemas/v3/app.json), so on Azure the App's private key still has to
  sit in an environment variable.
• Going through GCP Secret Manager: brings a second cloud into an Azure-only deployment.

Additional information

Sourcebot version: v5.1.12 (docker.sourcebot.dev/sourcebot-dev/sourcebot), also applies to
main at the time of writing. Running on AKS (Azure); no GCP.

I'm happy to send the PR: schemas/v3/shared.json (plus regenerated schemas and docs),
getTokenFromConfig, unit tests, and the Tokens section of
docs/docs/configuration/config-file.mdx. If you'd prefer it smaller, I can send file
first and Azure Key Vault in a follow-up.