Skip to content

advanced.md

Latest commit

 

History

History
363 lines (261 loc) · 55.1 KB

advanced.md

File metadata and controls

363 lines (261 loc) · 55.1 KB
Raw

Advanced reference

Home · 简体中文

Use the WebUI for everyday management. See deployment for startup methods and client configuration for examples.

Configuration and CLI

Precedence: explicit CLI flags > process environment > .env > saved SQLite settings > defaults. Hot settings apply immediately; restart-marked settings require a manual restart. Change locked options at their source; the WebUI does not edit .env. Standard keys can be revealed, while admin keys are stored only as hashes.

Compose loads every key in .env into the container, so commenting out a line is enough to unlock it; only CODEBUDDY_AUTH_DIR and CODEBUDDY2API_ALLOW_OPEN_NOAUTH are fixed by the Compose file. Recreate the container after changing these values.

Runtime keys in .env stay read-only in the WebUI until they are moved into the configuration store: stop the service and run python3 converter.py config migrate (see configuration migration). The Dashboard shows a one-time banner listing those keys; migrating or pressing "Do not remind me" records config_notice_muted in the store and stops the reminder, including for later .env additions.

language (auto|zh|en) is the only interface preference the WebUI and the CLI share: it has no environment mapping and is never locked. --lang writes it on serve, login and config migrate; config status is read-only and only affects that invocation's output.

Flag Default Description
--host / --port 127.0.0.1 / 8787 Local listener
--api-key unset External admin inference key and initial WebUI password; blank means unset, and database keys are never promoted to it
--admin-csrf [true/false] true Startup-only management Origin/CSRF checks; disabling does not bypass API-key or session authentication
--admin-allowed-origins none Extra trusted management Origins (comma-separated; bare domains mean HTTPS) for reverse-proxy sign-in; hot and WebUI-editable
--auth-file scan auth/ Explicit credential file, repeatable; disables scanning other files
--log retired Warns without writing a file; use SQLite logs in the WebUI
--desensitize off Adapt fixed CLI templates, compact runtime prompts and mask keywords with zero-width characters
--prompt-mode empty empty adds no gateway default system prompt; prepend inserts a concise official WorkBuddy/CodeBuddy prefix at the front; hot and WebUI-editable
--no-compact off With desensitization, retain fuller instructions while adapting templates and pruning runtime context; does not disable Responses projection
--keep-tool-metadata [true/false] false Retain tool descriptions and parameter-schema description/title, independently of prompt compaction
--skip-check off Skip startup preflight
--credit-price-cny 0.014 Domestic CNY per credit for billing conversion
--credit-price-usd 0.03 International USD per credit for billing conversion
--usd-rate 7.15 CNY per USD for billing conversion
--model-catalog-ttl 21600 Model catalog cache TTL, seconds
--no-model-guard off Disable the out-of-catalog guard; passthrough is limited to one product profile and still respects disabling, bindings and catalog readiness
--model-capability-guard [true/false] true Preflight declared image, tool, reasoning and mapped output limits; changes affect new requests
--max-images 16 Total images per request; 0 permits no images
--image-policy truncate Keep newest images; error rejects excess images with 413
--responses-projection-mode balanced|passthrough balanced Rewrite recognized harness blocks that have stable summaries; passthrough disables Responses projection
--responses-projection-max-bytes 40000 Complete per-item UTF-8 limit for assistant text, tool-argument JSON and tool results; 0 disables, otherwise valid range is 256..33554432
--tool-call-max-retry 3 Extra generations after malformed tool calls (each consumes credits); 0 disables retries
--max-inbound-bytes 67108864 Raw body limit for generation and token-count POSTs, before parsing (chunked included); other routes are not buffered; 413 beyond it
--max-collect-bytes 8388608 Total retained-output budget for aggregation and realtime validation (content + reasoning + tool arguments/metadata); response_too_large beyond it; 0 disables
--stream-mode compatible|realtime compatible compatible preserves aggregate/replay behavior; realtime incrementally streams all three protocols and does not regenerate tool arguments
--max-concurrent 64 Concurrency limit for the three generation endpoints only; excess requests get 503 with Retry-After; token counting is unaffected; 0 disables
--max-inflight-per-account 0 Per-process, per-account in-flight client inference limit; 0 disables, full accounts return 503
--upstream-keepalive [true/false] false Bounded connection reuse isolated by official origin; requires restart
--request-context-mode legacy scoped enables explicit sessions and per-attempt tracing; changes apply to new requests
--failover-max 0 Extra credentials tried when a request fails before the first response byte reaches the client; 0 keeps the upstream behaviour of surfacing the failure directly
--retry-write-timeout false Opt a request-body write timeout into replay (fresh connection and --failover-max), accepting that bytes already sent may have been processed
--max-request-bytes 33554432 Positive byte limit for processed upstream JSON, excluding gateway-only metadata
--log-body-limit 65536 Legacy text-preview option; text output is retired and SQLite diagnostics use their own budget

Every active environment variable is listed with its default in .env.example; startup examples are in deployment.

Responses projection

Configure these hot settings through the WebUI, CLI, process environment or .env. Precedence is CLI > process environment > .env > saved SQLite value > default; CLI/environment values lock the WebUI fields. Compose loads .env wholesale, so a commented-out line keeps the WebUI editable.

responses_projection_mode defaults to balanced and accepts only balanced or passthrough. Balanced mode rewrites only recognized harness blocks that have stable summaries; text outside those blocks is not budgeted or truncated. It also applies Codex-style head/tail truncation to generated assistant content, complete tool-argument JSON and tool results according to responses_projection_max_bytes. Text markers report original bytes, estimated tokens and total lines. Oversized tool arguments remain valid JSON and use a bounded object containing the original head, tail and size metadata. Passthrough disables Responses projection completely.

responses_projection_max_bytes defaults to 40000; valid values are 0 or 256..33554432. 0 disables per-item truncation only; global inbound/request and output gates still apply. Neither setting changes the client Base URL.

Downgrading to source that predates these keys requires the offline removal of settings.responses_projection_mode and settings.responses_projection_max_bytes described under Streaming modes.

Tool metadata retention

Responses projection no longer changes tool definitions or schemas. When desensitization is enabled, it strips tool descriptions and string description/title annotations by default; enabling this setting retains and processes that text across Chat, Responses and Messages. --no-compact does not change this setting.

  • WebUI: Settings → Keep tool descriptions; unlocked changes apply immediately and persist.
  • CLI: append --keep-tool-metadata or --keep-tool-metadata true to the existing command; explicit false overrides the environment.
  • Environment: set CODEBUDDY2API_KEEP_TOOL_METADATA=true in .env; comment the line out to remove the lock, and do not set an empty string.

Use an image or source build containing this feature; recreate the container after changing its environment. Retained descriptions may increase input tokens and content-filter risk; set false to restore the desensitization default. This setting does not change Responses balanced/passthrough mode or relax request-size limits.

Streaming modes

stream_mode defaults to compatible. Configure it through --stream-mode compatible|realtime, CODEBUDDY2API_STREAM_MODE, or the WebUI enum; explicit CLI/environment sources lock that field. Every generation request, including non-streaming, records the selected mode and freezes it with max_collect_bytes before routing. Hot changes affect only later requests, not in-flight failovers. Non-streaming responses remain aggregated JSON; there is no client mode override.

  • compatible preserves existing behavior: Responses streams aggregate first; Chat and Messages aggregate when tools are present and otherwise pass through upstream increments. Aggregated output is validated and replayed in fragments. Non-stream requests always use the validated aggregate path in either mode.
  • realtime forwards text, refusal, tool-argument and reasoning increments for all three protocols as they arrive, so reasoning timing stays visible to clients; Responses and Anthropic still deliver one logical thinking item/block with stable indexes. compatible keeps the older behaviour of coalescing reasoning into one delta before the first visible output, which hides the reasoning timeline and can inflate client-side TPS/TTFT. Adapters buffer tools with missing identity until the argument phase or terminal marker, append metadata fragments without guessing from prefixes, and reject identity changes after an item starts. max_collect_bytes bounds retained UTF-8 output; 0 disables that limit.

Realtime Messages keeps one content block open at a time. The active tool remains incremental; later tool, text or thinking blocks may wait until upstream completion. Deferred event bytes share max_collect_bytes. A tool still awaiting its identity does not block unrelated text before its block starts.

Realtime mode never regenerates malformed or incomplete tool arguments. Tool IDs, names, declared names, JSON-object arguments and tool_choice are checked at the terminal boundary before a success terminal is sent. In realtime, a tool-bearing completion must also carry the upstream tool_calls finish marker; a stop marker with tool calls is rejected, while compatible mode keeps its legacy acceptance behavior. Before any downstream byte, failures retain the upstream HTTP error and existing bounded pre-response failover rules. After any byte, malformed tools, disconnects, stream errors and budget overflow produce a protocol error terminal without credential replay or switching; valid length, refusal and content-filter results keep their native protocol distinctions (Responses reports truncation/filtering as incomplete, never completed) and are not regenerated. Clients must therefore accept partial output followed by an error rather than assuming every opened SSE stream completes successfully. Audit records retain only an allowlisted stream_mode marker plus available upstream usage.

Explicit filter-only terminals are valid even without output: realtime Responses emits response.incomplete with content_filter, retaining available usage. This does not permit an ordinary empty response, missing terminal or error frame to succeed.

For runtime fallback, select compatible to restore aggregate streaming and tool-argument repair without changing saved state. Before running older source, remove the new CLI/environment option, stop the gateway and back up the current data directory, including its SQLite/WAL/SHM generation. Do not restore a stale pre-upgrade database: that could roll back newer claims, sessions, revocations and account state. The following offline procedure deletes only the listed settings.<name> keys and increments the revision, then checks integrity; all other settings and tables remain intact. Pass one key or several. Never run it against a live database or mix SQLite generations.

python3 - /path/to/control.sqlite3 settings.stream_mode <<'PY'
import json, sqlite3, sys
path, names = sys.argv[1], sys.argv[2:]
keys = [name.split(".", 1)[1] for name in names if name.count(".") == 1 and name.split(".", 1)[1]]
if len(keys) != len(names):
    raise SystemExit("pass one or more settings.<name> keys")
con = sqlite3.connect(path)
try:
    con.execute("BEGIN IMMEDIATE")
    row = con.execute("SELECT revision,payload FROM control WHERE id=1").fetchone()
    if row is None:
        raise SystemExit("missing control row")
    revision, payload = row
    data = json.loads(payload)
    if set(data) != {"settings", "models", "credentials"} or not isinstance(data["settings"], dict):
        raise SystemExit("unexpected control payload")
    present = [key for key in keys if key in data["settings"]]
    if not present:
        raise SystemExit("no requested key is present; no write needed")
    for key in present:
        del data["settings"][key]
    con.execute("UPDATE control SET revision=?,payload=? WHERE id=1",
                (revision + 1, json.dumps(data, ensure_ascii=False, allow_nan=False)))
    con.commit()
    if con.execute("PRAGMA integrity_check").fetchone()[0] != "ok":
        raise SystemExit("integrity check failed")
finally:
    con.close()
print("ok")
PY

Older strict setting validators reject the unknown saved key, so changing only its value does not make old source compatible. This procedure is intentionally offline and operator-scoped; do not perform it on production without a current backup and a stopped service.

Connection reuse and account capacity

Both settings are available in the WebUI; their environment variables are CODEBUDDY2API_UPSTREAM_KEEPALIVE and CODEBUDDY2API_MAX_INFLIGHT_PER_ACCOUNT. Commenting a line out of .env keeps the WebUI in control. Connection reuse defaults to off; when enabled, each official origin permits 64 connections with 16 idle connections and a 30-second keepalive expiry. Authentication is request-scoped, upstream cookies are not stored, and shutdown closes the pools. Proxy environment, timeouts and replay rules are unchanged; disable and restart to restore fresh connections.

The account limit defaults to 0. Positive limits skip full accounts within existing routing and free-first rules; a full free tier never spills into paid accounts. No capacity returns 503 / credential_concurrency_limit with Retry-After: 3, without queueing or penalizing the account. Completion, cancellation and failed-account rotation release capacity. The credentials API exposes in_flight and max_in_flight. Only the three client generation endpoints count; limits are per process, not shared between instances. Setting 0 restores unlimited account capacity without interrupting active requests.

Request context

Generation responses carry a gateway-generated X-Request-ID for correlation with text logs and available audit details. Response-body and tool-call IDs are unchanged; client request IDs are not trusted or used for deduplication.

request_context_mode defaults to legacy, preserving legacy session-hint behavior. Legacy conversation IDs stay stable for the same account and session, but change when failover selects another account, even within one profile. Enable scoped in WebUI settings, with --request-context-mode scoped, or through CODEBUDDY2API_REQUEST_CONTEXT_MODE=scoped. Each client HTTP request keeps one root ID across existing retries/failover, with a new ID/span for every upstream attempt and account-isolated conversation IDs. This does not enable additional retries, server-side history or automatic cache keys.

In scoped mode, optionally send X-Codebuddy-Session-ID, metadata.conversation_id / metadata.conversationId, or top-level conversation_id / conversationId. Values must agree; conflicting, non-string, control-character or over-512-UTF-8-byte values return 400. Empty values fall back to a fingerprint of adapted instructions and the first user input, including image references; URLs are not fetched. Without reliable input a temporary session is used. Identical inputs without explicit IDs remain indistinguishable; user, metadata.user_id and prompt_cache_key are not session IDs. Raw hints are neither logged nor forwarded upstream.

Switch back to legacy to restore old behavior for new requests; in-flight requests retain their initial mode. Upstream client identities follow verified official packages: domestic CLI 2.165.0 / WorkBuddy 5.9.3 and international CLI 2.161.0 / WorkBuddy 5.8.4. Domestic WorkBuddy uses session-stable UUIDv7 conversation IDs, per-attempt request/tracing headers and gzip for JSON bodies of at least 1024 bytes; CLI and international WorkBuddy use their existing ID relationships and always gzip chat JSON. Headers are generated locally without reading hardware IDs.

Automation and rewards

Automatic check-in and Buddy travel are per-account switches (WebUI credentials page): on by default for domestic personal accounts, off internationally, applied live without restart. The growth center is personal-only, so enterprise accounts never travel and no longer surface the upstream 403. The growth-center sweep (auto_growth, on by default for domestic personal accounts) rides the hourly maintenance pass as pure accept/claim: it claims finished tasks for credit and energy and never starts a behavior chain by itself. Per-task growth_auto switches (off by default) opt specific tasks into automation, so the sweep also drives the matching chain — real conversations, app opens or expert summons that may spend credits. The credential drawer lists live tasks with one-click manual runs for a single task or all enabled ones, and task switches can be copied to selected accounts (POST /admin/credentials/growth-batch); GET /admin/credentials/{id}/growth-tasks, .../growth-tasks/auto and .../auto-all back the panel. Manual runs are server-side jobs, one per account: the request returns the job at once, closing the drawer or the page does not stop it, starting again while it runs returns the running job, and GET /admin/growth-jobs lists every account's current or last job with the task in progress for the page banner. Named expert tasks use verified task-specific expert IDs through the same opt-in real-conversation chain. Manual runs and sweep claims are audited; enterprise or international accounts never see the panel. Check-in, travel, growth and daily-chat results or exceptions carrying 11140 use the same persistent two-hour restricted state as inference. The current sweep stops immediately, including remaining growth claims, and later sweeps skip the account until the rest expires or an operator releases it; other codes retain their existing logging and retry behavior.

CODEBUDDY2API_AUTO_ACCEPT_BUDDY is startup-only and defaults to false. It preauthorizes enabled domestic accounts for first-Buddy onboarding, agreement and travel; automatic travel still respects its account switch. first_buddy needs no acceptance API: pending states, including not_accepted, allow one real domestic WorkBuddy conversation on that account. Prefer an eligible zero-rate model, otherwise the lowest known rate; request at most 32 output tokens with possible credit usage. Other reward tasks, paid boxes, pet switching and international trials are excluded.

Manual POST /admin/credentials/{id}/travel returns buddy_confirmation with official terms and a separate authorization scope. Submit {"confirm_buddy":true,"agreement_revision":"<returned revision>"} after consent; old adoption-only revisions are rejected. can_claim describes eligibility and never disables consent.

control.sqlite3 preserves consent and one actual onboarding conversation per account across restarts. A live preflight cancellation releases only its own unsent reservation; unknown or sent attempts are never released automatically. Historical acceptance records do not block an unsent conversation. Only official task completion permits adoption. Unconfirmed first-claim sends remain reserved beyond 24 hours and only reconcile through reads; pre-claim failures may resume after backoff. Keep this database when upgrading; travel-status and balance sync remain read-only.

Travel claims and departures share an account-scoped write reservation. Uncertain results do not expire or replay; fresh status reads reconcile them without issuing upstream writes. Store failures stop claims and departures, and local readback updates preserve receipt ownership across processes.

Trial credits are manual-only for eligible intl-work accounts through the credential drawer or POST /admin/credentials/{id}/trial; startup and maintenance never claim. Safe results, successful claims and reservations persist in control.sqlite3; failed attempts retain the 24-hour backoff. Response bodies are capped at 64 KiB and never returned to the browser. Preserve the database when upgrading.

CODEBUDDY2API_AUTO_TRIAL and --auto-trial are retired: old startup options warn and do nothing; saved Boolean auto_trial settings are ignored on load. Remove them from deployment configuration. Before reverting to older code, check these old settings to avoid re-enabling automatic claims.

APIs and authentication

Client endpoint Description
POST /v1/chat/completions OpenAI Chat Completions
POST /v1/responses OpenAI Responses
POST /v1/messages Anthropic Messages
POST /v1/messages/count_tokens Character-based heuristic token estimate for budgeting, not an exact count
GET /v1/models Available models, multipliers and safe per-profile declarations
GET /v1/dashboard/billing/subscription Converted credit totals; codebuddy_balance_usd is the remaining balance
GET /v1/dashboard/billing/usage total_usage in cents and daily breakdowns

hard_limit_usd is the converted sum of remaining and used credits, not the remaining balance. Without a date-range filter, balance equals hard_limit_usd - total_usage / 100. These are local conversions, not a redistribution billing system.

Management/shared endpoint Description
GET /health Public liveness only: {"status":"ok"}
GET /admin/credentials Credential inventory and runtime state
POST /admin/credentials Import an .info file from the server's controlled directory
DELETE /admin/credentials/{name} Delete the credential file by filename; returns 409 while referenced by model bindings
PATCH /admin/credentials/{id} One Boolean field per account (enabled, auto_checkin, auto_travel, auto_daily_chat or auto_growth) or a growth_auto task-code mapping; automation preference saves do not claim immediately
GET /admin/credentials/{id}/growth-tasks · POST .../growth-tasks/auto · .../auto-all · GET /admin/growth-jobs · POST /admin/credentials/growth-batch Live growth task list; start a background job for one task or every enabled one; current and last job per account; batch task switches for selected accounts
POST /admin/oauth/start · GET /admin/oauth/poll Start/poll login; site=cn (default), intl (international WorkBuddy) or intl-codebuddy (international CodeBuddy)
GET /admin/credits · POST /admin/checkin Inspect credits; daily-idempotent check-in followed by domestic travel when enabled
POST / GET / DELETE /admin/session · DELETE /admin/sessions Password login, session inspection/logout and revoke-all; remember-me sessions renew with the same SID
GET / PUT /admin/account Read or change the management username/password; account changes rotate the session epoch and CSRF token
GET / POST /admin/keys · PATCH / DELETE /admin/keys/{id} · POST .../rotate · POST .../reveal Create and manage inference keys; writes use optimistic versions, standard keys support reveal, and admin secrets are one-time only
POST /admin/sync Synchronize all enabled accounts' balances, catalogs and usage; no check-in or trial claims
POST /admin/credentials/{id}/{action} Single-account refresh, checkin, sync, travel-status (query only), travel (claim then dispatch), trial (one-time trial credits), or reset-cooldown (local-only)

Travel results include phase, optional safe error_kind/http_status/code, and snapshot remaining_seconds. claimed/departed remain true for confirmed writes even if a later query sets ok=false and stale=true; query status before another attempt.

reset-cooldown takes no request body and lifts every cooldown held by one account — both its 401/403 circuit breaker and its per-model 429 cooldowns — so an operator can recover from a cooldown recorded in error without restarting the gateway. It only edits local state: it never refreshes a token, contacts upstream, or queues synchronization, and it works for a manually disabled account. The result separates changed_in_memory from durable; ok is false when the write failed, in which case the in-memory reset is already effective but a restart would restore the stored row, and the request can simply be repeated.

Pages use /dashboard/*, management APIs use /admin/*, and clients retain /v1/*. /cn and /intl API prefixes are not registered. Automatic model routing requires no client URL changes.

OPEN_NOAUTH anonymous mode exposes both inference and account-pool billing (/v1/dashboard/billing/*). It applies only when there is no external key and no api_keys row, accepts missing or placeholder tokens, and must never be used for public deployments. Any key row, including disabled or expired rows, disables anonymous mode.

Management uses a username and password, never an API key. The HttpOnly Cookie authorizes only /admin/*; inference keys authorize only /v1/*. A normal login lasts 12 hours; Remember me lasts 400 days with sliding renewal. Account changes and Sign out all sessions revoke every session.

The external CODEBUDDY2API_KEY is the initial password and an unrestricted admin inference key. A legacy default key migrates to the initial password and a database admin key. Changing the password decouples login from later external-key changes. If password recovery is needed, run converter.py admin reset-password interactively.

Standard keys support expiry, IP/CIDR allowlists, RPM and source-model restrictions; they can be revealed and are stored in plaintext. Admin keys are unrestricted, limited to one, and stored only as hashes. Standard keys cannot read billing or access another key's video tasks. Public /v1/media/... assets remain available until media TTL expiry even after their creating key is revoked.

Management Origin / CSRF switch

Enabled by default. When OAuth polling omits both Origin and Sec-Fetch-Site, a same-origin Referer (matching scheme, host and port) is accepted, but a valid CSRF token is still required. An existing Origin takes precedence; without it, supplied Fetch Metadata must be same-origin and cannot fall back to Referer. Login and writes still require Origin.

Normal same-origin access does not require disabling protection. Behind a reverse proxy that rewrites the forwarded Host/scheme (for example HTTPS on a bound domain while the container sees HTTP), the browser Origin no longer matches what the server sees and login fails Origin checks. Add the public address to admin_allowed_origins (WebUI system settings, hot) or set CODEBUDDY2API_ADMIN_ORIGINS / --admin-allowed-origins: comma separated origins or bare domains (https://chat.example.com, chat.example.com; bare domains mean HTTPS), up to 32 entries. An explicit CLI or environment value locks the WebUI field. Prefer this over disabling protection; if errors persist, use a consistent access URL, check the proxy's forwarded Host/scheme, and refresh the page and log in again. Only for trusted local deployments, append --admin-csrf false to the startup command or set this in your existing .env:

CODEBUDDY2API_ADMIN_CSRF=false

This is startup-only, CLI takes precedence over the environment, and the WebUI cannot change it. Write the value in .env or pass the flag, then recreate the container; docker compose restart alone is insufficient.

Disabling skips login Origin checks and Origin/CSRF checks on Cookie-authenticated management writes and OAuth polling. API keys, session expiry, OAuth task ownership, official authorization-site validation and dangerous-action confirmations remain enforced; /v1/* is unaffected.

Disabling weakens browser cross-site request protection; do not expose this configuration directly to the public Internet. Set --admin-csrf true or the environment value to true, then restart to re-enable protection.

Server-side path imports

The WebUI supports direct uploads; these rules concern path imports through POST /admin/credentials:

  • Place files in auth/imports/ or the server directory set by CODEBUDDY_IMPORT_DIR.
  • Only regular .info files directly inside that directory are accepted; symlinks, subdirectories and files over 1 MiB are rejected.
  • Send {"path":"account.info"} or that file's absolute path. The same filename is updated under the import rules.
  • Identity includes product profile, UID and tenant. If another file already owns that identity, import returns 409; the same UID can coexist across products or tenants. Deletion takes a filename, while enable/disable takes an identity ID.

Models and scheduling

Select client models from the WebUI or GET /v1/models. Raw catalogs remain cached by account/tenant, region, product and client version in auth/control.sqlite3, with a default 6-hour TTL. New credentials trigger synchronization; failed refreshes retain that account's trusted cache. Legacy unscoped caches do not become international sharing sources.

International CLI and WorkBuddy use a deduplicated shared view from enabled, catalog-ready international accounts. A target account must have its own synchronized catalog; its existing model declarations win unchanged. Missing IDs inherit shared declarations, retaining catalog_source and safe source_variants. Conflicting inherited rates use the higher known rate, limits the smaller known value, reasoning options their intersection and differing descriptive fields are omitted; unknown prices never mean free. Domestic catalogs, credentials, balances, bindings and auto defaults remain independent. Shared rates are catalog references, not billing or permission guarantees.

Personal accounts first use the richer /v2/enterprises/personal/models table when it is available; its picker/root scopes are supplemented by missing IDs from /v3/config, while personal declarations win for duplicates. Enterprise accounts use /v3/config directly, and older deployments fall back there when the personal route is unavailable. Each refresh caches the merged picker subset as models and the merged account-root table as serves. Routing and GET /v1/models merge these candidates. Both scopes retain disabled and availableModels filtering; models without tool support are excluded.

The root table is not a guarantee that a backend serves every listed model; measured 11102 avoidance still applies. Unknown account catalogs do not authorize dispatch and must not trigger a premature all-backends-unsupported 404; they retain the retryable readiness state. Legacy cache entries without serves use the picker until the next refresh.

Beyond standard model fields, credits is the lowest source multiplier: 0.0 identifies a zero-multiplier source and null means no parseable multiplier was declared. credits_by_profile provides source details, such as {"intl-work":0.0,"cn-cli":0.03}. Compatible clients may ignore these fields; multipliers are not guaranteed to stay unchanged.

Model rules may include an ordered routes list. Each entry has an upstream_id, optional weight, enabled and source (builtin, cloud or manual). Empty weights use top-to-bottom failover; weights use weighted selection before the normal account scheduler. Disabled entries are skipped, and an all-disabled list makes the public model unavailable. Existing rules without routes keep their single upstream_id behavior.

Cloud catalog variants are grouped automatically only when their region, declared multiplier, context, output, image/reasoning/tool support and reasoning metadata match. A saved manual routes list takes precedence over that automatic group.

Credential domain / token issuer determine the product identity. Chat and refresh use fixed origins with separate product headers:

Profile Chat / refresh origin
cn-cli https://copilot.tencent.com
cn-work https://www.workbuddy.cn
intl-cli https://www.codebuddy.ai
intl-work https://www.workbuddy.ai
  • Domestic accounts use their own trusted catalogs; international accounts use the shared view above. Concrete zero-rate models take priority, followed by credit expiry, cooldowns and session stickiness. Balances are never borrowed.
  • Zero-balance accounts leave paid-model rotation but may use concrete zero-rate models in their effective catalog; they rejoin once balance recovers. International paid models require a known positive balance.
  • auto schedules an account's default, not any model. International accounts need positive balance and default-model in their catalog; domestic WorkBuddy must declare auto, and domestic CLI needs a known nonempty usable catalog. auto does not receive the concrete zero-multiplier balance exemption.
  • WebUI region, product and credential bindings strictly limit candidates; unavailable bindings never fall back to unselected accounts. Disabled models also reject direct requests. Renaming hides the original ID unless you choose to retain it.
  • Sent requests are not replayed against another account because of account availability or HTTP errors; later requests select again. Pending catalog/credential readiness usually returns 503 with Retry-After; unsupported or disabled models return 404.

Routing and catalog maintenance are switchable without reverting data:

  • routing_scheduler: shadow (default) runs the legacy scan and the indexed shards side by side, returns the legacy answer and records bounded, credential-free mismatches under GET /admin/performance; switch to indexed once the mismatch counter stays at zero; indexed reuses one eligibility shard per model, request-scoped alias and revision set; legacy re-evaluates every credential on every request. Shards are invalidated only when entry identities, the policy, the published catalogs or an account's solvency actually change, so token refreshes and maintenance writes keep the steady state.
  • catalog_probe_mode: account (default) refreshes every synced account each cycle; representative probes one credential per product profile per TTL cycle — accounts without a catalog first, then the current source, then the stalest — with exponential backoff after failures, and back-fills missing or dirty account catalogs under a bounded per-cycle quota so new accounts become routable without a manual scan. An unprobed or failed account keeps its last trusted catalog, /v1/models stays the union of current account catalogs, and only models newly advertised by the representative mark older accounts dirty.
  • POST /admin/models/scan refreshes every current account once, in the background; GET /admin/models/scan reports progress, successes, failures, catalog differences, the overall error and a per-profile breakdown of distinct catalog versions. It is the only way to calibrate all accounts at once and is never a periodic task.
  • housekeep_concurrency (default 4) bounds maintenance workers; balances, catalogs and usage each get an independent slot pool, 1 restores the previous serial cycle, and failed account syncs retry with exponential backoff plus jitter. Accounts synced within five minutes are skipped by the next full cycle, so an event-driven cycle never repeats them.

Model declarations and image compatibility

/v1/models retains its existing fields and adds capabilities, limits and metadata_by_profile; management and routing previews expose the same metadata. Capabilities use supported, unsupported, mixed or unknown. Limits carry state (known, mixed, unknown) and value; only unanimous known limits have a numeric value. Per-profile arrays preserve distinct account declarations without identities. Safe descriptions, capabilities, windows, reasoning options, related models and parameter suggestions are allowlisted; credentials, internal configuration and authenticated URLs are excluded. These are upstream declarations, not native-model or measured guarantees; suggestions do not override requests.

model_capability_guard defaults to true; use WebUI settings, --model-capability-guard false or CODEBUDDY2API_MODEL_CAPABILITY_GUARD=false to disable it. Explicit mismatches return 400 before sending, within existing bindings and the current free-first tier; unknown capabilities remain compatible. Requests retain their entry-time switch. Checks cover images, tools/history, declared reasoning options and the max_tokens output limit (including mapped Responses max_output_tokens); input tokens are not estimated, max_completion_tokens is not renamed or checked against this limit, and Anthropic thinking budgets are not converted. Disabling this guard leaves authentication, catalog authorization, capacity and size limits intact.

Both international profiles merge image-bearing consecutive user runs only after routing, preserving content order and image data. Domestic bodies, text-only runs and system/assistant/tool boundaries remain unchanged. Conflicting message attributes or unrepresentable content return 400 / image_user_run_not_mergeable; final byte limits still apply. This compatibility step remains enabled when capability preflight is disabled; it neither adds retries nor makes a text model natively visual.

Measured rates and output limits

Catalog multipliers are upstream declarations, and a model can keep declaring x0.00 after upstream starts charging for it. Every response's usage carries the credit actually charged, and the audit log keeps it per request, so the gateway measures billing instead of trusting the declaration:

  • A model that a product declares free counts as measured billing when any of its latest 20 requests in the past 7 days on that product was charged. It then stops being free everywhere: zero-balance accounts lose it, free-first ordering no longer prefers it and auto stops counting it as free. Twenty uncharged requests in a row make it free again.
  • Traffic proves that a model is billed but not its rate: upstream caches prompts per conversation and bills a hit at about a twentieth of fresh input, often without reporting it in usage, so per-request charges understate the per-token price by an order of magnitude. A measured-billing model therefore shows 实测计费 · 倍率待估 with its average charge per request until a probe calibrates it; it never reads as free.
  • A probe sends the same ≈8K-token request, each in a fresh conversation, to the target and to a model on the same product with a declared paid rate (the one nearest 0.1x). The target's rate is that declared rate scaled by the two per-token charges, so no token price is assumed. A target that is not charged needs no reference request.
  • GET /v1/models reports the effective rate in credits / credits_by_profile and adds credits_estimated: true when the lowest rate is such an estimate.
  • A manual rate per model and product (credits_override on the model rule, 0–1000; 0 confirms free) overrides both the declaration and the measurement for routing and /v1/models. Custom aliases carry no rate of their own; set it on the upstream model's rule.
  • POST /admin/models/{id}/rate-probe sends one ≈8K-token request for that model on every product that serves it, using an account with balance, and stores the charge upstream reported with the calibrated rate. POST /admin/models/rate-probe does the same for every model a product declares free. Both start the one shared background job (four probes at a time) and return it at once; starting while it runs returns the running job, and GET /admin/models/rate-probe reports progress with the model, product and step in flight, plus every result. A truly free model costs nothing to probe; a charged one costs its own request (≈0.6 credits for a ≈1x model) plus a few billing units for the reference. Every probe request, the reference included, is also written to the request log with its tokens and charge: protocol rate_probe, conversation ID prefixed rate-probe-. The calibrated samples are kept for 30 days in the control database and are not counted twice from the log.

Declared maxOutputTokens values are frequently higher than what a model emits, and a larger max_tokens is clamped silently rather than rejected. scripts/probe_max_tokens.py measures the real limit through the gateway: it asks each model to count far enough to fill a tier and tells upstream truncation (finish=length below the request) apart from a model that chose to stop. Tiers never exceed the declared limit, models run four at a time, each within a 600-second budget, and results append to a JSONL log that --resume continues.

python3 scripts/probe_max_tokens.py --key "$CODEBUDDY2API_KEY" --panel-out auth/output_probes.json
python3 scripts/probe_max_tokens.py --key "$CODEBUDDY2API_KEY" --models glm-5.2 --resume --panel-out auth/output_probes.json
python3 scripts/probe_max_tokens.py --key "$CODEBUDDY2API_KEY" --dry-run

--panel-out merges results into output_probes.json beside control.sqlite3; models not re-measured keep their previous record. GET /admin/models/output-probes serves the file read-only and picks up a rerun on the next request without a restart. Probing spends real credits.

Reasoning compatibility

Messages enabled / adaptive activate Chat reasoning; output_config.effort and Responses reasoning.effort map to reasoning_effort. An explicit top-level reasoning_effort takes precedence, except Messages disabled always selects none; model capability checks still apply. Omitted controls leave upstream defaults unchanged.

Without an explicit effort, Messages activation uses the selected account's reasoning.defaultEffort or legacy reasoning.effort, restricted to its declared options. Otherwise it prefers high, then an available option; unknown declarations fall back to high. Failover resolves the replacement account's default again.

Manual enabled requires an integer budget_tokens >= 1024, but the budget is not an exact upstream token limit; max_tokens is forwarded unchanged. This mapping does not reproduce native adaptive scheduling. Only display: summarized is supported.

Readable history is kept in reasoning_content, never ordinary answer text. Responses uses readable content before summary; summaries cannot reconstruct native hidden reasoning. Signatures are not forwarded. redacted_thinking, encrypted-only thinking and non-empty Responses encrypted_content return 400 before routing; upstreams decide which readable history they use.

Request boundaries

  • All three generation protocols normalize developer to system and move an existing system message first without mutating the caller's payload. Optional Responses projection and desensitization process content separately; the whole pipeline is not a verbatim pass-through by default.
  • prompt_mode is hot and configurable through WebUI, --prompt-mode or CODEBUDDY2API_PROMPT_MODE. Default empty adds no gateway system prompt and never clears caller instructions. prepend inserts exactly one prefix after routing: You are WorkBuddy, a powerful AI assistant. plus the official task goal for WorkBuddy profiles, or You are CodeBuddy Code. plus the official CLI task instructions for CLI profiles. The prefixes are distilled from the official templates, and retries do not duplicate them. DSH adaptation remains independent of this switch.
  • An always-on DSH adapter converts the official fixed identity, shared coding persona (preserving the model ID), minimal persona, direct You are DeepSeek Harness. and current/legacy DSH Web GUI role context to WorkBuddy wording in system, developer and assistant text and text blocks. It also rewrites the confirmed Web GUI whose URL the runtime context names. When the user refers fingerprint, including assistant quotations. It covers standard/PTC/Cordis/minimal/headless/web/SDK/ACP, leaves plan instructions, source-checkout descriptions and unrelated brand references unchanged, and remains active when desensitization is off. User/tool content is not globally rewritten.
  • Images count across all history and tool results, including duplicates, in message/content array order. The default keeps the newest 16, removing only excess images while retaining text and message structure; emptied image content receives a text placeholder.
  • --image-policy error returns local 413 / too_many_images. JSON still over budget after processing returns 413 / request_too_large, without further text truncation to fit the limit.
  • Image count does not guarantee acceptable individual image sizes or model vision support. URL/base64 images can be converted; Responses image file_id is unsupported.
  • With stream_mode=compatible (the default), streaming Responses and Chat/Messages with tools aggregate and validate before emitting SSE; Chat/Messages without tools pass through upstream increments. realtime streams all three incrementally, while omitted/non-stream requests remain complete validated JSON.
  • Inference errors follow the client protocol: OpenAI routes return a top-level error object and Messages returns {"type": "error", ...}. Status codes are retained; errors after streaming starts are reported through SSE without replay.
  • Valid upstream Retry-After values (0–86400 seconds or equivalent HTTP dates) are returned as seconds before streaming starts; 429 only cools the selected account/model. Invalid or expired values fall back to the body's reset time or 600 seconds. Pool-generated 429 responses include the remaining wait.
  • Chat and Responses preserve an explicit client prompt_cache_key without generating one; cache hits and savings depend on the upstream.
  • Unsupported capabilities are rejected rather than silently degraded: chat n other than 1 and the Responses state fields previous_response_id/conversation (this gateway keeps no server-side response state) return 400; length-truncated or content-filtered Responses are reported as incomplete, never disguised as completed.
  • SQLite auditing retains only bounded, redacted diagnostics, not complete original requests. Treat logs, credential exports and backups as private data.

Deployment exposure and credential intake

  • Compose maps loopback by default. Configure an external key or create a database key before exposing the port; do not rely on anonymous mode for public access.
  • With no external or database keys, CODEBUDDY2API_ALLOW_OPEN_NOAUTH=true anonymously exposes inference and account-pool billing, with or without a placeholder token. Any key row forces authentication; with no keys and the option off, every inference request returns 401 api_key_required.
  • Run one gateway process per data directory. Other-process key changes become visible within about one second, but RPM counters and last_used_at aggregation are process-local. Backups contain standard keys in plaintext and must be protected like credentials.
  • When a key is configured, generation and token-count POSTs verify request headers before buffering bodies or reserving inference capacity; invalid keys return 401 even while generation slots are full. Other routes retain their existing authentication and routing behavior.
  • Credential imports/uploads persist the normalized form (token aliases folded into the canonical fields); strict JSON parsing rejects NaN/Infinity, and expiresAt/lastRefreshTime must be plausible finite millisecond timestamps.

Billing data integrity

  • Balances and usage are paginated in full; when a page cap is hit or an account's sync fails, responses carry partial: true (and stale_accounts) instead of pretending to be exact.
  • A failed account keeps its last good snapshot; HTTP 200 responses with a failing business code or missing structure are treated as errors and never overwrite history.
  • If every account fails before a first snapshot, both billing endpoints still report partial data and stale accounts; quota-delta fallback remains in use. Existing account snapshots retain their original fetch time on failures.
  • daily_costs are priced per site per day at that site's price, not at one blended average.

Troubleshooting and retries

Symptom Behavior / action
Cannot sign in to WebUI Use the initial or current admin password, or run converter.py admin reset-password. Behind an HTTPS reverse proxy, trust the public origin via admin_allowed_origins (see above)
Local 401 / 403 / 429 Check that the inference key exists and is enabled and unexpired, then inspect its IP/model allowlist and RPM; authentication and permission failures use protocol-specific error envelopes
Upstream 401 / 403 Credential-level authentication circuit opens; inspect and log in again in the WebUI
429 Cool down that upstream model on the credential; later requests rebind automatically. All candidates cooling down still returns 429; with --failover-max the in-flight request is replayed on another credential instead
Channel rejection (11128) One same-account /console retry when available; a second channel refusal may move system instructions into a user message and retry once on the same account. A final refusal returns without cooling or replaying any credential. Search/MCP have no console detour and remain usable
Account restriction (11140) Mark the credential restricted for 2 hours, persist the evidence and exclude it from routing; after an appeal the credential page can re-enable it immediately. The current request is never replayed on another credential
Model restriction (11148) Cool only that account/model for 30 minutes; do not cool the account or replay the current request
Model-scoped 11129/11134/11150/11158 The business code takes precedence over HTTP status, including 500/503. No account/model cooldown or credential failover; return the upstream response
Upstream service info not found (11102) Confirmed 400/404 model rejection backs off by (backend, model) domestically and (account, product, model) internationally, without an account cooldown. Return 404 when no candidate remains; half-open after 6 h, up to 24 h on repeats, cleared on success. Console-confirmed model-missing responses may still fail over. Old international endpoint-wide entries no longer block accounts; inspect via GET /admin/model-blocks
Connection setup failure Retry once on a fresh connection: ConnectError and ConnectTimeout fail before the first body byte, so the upstream holds nothing and replaying cannot double-bill
Post-send disconnect, read timeout or protocol error No network replay, avoiding duplicate billing; logs include exception type and elapsed time
Client hangs up before a non-streaming response is ready The upstream call is cancelled and its concurrency slot returned at once; the request is audited as cancelled, never as a completed answer. Streaming already behaves this way
Streaming request fails before the first byte Reported with the real HTTP status, exactly like stream=false. A 200 carrying only an in-band error event is read by clients as an empty answer, so the session ends silently while the audit log records a success
Credential failover (--failover-max) Off by default. When enabled, a failure before any byte reached the client is retried on another credential up to N times and audited as success with a failover_recovered marker. Qualifying failures: upstream HTTP 401/403/429/502/503/504 rejections and bodies the upstream provably never received (ConnectError/ConnectTimeout). Content-filter rejections, 11128/11140/11148, 11129/11134/11150/11158, 502s from an already-open stream, read timeouts and protocol errors are never replayed; without another credential the original status surfaces. Billing note: 401/403/429/503 and transport failures happen at admission and cannot be billed; a 502/504 may already have been billed upstream, but its result never reached the client, so refusing to replay recovers no credit — it only turns a paid-for attempt into a broken session. Such replays are tagged 上游可能已处理该请求 in the log for reconciliation
Write-timeout replay (--retry-write-timeout) Off by default. A write timeout proves the body was not fully sent, not that the upstream ignored the bytes it received, so it stays excluded from connect retry and failover until enabled. Long cross-border sessions fail here more often than in the handshake; enable only when the upstream is confirmed not to bill partial bodies. These replays carry the same 上游可能已处理该请求 log tag
Malformed tool calls Compatible aggregate validation permits up to --tool-call-max-retry (default 3) additional generations, each consuming credits and recorded with its usage in the attempt details; exhaustion returns an error. Realtime mode never regenerates: it reports a protocol error before a success terminal
Empty or truncated upstream stream No valid output, a missing end marker or an error is not reported as success
Content-filter rejection With desensitization and --no-compact, a complete non-streaming filter-only rejection may receive one shorter-template retry on the same account. No streaming filter retry, circuit opening or account rotation
Slow responses Inspect timing and failed attempts in the WebUI, then choose a faster model supported by the account
Same account invalidated elsewhere Independent desktop/gateway refreshes may invalidate each other; prefer separate browser login or stop using the other client

Downgrades and rollback

Feature switches hold no hidden state: disabling a guard or mode stops it for new requests, and reverting source restores previous behavior. The exceptions are persisted settings and automation state: control.sqlite3 stores WebUI settings, model rules and reward reservations, and older code rejects unknown fields. Control schema v3 (with api_keys), audit schema v5 and media schema v3 are not readable by 1.4.x; downgrade requires restoring a pre-upgrade backup while stopped. For older schema-only changes, remove newly added startup options and use the narrowly scoped offline settings.<name> removal procedure above with a revision increment and integrity check. Do not restore a stale pre-upgrade database or mix WAL/SHM generations: doing so could roll back newer claims, sessions, revocations and account state. Rollback never undoes completed upstream check-ins, claims or travel dispatches.