Skip to content

fix(gastown): use the configured BYOK model for review-thread classification - #6035

Open
maphew wants to merge 4 commits into
Kilo-Org:mainfrom
maphew:kilo/twilight-bobcat-nsr
Open

fix(gastown): use the configured BYOK model for review-thread classification#6035
maphew wants to merge 4 commits into
Kilo-Org:mainfrom
maphew:kilo/twilight-bobcat-nsr

Conversation

@maphew

@maphew maphew commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Refs #4268 (does not close it — the issue also asks for disclosure/configurability of the auxiliary model for non-BYOK users).

Problem

areThreadsBlocking() in services/gastown/src/dos/town/town-scm.ts always classified unresolved PR review threads with a hardcoded Cloudflare Workers AI call (@cf/google/gemma-4-26b-a4b-it). A BYOK town therefore had one review-path call that ignored the town's configured model, with no UI control or disclosure.

This is a separate follow-up from the Code Reviewer credit-balance fix. The Workers AI call runs on the Cloudflare binding, not the Kilo gateway, so it does not draw Kilo credits; the motivation here is BYOK users expecting the configured model to be used throughout the flow (and the issue's request that the auxiliary call be honored/configurable).

Change

  • When role_models.refinery ?? default_model is a direct-BYOK model and the town has a kilocode_token, classify via POST {KILO_API_URL}/api/openrouter/chat/completions on that model. The call uses the town's Kilo token (the same token injected into the container as KILOCODE_TOKEN), sends X-KiloCode-Feature: gastown and, for org towns, X-KiloCode-OrganizationId. The gateway resolves the direct-BYOK model to the user's own provider key (getDirectByokModel), so the call bills that key.
  • Non-BYOK towns keep the existing Workers AI path. Managed users see no change.
  • A rejected/failed BYOK call throws and areThreadsBlocking() conservatively returns true (blocks auto-merge) rather than silently substituting a different model.
  • New @kilocode/worker-utils/direct-byok-model exports isDirectByokModelId() / DIRECT_BYOK_PROVIDER_IDS, plus a drift-guard test in apps/web that keeps the list equal to DIRECT_BYOK_PROVIDERS_META.

Adversarial review

An independent review pass and a second round of checks found two issues, both fixed in the second commit:

  1. Reasoning models could starve content. Reasoning-tagged BYOK models (e.g. zai-coding, the recommended neuralwatt models) can spend the 256-token budget on a thinking trace and return no JSON, which would block auto-merge for BYOK towns. The request now sends reasoning: { enabled: false, effort: 'none' } (the gateway's canonical "none" variant), mirroring the Gemma path's enable_thinking: false. Test asserts it.
  2. Failure-rationale wording was inaccurate. The docs/comment claimed Workers AI is "Kilo-billed"; it is infrastructure cost, not user credits. Reworded. Fail-closed is kept so a BYOK town never silently falls back to a different model than the one the user configured.

Also reviewed and accepted:

  • Token acceptance: the gateway requires audience KILO_GATEWAY_AUDIENCE, but isKiloResourceAudienceAllowed(..., mode: 'allow-legacy') accepts the town token's missing aud claim, so existing gastown tokens work.
  • Org header: X-KiloCode-OrganizationId matches ORGANIZATION_ID_HEADER (case-insensitive); verified against security-auto-analysis and review-memory callers.
  • Prefix-only detection: isDirectByokModelId() matches on the provider prefix, while the gateway routes to BYOK only for catalog model ids. The configured model comes from the same picker/catalog that mints these ids; an unknown provider/model under a direct-BYOK prefix is rejected (4xx) by the gateway and fails closed. Documented in the module.
  • Rig overrides: resolution is town-level (role_models.refinery ?? default_model), consistent with the rest of the auto-merge path; SCMContext carries no rig. Noted in a code comment.

Verification

  • services/gastown: tsgo --noEmit, oxlint (0 warnings/errors), vitest run — 344 passed (9 in the new town-scm.test.ts: BYOK→gateway incl. headers/body/reasoning, refinery-model preference, managed-refinery override of a BYOK default, org header, non-BYOK→Workers AI, no-token→Workers AI, 402→blocking without fallback, network error→blocking without fallback, empty content→blocking).
  • packages/worker-utils: tsgo --noEmit, oxlint, vitest run — 602 passed.
  • apps/web: tsgo --noEmit passes with the new drift guard. The drift test itself could not run in this sandbox because apps/web Jest's per-worker setup requires a live PostgreSQL (no Docker available); the list equality was validated directly and CI covers the Jest run.
  • oxfmt --list-different clean on all changed files.

Notes

maphew and others added 2 commits September 10, 2026 02:52
…ication

areThreadsBlocking() hardcoded a Workers AI Gemma call for the auto-merge
review-thread classifier. Direct-BYOK towns now run the classifier on
role_models.refinery ?? default_model through the Kilo gateway, so the call
bills the user's own provider key. Non-BYOK towns keep the Workers AI path.
A rejected BYOK call blocks without falling back to a Kilo-billed path.

Refs Kilo-Org#4268

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
A reasoning-tagged BYOK model can spend the 256-token budget on a thinking
trace and return no JSON content, which makes areThreadsBlocking()
conservatively block auto-merge. Send reasoning: { enabled: false,
effort: 'none' }, the gateway's "none" variant, and add tests for the
refinery-override and gateway network-failure paths.

Refs Kilo-Org#4268
@maphew
maphew marked this pull request as ready for review September 10, 2026 03:05
@kilo-code-bot

kilo-code-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (7 files)
  • apps/web/src/lib/ai-gateway/providers/direct-byok/direct-byok-provider-ids.drift.test.ts
  • packages/worker-utils/package.json
  • packages/worker-utils/src/direct-byok-model.test.ts
  • packages/worker-utils/src/direct-byok-model.ts
  • services/gastown/docs/local-debug-testing.md
  • services/gastown/src/dos/town/town-scm.test.ts
  • services/gastown/src/dos/town/town-scm.ts

Reviewed by grok-4.6 · Input: 100K · Output: 11.3K · Cached: 256.1K

Review guidance: REVIEW.md from base branch main

maphew and others added 2 commits September 10, 2026 12:04
…t-nsr

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants