Add add-llm-provider skill for onboarding new LLM providers/engines - #7903
Merged
Conversation
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add setup instructions for new LLM provider
Add add-llm-provider skill for onboarding new LLM providers/engines
Aug 31, 2026
lpcox
marked this pull request as ready for review
August 31, 2026 16:28
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Several instructions reference outdated architecture or could mishandle provider credentials.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds guidance for onboarding LLM providers and direct-API engines, addressing #7900.
Changes:
- Defines proxied-provider and direct-API integration paths.
- Documents allowlisting, credential, testing, and telemetry considerations.
File summaries
| File | Description |
|---|---|
.github/skills/add-llm-provider/SKILL.md |
Adds the provider-onboarding decision guide. |
Review details
Suppressed comments (1)
.github/skills/add-llm-provider/SKILL.md:45
- The API-proxy telemetry file is
token-usage.jsonl, nottoken_usage.jsonl; using the exact name is important because downstream artifacts can contain a separate underscored usage file.
4. **Telemetry caveat:** because traffic never passes through the api-proxy sidecar, AWF's `token_usage.jsonl` / token-tracking metrics will stay empty for this engine. Any downstream check that assumes all engines produce proxy telemetry (e.g. a "token usage present" CI gate) must exclude direct-API engines instead of trying to make them populate it.
- Files reviewed: 1/1 changed files
- Comments generated: 7
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
|
||
| ## Step 1 — Determine how the engine talks to its LLM backend | ||
|
|
||
| Ask: **does the engine call the provider's API directly (its own base URL, its own auth), or is it expected to route through AWF's universal `--enable-api-proxy` sidecar** (`http://172.30.0.30:1000{0,1,2,3}` for OpenAI/Anthropic/Copilot/Gemini)? |
|
|
||
| 1. Create `containers/api-proxy/providers/<name>.js` implementing the `ProviderAdapter` interface. | ||
| 2. Register it in `containers/api-proxy/providers/index.js`. | ||
| 3. Update `containers/api-proxy/Dockerfile` (`COPY` list + `EXPOSE` port). |
| 1. Create `containers/api-proxy/providers/<name>.js` implementing the `ProviderAdapter` interface. | ||
| 2. Register it in `containers/api-proxy/providers/index.js`. | ||
| 3. Update `containers/api-proxy/Dockerfile` (`COPY` list + `EXPOSE` port). | ||
| 4. Update `src/types/ports.ts` and `src/host-iptables-rules.ts` for the new port. |
| 2. Register it in `containers/api-proxy/providers/index.js`. | ||
| 3. Update `containers/api-proxy/Dockerfile` (`COPY` list + `EXPOSE` port). | ||
| 4. Update `src/types/ports.ts` and `src/host-iptables-rules.ts` for the new port. | ||
| 5. Forward any new env vars from the host in `src/docker-manager.ts`. |
| 5. Forward any new env vars from the host in `src/docker-manager.ts`. | ||
| 6. Add the upstream domain(s) to the allowlist wherever the caller configures `--allow-domains` (AWF itself does not hardcode per-provider domains). | ||
| 7. Document auth details in [docs/auth-matrix.md](../../../docs/auth-matrix.md) if it's a net-new auth pattern. | ||
| 8. Write adapter unit tests (`providers/<name>.test.js`) and run them per the memory note below. |
| awf --allow-domains api2.cursor.sh,api3.cursor.sh -- cursor-agent ... | ||
| ``` | ||
| or in the AWF config file under `network.allowDomains` (see [docs/awf-config-spec.md](../../../docs/awf-config-spec.md) and [docs/awf-config.schema.json](../../../docs/awf-config.schema.json)). | ||
| 3. If the engine's real API key must reach the agent container as an env var, treat it the same as any other secret: prefer `sensitiveAllowedDomains` handling and redaction (see credential-isolation memory below) rather than plumbing it through the api-proxy. |
This was referenced Aug 31, 2026
Collaborator
|
@copilot address review feedback |
Contributor
|
✅ Copilot review passed with no inline comments. @copilot Add the |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Setting up AWF support for a new LLM provider or agent engine (e.g. Cursor, which calls its own API endpoint directly rather than through AWF's api-proxy sidecar) had no clear guidance, leading to confusion about whether a proxy adapter is required.
Changes
.github/skills/add-llm-provider/SKILL.md— decision-tree guide for integrating a new LLM provider/engine, following the same format as existing skills (pr-finisher,debug-firewall, etc.)--enable-api-proxy— delegates to the existingcontainers/api-proxy/providers/ADDING-A-PROVIDER.mdadapter guide--allow-domains/network.allowDomains) is needed, no adapter requiredtoken_usage.jsonlstays empty, so any check assuming proxy telemetry for all engines must exclude direct-API enginesdocs/auth-matrix.md,docs/awf-config-spec.md,docs/quickstart.md, anddocs/squid_log_filtering.mdfor domain verification via Squid access logsExample
# Direct-API engine (Path B) — no proxy adapter, just allowlist its domains awf --allow-domains api2.cursor.sh,api3.cursor.sh -- cursor-agent ...