Skip to content

fix(provider): add Cheaper Inference as a built-in provider - #3893

Open
aiapienthusiast wants to merge 2 commits into
tailcallhq:mainfrom
aiapienthusiast:feat/cheaperinference-provider
Open

aiapienthusiast wants to merge 2 commits into
tailcallhq:mainfrom
aiapienthusiast:feat/cheaperinference-provider

Conversation

@aiapienthusiast

@aiapienthusiast aiapienthusiast commented Sep 17, 2026

Copy link
Copy Markdown

Adds Cheaper Inference as a built-in provider.

Cheaper Inference is an OpenAI-compatible gateway. One endpoint and one API key reach models from several upstream providers. The entry declares "response_type": "OpenAI" and uses the existing OpenAI request path, so this change needs no new response handling.

Cheaper Inference is one of the fastest-growing AI routers.
Each model costs 15–60% less than the list price of its lab.

The change follows #3623 (fix(provider): add OrcaRouter as a built-in provider), which added OrcaRouter in the same way.

Provider details

The repository has a provider-integration issue template that asks for these details:

Field Value
Provider name Cheaper Inference
API documentation https://cheaperinference.com/docs
API compatibility OpenAI-compatible
Authentication method API key, read from CHEAPERINFERENCE_API_KEY
Chat completions https://api.cheaperinference.com/v1/chat/completions
Model list https://api.cheaperinference.com/v1/models

Changes

File Change
crates/forge_repo/src/provider/provider.json New cheaper_inference entry: API key variable, OpenAI response type, chat-completions URL, models URL, api_key auth.
crates/forge_domain/src/provider.rs ProviderId::CHEAPER_INFERENCE, added to built_in_providers() and to the FromStr match. Plus a display_name arm, because the default conversion gives CheaperInference and the display name is Cheaper Inference. Three tests mirror test_orca_router_*. The new id joins the two shared assertion lists.
crates/forge_repo/src/provider/provider_repo.rs test_cheaper_inference_config, which mirrors test_orca_router_config.
README.md Provider env-var block, in the same shape as the surrounding <details> sections.

The model list stays dynamic

provider.json does not enumerate the models. The entry points models at the provider's /models endpoint. The catalogue and the per-token pricing are discovered at runtime, not pinned in the repository. dto::openai::Pricing already reads that pricing and treats it as optional.

#3623 did the same. test_cheaper_inference_config asserts the Models::Url variant, so a later switch to a hardcoded list fails loudly.

What this PR does not do

This PR does not touch crates/forge_app/src/dto/openai/model.rs and adds no /models fixture.

In #3623 that part was a schema relaxation, which OrcaRouter's response shape made necessary. Those fields are already optional on main. This PR changes no deserialisation. No response from this provider is captured, so a fixture would be invented, not recorded.

Testing

Run against 7ef8386acaa9564a76526828c53d430e1696730a, the current main:

  • cargo test --lib -p forge_domain618 passed, 0 failed, with the three new test_cheaper_inference_* cases
  • cargo test --lib -p forge_repo334 passed, 0 failed, 1 ignored, with the new test_cheaper_inference_config
  • cargo clippy -p forge_domain -p forge_repo --lib --all-targetsclean, 0 warnings
  • rustfmt --check on the two touched Rust files — no diff

Toolchain limit

The preparation environment cannot install the toolchain that rust-toolchain.toml pins (1.97), because static.rust-lang.org is unreachable there. The checks above ran on stable 1.95. Two points need a maintainer's attention:

  • rustfmt ran without the nightly-only options in .rustfmt.toml. Stable rustfmt ignores them with a warning. The new code adds no imports and wraps to 80 columns, which is what those options govern. Please re-run cargo fmt on the pinned toolchain.
  • cargo clippy ran clean on the two crates, but on the 1.95 lint set, and not across the whole workspace.

Not tested through forge

A direct check with a real key confirmed the endpoint. GET /v1/models returns the catalogue. POST /v1/chat/completions answers. The response carries the bare model id that was sent.

A reviewer should confirm the path through forge: the provider resolves from CHEAPERINFERENCE_API_KEY, and a completion succeeds from the app.

Cheaper Inference is an OpenAI-compatible gateway, so the entry declares
"response_type": "OpenAI" and reuses the existing OpenAI request path. Models are
discovered from the provider's /models endpoint rather than pinned, as in tailcallhq#3623.

- provider.json: the cheaper_inference entry (api key var, chat-completions URL,
  models URL, api_key auth)
- provider.rs: ProviderId::CHEAPER_INFERENCE, built_in_providers(), the FromStr
  arm and a display_name arm for "Cheaper Inference", plus three tests
- provider_repo.rs: test_cheaper_inference_config, which pins the Models::Url
  variant
- README.md: the env-var block, next to the other providers

cargo test --lib -p forge_domain: 618 passed, 0 failed.
cargo test --lib -p forge_repo: 334 passed, 0 failed, 1 ignored (pre-existing).
Both on stable 1.95, because the pinned 1.97 toolchain cannot be downloaded here;
the added code adds no imports and is wrapped to 80 columns, but please re-run
cargo fmt on the pinned toolchain.
@github-actions github-actions Bot added type: fix Iterations on existing features or infrastructure. type: provider Updates provider.json configuration. labels Sep 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: fix Iterations on existing features or infrastructure. type: provider Updates provider.json configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant