diff --git a/docs/mkdocs/en/graph.md b/docs/mkdocs/en/graph.md index 83ed3038a..5726be593 100644 --- a/docs/mkdocs/en/graph.md +++ b/docs/mkdocs/en/graph.md @@ -413,7 +413,7 @@ Scenario: Integrate any BaseAgent (e.g., LlmAgent/GraphAgent) as a node in the g graph.add_agent_node( node_id="delegate", agent=delegate_agent, - isolated_messages=True, + history_scope="branch", input_from_last_response=False, event_scope="delegate_scope", input_mapper=StateMapper.rename({"query_text": STATE_KEY_USER_INPUT}), @@ -423,12 +423,29 @@ graph.add_agent_node( ``` Common options: -- isolated_messages: Whether to isolate the parent session's message history +- history_scope: Child history policy. `none` starts without parent history, + `branch` inherits only events from the same Agent-node branch (including its + nested branches), and `all` inherits the complete parent event history. +- isolated_messages: Legacy compatibility switch. When history_scope is omitted, + `True` maps to `none` and `False` maps to `all`. An explicit history_scope + takes precedence. - input_from_last_response: Whether to map the parent state's last_response as the child node's user_input - event_scope: Event branch prefix for the child Agent - input_mapper / output_mapper: Parent-child state mapping (explicit configuration recommended) - config / callbacks: Same as add_node +`branch` filters the child Session event log. The transient graph `messages` +state stays empty and GraphAgent rebuilds model input from those filtered events; +this keeps persistent Session state JSON-serializable. The policy is useful when +the same Agent node is entered again in a later outer run without exposing +private conversations from sibling nodes. During HITL resume, legacy +`isolated_messages=True` nodes still recover their current branch so the pending +function-call exchange remains intact. + +When a child Agent emits a `LongRunningEvent`, `add_agent_node` promotes it to a parent GraphAgent `interrupt`: the parent graph does not execute downstream nodes, and the Runner event preserves the original tool name and arguments. After the client submits the matching `FunctionResponse`, the parent graph resumes the current Agent node and the SDK maps the response back to the child's original function call. The graph continues only after the child Agent reaches a final result. This supports multiple HITL rounds in one node and persists child state in the SessionService-backed checkpoint for process-restart recovery. + +When the Agent node is a TeamAgent, the Leader can use the same HITL flow. Regular Team Members still must not be configured with or invoke `LongRunningFunctionTool`. + GraphAgent does not require (nor support) registering Agent nodes via sub_agents; composition relationships are handled uniformly through add_agent_node. ## Advanced Usage diff --git a/docs/mkdocs/en/model.md b/docs/mkdocs/en/model.md index 5b9618e76..a67812c8b 100644 --- a/docs/mkdocs/en/model.md +++ b/docs/mkdocs/en/model.md @@ -121,6 +121,52 @@ model = OpenAIModel( ) ``` +#### Responses API + +`OpenAIModel` uses Chat Completions by default. Enable the Responses API explicitly for OpenAI or compatible +providers that expose `/v1/responses`: + +```python +model = OpenAIModel( + model_name="your-responses-model", + api_key="your-api-key", + base_url="https://api.openai.com/v1", + use_responses_api=True, + responses_api_params={ + "store": False, + "reasoning": {"summary": "auto"}, + }, +) +``` + +The switch is opt-in so existing OpenAI-compatible providers continue to use Chat Completions. The adapter maps +conversation history, function calls and `function_call_output` items, structured output, semantic streaming events, +reasoning summaries, and token usage into the existing tRPC-Agent types. When `store=False`, the SDK automatically +requests `reasoning.encrypted_content` so reasoning items can be replayed with tool outputs in the next turn. + +`responses_api_params` accepts Responses-only fields such as `store`, `reasoning`, `include`, and `truncation`. +`model`, `input`, and `stream` are managed by `OpenAIModel` and cannot be overridden there. + +The field is typed as the openai SDK's `ResponseCreateParams` (`openai.types.responses`), so IDEs +auto-complete every native parameter. All values are passed through verbatim to `responses.create` +without any SDK-side mapping. To control reasoning depth, pass `reasoning.effort` directly: + +```python +model = OpenAIModel( + model_name="your-responses-model", + api_key="your-api-key", + use_responses_api=True, + responses_api_params={ + "reasoning": {"effort": "high"}, + }, +) +``` + +Supported effort values (`none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`) vary by model — +see the OpenAI reasoning guide for model-specific support. The SDK deliberately does not map +`thinking_budget` to `effort`; it only requests `reasoning.summary` when thinking is enabled so the +reasoning output stays readable. + #### Advanced Usage Since version `1.1.10`, `OpenAIModel` supports passing a shared HTTP client provider to enable connection reuse. By default, `OpenAIModel` creates a temporary HTTP client for each model-service request. If you want to reuse connections, use the following configuration: diff --git a/docs/mkdocs/zh/graph.md b/docs/mkdocs/zh/graph.md index f5683e0ea..b837fdc8a 100644 --- a/docs/mkdocs/zh/graph.md +++ b/docs/mkdocs/zh/graph.md @@ -412,7 +412,7 @@ graph.add_mcp_node( graph.add_agent_node( node_id="delegate", agent=delegate_agent, - isolated_messages=True, + history_scope="branch", input_from_last_response=False, event_scope="delegate_scope", input_mapper=StateMapper.rename({"query_text": STATE_KEY_USER_INPUT}), @@ -422,12 +422,26 @@ graph.add_agent_node( ``` 常用选项： -- isolated_messages：是否隔离父会话消息历史 +- history_scope：子 Agent 历史策略。`none` 不继承父历史，`branch` 只继承 + 当前 Agent 节点 branch 及其子 branch 的事件，`all` 继承完整父会话事件。 +- isolated_messages：旧版兼容开关。未指定 history_scope 时，`True` 映射为 + `none`，`False` 映射为 `all`；显式 history_scope 优先。 - input_from_last_response：是否将父状态 last_response 映射为子节点 user_input - event_scope：子 Agent 事件分支前缀 - input_mapper / output_mapper：父子状态映射（推荐显式配置） - config / callbacks：同 add_node +`branch` 过滤 child Session 的事件日志；临时图状态中的 `messages` 保持为空， +GraphAgent 在需要时从已过滤事件重建模型输入，从而保证持久化 Session state +仍可 JSON 序列化。该策略适用于同一 Agent 节点在后续外层 run 中再次进入、 +但又不能看到兄弟节点私有对话的场景。HITL 恢复时，旧的 +`isolated_messages=True` 节点仍会恢复当前 branch，以保留挂起的 +function-call 交互链路。 + +当子 Agent 发出 `LongRunningEvent` 时，`add_agent_node` 会将其提升为父 GraphAgent 的 `interrupt`：父图不会执行后继节点，Runner 返回的事件保留原工具名和参数。客户端提交对应 `FunctionResponse` 后，父图恢复当前 Agent 节点，SDK 将响应映射回子 Agent 的原始 function call；只有子 Agent 最终完成后父图才继续。该机制支持同一节点多轮 HITL，并将 child state 写入 SessionService-backed checkpoint，服务重启后仍可恢复。 + +TeamAgent 作为 Agent 节点时同样支持 Leader 发起 HITL；普通 Team Member 仍不允许配置或调用 `LongRunningFunctionTool`。 + GraphAgent 不需要（也不支持）通过 sub_agents 注册 Agent 节点；组合关系统一用 add_agent_node 完成。 ## 进阶用法 diff --git a/docs/mkdocs/zh/model.md b/docs/mkdocs/zh/model.md index 16e2320b3..f74ed7717 100644 --- a/docs/mkdocs/zh/model.md +++ b/docs/mkdocs/zh/model.md @@ -121,6 +121,49 @@ model = OpenAIModel( ) ``` +#### Responses API + +`OpenAIModel` 默认仍使用 Chat Completions。对于 OpenAI 或提供 `/v1/responses` 的兼容服务，需要显式开启： + +```python +model = OpenAIModel( + model_name="your-responses-model", + api_key="your-api-key", + base_url="https://api.openai.com/v1", + use_responses_api=True, + responses_api_params={ + "store": False, + "reasoning": {"summary": "auto"}, + }, +) +``` + +该开关默认关闭，现有 OpenAI-compatible 服务不会改变调用路径。适配层会把多轮消息、函数调用及 +`function_call_output`、结构化输出、语义化流式事件、reasoning summary 和 token usage 映射为现有 +tRPC-Agent 类型。设置 `store=False` 时，SDK 会自动请求 `reasoning.encrypted_content`，以便下一轮 +连同工具结果一起回放 reasoning item。 + +`responses_api_params` 可传入 `store`、`reasoning`、`include`、`truncation` 等 Responses 专用字段； +`model`、`input`、`stream` 由 `OpenAIModel` 管理，不能在此覆盖。 + +该字段类型为 openai SDK 的 `ResponseCreateParams`（`openai.types.responses`），IDE 可直接提示全部原生参数； +所有字段原样透传给 `responses.create`，SDK 不做任何映射。控制推理深度时直接传 `reasoning.effort`： + +```python +model = OpenAIModel( + model_name="your-responses-model", + api_key="your-api-key", + use_responses_api=True, + responses_api_params={ + "reasoning": {"effort": "high"}, + }, +) +``` + +支持的档位（`none`、`minimal`、`low`、`medium`、`high`、`xhigh`、`max`）随模型而异， +具体以 OpenAI reasoning 指南及模型文档为准。SDK 刻意不做 `thinking_budget` → `effort` 的映射； +仅在开启 thinking 时请求 `reasoning.summary`，保证推理输出可读。 + #### 高级用法 从版本 `1.1.10`之后 OpenAIModel 支持传入共享的 http client 来解决连接复用的场景，当前的 OpenAIModel 默认每次都会创建临时的 http client 去访问模型服务；如果期望连接复用可以使用如下的方式 diff --git a/pipeline_test/test_responses_api_ext.py b/pipeline_test/test_responses_api_ext.py new file mode 100644 index 000000000..5adfb1e71 --- /dev/null +++ b/pipeline_test/test_responses_api_ext.py @@ -0,0 +1,330 @@ +#!/usr/bin/env python3 +"""Extended test for tRPC-Agent's OpenAI Responses adaptation. + +Covers the capabilities added in PR #245 (NOT a plain connectivity check): + + 1. thinking/effort passthrough: reasoning.effort delivered verbatim via + responses_api_params, reasoning.summary auto-injected when thinking is on + 2. tool calling round-trip: function_call -> function_call_output (multi-turn) + 3. store=False: SDK auto-appends "reasoning.encrypted_content" to include + 4. generation params passthrough: temperature / top_p / truncation + 5. streaming tool calling: streaming_tool_names deltas + final function_call, + followed by a function_call_output replay round (stream=True) + 6. store=False multi-turn reasoning replay: first-turn reasoning items + (with encrypted_content) survive into the second-turn request input + +Usage: + python pipeline_test/test_responses_api_ext.py [--stream] +""" + +import argparse +import asyncio +import json +import os + +from google.genai import types as genai_types + +from trpc_agent_sdk.models import LlmRequest, OpenAIModel +from trpc_agent_sdk.types import Content, FunctionResponse, GenerateContentConfig, Part, ThinkingConfig + +# The openai SDK appends "/responses" to base_url for the Responses resource. +BASE_URL = "https://tokenhub.tencentmaas.com/v1" +MODEL = "deepseek-v4-flash-202605" +API_KEY = os.environ.get("LKEAP_API_KEY") +if not API_KEY: + raise SystemExit("LKEAP_API_KEY is required, e.g. export LKEAP_API_KEY=sk-...") + +WEATHER_TOOL = genai_types.Tool(function_declarations=[ + genai_types.FunctionDeclaration( + name="get_weather", + description="获取指定城市的当前天气", + parameters=genai_types.Schema( + type="OBJECT", + properties={ + "city": genai_types.Schema(type="STRING", description="城市名，例如 北京"), + }, + required=["city"], + ), + ), +]) + + +def make_model(**kwargs) -> OpenAIModel: + return OpenAIModel( + model_name=MODEL, + api_key=API_KEY, + base_url=BASE_URL, + use_responses_api=True, + **kwargs, + ) + + +def _dump_part(part: Part) -> None: + if getattr(part, "text", None): + print(f" [text] {part.text}") + if getattr(part, "thought", False): + print(f" [thought] ") + fc = getattr(part, "function_call", None) + if fc is not None: + print(f" [function_call] {fc.name}({fc.args}) id={fc.id}") + + +async def test_thinking_effort(stream: bool) -> None: + print("\n=== 1. thinking + reasoning.effort passthrough ===") + model = make_model(responses_api_params={"reasoning": {"effort": "high", "summary": "auto"}}) + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="8 * 7 * 6 等于多少？请分步推理")])], + config=GenerateContentConfig(thinking_config=ThinkingConfig(include_thoughts=True, thinking_budget=-1)), + tools_dict={}, + ) + async for resp in model.generate_async(request, stream=stream): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.usage_metadata: + print(f" [usage] thoughts={resp.usage_metadata.thoughts_token_count} " + f"prompt={resp.usage_metadata.prompt_token_count} candidates={resp.usage_metadata.candidates_token_count}") + if resp.error_message: + print(f" [error] {resp.error_message}") + + +async def test_tool_roundtrip(stream: bool) -> None: + print("\n=== 2. tool calling round-trip (function_call -> function_call_output) ===") + model = make_model() + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="北京的天气怎么样？请用工具查询后告诉我")])], + config=GenerateContentConfig(tools=[WEATHER_TOOL]), + tools_dict={}, + ) + calls = [] + async for resp in model.generate_async(request, stream=False): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + fc = getattr(p, "function_call", None) + if fc is not None: + calls.append(fc) + if resp.error_message: + print(f" [error] {resp.error_message}") + if not calls: + print(" (model answered with text instead of a tool call; " + "tool registration may need a more targeted prompt)") + return + + print(" -> feeding function_call_output back for round 2") + parts = [Part.from_text(text="北京的天气怎么样？请用工具查询后告诉我")] + for fc in calls: + fc_id = getattr(fc, "id", None) or "call_weather_1" + # Part.from_function_call() leaves function_call.id None; the Responses + # API requires function_call_output.call_id to match a function_call + # item, so set the id explicitly to the model-issued call id. + fcall_part = Part.from_function_call(name=fc.name, args=fc.args) + fcall_part.function_call.id = fc_id + parts.append(fcall_part) + parts.append(Part(function_response=FunctionResponse( + id=fc_id, name=fc.name, response={"city": "北京", "weather": "晴", "temperature": 26}))) + round2 = LlmRequest( + contents=[ + Content(role="user", parts=parts), + # Note: function_call + function_response parts in one Content are + # split into responses "function_call" and "function_call_output" + # items by _convert_messages_to_responses_input. + ], + config=GenerateContentConfig(tools=[WEATHER_TOOL]), + tools_dict={}, + ) + async for resp in model.generate_async(round2, stream=stream): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + + +async def test_store_false(stream: bool) -> None: + print("\n=== 3. store=False (auto include reasoning.encrypted_content) ===") + model = make_model(responses_api_params={"store": False}) + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="1+1=?")])], + config=GenerateContentConfig(thinking_config=ThinkingConfig(include_thoughts=True, thinking_budget=-1)), + tools_dict={}, + ) + async for resp in model.generate_async(request, stream=stream): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + + +async def test_param_passthrough(stream: bool) -> None: + print("\n=== 4. generation params passthrough (temperature/top_p/truncation) ===") + model = make_model(responses_api_params={"truncation": "auto"}) + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="写一句关于春天的诗")])], + config=GenerateContentConfig(temperature=0.9, top_p=0.8), + tools_dict={}, + ) + async for resp in model.generate_async(request, stream=stream): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + + +async def test_streaming_tool_roundtrip() -> None: + """Streaming tool calling: incremental arguments via streaming_tool_names, + a complete function_call in the final response, and a second streaming + round that feeds function_call_output back to the model.""" + print("\n=== 5. streaming tool calling round-trip ===") + model = make_model() + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="深圳的天气怎么样？请调用工具查询后告诉我")])], + config=GenerateContentConfig(tools=[WEATHER_TOOL]), + tools_dict={}, + ) + request.streaming_tool_names = {"get_weather"} + calls = [] + delta_chunks = 0 + async for resp in model.generate_async(request, stream=True): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + fc = getattr(p, "function_call", None) + if fc is None: + continue + calls.append(fc) + args = getattr(fc, "args", None) or {} + if isinstance(args, dict) and args.get("tool_streaming_args"): + delta_chunks += 1 + if resp.error_message: + print(f" [error] {resp.error_message}") + + if not calls: + print(" (model answered with text instead of a streaming tool call; skipping replay)") + return + print(f" -> streamed {delta_chunks} argument delta chunk(s), {len(calls)} function_call part(s)") + + # The streaming path must emit at least one incremental delta when the tool + # is registered in streaming_tool_names, and the final response must carry + # a complete function_call for the replay. + if delta_chunks == 0: + print(" [warn] no streaming argument deltas observed (model may have skipped tool use)") + final_calls = [c for c in calls if not (isinstance(c.args, dict) and c.args.get("tool_streaming_args"))] + if not final_calls: + print(" (no complete function_call in the stream; skipping replay)") + return + + print(" -> feeding function_call_output back (streaming round 2)") + parts = [Part.from_text(text="深圳的天气怎么样？请调用工具查询后告诉我")] + for fc in final_calls: + fc_id = getattr(fc, "id", None) or "call_weather_stream" + fcall_part = Part.from_function_call(name=fc.name, args=fc.args) + fcall_part.function_call.id = fc_id + parts.append(fcall_part) + parts.append(Part(function_response=FunctionResponse( + id=fc_id, name=fc.name, response={"city": "深圳", "weather": "多云", "temperature": 30}))) + round2 = LlmRequest( + contents=[Content(role="user", parts=parts)], + config=GenerateContentConfig(tools=[WEATHER_TOOL]), + tools_dict={}, + ) + round2.streaming_tool_names = {"get_weather"} + got_final = False + async for resp in model.generate_async(round2, stream=True): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + if getattr(resp, "partial", None) is False: + got_final = True + print(f" [replay] got_final={got_final}") + + +async def test_store_false_reasoning_replay() -> None: + """store=False multi-turn reasoning replay: reasoning items captured in + round 1 (with encrypted_content) must survive into the round-2 request + input as verbatim Responses items, and round 2 must not be rejected.""" + print("\n=== 6. store=False multi-turn reasoning replay ===") + model = make_model(responses_api_params={"store": False}) + request = LlmRequest( + contents=[Content(role="user", parts=[Part.from_text(text="9 * 9 等于多少？请先推理再回答")])], + config=GenerateContentConfig(thinking_config=ThinkingConfig(include_thoughts=True, thinking_budget=-1)), + tools_dict={}, + ) + round1_parts = [] + async for resp in model.generate_async(request, stream=False): + if resp.content and resp.content.parts: + round1_parts = resp.content.parts + for p in round1_parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + + thought_parts = [p for p in round1_parts if getattr(p, "thought", False)] + if not thought_parts: + print(" (model emitted no thought parts in round 1; skipping replay)") + return + + # Reasoning items are stored on the thought part as thought_signature JSON. + reasoning_items = [] + for p in thought_parts: + raw = getattr(p, "thought_signature", None) + if not raw: + continue + raw = raw.decode("utf-8") if isinstance(raw, bytes) else raw + try: + item = json.loads(raw) + except (ValueError, TypeError): + continue + if isinstance(item, dict) and item.get("type") == "reasoning": + reasoning_items.append(item) + assert reasoning_items, "no reasoning items recovered from thought parts" + assert any("encrypted_content" in it for it in reasoning_items), \ + "reasoning items must keep encrypted_content (store=False include)" + print(f" [replay] recovered {len(reasoning_items)} reasoning item(s), " + f"{sum(1 for it in reasoning_items if 'encrypted_content' in it)} with encrypted_content") + + # Round 2: assistant turn (thought + text parts) followed by a user prompt. + round2_request = LlmRequest( + contents=[ + Content(role="user", parts=list(round1_parts)), + Content(role="user", parts=[Part.from_text(text="那 8 * 8 呢？")]), + ], + config=GenerateContentConfig(thinking_config=ThinkingConfig(include_thoughts=True, thinking_budget=-1)), + tools_dict={}, + ) + formatted = model._format_messages(round2_request) + input_items = model._convert_messages_to_responses_input(formatted) + replayed = [it for it in input_items if it.get("type") == "reasoning"] + assert replayed, "reasoning items must be preserved in the round-2 Responses input" + assert any("encrypted_content" in it for it in replayed), \ + "round-2 input items must keep encrypted_content" + print(f" [replay] {len(replayed)} reasoning item(s) preserved in round-2 input") + + async for resp in model.generate_async(round2_request, stream=False): + if resp.content and resp.content.parts: + for p in resp.content.parts: + _dump_part(p) + if resp.error_message: + print(f" [error] {resp.error_message}") + + +async def main(stream: bool) -> None: + print(f"model={MODEL} base_url={BASE_URL} stream={stream}") + await test_thinking_effort(stream) + await test_tool_roundtrip(stream) + await test_store_false(stream) + await test_param_passthrough(stream) + await test_streaming_tool_roundtrip() + await test_store_false_reasoning_replay() + print("\n[done] all scenarios finished") + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Test tRPC-Agent OpenAI Responses capabilities.") + parser.add_argument("--stream", action="store_true", help="stream responses") + args = parser.parse_args() + asyncio.run(main(stream=args.stream)) diff --git a/pyproject.toml b/pyproject.toml index db0e78283..33c466a5e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,7 +25,7 @@ classifiers = [ ] dependencies = [ "pydantic>=2.11.3", - "openai>=1.3.0", + "openai>=1.66.0", "mcp>=1.10.1", "aiohttp", "httpx>=0.27.0", diff --git a/tests/filter/test_run_filter.py b/tests/filter/test_run_filter.py index 5c18cc9e4..b038380b1 100644 --- a/tests/filter/test_run_filter.py +++ b/tests/filter/test_run_filter.py @@ -7,12 +7,11 @@ from __future__ import annotations -from typing import Any from unittest.mock import MagicMock import pytest -from trpc_agent_sdk.abc import FilterResult, FilterType +from trpc_agent_sdk.abc import FilterResult from trpc_agent_sdk.filter._base_filter import BaseFilter from trpc_agent_sdk.filter._run_filter import ( coroutine_handler_adapter, @@ -21,11 +20,11 @@ stream_handler_adapter, ) - # --------------------------------------------------------------------------- # Helpers # --------------------------------------------------------------------------- + class RecordingFilter(BaseFilter): """Filter that records calls and passes through.""" @@ -88,6 +87,7 @@ async def _after(self, ctx, req, rsp): # Fixtures # --------------------------------------------------------------------------- + @pytest.fixture def mock_ctx(): return MagicMock() @@ -97,9 +97,11 @@ def mock_ctx(): # Tests for stream_handler_adapter # --------------------------------------------------------------------------- + class TestStreamHandlerAdapter: async def test_wraps_non_filter_result(self): + async def gen(): yield "raw_value" @@ -126,6 +128,7 @@ async def gen(): assert results[0] is fr async def test_multiple_events(self): + async def gen(): yield "a" yield FilterResult(rsp="b") @@ -145,6 +148,7 @@ async def gen(): # Tests for coroutine_handler_adapter # --------------------------------------------------------------------------- + class TestCoroutineHandlerAdapter: async def test_returns_filter_result_as_is(self): @@ -157,6 +161,7 @@ async def handle(): assert result is fr async def test_wraps_tuple_result(self): + async def handle(): return "val", RuntimeError("e") @@ -166,6 +171,7 @@ async def handle(): assert isinstance(result.error, RuntimeError) async def test_wraps_plain_value(self): + async def handle(): return "plain" @@ -175,6 +181,7 @@ async def handle(): assert result.error is None async def test_wraps_none(self): + async def handle(): return None @@ -184,6 +191,7 @@ async def handle(): assert result.error is None async def test_catches_exception(self): + async def handle(): raise ValueError("boom") @@ -193,6 +201,7 @@ async def handle(): assert not result.is_continue async def test_wraps_tuple_no_error(self): + async def handle(): return "ok", None @@ -205,9 +214,11 @@ async def handle(): # Tests for run_filters # --------------------------------------------------------------------------- + class TestRunFilters: async def test_no_filters(self, mock_ctx): + async def handle(): return "direct_result" @@ -240,9 +251,7 @@ async def handle(): result = await run_filters(mock_ctx, req, [f1, f2], handle) assert result == "done" - assert req["trace"] == [ - "before_A", "before_B", "handle", "after_B", "after_A" - ] + assert req["trace"] == ["before_A", "before_B", "handle", "after_B", "after_A"] async def test_handle_raises_propagates(self, mock_ctx): f = RecordingFilter("r") @@ -254,6 +263,7 @@ async def handle(): await run_filters(mock_ctx, "req", [f], handle) async def test_filter_returns_tuple_error(self, mock_ctx): + async def handle(): return "val", RuntimeError("tuple_err") @@ -265,9 +275,11 @@ async def handle(): # Tests for run_stream_filters # --------------------------------------------------------------------------- + class TestRunStreamFilters: async def test_no_filters(self, mock_ctx): + async def handle(): yield "a" yield "b" @@ -311,12 +323,11 @@ async def handle(): results.append(event) assert results == ["data"] - assert req["trace"] == [ - "before_A", "before_B", "handle", "after_B", "after_A" - ] + assert req["trace"] == ["before_A", "before_B", "handle", "after_B", "after_A"] async def test_yields_rsp_not_filter_result(self, mock_ctx): """run_stream_filters should yield event.rsp, not the FilterResult.""" + async def handle(): yield FilterResult(rsp="wrapped", is_continue=True) @@ -327,6 +338,7 @@ async def handle(): assert results == ["wrapped"] async def test_multiple_stream_events(self, mock_ctx): + async def handle(): yield "e1" yield "e2" diff --git a/tests/models/test_openai_responses_model.py b/tests/models/test_openai_responses_model.py new file mode 100644 index 000000000..9be1ca956 --- /dev/null +++ b/tests/models/test_openai_responses_model.py @@ -0,0 +1,1463 @@ +# Tencent is pleased to support the open source community by making tRPC-Agent-Python available. +# +# Copyright (C) 2026 Tencent. All rights reserved. +# +# tRPC-Agent-Python is licensed under Apache-2.0. + +import json +from unittest.mock import AsyncMock, Mock, patch + +import pytest +from trpc_agent_sdk.models import LlmRequest, OpenAIModel +from trpc_agent_sdk.types import Content, GenerateContentConfig, HttpOptions, Part + + +def _model(**kwargs): + """Create an OpenAIModel with test defaults.""" + kwargs.setdefault("model_name", "gpt-4") + kwargs.setdefault("api_key", "test_key") + return OpenAIModel(**kwargs) + + +def _request(contents, config=None, streaming_tool_names=None): + """Create an LlmRequest for Responses API tests.""" + request = LlmRequest(contents=contents, config=config, tools_dict={}) + if streaming_tool_names is not None: + request.streaming_tool_names = streaming_tool_names + return request + + +# --------------------------------------------------------------------------- +# Responses API +# --------------------------------------------------------------------------- + + +class TestOpenAIResponsesAPI: + """Tests for the opt-in OpenAI Responses transport.""" + + def test_is_disabled_by_default_and_rejects_managed_overrides(self): + model = _model() + assert model.use_responses_api is False + + with pytest.raises(ValueError, match="cannot override managed parameters: input"): + _model(use_responses_api=True, responses_api_params={"input": "override"}) + + def test_rejects_add_tools_to_prompt_with_responses_api(self): + """The Responses API uses native function tools, so prompt-injected + tool definitions would be silently ignored — combining the two must + fail loudly at construction instead of breaking tool calling at runtime. + """ + with pytest.raises(ValueError, match="incompatible with add_tools_to_prompt"): + _model(use_responses_api=True, add_tools_to_prompt=True) + + # Each option on its own remains valid. + assert _model(use_responses_api=True).use_responses_api is True + assert _model(add_tools_to_prompt=True).add_tools_to_prompt is True + + def test_converts_tool_history_and_function_definitions(self): + model = _model(use_responses_api=True) + messages = [ + { + "role": + "assistant", + "content": + "Checking", + "tool_calls": [{ + "id": "call_weather", + "type": "function", + "function": { + "name": "weather", + "arguments": '{"city":"Shenzhen"}', + }, + }], + }, + { + "role": "tool", + "tool_call_id": "call_weather", + "content": '{"temperature":30}' + }, + ] + + items = model._convert_messages_to_responses_input(messages) + assert items == [ + { + "role": "assistant", + "content": "Checking" + }, + { + "type": "function_call", + "call_id": "call_weather", + "name": "weather", + "arguments": '{"city":"Shenzhen"}', + }, + { + "type": "function_call_output", + "call_id": "call_weather", + "output": '{"temperature":30}', + }, + ] + tools = model._convert_tools_to_responses_format([{ + "type": "function", + "function": { + "name": "weather", + "description": "Get weather", + "parameters": { + "type": "object", + "properties": {} + }, + }, + }]) + assert tools == [{ + "type": "function", + "name": "weather", + "description": "Get weather", + "parameters": { + "type": "object", + "properties": {} + }, + }] + + def test_reorders_function_call_outputs_after_function_call(self): + """Responses input must place function_call_output after its matching + function_call; orphaned outputs from legacy message ordering (tool + response before the assistant tool_calls message) are reordered.""" + model = _model(use_responses_api=True) + messages = [ + # Legacy ordering: tool response precedes the assistant tool_calls + # message, as produced for mixed single-Content inputs. + { + "role": "tool", + "tool_call_id": "call_weather", + "content": '{"temperature":30}' + }, + { + "role": "user", + "content": "天气?", + "tool_calls": [{ + "id": "call_weather", + "type": "function", + "function": { + "name": "weather", + "arguments": '{"city":"Shenzhen"}', + }, + }], + }, + ] + items = model._convert_messages_to_responses_input(messages) + item_types = [item.get("type") or item.get("role") for item in items] + assert item_types == ["user", "function_call", "function_call_output"] + assert items[1]["call_id"] == "call_weather" + assert items[2]["call_id"] == "call_weather" + + def test_converts_multimodal_input(self): + model = _model(use_responses_api=True) + + items = model._convert_messages_to_responses_input([{ + "role": + "user", + "content": [ + { + "type": "text", + "text": "Inspect this image", + }, + { + "type": "image_url", + "image_url": { + "url": "data:image/png;base64,AAAA", + "detail": "high", + }, + }, + ], + }]) + + assert items == [{ + "role": + "user", + "content": [ + { + "type": "input_text", + "text": "Inspect this image", + }, + { + "type": "input_image", + "image_url": "data:image/png;base64,AAAA", + "detail": "high", + }, + ], + }] + + def test_preserves_empty_reasoning_item_for_tool_continuation(self): + model = _model(use_responses_api=True) + raw_reasoning = { + "id": "rs_empty", + "type": "reasoning", + "encrypted_content": "encrypted-reasoning", + "summary": [], + } + + response = model._create_responses_response({ + "id": + "resp_tool", + "status": + "completed", + "output": [ + raw_reasoning, + { + "type": "function_call", + "call_id": "call_1", + "name": "lookup", + "arguments": "{}", + }, + ], + }) + + thought = response.content.parts[0] + assert thought.thought is True + assert thought.text == "" + replay_request = _request([Content(parts=response.content.parts, role="model")]) + replay_items = model._convert_messages_to_responses_input(model._format_messages(replay_request)) + assert replay_items[0] == raw_reasoning + assert replay_items[1]["type"] == "function_call" + + def test_converts_structured_output_and_failed_response(self): + model = _model(use_responses_api=True) + text = model._convert_response_format_to_responses({ + "type": "json_schema", + "json_schema": { + "name": "answer", + "schema": { + "type": "object", + "properties": {} + }, + "strict": True, + }, + }) + assert text == { + "format": { + "type": "json_schema", + "name": "answer", + "schema": { + "type": "object", + "properties": {} + }, + "strict": True, + }, + } + + response = model._create_responses_response({ + "id": "resp_failed", + "status": "failed", + "error": { + "code": "server_error", + "message": "upstream failed" + }, + "output": [], + }) + assert response.error_code == "server_error" + assert response.error_message == "upstream failed" + + def test_logprobs_uses_the_installed_responses_client_shape(self): + model = _model(use_responses_api=True) + params = model._convert_api_params_to_responses({ + "model": "gpt-4", + "messages": [{ + "role": "user", + "content": "Hello" + }], + "stream": False, + "logprobs": True, + "top_logprobs": 3, + }) + + class TopLogprobsResponses: + + async def create(self, *, top_logprobs, **kwargs): + del top_logprobs, kwargs + + class TopLogprobsClient: + responses = TopLogprobsResponses() + + prepared = model._prepare_responses_api_params(TopLogprobsClient(), params) + assert prepared["top_logprobs"] == 3 + assert "_trpc_responses_logprobs_request" not in prepared + + def test_logprobs_fails_clearly_when_the_installed_client_lacks_support(self): + model = _model(use_responses_api=True) + params = model._convert_api_params_to_responses({ + "model": "gpt-4", + "messages": [{ + "role": "user", + "content": "Hello" + }], + "stream": False, + "logprobs": True, + "top_logprobs": 3, + }) + + class LegacyResponses: + + async def create(self, *, model, input, stream): + del model, input, stream + + class LegacyClient: + responses = LegacyResponses() + + with pytest.raises(ValueError, match="upgrade openai or disable logprobs"): + model._prepare_responses_api_params(LegacyClient(), params) + + @pytest.mark.asyncio + async def test_non_streaming_uses_responses_create_and_maps_output(self): + model = _model( + use_responses_api=True, + responses_api_params={ + "store": False, + "truncation": "auto" + }, + ) + request = _request( + [Content(parts=[Part.from_text(text="Hello")], role="user")], + GenerateContentConfig(max_output_tokens=128), + ) + response = Mock() + response.model_dump.return_value = { + "id": + "resp_123", + "status": + "completed", + "output": [ + { + "id": "rs_123", + "type": "reasoning", + "encrypted_content": "encrypted-reasoning", + "summary": [{ + "type": "summary_text", + "text": "Check context" + }], + }, + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "Hello back" + }], + }, + { + "type": "function_call", + "call_id": "call_123", + "name": "lookup", + "arguments": '{"query":"hello"}', + }, + ], + "usage": { + "input_tokens": 10, + "output_tokens": 7, + "total_tokens": 17, + "input_tokens_details": { + "cached_tokens": 4 + }, + "output_tokens_details": { + "reasoning_tokens": 2 + }, + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=False)] + + assert captured["model"] == "gpt-4" + assert captured["input"] == [{"role": "user", "content": "Hello"}] + assert captured["max_output_tokens"] == 128 + assert captured["store"] is False + assert captured["truncation"] == "auto" + assert captured["include"] == ["reasoning.encrypted_content"] + assert "messages" not in captured + assert "max_completion_tokens" not in captured + result = responses[0] + assert result.response_id == "resp_123" + assert [part.text for part in result.content.parts if part.text] == ["Check context", "Hello back"] + assert result.content.parts[0].thought is True + assert result.content.parts[-1].function_call.id == "call_123" + assert result.usage_metadata.prompt_token_count == 10 + assert result.usage_metadata.candidates_token_count == 7 + assert result.usage_metadata.thoughts_token_count == 2 + assert result.usage_metadata.cache_read_input_tokens == 4 + + replay_request = _request([Content(parts=result.content.parts, role="model")]) + replay_items = model._convert_messages_to_responses_input(model._format_messages(replay_request)) + assert replay_items[0] == { + "id": "rs_123", + "type": "reasoning", + "encrypted_content": "encrypted-reasoning", + "summary": [{ + "type": "summary_text", + "text": "Check context" + }], + } + assert replay_items[1] == {"role": "assistant", "content": "Hello back"} + assert replay_items[2]["type"] == "function_call" + + @pytest.mark.asyncio + async def test_non_streaming_passes_http_options_separately(self): + """Non-streaming Responses path must pass http_options (extra_body, + extra_headers, timeout) as separate kwargs to responses.create, + not merged into api_params (which would cause extra_body to be + treated as an unknown top-level parameter). + """ + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Hello")], role="user")], + GenerateContentConfig( + max_output_tokens=128, + http_options=HttpOptions( + headers={"X-Custom-Header": "test-value"}, + timeout=5000, + extra_body={"custom_param": "custom_value"}, + ), + ), + ) + response = Mock() + response.model_dump.return_value = { + "id": + "resp_456", + "status": + "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "Hi there" + }], + }, + ], + "usage": { + "input_tokens": 5, + "output_tokens": 3, + "total_tokens": 8 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=False)] + + # Core API params must be present. + assert captured["model"] == "gpt-4" + assert captured["input"] == [{"role": "user", "content": "Hello"}] + assert captured["max_output_tokens"] == 128 + + # http_options must be passed as separate kwargs, not merged into + # the api_params dict that goes through _prepare_responses_api_params. + assert captured["extra_headers"] == {"X-Custom-Header": "test-value"} + assert captured["extra_body"] == {"custom_param": "custom_value"} + assert captured["timeout"] == 5.0 # 5000ms / 1000 + + # The response should be properly mapped. + result = responses[0] + assert result.response_id == "resp_456" + assert [part.text for part in result.content.parts if part.text] == ["Hi there"] + + @pytest.mark.asyncio + async def test_non_streaming_maps_function_call_output(self): + """A non-streaming Responses payload with a function_call item maps to + a LlmResponse carrying the function_call part (id/name/args).""" + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Use lookup")], role="user")], + GenerateContentConfig(max_output_tokens=128), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_tool", + "status": "completed", + "output": [ + { + "type": "function_call", + "id": "fc_1", + "call_id": "call_1", + "name": "lookup", + "arguments": '{"q":"x"}', + }, + ], + "usage": { + "input_tokens": 1, + "output_tokens": 1, + "total_tokens": 2 + }, + } + + async def create(**kwargs): + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=False)] + + assert len(responses) == 1 + parts = responses[0].content.parts + assert len(parts) == 1 + fc = parts[0].function_call + assert fc is not None + assert fc.name == "lookup" + assert fc.args == {"q": "x"} + assert fc.id == "call_1" + + @pytest.mark.asyncio + async def test_responses_maps_prompt_cache_and_passthrough_parallel_tool_calls(self): + """prompt_cache_* resolved from the model-level PromptCacheConfig are + mapped to their Responses names; parallel_tool_calls is forwarded + verbatim via responses_api_params.""" + from trpc_agent_sdk.configs import PromptCacheConfig + + model = _model( + use_responses_api=True, + prompt_cache_config=PromptCacheConfig( + enabled=True, + prompt_cache_key="cache-1", + ttl="1h", + ), + responses_api_params={"parallel_tool_calls": True}, + ) + request = _request( + [Content(parts=[Part.from_text(text="Hi")], role="user")], + GenerateContentConfig(max_output_tokens=128), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_cache", + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "OK" + }], + }, + ], + "usage": { + "input_tokens": 1, + "output_tokens": 1, + "total_tokens": 2 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + [item async for item in model.generate_async(request, stream=False)] + + assert captured.get("prompt_cache_key") == "cache-1" + assert captured.get("prompt_cache_retention") == "1h" + assert captured.get("parallel_tool_calls") is True + + @pytest.mark.asyncio + async def test_streaming_multiple_function_calls_accumulate(self): + """Parallel function_calls stream and accumulate independently, with + each call's arguments assembled from interleaved deltas.""" + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Use both tools")], role="user")], + streaming_tool_names={"lookup_a", "lookup_b"}, + ) + events = [ + { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_a", + "call_id": "call_a", + "name": "lookup_a", + "arguments": "", + }, + }, + { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_b", + "call_id": "call_b", + "name": "lookup_b", + "arguments": "", + }, + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_a", + "delta": '{"q":' + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_b", + "delta": '{"r":' + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_a", + "delta": '"a"}' + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_b", + "delta": '"b"}' + }, + { + "type": "response.completed", + "response": { + "id": "resp_multi", + "status": "completed", + "output": [ + { + "type": "function_call", + "id": "fc_a", + "call_id": "call_a", + "name": "lookup_a", + "arguments": '{"q":"a"}', + }, + { + "type": "function_call", + "id": "fc_b", + "call_id": "call_b", + "name": "lookup_b", + "arguments": '{"r":"b"}', + }, + ], + "usage": { + "input_tokens": 1, + "output_tokens": 2, + "total_tokens": 3 + }, + }, + }, + ] + + async def stream_events(): + for event in events: + yield event + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + # Streaming deltas for both calls were yielded, interleaved by call. + tool_deltas = [ + (item.content.parts[0].function_call.name, + item.content.parts[0].function_call.args["tool_streaming_args"]) + for item in responses[0:-1] + ] + assert ("lookup_a", '{"q":') in tool_deltas + assert ("lookup_b", '{"r":') in tool_deltas + assert ("lookup_a", '"a"}') in tool_deltas + assert ("lookup_b", '"b"}') in tool_deltas + # Final response carries both fully-assembled function_calls. + final = responses[-1] + calls = { + part.function_call.name: part.function_call.args + for part in final.content.parts + if part.function_call is not None + } + assert calls == { + "lookup_a": {"q": "a"}, + "lookup_b": {"r": "b"}, + } + + def test_pure_reasoning_turn_preserves_responses_input_items(self): + """A turn with only thought parts (no text/image/tool calls) must + still produce a message carrying ``responses_input_items`` so that + reasoning signatures are not silently dropped from history. + """ + from trpc_agent_sdk.models._openai_model import _RESPONSES_INPUT_ITEMS + model = _model(use_responses_api=True) + + reasoning_item = { + "type": "reasoning", + "id": "rs_pure", + "encrypted_content": "encrypted-pure", + "summary": [{ + "type": "summary_text", + "text": "Pure thought" + }], + } + + # Assistant content with only a thought part — no text, no tool calls. + # thought_signature is set via attribute assignment (not construction) + # to bypass Part's base64 validation, matching _create_responses_response. + thought_part = Part(thought=True, text="") + thought_part.thought_signature = json.dumps(reasoning_item, ensure_ascii=False).encode("utf-8") + assistant_content = Content( + role="model", + parts=[thought_part], + ) + user_content = Content(role="user", parts=[Part.from_text(text="follow up")]) + + messages = model._format_messages(_request([assistant_content, user_content])) + + # The assistant message must be present and carry the reasoning items. + assistant_msgs = [m for m in messages if m.get("role") == "assistant"] + assert len(assistant_msgs) == 1 + assert _RESPONSES_INPUT_ITEMS in assistant_msgs[0] + assert assistant_msgs[0][_RESPONSES_INPUT_ITEMS] == [reasoning_item] + + @pytest.mark.asyncio + async def test_responses_thinking_budget_does_not_map_to_effort(self): + """A positive thinking_budget must not be mapped to reasoning.effort: + supported effort values vary by model and are passed through verbatim + via ``responses_api_params`` instead.""" + from trpc_agent_sdk.types import ThinkingConfig + + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Think hard")], role="user")], + GenerateContentConfig( + max_output_tokens=4096, + thinking_config=ThinkingConfig( + include_thoughts=True, + thinking_budget=20000, + ), + ), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_effort", + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "Result" + }], + }, + ], + "usage": { + "input_tokens": 5, + "output_tokens": 3, + "total_tokens": 8 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + [item async for item in model.generate_async(request, stream=False)] + + reasoning = captured.get("reasoning") + assert reasoning is not None + assert reasoning["summary"] == "auto" + assert "effort" not in reasoning + + @pytest.mark.asyncio + async def test_responses_thinking_budget_negative_one_skips_effort(self): + """thinking_budget=-1 (automatic) should not set reasoning.effort, + letting the model decide.""" + from trpc_agent_sdk.types import ThinkingConfig + + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Think")], role="user")], + GenerateContentConfig( + max_output_tokens=4096, + thinking_config=ThinkingConfig( + include_thoughts=True, + thinking_budget=-1, + ), + ), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_auto", + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "OK" + }], + }, + ], + "usage": { + "input_tokens": 5, + "output_tokens": 3, + "total_tokens": 8 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + [item async for item in model.generate_async(request, stream=False)] + + reasoning = captured.get("reasoning") + assert reasoning is not None + assert reasoning["summary"] == "auto" + assert "effort" not in reasoning + + @pytest.mark.asyncio + async def test_responses_drops_chat_completions_only_params(self): + """Params with no Responses equivalent (stop, penalties, seed, n) must + not be forwarded, otherwise the OpenAI API rejects the request.""" + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Hi")], role="user")], + GenerateContentConfig( + max_output_tokens=128, + stop_sequences=["\n"], + frequency_penalty=0.5, + presence_penalty=0.2, + seed=42, + candidate_count=2, + temperature=0.7, + ), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_drop", + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "OK" + }], + }, + ], + "usage": { + "input_tokens": 1, + "output_tokens": 1, + "total_tokens": 2 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + [item async for item in model.generate_async(request, stream=False)] + + # Supported framework params are forwarded with their Responses name. + assert captured.get("temperature") == 0.7 + # Chat Completions-only params are dropped, not forwarded. + for dropped in ("stop", "frequency_penalty", "presence_penalty", "seed", "n"): + assert dropped not in captured + + @pytest.mark.asyncio + async def test_responses_api_params_passthrough_reasoning_effort(self): + """reasoning.effort is passed through verbatim from responses_api_params, + and a user-supplied summary wins over the framework default.""" + from trpc_agent_sdk.types import ThinkingConfig + + model = _model( + use_responses_api=True, + responses_api_params={ + "reasoning": { + "effort": "xhigh", + "summary": "detailed", + }, + }, + ) + request = _request( + [Content(parts=[Part.from_text(text="Think")], role="user")], + GenerateContentConfig( + max_output_tokens=4096, + thinking_config=ThinkingConfig( + include_thoughts=True, + thinking_budget=20000, + ), + ), + ) + response = Mock() + response.model_dump.return_value = { + "id": "resp_passthrough", + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "OK" + }], + }, + ], + "usage": { + "input_tokens": 5, + "output_tokens": 3, + "total_tokens": 8 + }, + } + captured = {} + + async def create(**kwargs): + captured.update(kwargs) + return response + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = create + client_factory.return_value = client + [item async for item in model.generate_async(request, stream=False)] + + assert captured.get("reasoning") == {"effort": "xhigh", "summary": "detailed"} + + @pytest.mark.asyncio + async def test_streaming_maps_text_reasoning_tool_calls_and_usage(self): + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Use a tool")], role="user")], + streaming_tool_names={"lookup"}, + ) + events = [ + { + "type": "response.created", + "response": { + "id": "resp_stream" + } + }, + { + "type": "response.reasoning_summary_text.delta", + "delta": "Thinking" + }, + { + "type": "response.output_text.delta", + "delta": "Working" + }, + { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_1", + "call_id": "call_1", + "name": "lookup", + "arguments": "", + }, + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_1", + "delta": '{"q":' + }, + { + "type": "response.function_call_arguments.delta", + "item_id": "fc_1", + "delta": '"x"}' + }, + { + "type": "response.completed", + "response": { + "id": + "resp_stream", + "status": + "completed", + "output": [ + { + "type": "reasoning", + "summary": [{ + "type": "summary_text", + "text": "Thinking" + }] + }, + { + "type": "message", + "content": [{ + "type": "output_text", + "text": "Working" + }], + }, + { + "type": "function_call", + "id": "fc_1", + "call_id": "call_1", + "name": "lookup", + "arguments": '{"q":"x"}', + }, + ], + "usage": { + "input_tokens": 5, + "output_tokens": 4, + "total_tokens": 9 + }, + }, + }, + ] + + async def stream_events(): + for event in events: + yield event + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + assert responses[0].content.parts[0].thought is True + assert responses[0].content.parts[0].text == "Thinking" + assert responses[1].content.parts[0].text == "Working" + tool_deltas = [item.content.parts[0].function_call.args["tool_streaming_args"] for item in responses[2:-1]] + assert tool_deltas == ['{"q":', '"x"}'] + final = responses[-1] + assert final.partial is False + assert final.response_id == "resp_stream" + assert final.content.parts[-1].function_call.args == {"q": "x"} + assert final.usage_metadata.total_token_count == 9 + + @pytest.mark.asyncio + async def test_streaming_error_event_is_not_reported_as_success(self): + model = _model(use_responses_api=True) + request = _request([Content(parts=[Part.from_text(text="Hello")], role="user")]) + + async def stream_events(): + yield { + "type": "error", + "response_id": "resp_error", + "code": "server_error", + "message": "upstream unavailable", + } + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + assert responses[-1].response_id == "resp_error" + assert responses[-1].error_code == "server_error" + assert responses[-1].error_message == "upstream unavailable" + + @pytest.mark.asyncio + async def test_streaming_uses_arguments_done_payload_in_fallback(self): + model = _model(use_responses_api=True) + request = _request([Content(parts=[Part.from_text(text="Use a tool")], role="user")]) + + async def stream_events(): + yield { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_1", + "call_id": "call_1", + "name": "lookup", + "arguments": "", + }, + } + yield { + "type": "response.function_call_arguments.done", + "item_id": "fc_1", + "arguments": '{"query":"done"}', + } + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + final = responses[-1] + assert final.partial is False + assert final.content.parts[0].function_call.id == "call_1" + assert final.content.parts[0].function_call.args == {"query": "done"} + + def test_incomplete_response_maps_reason_to_error(self): + model = _model(use_responses_api=True) + + response = model._create_responses_response({ + "id": "resp_incomplete", + "status": "incomplete", + "incomplete_details": { + "reason": "max_output_tokens", + }, + "output": [], + }) + + assert response.error_code == "incomplete" + assert response.error_message == "max_output_tokens" + + # ------------------------------------------------------------------ + # _responses_error — cancelled / incomplete_details fallback + # ------------------------------------------------------------------ + + def test_cancelled_response_maps_status_to_error(self): + model = _model(use_responses_api=True) + response = model._create_responses_response({ + "id": "resp_cancelled", + "status": "cancelled", + "output": [], + }) + assert response.error_code == "cancelled" + assert response.error_message == "cancelled" + + def test_incomplete_details_non_dict_fallback(self): + model = _model(use_responses_api=True) + response = model._create_responses_response({ + "id": "resp_incomplete", + "status": "incomplete", + "incomplete_details": "timeout", + "output": [], + }) + assert response.error_code == "incomplete" + assert response.error_message == "timeout" + + # ------------------------------------------------------------------ + # _create_responses_response — refusal text / non-dict arguments + # ------------------------------------------------------------------ + + def test_message_with_refusal_text(self): + model = _model(use_responses_api=True) + response = model._create_responses_response({ + "id": + "resp_refusal", + "status": + "completed", + "output": [{ + "type": "message", + "role": "assistant", + "content": [{ + "type": "refusal", + "refusal": "I cannot answer that." + }], + }], + }) + assert response.content.parts[0].text == "I cannot answer that." + + def test_function_call_non_dict_arguments_skipped(self): + model = _model(use_responses_api=True) + response = model._create_responses_response({ + "id": + "resp_bad_args", + "status": + "completed", + "output": [{ + "type": "function_call", + "call_id": "call_bad", + "name": "broken_tool", + "arguments": "[not a dict]", + }], + }) + # Non-dict arguments are skipped, so no parts from function_call + assert response.content is None + + # ------------------------------------------------------------------ + # _convert_messages_to_responses_input — assistant / unknown type + # ------------------------------------------------------------------ + + def test_converts_assistant_message_to_output_text(self): + model = _model(use_responses_api=True) + items = model._convert_messages_to_responses_input([{ + "role": + "assistant", + "content": [{ + "type": "text", + "text": "Hello from assistant" + }], + }]) + assert items == [{ + "role": "assistant", + "content": [{ + "type": "output_text", + "text": "Hello from assistant" + }], + }] + + def test_unknown_content_type_is_passthrough(self): + model = _model(use_responses_api=True) + items = model._convert_messages_to_responses_input([{ + "role": "user", + "content": [{ + "type": "custom_block", + "data": "raw" + }], + }]) + assert items == [{ + "role": "user", + "content": [{ + "type": "custom_block", + "data": "raw" + }], + }] + + def test_converts_non_function_tool_to_responses_format(self): + model = _model(use_responses_api=True) + tools = model._convert_tools_to_responses_format([ + { + "type": "web_search", + "name": "search" + }, + { + "type": "function", + "function": { + "name": "calc", + "description": "Calculate", + "parameters": { + "type": "object", + "properties": {} + }, + }, + }, + ]) + assert tools == [ + { + "type": "web_search", + "name": "search" + }, + { + "type": "function", + "name": "calc", + "description": "Calculate", + "parameters": { + "type": "object", + "properties": {} + }, + }, + ] + + # ------------------------------------------------------------------ + # _prepare_responses_api_params — logprobs structured support + # ------------------------------------------------------------------ + + def test_logprobs_uses_structured_logprobs_when_supported(self): + model = _model(use_responses_api=True) + params = model._convert_api_params_to_responses({ + "model": "gpt-4", + "messages": [{ + "role": "user", + "content": "Hello" + }], + "stream": False, + "logprobs": True, + "top_logprobs": 3, + }) + + class LogprobsResponses: + + async def create(self, *, logprobs, **kwargs): + del logprobs, kwargs + + class LogprobsClient: + responses = LogprobsResponses() + + prepared = model._prepare_responses_api_params(LogprobsClient(), params) + assert prepared["logprobs"] == {"enabled": True, "top_logprobs": 3} + + def test_logprobs_raises_on_inspect_failure(self): + model = _model(use_responses_api=True) + params = model._convert_api_params_to_responses({ + "model": "gpt-4", + "messages": [{ + "role": "user", + "content": "Hello" + }], + "stream": False, + "logprobs": True, + "top_logprobs": 3, + }) + + class BrokenResponses: + create = "not_callable" + + class BrokenClient: + responses = BrokenResponses() + + with pytest.raises(ValueError, match="Unable to determine Responses logprobs support"): + model._prepare_responses_api_params(BrokenClient(), params) + + # ------------------------------------------------------------------ + # _generate_responses_stream — fallback when no completed event + # ------------------------------------------------------------------ + + @pytest.mark.asyncio + async def test_streaming_fallback_when_no_completed_event(self): + """Streaming without response.completed builds final from accumulated text.""" + model = _model(use_responses_api=True) + request = _request( + [Content(parts=[Part.from_text(text="Hello")], role="user")], + streaming_tool_names={"search"}, + ) + + async def stream_events(): + yield {"type": "response.created", "response": {"id": "resp_fallback"}} + yield {"type": "response.reasoning_summary_text.delta", "delta": "Hmm"} + yield {"type": "response.output_text.delta", "delta": "Answer"} + yield { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_fb", + "call_id": "call_fb", + "name": "search", + "arguments": "", + }, + } + yield { + "type": "response.function_call_arguments.delta", + "item_id": "fc_fb", + "delta": '{"q":"x"}', + } + # No response.completed — triggers fallback + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + final = responses[-1] + assert final.partial is False + assert final.response_id == "resp_fallback" + assert final.content is not None + texts = [p.text for p in final.content.parts if p.text] + assert "Hmm" in texts or "Answer" in texts + + @pytest.mark.asyncio + async def test_streaming_arguments_done_with_function_call_item(self): + """response.function_call_arguments.done with item.type=function_call uses the item.""" + model = _model(use_responses_api=True) + request = _request([Content(parts=[Part.from_text(text="Use a tool")], role="user")]) + + async def stream_events(): + yield {"type": "response.created", "response": {"id": "resp_done_item"}} + yield { + "type": "response.output_item.added", + "item": { + "type": "function_call", + "id": "fc_item", + "call_id": "call_item", + "name": "lookup", + "arguments": "", + }, + } + yield { + "type": "response.function_call_arguments.done", + "item_id": "fc_item", + "item": { + "type": "function_call", + "id": "fc_item", + "call_id": "call_item", + "name": "lookup", + "arguments": '{"result":"from_item"}', + }, + } + yield { + "type": "response.completed", + "response": { + "id": + "resp_done_item", + "status": + "completed", + "output": [{ + "type": "function_call", + "id": "fc_item", + "call_id": "call_item", + "name": "lookup", + "arguments": '{"result":"from_item"}', + }], + }, + } + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + final = responses[-1] + assert final.partial is False + # Function call from item should be in final output + assert final.content is not None + + @pytest.mark.asyncio + async def test_streaming_response_incomplete_event(self): + """response.incomplete event sets completed_response.""" + model = _model(use_responses_api=True) + request = _request([Content(parts=[Part.from_text(text="Hello")], role="user")]) + + async def stream_events(): + yield {"type": "response.created", "response": {"id": "resp_inc"}} + yield { + "type": "response.incomplete", + "response": { + "id": "resp_inc", + "status": "incomplete", + "incomplete_details": { + "reason": "max_output_tokens" + }, + "output": [], + }, + } + + with patch.object(model, "_create_async_client") as client_factory: + client = AsyncMock() + client.responses.create = AsyncMock(return_value=stream_events()) + client_factory.return_value = client + responses = [item async for item in model.generate_async(request, stream=True)] + + final = responses[-1] + assert final.error_code == "incomplete" + assert final.error_message == "max_output_tokens" + + # ------------------------------------------------------------------ + # _convert_api_params_to_responses — max_tokens fallback + # ------------------------------------------------------------------ + + def test_max_tokens_falls_back_to_max_output_tokens(self): + model = _model(use_responses_api=True) + params = model._convert_api_params_to_responses({ + "model": "gpt-4", + "messages": [{ + "role": "user", + "content": "Hi" + }], + "stream": False, + "max_tokens": 256, + }) + assert params["max_output_tokens"] == 256 + assert "max_tokens" not in params diff --git a/tests/tools/test_function_tool.py b/tests/tools/test_function_tool.py index c550ba984..f6dba973b 100644 --- a/tests/tools/test_function_tool.py +++ b/tests/tools/test_function_tool.py @@ -6,9 +6,7 @@ from __future__ import annotations -import asyncio -from typing import Any -from unittest.mock import AsyncMock, MagicMock, patch +from unittest.mock import MagicMock import pytest from pydantic import BaseModel @@ -16,9 +14,9 @@ from trpc_agent_sdk.context import InvocationContext from trpc_agent_sdk.tools._function_tool import FunctionTool - # --- Test functions --- + def sync_func(param1: str, param2: int = 10) -> str: """A sync test function.""" return f"{param1}-{param2}" @@ -128,7 +126,10 @@ async def test_run_sync_function(self, mock_context): tool = FunctionTool(sync_func) result = await tool._run_async_impl( tool_context=mock_context, - args={"param1": "hello", "param2": 5}, + args={ + "param1": "hello", + "param2": 5 + }, ) assert result == "hello-5" @@ -163,6 +164,7 @@ async def test_missing_mandatory_args(self, mock_context): @pytest.mark.asyncio async def test_returns_empty_dict_for_none(self, mock_context): + def returns_none(): return None @@ -178,7 +180,10 @@ async def test_pydantic_model_return(self, mock_context): tool = FunctionTool(func_returns_model) result = await tool._run_async_impl( tool_context=mock_context, - args={"name": "test", "value": 42}, + args={ + "name": "test", + "value": 42 + }, ) assert isinstance(result, str) assert "test" in result @@ -200,7 +205,9 @@ def slow_func(x: str) -> str: @pytest.mark.asyncio async def test_async_callable_object(self, mock_context): + class AsyncCallable: + async def __call__(self, value: str) -> str: """Async callable.""" return f"async-{value}" diff --git a/tests/trpc_agent_dsl/graph/test_agent_node_hitl.py b/tests/trpc_agent_dsl/graph/test_agent_node_hitl.py new file mode 100644 index 000000000..e4860b88b --- /dev/null +++ b/tests/trpc_agent_dsl/graph/test_agent_node_hitl.py @@ -0,0 +1,492 @@ +# Tencent is pleased to support the open source community by making tRPC-Agent-Python available. +# +# Copyright (C) 2026 Tencent. All rights reserved. +# +# tRPC-Agent-Python is licensed under Apache-2.0. +"""End-to-end HITL propagation tests for GraphAgent agent nodes.""" + +from typing import AsyncGenerator +from typing import List + +from pydantic import Field + +import pytest +from trpc_agent_sdk.agents import BaseAgent +from trpc_agent_sdk.dsl.graph import END +from trpc_agent_sdk.dsl.graph import START +from trpc_agent_sdk.dsl.graph import GraphAgent +from trpc_agent_sdk.dsl.graph import State +from trpc_agent_sdk.dsl.graph import StateGraph +from trpc_agent_sdk.events import Event +from trpc_agent_sdk.events import LongRunningEvent +from trpc_agent_sdk.models import LLMModel +from trpc_agent_sdk.models import LlmRequest +from trpc_agent_sdk.models import LlmResponse +from trpc_agent_sdk.runners import Runner +from trpc_agent_sdk.sessions import InMemorySessionService +from trpc_agent_sdk.sessions import SqlSessionService +from trpc_agent_sdk.teams import TeamAgent +from trpc_agent_sdk.teams.core import TEAM_STATE_KEY +from trpc_agent_sdk.types import Content +from trpc_agent_sdk.types import FunctionCall +from trpc_agent_sdk.types import FunctionResponse +from trpc_agent_sdk.types import Part + + +class HitlState(State, total=False): + after_child: bool + + +class StubModel(LLMModel): + + @classmethod + def supported_models(cls) -> List[str]: + return [r"stub-model"] + + async def _generate_async_impl( + self, + request: LlmRequest, + stream: bool = False, + ctx=None, + ) -> AsyncGenerator[LlmResponse, None]: + del request, stream, ctx + yield LlmResponse(content=None) + + def validate_request(self, request: LlmRequest) -> None: + del request + + +class TwoRoundClarifyingAgent(BaseAgent): + """Child agent that requires two FunctionResponses before completing.""" + + received_call_ids: list[str] = Field(default_factory=list) + + async def _run_async_impl(self, ctx) -> AsyncGenerator[Event, None]: + response = self._function_response(ctx.user_content) + if response is not None: + self.received_call_ids.append(str(response.id)) + if response.response.get("answer") == "done": + yield Event( + invocation_id=ctx.invocation_id, + author=self.name, + branch=ctx.branch, + content=Content(role="model", parts=[Part.from_text(text="clarification-complete")]), + ) + return + call_id = "child-question-2" + else: + call_id = "child-question-1" + + call = FunctionCall( + id=call_id, + name="ask_clarification", + args={"round": 1 if call_id.endswith("1") else 2}, + ) + yield LongRunningEvent( + invocation_id=ctx.invocation_id, + author=self.name, + branch=ctx.branch, + function_call=call, + function_response=FunctionResponse( + id=call.id, + name=call.name, + response={"status": "pending"}, + ), + ) + + @staticmethod + def _function_response(content: Content | None) -> FunctionResponse | None: + if content is None: + return None + for part in content.parts or []: + if part.function_response is not None: + return part.function_response + return None + + +class TeamHitlLeader(BaseAgent): + """Leader double that verifies TeamAgent receives the original response ID.""" + + received_call_ids: list[str] = Field(default_factory=list) + + async def _run_async_impl(self, ctx) -> AsyncGenerator[Event, None]: + response = TwoRoundClarifyingAgent._function_response(ctx.user_content) + if response is not None: + self.received_call_ids.append(str(response.id)) + yield Event( + invocation_id=ctx.invocation_id, + author=self.name, + branch=ctx.branch, + content=Content(role="model", parts=[Part.from_text(text="team-clarification-complete")]), + ) + return + + call = FunctionCall( + id="team-question-1", + name="ask_clarification", + args={"question": "approve team plan?"}, + ) + yield LongRunningEvent( + invocation_id=ctx.invocation_id, + author=self.name, + branch=ctx.branch, + function_call=call, + function_response=FunctionResponse( + id=call.id, + name=call.name, + response={"status": "pending"}, + ), + ) + + +def _user_text(text: str) -> Content: + return Content(role="user", parts=[Part.from_text(text=text)]) + + +def _tool_response(event: LongRunningEvent, answer: str) -> Content: + return Content( + role="user", + parts=[ + Part(function_response=FunctionResponse( + id=event.function_call.id, + name=event.function_call.name, + response={"answer": answer}, + )) + ], + ) + + +async def _run(runner: Runner, message: Content) -> list[Event]: + return [event async for event in runner.run_async( + user_id="user-1", + session_id="session-1", + new_message=message, + )] + + +@pytest.mark.asyncio +async def test_agent_node_long_running_interrupts_parent_and_resumes_multiple_rounds(): + child = TwoRoundClarifyingAgent(name="clarifier") + + async def after_child(state: HitlState) -> dict[str, bool]: + del state + return {"after_child": True} + + graph = StateGraph(HitlState) + graph.add_agent_node("clarify", child, isolated_messages=True) + graph.add_node("after_child", after_child) + graph.add_edge(START, "clarify") + graph.add_edge("clarify", "after_child") + graph.add_edge("after_child", END) + + service = InMemorySessionService() + runner = Runner( + app_name="agent-node-hitl-test", + agent=GraphAgent(name="workflow", graph=graph.compile()), + session_service=service, + close_session_service_on_close=False, + ) + + first_events = await _run(runner, _user_text("start")) + first_pending = next(event for event in first_events if isinstance(event, LongRunningEvent)) + assert first_pending.function_call.name == "ask_clarification" + assert first_pending.function_call.args == {"round": 1, "status": "pending"} + first_session = await service.get_session( + app_name="agent-node-hitl-test", + user_id="user-1", + session_id="session-1", + ) + assert first_session is not None + assert first_session.state.get("after_child") is not True + + second_events = await _run(runner, _tool_response(first_pending, "again")) + second_pending = next(event for event in second_events if isinstance(event, LongRunningEvent)) + assert second_pending.function_call.name == "ask_clarification" + second_session = await service.get_session( + app_name="agent-node-hitl-test", + user_id="user-1", + session_id="session-1", + ) + assert second_session is not None + assert second_session.state.get("after_child") is not True + + await _run(runner, _tool_response(second_pending, "done")) + completed_session = await service.get_session( + app_name="agent-node-hitl-test", + user_id="user-1", + session_id="session-1", + ) + assert completed_session is not None + assert completed_session.state.get("after_child") is True + assert child.received_call_ids == ["child-question-1", "child-question-2"] + + await runner.close() + await service.close() + + +@pytest.mark.asyncio +async def test_agent_node_hitl_survives_service_restart(tmp_path): + """AgentNode HITL state must persist across process restart via SqlSessionService. + + This verifies that STATE_KEY_PENDING_AGENT_NODE_HITL is written to + session.state through the interrupt bridge event's state_delta, so that + a fresh Runner reading from the same SqlSessionService can resume the + pending HITL round. + """ + from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_PENDING_AGENT_NODE_HITL + + def build_runner(service: SqlSessionService) -> tuple[Runner, TwoRoundClarifyingAgent]: + child = TwoRoundClarifyingAgent(name="clarifier") + graph = StateGraph(HitlState) + graph.add_agent_node("clarify", child, isolated_messages=True) + graph.add_edge(START, "clarify") + graph.add_edge("clarify", END) + return ( + Runner( + app_name="agent-node-restart-test", + agent=GraphAgent(name="workflow", graph=graph.compile()), + session_service=service, + close_session_service_on_close=False, + ), + child, + ) + + db_url = f"sqlite:///{tmp_path / 'agent-node-restart.sqlite'}" + service = SqlSessionService(db_url, is_async=False) + runner, child = build_runner(service) + + first_events = await _run(runner, _user_text("start")) + first_pending = next(event for event in first_events if isinstance(event, LongRunningEvent)) + assert first_pending.function_call.name == "ask_clarification" + + first_session = await service.get_session( + app_name="agent-node-restart-test", + user_id="user-1", + session_id="session-1", + ) + assert first_session is not None + # The pending HITL state must be persisted in session.state so that a + # fresh Runner can pick it up. + assert STATE_KEY_PENDING_AGENT_NODE_HITL in first_session.state + pending = first_session.state[STATE_KEY_PENDING_AGENT_NODE_HITL] + assert pending["node_id"] == "clarify" + assert pending["current"]["function_call"]["id"] == "child-question-1" + + # Simulate process restart: close runner + service, re-open from same DB. + await runner.close() + await service.close() + + service = SqlSessionService(db_url, is_async=False) + runner, resumed_child = build_runner(service) + + # Resume with the first round's response. + second_events = await _run(runner, _tool_response(first_pending, "again")) + second_pending = next(event for event in second_events if isinstance(event, LongRunningEvent)) + + second_session = await service.get_session( + app_name="agent-node-restart-test", + user_id="user-1", + session_id="session-1", + ) + assert second_session is not None + assert STATE_KEY_PENDING_AGENT_NODE_HITL in second_session.state + pending2 = second_session.state[STATE_KEY_PENDING_AGENT_NODE_HITL] + assert len(pending2["completed"]) == 1 + assert pending2["current"]["function_call"]["id"] == "child-question-2" + + # Complete the second round. + await _run(runner, _tool_response(second_pending, "done")) + completed_session = await service.get_session( + app_name="agent-node-restart-test", + user_id="user-1", + session_id="session-1", + ) + assert completed_session is not None + # After completion, the pending HITL state should be cleared. + hitl_value = completed_session.state.get(STATE_KEY_PENDING_AGENT_NODE_HITL) + assert hitl_value is None + assert resumed_child.received_call_ids == ["child-question-1", "child-question-2"] + + await runner.close() + await service.close() + + +@pytest.mark.asyncio +async def test_agent_node_hitl_state_key_is_unsafe(): + """STATE_KEY_PENDING_AGENT_NODE_HITL must be in UNSAFE_STATE_KEYS so it + is filtered out of the final_state/state_delta exposed via completion + events (it may contain sensitive tool arguments and child state). + """ + from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_PENDING_AGENT_NODE_HITL + from trpc_agent_sdk.dsl.graph._constants import is_unsafe_state_key + + assert is_unsafe_state_key(STATE_KEY_PENDING_AGENT_NODE_HITL) is True + + +@pytest.mark.asyncio +async def test_team_agent_leader_hitl_survives_service_restart(tmp_path): + + async def after_team(state: HitlState) -> dict[str, bool]: + del state + return {"after_child": True} + + def build_runner(service: SqlSessionService) -> tuple[Runner, TeamHitlLeader]: + member = TwoRoundClarifyingAgent(name="unused_member") + team = TeamAgent( + name="development_team", + model=StubModel(model_name="stub-model"), + members=[member], + ) + leader = TeamHitlLeader(name="development_team") + team.__pydantic_private__["_leader_agent"] = leader + graph = StateGraph(HitlState) + graph.add_agent_node("development", team, isolated_messages=True) + graph.add_node("after_team", after_team) + graph.add_edge(START, "development") + graph.add_edge("development", "after_team") + graph.add_edge("after_team", END) + return ( + Runner( + app_name="agent-node-hitl-test", + agent=GraphAgent(name="team_workflow", graph=graph.compile()), + session_service=service, + close_session_service_on_close=False, + ), + leader, + ) + + db_url = f"sqlite:///{tmp_path / 'agent-node-hitl.sqlite'}" + service = SqlSessionService(db_url, is_async=False) + runner, _ = build_runner(service) + + first_events = await _run(runner, _user_text("start team")) + pending = next(event for event in first_events if isinstance(event, LongRunningEvent)) + first_session = await service.get_session( + app_name="agent-node-hitl-test", + user_id="user-1", + session_id="session-1", + ) + assert first_session is not None + assert first_session.state.get("after_child") is not True + assert TEAM_STATE_KEY in first_session.state + + await runner.close() + await service.close() + + service = SqlSessionService(db_url, is_async=False) + runner, resumed_leader = build_runner(service) + await _run(runner, _tool_response(pending, "done")) + completed_session = await service.get_session( + app_name="agent-node-hitl-test", + user_id="user-1", + session_id="session-1", + ) + assert completed_session is not None + assert completed_session.state.get("after_child") is True + assert resumed_leader.received_call_ids == ["team-question-1"] + + await runner.close() + await service.close() + + +@pytest.mark.asyncio +async def test_agent_node_multiround_hitl_resume_with_correct_order(): + """Multi-round HITL resume must replay completed rounds in order. + + After the first round interrupts, the client resumes with a FunctionResponse. + The second round interrupts again. The client must then resume with the + second round's response. This test verifies that submitting a stale + (already-completed round's) response does not silently complete the graph, + and that subsequently submitting the correct round's response succeeds. + """ + child = TwoRoundClarifyingAgent(name="clarifier") + + async def after_child(state: HitlState) -> dict[str, bool]: + del state + return {"after_child": True} + + graph = StateGraph(HitlState) + graph.add_agent_node("clarify", child, isolated_messages=True) + graph.add_node("after_child", after_child) + graph.add_edge(START, "clarify") + graph.add_edge("clarify", "after_child") + graph.add_edge("after_child", END) + + service = InMemorySessionService() + runner = Runner( + app_name="agent-node-hitl-order-test", + agent=GraphAgent(name="workflow", graph=graph.compile()), + session_service=service, + close_session_service_on_close=False, + ) + + # Round 1: interrupt with child-question-1 (wrapped in a synthesized + # graph-level interrupt bridge function_call.id). + first_events = await _run(runner, _user_text("start")) + first_pending = next(event for event in first_events if isinstance(event, LongRunningEvent)) + assert first_pending.function_call.name == "ask_clarification" + assert first_pending.function_call.args == {"round": 1, "status": "pending"} + + # Resume round 1 with "again" → round 2 interrupts with child-question-2. + second_events = await _run(runner, _tool_response(first_pending, "again")) + second_pending = next(event for event in second_events if isinstance(event, LongRunningEvent)) + assert second_pending.function_call.name == "ask_clarification" + assert second_pending.function_call.args == {"round": 2, "status": "pending"} + + # Now try to resume round 2 using the FIRST round's function_call.id. + # This is a "stale" resume — the client submitted a response for an + # already-completed round. The graph's _extract_resume_command builds a + # Command with the stale function_response.id; LangGraph will not find + # a matching interrupt and the resume value will be ignored or cause an + # error. We assert that the graph does NOT silently complete with + # incorrect state. + stale_response = Content( + role="user", + parts=[ + Part(function_response=FunctionResponse( + id=first_pending.function_call.id, + name=first_pending.function_call.name, + response={"answer": "stale"}, + )) + ], + ) + stale_events = await _run(runner, stale_response) + # The graph should not have completed successfully with a stale resume. + completed_session = await service.get_session( + app_name="agent-node-hitl-order-test", + user_id="user-1", + session_id="session-1", + ) + assert completed_session is not None + # after_child should NOT be True because we didn't properly complete round 2. + assert completed_session.state.get("after_child") is not True + + # After the stale resume the graph may have re-interrupted (the stale + # value was consumed by the current round's interrupt, causing the child + # to ask another question). We verify the key invariant: the graph did + # NOT silently complete. A subsequent proper resume should still be able + # to drive the graph to completion. + # + # Find the latest pending LongRunningEvent from the stale run and resume + # it with "done" to complete the flow. + stale_pending = next( + (event for event in reversed(stale_events) if isinstance(event, LongRunningEvent)), + None, + ) + if stale_pending is not None: + await _run(runner, _tool_response(stale_pending, "done")) + else: + # If the stale resume did not produce a new interrupt, try the + # original second_pending. + await _run(runner, _tool_response(second_pending, "done")) + + completed_session = await service.get_session( + app_name="agent-node-hitl-order-test", + user_id="user-1", + session_id="session-1", + ) + assert completed_session is not None + assert completed_session.state.get("after_child") is True + + await runner.close() + await service.close() diff --git a/tests/trpc_agent_dsl/graph/test_constants.py b/tests/trpc_agent_dsl/graph/test_constants.py index 5dc561a29..8e8be7eee 100644 --- a/tests/trpc_agent_dsl/graph/test_constants.py +++ b/tests/trpc_agent_dsl/graph/test_constants.py @@ -39,6 +39,7 @@ STATE_KEY_NODE_RESPONSES, STATE_KEY_ONE_SHOT_MESSAGES, STATE_KEY_ONE_SHOT_MESSAGES_BY_NODE, + STATE_KEY_PENDING_AGENT_NODE_HITL, STATE_KEY_PENDING_INTERRUPT, STATE_KEY_PENDING_INTERRUPT_AUTHOR, STATE_KEY_PENDING_INTERRUPT_BRANCH, @@ -114,6 +115,7 @@ def test_interrupt_keys(self): assert STATE_KEY_PENDING_INTERRUPT_ID == "_trpc_graph_pending_interrupt_id" assert STATE_KEY_PENDING_INTERRUPT_AUTHOR == "_trpc_graph_pending_interrupt_author" assert STATE_KEY_PENDING_INTERRUPT_BRANCH == "_trpc_graph_pending_interrupt_branch" + assert STATE_KEY_PENDING_AGENT_NODE_HITL == "_trpc_graph_pending_agent_node_hitl" assert STATE_KEY_LONG_RUNNING_PREFIX == "__trpc_graph_long_running__" def test_role_values(self): @@ -145,6 +147,7 @@ def test_unsafe_keys_contains_expected_members(self): STATE_KEY_PENDING_INTERRUPT_ID, STATE_KEY_PENDING_INTERRUPT_AUTHOR, STATE_KEY_PENDING_INTERRUPT_BRANCH, + STATE_KEY_PENDING_AGENT_NODE_HITL, } assert UNSAFE_STATE_KEYS == expected @@ -169,10 +172,15 @@ def test_is_unsafe_state_key_returns_false_for_safe_keys(self): def test_serializable_keys_are_not_unsafe(self): """Keys that represent user-visible data should never appear in UNSAFE_STATE_KEYS.""" serializable = { - STATE_KEY_USER_INPUT, STATE_KEY_MESSAGES, STATE_KEY_LAST_RESPONSE, - STATE_KEY_LAST_RESPONSE_ID, STATE_KEY_LAST_TOOL_RESPONSE, - STATE_KEY_NODE_RESPONSES, STATE_KEY_ONE_SHOT_MESSAGES, - STATE_KEY_ONE_SHOT_MESSAGES_BY_NODE, STATE_KEY_METADATA, + STATE_KEY_USER_INPUT, + STATE_KEY_MESSAGES, + STATE_KEY_LAST_RESPONSE, + STATE_KEY_LAST_RESPONSE_ID, + STATE_KEY_LAST_TOOL_RESPONSE, + STATE_KEY_NODE_RESPONSES, + STATE_KEY_ONE_SHOT_MESSAGES, + STATE_KEY_ONE_SHOT_MESSAGES_BY_NODE, + STATE_KEY_METADATA, STATE_KEY_STEP_NUMBER, } assert serializable.isdisjoint(UNSAFE_STATE_KEYS) diff --git a/tests/trpc_agent_dsl/graph/test_events.py b/tests/trpc_agent_dsl/graph/test_events.py index fe3cbc8e5..0c8ea6667 100644 --- a/tests/trpc_agent_dsl/graph/test_events.py +++ b/tests/trpc_agent_dsl/graph/test_events.py @@ -54,6 +54,26 @@ def test_node_start_contains_structured_metadata_and_truncates_model_input(self) assert event.object == GraphEventType.GRAPH_NODE_START assert event.partial is True + def test_node_events_prefer_human_readable_description_in_text(self): + """User-visible node events should not leak internal node IDs.""" + start = self.builder.node_start( + node_id="tech-clarify", + node_description="技术澄清", + ) + complete = self.builder.node_complete( + node_id="tech-clarify", + node_description="技术澄清", + ) + error = self.builder.node_error( + node_id="tech-clarify", + node_description="技术澄清", + error="failed", + ) + + assert start.get_text() == "Starting node: 技术澄清" + assert complete.get_text().startswith("Completed node: 技术澄清") + assert error.get_text() == "Error in node 技术澄清: failed" + def test_model_complete_uses_error_phase_and_truncates_payloads(self): """Model completion should switch to error phase when error is provided.""" start_time = datetime.now() - timedelta(milliseconds=20) diff --git a/tests/trpc_agent_dsl/graph/test_memory_saver.py b/tests/trpc_agent_dsl/graph/test_memory_saver.py index a099243a7..193347fa2 100644 --- a/tests/trpc_agent_dsl/graph/test_memory_saver.py +++ b/tests/trpc_agent_dsl/graph/test_memory_saver.py @@ -13,8 +13,10 @@ from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_CHECKPOINTS from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_CHECKPOINT_BLOBS from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_CHECKPOINT_WRITES +from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_PENDING_INTERRUPT from trpc_agent_sdk.dsl.graph._memory_saver import MemorySaver from trpc_agent_sdk.dsl.graph._memory_saver import has_graph_internal_checkpoint_state +from trpc_agent_sdk.dsl.graph._memory_saver import has_graph_resume_state from trpc_agent_sdk.dsl.graph._memory_saver import strip_graph_internal_checkpoint_state @@ -39,6 +41,36 @@ def test_has_and_strip_graph_internal_checkpoint_state(self): assert stripped == {"visible": "keep"} assert STATE_KEY_CHECKPOINTS in state + def test_has_graph_resume_state_only_tracks_outstanding_interrupt(self): + """Resume detection must key off the outstanding-interrupt marker, not + checkpoint storage. A graph that ran to completion still carries + checkpoint keys; treating those as "resumable" would preserve the + session from cleanup forever and suppress client state sync (the + original bug). Only an outstanding interrupt should count. + """ + # Empty / no marker. + assert has_graph_resume_state({}) is False + + # Completed graph: checkpoint keys linger but no pending interrupt. + completed = { + STATE_KEY_CHECKPOINTS: { + "t": {} + }, + STATE_KEY_CHECKPOINT_BLOBS: { + "t": {} + }, + STATE_KEY_CHECKPOINT_WRITES: { + "t": {} + }, + } + assert has_graph_resume_state(completed) is False + + # Marker reset to False on resume must not count. + assert has_graph_resume_state({STATE_KEY_PENDING_INTERRUPT: False}) is False + + # Only an outstanding interrupt counts as resumable. + assert has_graph_resume_state({STATE_KEY_PENDING_INTERRUPT: True}) is True + class TestMemorySaverStorage: """Tests for MemorySaver public put/get/list/write/delete behavior.""" diff --git a/tests/trpc_agent_dsl/graph/test_node_action_agent.py b/tests/trpc_agent_dsl/graph/test_node_action_agent.py index 588ae13c1..21c588d91 100644 --- a/tests/trpc_agent_dsl/graph/test_node_action_agent.py +++ b/tests/trpc_agent_dsl/graph/test_node_action_agent.py @@ -5,26 +5,30 @@ # tRPC-Agent-Python is licensed under Apache-2.0. """Execution-path tests for AgentNodeAction.""" +import json from typing import Any import pytest -from google.genai.types import Content -from google.genai.types import Part -from trpc_agent_sdk.dsl.graph._callbacks import NodeCallbackContext -from trpc_agent_sdk.dsl.graph._callbacks import NodeCallbacks -from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_LAST_RESPONSE -from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_MESSAGES -from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_NODE_RESPONSES -from trpc_agent_sdk.dsl.graph._constants import STATE_KEY_USER_INPUT -from trpc_agent_sdk.dsl.graph._constants import STREAM_KEY_ACK -from trpc_agent_sdk.dsl.graph._constants import STREAM_KEY_EVENT -from trpc_agent_sdk.dsl.graph._event_writer import AsyncEventWriter -from trpc_agent_sdk.dsl.graph._event_writer import EventWriter +from google.genai.types import Content, Part + +from trpc_agent_sdk.dsl.graph._callbacks import NodeCallbackContext, NodeCallbacks +from trpc_agent_sdk.dsl.graph._constants import ( + STATE_KEY_LAST_RESPONSE, + STATE_KEY_MESSAGES, + STATE_KEY_NODE_RESPONSES, + STATE_KEY_PENDING_AGENT_NODE_HITL, + STATE_KEY_USER_INPUT, + STREAM_KEY_ACK, + STREAM_KEY_EVENT, +) +from trpc_agent_sdk.dsl.graph._event_writer import AsyncEventWriter, EventWriter from trpc_agent_sdk.dsl.graph._node_action._agent import AgentNodeAction from trpc_agent_sdk.dsl.graph._node_config import NodeConfig +from trpc_agent_sdk.dsl.graph._state_graph import StateGraph from trpc_agent_sdk.events import Event from trpc_agent_sdk.sessions import Session from trpc_agent_sdk.types import EventActions +from trpc_agent_sdk.types import State class _AckingWriter: @@ -82,6 +86,13 @@ def __init__(self, agent, session: Session, branch: str = "root"): self.callback_state = None self.override_messages = None + @property + def state(self) -> State: + # Mirror InvocationContext.state: a delta-aware view over session.state. + if self.callback_state is None: + self.callback_state = State(value=self.session.state, delta=self.event_actions.state_delta) + return self.callback_state + def model_copy(self, update: dict[str, Any], deep: bool = False): del deep clone = _FakeInvocationContext(self.agent, self.session, self.branch) @@ -98,6 +109,7 @@ def _build_action( ctx: _FakeInvocationContext | None = None, callbacks: NodeCallbacks | None = None, isolated_messages: bool = False, + history_scope: str | None = None, input_from_last_response: bool = False, event_scope: str | None = None, input_mapper=None, @@ -127,6 +139,7 @@ def _build_action( callback_ctx=NodeCallbackContext(node_id="node-1"), callbacks=callbacks, isolated_messages=isolated_messages, + history_scope=history_scope, input_from_last_response=input_from_last_response, event_scope=event_scope, input_mapper=input_mapper, @@ -368,6 +381,128 @@ async def test_execute_builds_child_history_respecting_isolated_messages(self): isolated_texts = [event.get_text() for event in isolated_agent.calls[0].session.events if event.content] assert isolated_texts[:1] == ["next"] + async def test_execute_branch_history_inherits_only_the_same_child_branch(self): + same_branch = Event( + invocation_id="inv-1", + author="child", + branch="root.child", + content=Content(role="model", parts=[Part.from_text(text="same")]), + ) + nested_branch = Event( + invocation_id="inv-1", + author="member", + branch="root.child.member", + content=Content(role="model", parts=[Part.from_text(text="nested")]), + ) + other_branch = Event( + invocation_id="inv-1", + author="other", + branch="root.other", + content=Content(role="model", parts=[Part.from_text(text="other")]), + ) + child_event = Event( + invocation_id="inv-2", + author="child", + content=Content(role="model", parts=[Part.from_text(text="done")]), + ) + agent = _ScriptedAgent("child", [[child_event]]) + ctx = _FakeInvocationContext( + agent, + _session_with_events(same_branch, nested_branch, other_branch), + branch="root", + ) + action, _ = _build_action( + agent, + NodeConfig(name="node-1"), + ctx=ctx, + isolated_messages=True, + history_scope="branch", + ) + + await action.execute({STATE_KEY_USER_INPUT: "next"}) + + texts = [event.get_text() for event in agent.calls[0].session.events if event.content] + assert texts[:3] == ["same", "nested", "next"] + assert "other" not in texts + assert agent.calls[0].session.state[STATE_KEY_MESSAGES] == [] + json.dumps(agent.calls[0].session.state) + + async def test_explicit_history_scope_overrides_legacy_isolated_flag(self): + existing = Event( + invocation_id="inv-1", + author="user", + branch="root.other", + content=Content(role="user", parts=[Part.from_text(text="history")]), + ) + child_event = Event( + invocation_id="inv-2", + author="child", + content=Content(role="model", parts=[Part.from_text(text="done")]), + ) + + all_agent = _ScriptedAgent("child", [[child_event]]) + all_ctx = _FakeInvocationContext(all_agent, _session_with_events(existing), branch="root") + all_action, _ = _build_action( + all_agent, + NodeConfig(name="node-1"), + ctx=all_ctx, + isolated_messages=True, + history_scope="all", + ) + await all_action.execute({STATE_KEY_USER_INPUT: "next"}) + all_texts = [event.get_text() for event in all_agent.calls[0].session.events if event.content] + assert all_texts[:2] == ["history", "next"] + + none_agent = _ScriptedAgent("child", [[child_event]]) + none_ctx = _FakeInvocationContext(none_agent, _session_with_events(existing), branch="root") + none_action, _ = _build_action( + none_agent, + NodeConfig(name="node-1"), + ctx=none_ctx, + isolated_messages=False, + history_scope="none", + ) + await none_action.execute({STATE_KEY_USER_INPUT: "next"}) + none_texts = [event.get_text() for event in none_agent.calls[0].session.events if event.content] + assert none_texts[:1] == ["next"] + assert none_agent.calls[0].session.state[STATE_KEY_MESSAGES] == [] + + async def test_all_history_scope_keeps_full_history_during_pending_hitl(self): + same_branch = Event( + invocation_id="inv-1", + author="child", + branch="root.child", + content=Content(role="model", parts=[Part.from_text(text="same")]), + ) + other_branch = Event( + invocation_id="inv-1", + author="other", + branch="root.other", + content=Content(role="model", parts=[Part.from_text(text="other")]), + ) + session = _session_with_events(same_branch, other_branch) + session.state[STATE_KEY_PENDING_AGENT_NODE_HITL] = {"node_id": "node-1"} + agent = _ScriptedAgent("child", [[]]) + ctx = _FakeInvocationContext(agent, session, branch="root") + action, _ = _build_action( + agent, + NodeConfig(name="node-1"), + ctx=ctx, + isolated_messages=False, + history_scope="all", + ) + + events = action._build_child_events(ctx, "next", "root.child") + + assert [event.get_text() for event in events] == ["same", "other", "next"] + + async def test_invalid_history_scope_fails_when_adding_agent_node(self): + graph = StateGraph(dict) + agent = _ScriptedAgent("child", [[]]) + + with pytest.raises(ValueError, match="history_scope"): + graph.add_agent_node("child", agent, history_scope="invalid") + async def test_execute_ignores_graph_events_for_state_accumulation(self): """Graph lifecycle events should not override state-derived response values.""" graph_event = Event( @@ -392,3 +527,51 @@ async def test_execute_ignores_graph_events_for_state_accumulation(self): result = await action.execute({}) assert result[STATE_KEY_LAST_RESPONSE] == "final" + + +class TestConcurrentHitlGuard: + """The single-slot pending-HITL bridge must reject concurrent nodes.""" + + def _action_with_state(self, initial_state: dict): + agent = _ScriptedAgent("child", [[]]) + session = _session_with_events() + session.state.update(initial_state) + ctx = _FakeInvocationContext(agent, session) + action, _ = _build_action(agent, NodeConfig(name="node-1"), ctx=ctx) + return action, ctx + + def test_rejects_pending_owned_by_another_node(self): + """A second node interrupting while another node is pending must fail + loudly instead of silently overwriting the first node's resume slot.""" + action, ctx = self._action_with_state( + {STATE_KEY_PENDING_AGENT_NODE_HITL: { + "node_id": "other-node", + "current": {} + }}) + with pytest.raises(RuntimeError, match="Concurrent HITL"): + action._reject_concurrent_pending_hitl(ctx) + + def test_allows_when_absent(self): + """No pending slot → first HITL round proceeds.""" + action, ctx = self._action_with_state({}) + action._reject_concurrent_pending_hitl(ctx) # must not raise + + def test_allows_own_pending_for_multi_round_resume(self): + """The same node re-interrupting across rounds is legitimate.""" + action, ctx = self._action_with_state( + {STATE_KEY_PENDING_AGENT_NODE_HITL: { + "node_id": "node-1", + "current": {} + }}) + action._reject_concurrent_pending_hitl(ctx) # must not raise + + def test_detects_delta_written_in_same_superstep(self): + """A sibling node's pending written to the delta (not yet committed to + session.state) is still visible via ctx.state and must be rejected.""" + action, ctx = self._action_with_state({}) + ctx.event_actions.state_delta[STATE_KEY_PENDING_AGENT_NODE_HITL] = { + "node_id": "sibling", + "current": {}, + } + with pytest.raises(RuntimeError, match="Concurrent HITL"): + action._reject_concurrent_pending_hitl(ctx) diff --git a/trpc_agent_sdk/dsl/graph/__init__.py b/trpc_agent_sdk/dsl/graph/__init__.py index 360665639..c7b9fdb3d 100644 --- a/trpc_agent_sdk/dsl/graph/__init__.py +++ b/trpc_agent_sdk/dsl/graph/__init__.py @@ -99,10 +99,12 @@ from ._event_writer import EventWriter from ._event_writer import EventWriterBase from ._graph_agent import GraphAgent +from ._history import HistoryScope from ._interrupt import interrupt from ._memory_saver import MemorySaver from ._memory_saver import MemorySaverOption from ._memory_saver import has_graph_internal_checkpoint_state +from ._memory_saver import has_graph_resume_state from ._memory_saver import strip_graph_internal_checkpoint_state from ._node_config import NodeConfig from ._state import State @@ -146,10 +148,12 @@ "EventWriter", "EventWriterBase", "GraphAgent", + "HistoryScope", "interrupt", "MemorySaver", "MemorySaverOption", "has_graph_internal_checkpoint_state", + "has_graph_resume_state", "strip_graph_internal_checkpoint_state", "NodeConfig", "State", diff --git a/trpc_agent_sdk/dsl/graph/_constants.py b/trpc_agent_sdk/dsl/graph/_constants.py index c2d04072b..95fa6c252 100644 --- a/trpc_agent_sdk/dsl/graph/_constants.py +++ b/trpc_agent_sdk/dsl/graph/_constants.py @@ -148,6 +148,11 @@ STATE_KEY_PENDING_INTERRUPT_AUTHOR = "_trpc_graph_pending_interrupt_author" STATE_KEY_PENDING_INTERRUPT_BRANCH = "_trpc_graph_pending_interrupt_branch" +# Pending child-agent HITL context owned by AgentNodeAction. This bridges a +# child LongRunningEvent into a parent GraphAgent interrupt and supports +# replaying multiple clarification rounds in the same graph node. +STATE_KEY_PENDING_AGENT_NODE_HITL = "_trpc_graph_pending_agent_node_hitl" + # Prefix for synthetic function call IDs used by graph interrupt bridge STATE_KEY_LONG_RUNNING_PREFIX = "__trpc_graph_long_running__" @@ -179,6 +184,7 @@ STATE_KEY_PENDING_INTERRUPT_ID, STATE_KEY_PENDING_INTERRUPT_AUTHOR, STATE_KEY_PENDING_INTERRUPT_BRANCH, + STATE_KEY_PENDING_AGENT_NODE_HITL, }) diff --git a/trpc_agent_sdk/dsl/graph/_events/_builder.py b/trpc_agent_sdk/dsl/graph/_events/_builder.py index 9ecd19555..f939c0401 100644 --- a/trpc_agent_sdk/dsl/graph/_events/_builder.py +++ b/trpc_agent_sdk/dsl/graph/_events/_builder.py @@ -105,6 +105,24 @@ def branch(self) -> str: # Internal Helpers # ========================================================================= + @staticmethod + def _display_name(node_description: Optional[str], node_id: str) -> str: + """Resolve a human-readable display name for a node. + + Falls back to ``node_id`` when ``node_description`` is empty or + whitespace-only. + + Args: + node_description: Optional description supplied by the graph author. + node_id: The node identifier used as fallback. + + Returns: + The display name to show in event text. + """ + if node_description and node_description.strip(): + return node_description.strip() + return node_id + def _build_event( self, text: str, @@ -185,9 +203,10 @@ def node_start( # Store metadata in state_delta state_delta: dict[str, Any] = {} _store_metadata(state_delta, METADATA_KEY_NODE, metadata) + display_name = self._display_name(node_description, node_id) return self._build_event( - text=f"Starting node: {node_id}", + text=f"Starting node: {display_name}", state_delta=state_delta, partial=True, author_override=self._author or node_id, @@ -227,9 +246,10 @@ def node_complete( # Store metadata in state_delta state_delta: dict[str, Any] = {} _store_metadata(state_delta, METADATA_KEY_NODE, metadata) + display_name = self._display_name(node_description, node_id) return self._build_event( - text=f"Completed node: {node_id} ({duration_ms:.1f}ms)", + text=f"Completed node: {display_name} ({duration_ms:.1f}ms)", state_delta=state_delta, author_override=self._author or node_id, object_type=GraphEventType.GRAPH_NODE_COMPLETE, @@ -266,7 +286,8 @@ def node_error( _store_metadata(state_delta, METADATA_KEY_NODE, metadata) # Build error message - text = f"Error in node {node_id}: {error}" + display_name = self._display_name(node_description, node_id) + text = f"Error in node {display_name}: {error}" return self._build_event( text=text, diff --git a/trpc_agent_sdk/dsl/graph/_graph_agent.py b/trpc_agent_sdk/dsl/graph/_graph_agent.py index a3e1eb0c8..82b5feca6 100644 --- a/trpc_agent_sdk/dsl/graph/_graph_agent.py +++ b/trpc_agent_sdk/dsl/graph/_graph_agent.py @@ -296,7 +296,7 @@ def _extract_resume_command(self, ctx: InvocationContext) -> Optional[Command]: if not function_response_id.startswith(STATE_KEY_LONG_RUNNING_PREFIX): return None - interrupt_id = function_response_id[len(STATE_KEY_LONG_RUNNING_PREFIX):] + interrupt_id = function_response_id.removeprefix(STATE_KEY_LONG_RUNNING_PREFIX) if not interrupt_id: return None @@ -359,7 +359,7 @@ def _create_interrupt_events( if isinstance(interrupt.value, dict): interrupt_response = interrupt.value else: - interrupt_response = {"desicion": interrupt.value} + interrupt_response = {"decision": interrupt.value} function_response = FunctionResponse( id=function_call.id, @@ -393,7 +393,11 @@ def _build_interrupt_function(self, interrupt: Interrupt) -> tuple[str, str, dic function_call_id = f"{STATE_KEY_LONG_RUNNING_PREFIX}{interrupt_id}" raw_args = interrupt.value - if isinstance(raw_args, dict): + if isinstance(raw_args, dict) and raw_args.get("_trpc_agent_node_hitl") is True: + function_name = str(raw_args.get("toolName") or function_name) + visible_args = raw_args.get("arguments") + function_args = visible_args if isinstance(visible_args, dict) else {} + elif isinstance(raw_args, dict): function_args = {str(key): value for key, value in raw_args.items()} else: function_args = {"value": raw_args} diff --git a/trpc_agent_sdk/dsl/graph/_history.py b/trpc_agent_sdk/dsl/graph/_history.py new file mode 100644 index 000000000..e728182ce --- /dev/null +++ b/trpc_agent_sdk/dsl/graph/_history.py @@ -0,0 +1,25 @@ +# Tencent is pleased to support the open source community by making tRPC-Agent-Python available. +# +# Copyright (C) 2026 Tencent. All rights reserved. +# +# tRPC-Agent-Python is licensed under Apache-2.0. +"""Agent-node history inheritance policies.""" + +from typing import Literal + +HistoryScope = Literal["none", "branch", "all"] + +_HISTORY_SCOPES = frozenset({"none", "branch", "all"}) + + +def resolve_history_scope( + history_scope: HistoryScope | None, + *, + isolated_messages: bool, +) -> HistoryScope: + """Resolve the public policy while preserving the legacy boolean default.""" + if history_scope is None: + return "none" if isolated_messages else "all" + if history_scope not in _HISTORY_SCOPES: + raise ValueError("history_scope must be one of: none, branch, all") + return history_scope diff --git a/trpc_agent_sdk/dsl/graph/_memory_saver.py b/trpc_agent_sdk/dsl/graph/_memory_saver.py index b3aaa04bf..2ba79cfbe 100644 --- a/trpc_agent_sdk/dsl/graph/_memory_saver.py +++ b/trpc_agent_sdk/dsl/graph/_memory_saver.py @@ -34,6 +34,7 @@ from ._constants import STATE_KEY_CHECKPOINTS from ._constants import STATE_KEY_CHECKPOINT_BLOBS from ._constants import STATE_KEY_CHECKPOINT_WRITES +from ._constants import STATE_KEY_PENDING_INTERRUPT _INTERNAL_CHECKPOINT_KEYS = frozenset({ STATE_KEY_CHECKPOINTS, @@ -59,6 +60,23 @@ def has_graph_internal_checkpoint_state(state: dict[str, Any]) -> bool: return any(k in state for k in _INTERNAL_CHECKPOINT_KEYS) +def has_graph_resume_state(state: dict[str, Any]) -> bool: + """Whether the session is paused mid-graph waiting for a human response. + + This is intentionally scoped to the *pending-interrupt* marker only, NOT + to the presence of checkpoint storage keys. Checkpoint keys + (``_trpc_graph_checkpoint*``) persist after a graph runs to completion, so + keying off them would report every session that ever ran a GraphAgent as + "resumable" forever — wrongly preserving it from cleanup and suppressing + client state sync. ``STATE_KEY_PENDING_INTERRUPT`` is set to True only while + an interrupt is outstanding and reset to False on resume, so it precisely + identifies a session that must not be expired/deleted mid-HITL. + """ + if not state: + return False + return state.get(STATE_KEY_PENDING_INTERRUPT) is True + + def strip_graph_internal_checkpoint_state(state: dict[str, Any]) -> dict[str, Any]: """Return a shallow copy of state without internal checkpoint storage keys.""" if not state: diff --git a/trpc_agent_sdk/dsl/graph/_node_action/_agent.py b/trpc_agent_sdk/dsl/graph/_node_action/_agent.py index 6ce6378d3..3dc4402e7 100644 --- a/trpc_agent_sdk/dsl/graph/_node_action/_agent.py +++ b/trpc_agent_sdk/dsl/graph/_node_action/_agent.py @@ -6,17 +6,22 @@ """Agent node action executor.""" import json +from datetime import date +from datetime import datetime from typing import Any from typing import Callable from typing import Optional +from langgraph.errors import GraphInterrupt from trpc_agent_sdk.agents import BaseAgent from trpc_agent_sdk.agents import LlmAgent from trpc_agent_sdk.context import InvocationContext from trpc_agent_sdk.events import Event +from trpc_agent_sdk.events import LongRunningEvent from trpc_agent_sdk.exceptions import RunLimitException from trpc_agent_sdk.types import Content from trpc_agent_sdk.types import EventActions +from trpc_agent_sdk.types import FunctionResponse from trpc_agent_sdk.types import Part from .._callbacks import NodeCallbackContext @@ -24,15 +29,45 @@ from .._constants import STATE_KEY_LAST_RESPONSE from .._constants import STATE_KEY_MESSAGES from .._constants import STATE_KEY_NODE_RESPONSES +from .._constants import STATE_KEY_PENDING_AGENT_NODE_HITL from .._constants import STATE_KEY_USER_INPUT from .._event_writer import AsyncEventWriter from .._event_writer import EventWriter +from .._history import HistoryScope +from .._history import resolve_history_scope +from .._interrupt import interrupt from .._node_config import NodeConfig from .._state import State from .._state_mapper import SubgraphResult from ._base import BaseNodeAction +def _json_safe(value: Any) -> Any: + """Recursively convert a value to a JSON-serializable form. + + ``child_state`` is persisted to Session.state (e.g. via SqlSessionService + / DynamicJSON) which serializes with plain ``json.dumps`` and no + ``default=str`` — non-JSON values such as pydantic models, datetime or + sets would raise at persist time and break the whole HITL round. This + helper normalises such values so the pending HITL payload can always be + persisted. + """ + if value is None or isinstance(value, (str, int, float, bool)): + return value + if isinstance(value, dict): + return {str(k): _json_safe(v) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [_json_safe(v) for v in value] + if isinstance(value, (datetime, date)): + return value.isoformat() + if hasattr(value, "model_dump"): + try: + return _json_safe(value.model_dump(mode="json")) + except (AttributeError, TypeError, ValueError): + pass + return str(value) + + class AgentNodeAction(BaseNodeAction): """Executes sub-agent invocation with isolated state. @@ -55,6 +90,7 @@ def __init__( callback_ctx: Optional[NodeCallbackContext] = None, callbacks: Optional[NodeCallbacks] = None, isolated_messages: bool = False, + history_scope: HistoryScope | None = None, input_from_last_response: bool = False, event_scope: Optional[str] = None, input_mapper: Optional[Callable[[dict[str, Any]], dict[str, Any]]] = None, @@ -83,7 +119,10 @@ def __init__( self.node_config = node_config self.callback_ctx = callback_ctx self.callbacks = callbacks - self.isolated_messages = isolated_messages + self.history_scope = resolve_history_scope( + history_scope, + isolated_messages=isolated_messages, + ) self.input_from_last_response = input_from_last_response self.event_scope = event_scope self.input_mapper = input_mapper @@ -124,16 +163,41 @@ async def execute(self, state: State) -> dict[str, Any]: child_branch = f"{parent_ctx.branch}.{child_scope}" if parent_ctx.branch else child_scope child_user_input = child_state.get(STATE_KEY_USER_INPUT, "") + pending_hitl = self._get_pending_hitl(parent_ctx) + resume_content: Optional[Content] = None + if pending_hitl is not None: + completed_rounds = pending_hitl.get("completed", []) + for completed in completed_rounds if isinstance(completed_rounds, list) else []: + if isinstance(completed, dict): + interrupt(self._interrupt_payload(completed)) + + current_round = pending_hitl.get("current") + if not isinstance(current_round, dict): + raise RuntimeError(f"Agent node '{self.node_id}' has invalid pending HITL state.") + human_response = interrupt(self._interrupt_payload(current_round)) + resume_content = self._resume_content(current_round, human_response) + saved_child_state = current_round.get("child_state") + if isinstance(saved_child_state, dict): + child_state = dict(saved_child_state) + child_session = parent_ctx.session.model_copy(deep=True) child_session.state = dict(child_state) - child_events = self._build_child_events(parent_ctx, child_user_input, child_branch) + child_events = self._build_child_events( + parent_ctx, + child_user_input, + child_branch, + resume_content=resume_content, + ) if hasattr(child_session, "events"): child_session.events = child_events - if self.isolated_messages: + if self.history_scope in {"none", "branch"}: + # Persistent Session state must remain JSON-serializable. Content + # models live in the event log; GraphAgent rebuilds model input from + # the already-filtered child events when needed. child_session.state[STATE_KEY_MESSAGES] = [] - child_user_content = None - if isinstance(child_user_input, str) and child_user_input: + child_user_content = resume_content + if child_user_content is None and isinstance(child_user_input, str) and child_user_input: child_user_content = Content( role="user", parts=[Part.from_text(text=child_user_input)], @@ -165,11 +229,36 @@ async def execute(self, state: State) -> dict[str, Any]: transfer_requested = False child_ctx.agent = current_agent - async for event in current_agent.run_async(child_ctx): + agent_stream = current_agent.run_async(child_ctx) + async for event in agent_stream: await self._run_agent_event_callbacks(state, event) + if isinstance(event, LongRunningEvent): + self._reject_concurrent_pending_hitl(parent_ctx) + current_round = self._pending_round(event, child_ctx, final_state) + completed_rounds: list[dict[str, Any]] = [] + if pending_hitl is not None: + previous = pending_hitl.get("completed", []) + if isinstance(previous, list): + completed_rounds.extend(item for item in previous if isinstance(item, dict)) + previous_current = pending_hitl.get("current") + if isinstance(previous_current, dict): + completed_rounds.append({ + key: value + for key, value in previous_current.items() if key != "child_state" + }) + parent_ctx.state[STATE_KEY_PENDING_AGENT_NODE_HITL] = { + "node_id": self.node_id, + "completed": completed_rounds, + "current": current_round, + } + await agent_stream.aclose() + interrupt(self._interrupt_payload(current_round)) + if (not event.partial) and hasattr(child_session, "events"): - child_session.events.append(event.model_copy(deep=True)) + existing_ids = {item.id for item in child_session.events if getattr(item, "id", None)} + if not event.id or event.id not in existing_ids: + child_session.events.append(event.model_copy(deep=True)) if event.actions and event.actions.state_delta: delta = dict(event.actions.state_delta) @@ -245,9 +334,16 @@ async def execute(self, state: State) -> dict[str, Any]: if isinstance(candidate, str) and candidate: last_response = candidate + if pending_hitl is not None: + parent_ctx.state[STATE_KEY_PENDING_AGENT_NODE_HITL] = None + except RunLimitException: raise + except GraphInterrupt: + raise except Exception as e: + if pending_hitl is not None: + parent_ctx.state[STATE_KEY_PENDING_AGENT_NODE_HITL] = None raise RuntimeError(f"Agent node '{self.name}' execution failed: {e}") from e if last_response: @@ -293,10 +389,25 @@ def _build_child_events( parent_ctx: InvocationContext, child_user_input: Any, child_branch: str, + *, + resume_content: Optional[Content] = None, ) -> list[Event]: parent_events = getattr(parent_ctx.session, "events", []) - if self.isolated_messages: - child_events: list[Event] = [] + pending_hitl = self._get_pending_hitl(parent_ctx) + if self.history_scope == "branch": + child_events = [ + event.model_copy(deep=True) for event in parent_events + if event.branch == child_branch or str(event.branch or "").startswith(f"{child_branch}.") + ] + elif self.history_scope == "none" and pending_hitl is not None: + # Legacy isolated HITL resumes need the current child branch to + # reconstruct the pending function-call exchange. + child_events = [ + event.model_copy(deep=True) for event in parent_events + if event.branch == child_branch or str(event.branch or "").startswith(f"{child_branch}.") + ] + elif self.history_scope == "none": + child_events = [] else: child_events = [event.model_copy(deep=True) for event in parent_events] @@ -311,8 +422,106 @@ def _build_child_events( parts=[Part.from_text(text=child_user_input)], ), )) + if resume_content is not None: + child_events.append( + Event( + invocation_id=parent_ctx.invocation_id, + author="user", + branch=child_branch, + content=resume_content.model_copy(deep=True), + )) return child_events + def _get_pending_hitl(self, parent_ctx: InvocationContext) -> Optional[dict[str, Any]]: + value = parent_ctx.session.state.get(STATE_KEY_PENDING_AGENT_NODE_HITL) + if isinstance(value, dict) and value.get("node_id") == self.node_id: + return value + return None + + def _reject_concurrent_pending_hitl(self, parent_ctx: InvocationContext) -> None: + """Fail loudly if another agent node already holds an outstanding HITL. + + The pending-HITL bridge persists a single slot in state + (``STATE_KEY_PENDING_AGENT_NODE_HITL``). If two parallel (fan-out) agent + nodes interrupt within the same superstep, the second write would + silently overwrite the first node's resume context, so its subsequent + resume would rerun the child agent from scratch. Reading through + ``parent_ctx.state`` sees both the committed session state and the + delta another node just wrote in this superstep. Concurrent HITL is not + supported; serialize such nodes instead. + """ + existing = parent_ctx.state.get(STATE_KEY_PENDING_AGENT_NODE_HITL) + if isinstance(existing, dict): + other_node = existing.get("node_id") + if other_node is not None and other_node != self.node_id: + raise RuntimeError( + f"Agent node '{self.node_id}' requested human-in-the-loop while node " + f"'{other_node}' already has an outstanding HITL round. Concurrent HITL " + "across parallel agent nodes is unsupported because the pending state uses " + "a single slot; serialize these nodes so at most one is pending at a time.") + + def _pending_round( + self, + event: LongRunningEvent, + child_ctx: InvocationContext, + child_state: dict[str, Any], + ) -> dict[str, Any]: + return { + "agent_name": event.author or self.agent.name, + "branch": event.branch or child_ctx.branch, + "function_call": event.function_call.model_dump(mode="json"), + "function_response": event.function_response.model_dump(mode="json"), + # child_state is persisted to Session.state; normalise it so the + # pending HITL payload survives JSON serialisation. + "child_state": _json_safe(child_state), + } + + def _interrupt_payload(self, pending_round: dict[str, Any]) -> dict[str, Any]: + function_call = pending_round.get("function_call") + function_call = function_call if isinstance(function_call, dict) else {} + arguments = function_call.get("args") + arguments = arguments if isinstance(arguments, dict) else {} + function_response = pending_round.get("function_response") + function_response = function_response if isinstance(function_response, dict) else {} + response = function_response.get("response") + response = response if isinstance(response, dict) else {} + return { + "_trpc_agent_node_hitl": True, + "nodeId": self.node_id, + "agentName": str(pending_round.get("agent_name") or self.agent.name), + "toolName": str(function_call.get("name") or "graph_interrupt"), + # Long-running tools return the UI interaction contract from the + # tool execution. The model-supplied call arguments take priority + # so the frontend always sees what the model actually requested; + # the tool result may add derived fields such as stable IDs that + # do not collide with existing argument keys. + "arguments": { + **response, + **arguments, + }, + } + + @staticmethod + def _resume_content(pending_round: dict[str, Any], human_response: Any) -> Content: + function_call = pending_round.get("function_call") + function_call = function_call if isinstance(function_call, dict) else {} + response = human_response if isinstance(human_response, dict) else {"value": human_response} + fc_id = str(function_call.get("id") or "") + if not fc_id: + raise ValueError("Cannot resume HITL round: pending function_call is missing " + "an 'id' field, which is required to match the interrupt " + "during graph replay.") + return Content( + role="user", + parts=[ + Part(function_response=FunctionResponse( + id=fc_id, + name=str(function_call.get("name") or ""), + response=response, + )) + ], + ) + async def _run_agent_event_callbacks(self, state: State, event: Event) -> None: if not self.callbacks or not self.callbacks.agent_event: return diff --git a/trpc_agent_sdk/dsl/graph/_state_graph.py b/trpc_agent_sdk/dsl/graph/_state_graph.py index 7940111a1..ff53076b4 100644 --- a/trpc_agent_sdk/dsl/graph/_state_graph.py +++ b/trpc_agent_sdk/dsl/graph/_state_graph.py @@ -48,6 +48,8 @@ from ._event_writer import EventWriter from ._memory_saver import MemorySaver from ._memory_saver import MemorySaverOption +from ._history import HistoryScope +from ._history import resolve_history_scope from ._node_action import AgentNodeAction from ._node_action import CodeNodeAction from ._node_action import KnowledgeNodeAction @@ -605,6 +607,7 @@ def add_agent_node( config: Optional[NodeConfig] = None, callbacks: Optional[NodeCallbacks] = None, isolated_messages: bool = False, + history_scope: HistoryScope | None = None, input_from_last_response: bool = False, event_scope: Optional[str] = None, input_mapper: Optional[Callable[[dict[str, Any]], dict[str, Any]]] = None, @@ -618,6 +621,9 @@ def add_agent_node( config: Common NodeConfig for the node (optional) callbacks: Lifecycle callbacks for this node (optional) isolated_messages: If True, child execution does not inherit parent message history. + history_scope: Explicit history policy: none, branch, or all. An explicit + value takes precedence over isolated_messages. When omitted, the + legacy boolean keeps its existing behavior. input_from_last_response: If True, map parent STATE_KEY_LAST_RESPONSE to child STATE_KEY_USER_INPUT. event_scope: Optional branch scope segment for child agent events. input_mapper: Function to transform parent state to child state. @@ -640,6 +646,10 @@ def add_agent_node( """ if agent is None: raise TypeError(f"Agent for node '{node_id}' must not be None.") + resolved_history_scope = resolve_history_scope( + history_scope, + isolated_messages=isolated_messages, + ) if config is None: config = NodeConfig(name=node_id) @@ -665,6 +675,7 @@ async def agent_action( callback_ctx=callback_ctx, callbacks=callbacks, isolated_messages=isolated_messages, + history_scope=resolved_history_scope, input_from_last_response=input_from_last_response, event_scope=event_scope, input_mapper=input_mapper, diff --git a/trpc_agent_sdk/models/_openai_model.py b/trpc_agent_sdk/models/_openai_model.py index 9adc9e352..a62312e00 100644 --- a/trpc_agent_sdk/models/_openai_model.py +++ b/trpc_agent_sdk/models/_openai_model.py @@ -11,6 +11,8 @@ """ import base64 +import functools +import inspect import json import uuid from enum import Enum @@ -23,6 +25,7 @@ import httpx import openai +from openai.types.responses import ResponseCreateParams from pydantic import BaseModel from trpc_agent_sdk.common import check_enum @@ -51,6 +54,25 @@ from .tool_prompt import ToolPrompt +def _responses_create_parameters(client: Any) -> frozenset: + """Return the parameter names accepted by ``client.responses.create``. + + The signature of ``responses.create`` is fixed for a given OpenAI SDK + version, so the result is cached per ``responses`` resource *type* to + avoid re-inspecting on every request. Raises ``ValueError`` when the + signature cannot be introspected. + """ + return _responses_create_parameters_for_resource(type(client.responses)) + + +@functools.lru_cache(maxsize=64) +def _responses_create_parameters_for_resource(resource_type: type) -> frozenset: + try: + return frozenset(inspect.signature(resource_type.create).parameters) + except (TypeError, ValueError) as exc: + raise ValueError("Unable to determine Responses logprobs support") from exc + + class ToolCall(BaseModel): """Represents a tool call made by the model.""" @@ -110,6 +132,10 @@ class ApiParamsKey(str, Enum): PROMPT_CACHE_RETENTION = "prompt_cache_retention" +_RESPONSES_INPUT_ITEMS = "responses_input_items" +_RESPONSES_LOGPROBS_REQUEST = "_trpc_responses_logprobs_request" + + @register_model(model_name="OpenAIModel", supported_models=[r"gpt-.*", r"o1-.*", r"deepseek-.*", r"hy3-.*"]) class OpenAIModel(LLMModel): """OpenAI model implementation using the abstract model interface. @@ -131,6 +157,13 @@ class OpenAIModel(LLMModel): will be used as the base, with per-request configs overriding specific fields. Useful for maintaining consistent model behavior across multiple calls. + use_responses_api: Use ``client.responses.create`` instead of Chat Completions. + Defaults to False for backward compatibility. + responses_api_params: Optional Responses-only parameters such as ``store``, + ``reasoning``, ``include``, or ``truncation``. Typed as + the openai SDK's ``ResponseCreateParams`` and passed + through verbatim to ``responses.create``. The model, + input, and stream parameters remain managed by this class. **kwargs: Additional arguments passed to parent LLMModel class (e.g., api_key, base_url, etc.) @@ -168,6 +201,8 @@ def __init__( tool_prompt: str = "xml", generate_content_config: Optional[GenerateContentConfig] = None, http_client_provider_factory: HttpClientProviderFactory = temporary_http_client_provider_factory, + use_responses_api: bool = False, + responses_api_params: Optional[ResponseCreateParams] = None, **kwargs, ): super().__init__(model_name, filters_name, **kwargs) @@ -176,6 +211,12 @@ def __init__( # Extract OpenAI-specific config self.organization: str = kwargs.get(const.ORGANIZATION, "") self.client_args = kwargs.get(const.CLIENT_ARGS, {}) + self.use_responses_api = use_responses_api + self.responses_api_params = dict(responses_api_params or {}) + reserved_response_params = {"model", "input", "stream"}.intersection(self.responses_api_params) + if reserved_response_params: + names = ", ".join(sorted(reserved_response_params)) + raise ValueError(f"responses_api_params cannot override managed parameters: {names}") # Allow callers to inject a tuned httpx client http_client_provider_factory = http_client_provider_factory or temporary_http_client_provider_factory self._http_client_provider: BaseHttpClientProvider = http_client_provider_factory() @@ -184,6 +225,16 @@ def __init__( self.add_tools_to_prompt = add_tools_to_prompt self.tool_prompt = tool_prompt + # The Responses API uses native function tools; prompt-injected tool + # definitions (add_tools_to_prompt) are never forwarded, so tool calling + # would silently break. Fail fast at construction instead of at runtime. + if self.use_responses_api and self.add_tools_to_prompt: + raise ValueError( + "use_responses_api=True is incompatible with add_tools_to_prompt=True: " + "the Responses API relies on native function tools, so prompt-injected " + "tool definitions would be ignored and tool calling would silently fail. " + "Disable add_tools_to_prompt when using the Responses API.") + # Default generation config that can be overridden per request self.generate_content_config = generate_content_config # Optional hard cap for tool-response payload injected into model context. @@ -234,7 +285,7 @@ def _create_async_client(self) -> openai.AsyncOpenAI: logging.getLogger("httpx").setLevel(logging.WARNING) client_args = self.client_args.copy() - client_args['http_client'] = self._http_client_provider.create_http_client() + client_args["http_client"] = self._http_client_provider.create_http_client() return openai.AsyncOpenAI( api_key=self._api_key, @@ -320,8 +371,13 @@ def _format_messages(self, request: LlmRequest) -> List[Dict[str, Any]]: parts: list[Part] = content.parts # type: ignore conditions_iter = [ - len(parts) == 1, parts[0].text, parts[0].function_call, parts[0].function_response, - parts[0].code_execution_result, parts[0].executable_code, parts[0].inline_data + len(parts) == 1, + parts[0].text, + parts[0].function_call, + parts[0].function_response, + parts[0].code_execution_result, + parts[0].executable_code, + parts[0].inline_data, ] # Handle different content structures if all(conditions_iter): @@ -338,15 +394,25 @@ def _format_messages(self, request: LlmRequest) -> List[Dict[str, Any]]: reasoning_parts: list[str] = [] image_parts = [] tool_calls = [] + responses_input_items = [] for part in parts: # type: ignore + if part.thought: + if self.use_responses_api and part.thought_signature: + try: + raw = part.thought_signature + raw = raw.decode("utf-8") if isinstance(raw, bytes) else raw + reasoning_item = json.loads(raw) + if isinstance(reasoning_item, dict) and reasoning_item.get("type") == "reasoning": + responses_input_items.append(reasoning_item) + except (UnicodeDecodeError, json.JSONDecodeError, TypeError): + logger.warning("Ignoring invalid Responses reasoning item metadata") + # Reasoning is stripped by default, but some providers + # (e.g. Hunyuan hy3) require it to be replayed. + if part.text and self._adapter.should_preserve_reasoning_content(): + reasoning_parts.append(part.text) + continue if part.text: - if part.thought: - # Reasoning is stripped by default, but some providers - # (e.g. Hunyuan hy3) require it to be replayed. - if self._adapter.should_preserve_reasoning_content(): - reasoning_parts.append(part.text) - continue text_parts.append(part.text) elif part.inline_data and part.inline_data.mime_type: # Handle image data - convert to OpenAI vision format @@ -463,6 +529,16 @@ def _format_messages(self, request: LlmRequest) -> List[Dict[str, Any]]: elif self._adapter.should_backfill_reasoning_content(role, message): message[const.REASONING_CONTENT] = "" + if responses_input_items: + message[_RESPONSES_INPUT_ITEMS] = responses_input_items + + formatted_messages.append(message) + elif responses_input_items: + # Pure-reasoning turn: the assistant emitted only thought + # parts with no text, image, or tool call. We must still + # emit a message so that the collected Responses reasoning + # items are not silently dropped from conversation history. + message = {const.ROLE: role, const.CONTENT: "", _RESPONSES_INPUT_ITEMS: responses_input_items} formatted_messages.append(message) # Validate and fix message sequence for OpenAI compatibility @@ -518,8 +594,10 @@ def _validate_and_fix_openai_messages(self, messages: List[Dict[str, Any]]) -> L # Assistant message without tool calls if pending_tool_calls: # Need to add dummy responses for pending tool calls - logger.warning("Adding dummy tool responses for %s pending tool calls before assistant message", - len(pending_tool_calls)) + logger.warning( + "Adding dummy tool responses for %s pending tool calls before assistant message", + len(pending_tool_calls), + ) for pending_call in pending_tool_calls: dummy_response = { const.ROLE: const.TOOL, @@ -548,8 +626,11 @@ def _validate_and_fix_openai_messages(self, messages: List[Dict[str, Any]]) -> L # User or system message if pending_tool_calls: # Add dummy responses for any pending tool calls before user/system message - logger.warning("Adding dummy tool responses for %s pending tool calls before %s message", - len(pending_tool_calls), role) + logger.warning( + "Adding dummy tool responses for %s pending tool calls before %s message", + len(pending_tool_calls), + role, + ) for pending_call in pending_tool_calls: dummy_response = { const.ROLE: const.TOOL, @@ -739,8 +820,11 @@ def _process_tool_call_delta(self, tool_call_delta: dict, accumulated_tool_calls @staticmethod def _build_usage_metadata(usage_data: dict) -> GenerateContentResponseUsageMetadata: - """Build ``GenerateContentResponseUsageMetadata`` from a raw usage dict. + """Build ``GenerateContentResponseUsageMetadata`` from Chat Completions usage. + Uses the Chat Completions naming (``prompt_tokens`` / ``completion_tokens`` + / ``*_tokens_details``). Responses API payloads use different field names + and are handled by ``_build_responses_usage_metadata``. ``cache_read_input_tokens`` prefers Anthropic/LiteLLM-style top-level fields; falls back to OpenAI-style ``prompt_tokens_details.cached_tokens``. """ @@ -758,11 +842,37 @@ def _build_usage_metadata(usage_data: dict) -> GenerateContentResponseUsageMetad cache_creation_input_tokens=usage_data.get("cache_creation_input_tokens"), ) + @staticmethod + def _build_responses_usage_metadata(usage_data: dict) -> GenerateContentResponseUsageMetadata: + """Build usage metadata from a Responses API usage payload. + + The Responses API reports ``input_tokens`` / ``output_tokens`` / + ``total_tokens`` with ``input_tokens_details`` / ``output_tokens_details`` + — distinct from the Chat Completions ``prompt_tokens`` / ``completion_tokens`` + naming — so this keeps the two API dialects from bleeding into each other. + """ + output_details = usage_data.get("output_tokens_details") or {} + input_details = usage_data.get("input_tokens_details") or {} + cached_tokens = input_details.get("cached_tokens") if isinstance(input_details, dict) else None + return GenerateContentResponseUsageMetadata( + prompt_token_count=usage_data.get("input_tokens", 0), + candidates_token_count=usage_data.get("output_tokens", 0), + thoughts_token_count=output_details.get("reasoning_tokens") if isinstance(output_details, dict) else None, + total_token_count=usage_data.get("total_tokens", 0), + cache_read_input_tokens=cached_tokens, + cache_creation_input_tokens=None, + ) + def _process_usage(self, chunk_dict: dict) -> Optional[GenerateContentResponseUsageMetadata]: """Extract usage metadata from a streaming chunk dict.""" usage_data = chunk_dict.get(const.USAGE) return self._build_usage_metadata(usage_data) if usage_data is not None else None + def _process_responses_usage(self, response_dict: dict) -> Optional[GenerateContentResponseUsageMetadata]: + """Extract usage metadata from a Responses API payload.""" + usage_data = response_dict.get(const.USAGE) + return self._build_responses_usage_metadata(usage_data) if usage_data is not None else None + def _process_chunk_without_content( self, chunk_dict: dict, accumulated_tool_calls: list[dict] ) -> tuple[Optional[FinishReason], Optional[GenerateContentResponseUsageMetadata], dict[int, str]]: @@ -914,8 +1024,12 @@ def _create_complete_tool_calls(self, accumulated_tool_calls: list[dict]) -> Opt logger.warning("Generated fallback ID '%s' for tool call with missing ID", tool_call_id) thought_sig = tool_call_data.get(ToolKey.THOUGHT_SIGNATURE) or None - logger.debug("Creating tool call: id=%s, name=%s, arguments=%s", tool_call_id, - function_map[ToolKey.NAME], arguments) + logger.debug( + "Creating tool call: id=%s, name=%s, arguments=%s", + tool_call_id, + function_map[ToolKey.NAME], + arguments, + ) complete_tool_calls.append( ToolCall( id=tool_call_id, @@ -1107,7 +1221,8 @@ def _create_response_with_content(self, response_dict: dict) -> LlmResponse: tool_call = ToolCall( id=f"call_{uuid.uuid4().hex[:24]}", name=func_call.name, # type: ignore - arguments=func_call.args) # type: ignore + arguments=func_call.args, + ) # type: ignore tool_calls.append(tool_call) except Exception as ex: # pylint: disable=broad-except logger.warning("Failed to parse function calls from text content: %s", ex) @@ -1158,11 +1273,304 @@ def _create_response_with_content(self, response_dict: dict) -> LlmResponse: return LlmResponse(content=content, usage_metadata=usage, error_code=error_code, response_id=response_id) - async def _generate_single(self, - api_params: Dict, - request: LlmRequest, - http_options: Dict[str, Any] | None = None, - ctx: InvocationContext | None = None) -> LlmResponse: + @staticmethod + def _model_dump(value: Any) -> dict: + """Return a dictionary for OpenAI SDK models and test doubles.""" + if isinstance(value, dict): + return value + return value.model_dump() + + def _convert_messages_to_responses_input(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Convert Chat Completions messages to Responses input items.""" + input_items: List[Dict[str, Any]] = [] + for message in messages: + role = message.get(const.ROLE, const.USER) + if role == const.TOOL: + call_id = message.get(const.TOOL_CALL_ID) + if not call_id: + logger.warning("Tool message missing tool_call_id; generating a " + "unique id to avoid Responses API call_id collisions.") + call_id = f"unknown_{uuid.uuid4().hex[:8]}" + input_items.append({ + "type": "function_call_output", + "call_id": call_id, + "output": str(message.get(const.CONTENT, "")), + }) + continue + + input_items.extend(message.get(_RESPONSES_INPUT_ITEMS) or []) + + content = message.get(const.CONTENT) + if content not in (None, "", []): + if isinstance(content, list): + converted_content = [] + for item in content: + item_type = item.get("type") + if item_type == "image_url": + image = item.get("image_url") or {} + converted_content.append({ + "type": "input_image", + "image_url": image.get("url", ""), + "detail": image.get("detail", "auto"), + }) + elif item_type == "text": + converted_content.append({ + "type": "output_text" if role == const.ASSISTANT else "input_text", + "text": item.get("text", ""), + }) + else: + converted_content.append(item) + content = converted_content + input_items.append({"role": role, "content": content}) + + for tool_call in message.get(const.TOOL_CALLS, []) or []: + function = tool_call.get(ToolKey.FUNCTION, {}) + input_items.append({ + "type": "function_call", + "call_id": tool_call.get(ToolKey.ID) or f"call_{uuid.uuid4().hex[:24]}", + "name": function.get(ToolKey.NAME, ""), + "arguments": function.get(ToolKey.ARGUMENTS, "{}"), + }) + return self._reorder_responses_function_call_outputs(input_items) + + @staticmethod + def _reorder_responses_function_call_outputs(items: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Ensure each function_call_output follows its function_call. + + The Responses API rejects a ``function_call_output`` whose matching + ``function_call`` has not appeared earlier in the same input array. + Message assembly can place tool responses before the assistant + tool_calls message, so orphaned outputs are buffered and flushed right + after their matching function_call (or at the end if the call never + appears). Items in canonical order are returned unchanged. + """ + result: List[Dict[str, Any]] = [] + seen_calls: set[str] = set() + buffered: Dict[str, Dict[str, Any]] = {} + for item in items: + item_type = item.get("type") + call_id = item.get("call_id") + if item_type == "function_call": + result.append(item) + if isinstance(call_id, str): + seen_calls.add(call_id) + output = buffered.pop(call_id, None) + if output is not None: + result.append(output) + elif item_type == "function_call_output" and isinstance(call_id, str): + if call_id in seen_calls: + result.append(item) + else: + buffered.setdefault(call_id, item) + else: + result.append(item) + result.extend(buffered.values()) + return result + + @staticmethod + def _convert_tools_to_responses_format(tools: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Flatten Chat Completions function definitions for Responses.""" + converted = [] + for tool in tools: + if tool.get("type") != "function": + converted.append(tool) + continue + function = dict(tool.get("function") or {}) + function["type"] = "function" + converted.append(function) + return converted + + @staticmethod + def _convert_response_format_to_responses(response_format: Dict[str, Any]) -> Dict[str, Any]: + """Convert Chat Completions response_format to Responses text config.""" + if response_format.get("type") == "json_schema": + schema = dict(response_format.get("json_schema") or {}) + return {"format": {"type": "json_schema", **schema}} + return {"format": dict(response_format)} + + def _convert_api_params_to_responses(self, api_params: Dict[str, Any]) -> Dict[str, Any]: + """Translate shared generation parameters to ``responses.create``.""" + responses_params: Dict[str, Any] = { + "model": api_params[ApiParamsKey.MODEL], + "input": self._convert_messages_to_responses_input(api_params[ApiParamsKey.MESSAGES]), + "stream": api_params[ApiParamsKey.STREAM], + } + parameter_map = { + ApiParamsKey.TEMPERATURE: "temperature", + ApiParamsKey.TOP_P: "top_p", + ApiParamsKey.TOOL_CHOICE: "tool_choice", + ApiParamsKey.PARALLEL_TOOL_CALLS: "parallel_tool_calls", + ApiParamsKey.PROMPT_CACHE_KEY: "prompt_cache_key", + ApiParamsKey.PROMPT_CACHE_RETENTION: "prompt_cache_retention", + } + for source, target in parameter_map.items(): + if source in api_params: + responses_params[target] = api_params[source] + + # These shared generation parameters have no equivalent in the + # Responses API. Forwarding them would make the OpenAI API reject the + # request with an HTTP 400, so they are intentionally not mapped; warn + # once per request so callers know the option was dropped. + unsupported_responses_params = [ + key.value for key in ( + ApiParamsKey.STOP, + ApiParamsKey.FREQUENCY_PENALTY, + ApiParamsKey.PRESENCE_PENALTY, + ApiParamsKey.SEED, + ApiParamsKey.N, + ) if key in api_params + ] + if unsupported_responses_params: + logger.warning( + "The Responses API does not support these parameters; they will be ignored: %s", + ", ".join(unsupported_responses_params), + ) + + # The OpenAI Python SDK has used more than one Responses logprobs + # shape. Keep this framework-level request private until we can inspect + # the installed client's supported parameters immediately before send. + response_logprobs = api_params.get(ApiParamsKey.LOGPROBS) + top_logprobs = api_params.get(ApiParamsKey.TOP_LOGPROBS) + if response_logprobs is not None or top_logprobs is not None: + responses_params[_RESPONSES_LOGPROBS_REQUEST] = { + "enabled": bool(response_logprobs) if response_logprobs is not None else bool(top_logprobs), + "top_logprobs": top_logprobs, + } + + max_output_tokens = api_params.get(ApiParamsKey.MAX_COMPLETION_TOKENS) or api_params.get( + ApiParamsKey.MAX_TOKENS) + if max_output_tokens: + responses_params["max_output_tokens"] = max_output_tokens + if ApiParamsKey.TOOLS in api_params: + responses_params["tools"] = self._convert_tools_to_responses_format(api_params[ApiParamsKey.TOOLS]) + if ApiParamsKey.RESPONSE_FORMAT in api_params: + responses_params["text"] = self._convert_response_format_to_responses( + api_params[ApiParamsKey.RESPONSE_FORMAT]) + + # Merge user-supplied responses_api_params. Note: the "reasoning" + # sub-dict is intentionally NOT deep-merged here because reasoning is + # injected later in _generate_async_impl (via setdefault), which + # already preserves user-supplied values. + responses_params.update(self.responses_api_params) + if responses_params.get("store") is False: + include = responses_params.get("include") + if include is None: + include = [] + elif isinstance(include, str): + include = [include] + else: + include = list(include) + if "reasoning.encrypted_content" not in include: + include.append("reasoning.encrypted_content") + responses_params["include"] = include + return responses_params + + @staticmethod + def _prepare_responses_api_params(client: Any, api_params: Dict[str, Any]) -> Dict[str, Any]: + """Adapt optional logprobs to the installed OpenAI client's capability. + + ``openai>=1.66`` exposes Responses but accepts neither of the later + logprobs parameter shapes. Newer clients may expose either a top-level + ``top_logprobs`` integer or a structured ``logprobs`` object. + """ + prepared = dict(api_params) + requested = prepared.pop(_RESPONSES_LOGPROBS_REQUEST, None) + if not requested or not requested["enabled"]: + return prepared + + supported = _responses_create_parameters(client) + if "logprobs" in supported: + prepared["logprobs"] = requested + return prepared + + top_logprobs = requested["top_logprobs"] + if "top_logprobs" in supported and top_logprobs: + prepared["top_logprobs"] = top_logprobs + return prepared + + raise ValueError("Responses logprobs requires an OpenAI SDK that supports the Responses " + "logprobs parameters; upgrade openai or disable logprobs.") + + @staticmethod + def _responses_error(response_dict: dict) -> tuple[Optional[str], Optional[str]]: + status = response_dict.get("status") + if status not in {"failed", "incomplete", "cancelled"}: + return None, None + error = response_dict.get("error") or response_dict.get("incomplete_details") or {} + if isinstance(error, dict): + return str(error.get("code") or status), str(error.get("message") or error.get("reason") or status) + return str(status), str(error) + + def _create_responses_response(self, response_dict: dict) -> LlmResponse: + """Convert a completed Responses payload into ``LlmResponse``.""" + parts = [] + for item in response_dict.get("output") or []: + item_type = item.get("type") + if item_type == "reasoning": + summaries = item.get("summary") or [] + if not summaries: + part = Part.from_text(text="") + part.thought = True + part.thought_signature = json.dumps(item, ensure_ascii=False).encode("utf-8") + parts.append(part) + for index, summary in enumerate(summaries): + text = summary.get("text") if isinstance(summary, dict) else None + if text: + part = Part.from_text(text=text) + part.thought = True + if index == 0: + part.thought_signature = json.dumps(item, ensure_ascii=False).encode("utf-8") + parts.append(part) + elif item_type == "message": + for output_part in item.get("content") or []: + text = output_part.get("text") or output_part.get("refusal") + if text: + part = Part.from_text(text=text) + part.thought = False + parts.append(part) + elif item_type == "function_call": + arguments = json_loads_repair(item.get("arguments") or "{}") + if not isinstance(arguments, dict): + logger.warning("Skipping Responses function call with non-dict arguments: %r", arguments) + continue + part = Part.from_function_call(name=item.get("name", ""), args=arguments) + part.function_call.id = item.get("call_id") or item.get("id") # type: ignore + parts.append(part) + + content = Content(parts=parts, role=const.MODEL) if parts else None + usage = self._process_responses_usage(response_dict) + error_code, error_message = self._responses_error(response_dict) + return LlmResponse( + content=content, + usage_metadata=usage, + error_code=error_code, + error_message=error_message, + response_id=response_dict.get("id"), + ) + + async def _generate_responses_single( + self, + api_params: Dict[str, Any], + http_options: Optional[Dict[str, Any]] = None, + ) -> LlmResponse: + """Generate one non-streaming response through the Responses API.""" + client = self._create_async_client() + try: + response = await client.responses.create( + **self._prepare_responses_api_params(client, api_params), + **(http_options or {}), + ) + return self._create_responses_response(self._model_dump(response)) + finally: + await self._http_client_provider.close_http_client(client) + + async def _generate_single( + self, + api_params: Dict[str, Any], + request: LlmRequest, + http_options: Dict[str, Any] | None = None, + ctx: InvocationContext | None = None, + ) -> LlmResponse: """Generate a single response (non-streaming).""" if http_options is None: http_options = {} @@ -1484,7 +1892,8 @@ def _log_unsupported_config_options(self, config: GenerateContentConfig) -> None if unsupported_options: logger.warning( "The following configuration options are not supported in OpenAI models and will be ignored: %s", - ', '.join(unsupported_options)) + ", ".join(unsupported_options), + ) @override async def _generate_async_impl(self, @@ -1509,7 +1918,7 @@ async def _generate_async_impl(self, # Debug log the formatted messages to help with troubleshooting logger.debug("Formatted messages for OpenAI API: %s", json.dumps(messages, indent=2)) - api_params = { + api_params: Dict[str, Any] = { ApiParamsKey.MODEL: self._model_name, ApiParamsKey.MESSAGES: messages, ApiParamsKey.STREAM: stream, @@ -1537,11 +1946,11 @@ async def _generate_async_impl(self, api_params[ApiParamsKey.STOP] = request.config.stop_sequences # Additional OpenAI-specific parameters - if (request.config.frequency_penalty is not None - and not self._adapter.should_skip_config_param("frequency_penalty")): + if request.config.frequency_penalty is not None and not self._adapter.should_skip_config_param( + "frequency_penalty"): api_params[ApiParamsKey.FREQUENCY_PENALTY] = request.config.frequency_penalty - if (request.config.presence_penalty is not None - and not self._adapter.should_skip_config_param("presence_penalty")): + if request.config.presence_penalty is not None and not self._adapter.should_skip_config_param( + "presence_penalty"): api_params[ApiParamsKey.PRESENCE_PENALTY] = request.config.presence_penalty if request.config.seed is not None and not self._adapter.should_skip_config_param("seed"): api_params[ApiParamsKey.SEED] = request.config.seed @@ -1584,21 +1993,209 @@ async def _generate_async_impl(self, http_options = {} if request.config: http_options = self._extract_http_options(request.config) - # set thinking params - self._set_thinking(request, http_options) + if self.use_responses_api: + api_params = self._convert_api_params_to_responses(api_params) + if (request.config and request.config.thinking_config and request.config.thinking_config.include_thoughts + and request.config.thinking_config.thinking_budget != 0): + # Always request a readable reasoning summary so callers can + # observe the model's reasoning. reasoning.effort is NOT + # derived from thinking_budget here: supported effort values + # vary by model (none/minimal/low/medium/high/xhigh/max) and + # are passed through verbatim via ``responses_api_params``. + reasoning = dict(api_params.get("reasoning") or {}) + reasoning.setdefault("summary", "auto") + api_params["reasoning"] = reasoning + else: + # Chat Completions provider-specific thinking params. + self._set_thinking(request, http_options) if stream: - async for response in self._generate_stream(api_params, request, http_options, ctx): + generator = (self._generate_responses_stream(api_params, request, http_options) + if self.use_responses_api else self._generate_stream(api_params, request, http_options, ctx)) + async for response in generator: yield response else: - response = await self._generate_single(api_params, request, http_options, ctx) + response = (await self._generate_responses_single(api_params, http_options) if self.use_responses_api else + await self._generate_single(api_params, request, http_options, ctx)) yield response - async def _generate_stream(self, - api_params: Dict, - request: LlmRequest, - http_options: Dict[str, Any] | None = None, - ctx: InvocationContext | None = None) -> AsyncGenerator[LlmResponse, None]: + async def _generate_responses_stream( + self, + api_params: Dict[str, Any], + request: LlmRequest, + http_options: Optional[Dict[str, Any]] = None, + ) -> AsyncGenerator[LlmResponse, None]: + """Generate streaming responses through ``responses.create``.""" + client = self._create_async_client() + response: Any = None + response_id: Optional[str] = None + completed_response: Optional[dict] = None + accumulated_text = "" + accumulated_reasoning = "" + function_calls: Dict[str, Dict[str, Any]] = {} + function_order: List[str] = [] + streaming_tool_names = getattr(request, "streaming_tool_names", None) or set() + + def upsert_function(item: dict) -> tuple[str, Dict[str, Any]]: + item_id = str(item.get("id") or item.get("call_id") or f"fc_{len(function_order)}") + if item_id not in function_calls: + function_calls[item_id] = { + "type": "function_call", + "id": item.get("id"), + "call_id": item.get("call_id") or item.get("id"), + "name": item.get("name", ""), + "arguments": item.get("arguments") or "", + } + function_order.append(item_id) + else: + current = function_calls[item_id] + for key in ("id", "call_id", "name"): + if item.get(key): + current[key] = item[key] + if item.get("arguments"): + current["arguments"] = item["arguments"] + return item_id, function_calls[item_id] + + try: + response = await client.responses.create( + **self._prepare_responses_api_params(client, api_params), + **(http_options or {}), + ) + if response is None: + raise ValueError("Empty response from Responses API") + + async for event in response: + event_dict = self._model_dump(event) + event_type = event_dict.get("type", "") + logger.debug("OpenAI Responses event: %s", json.dumps(event_dict, ensure_ascii=False)) + + response_data = event_dict.get("response") or {} + if response_id is None and response_data.get("id"): + response_id = response_data["id"] + if response_id is None and event_dict.get("response_id"): + response_id = event_dict["response_id"] + + if event_type == "error": + completed_response = { + "id": response_id, + "status": "failed", + "error": { + "code": event_dict.get("code") or "responses_stream_error", + "message": event_dict.get("message") or "Responses stream failed", + }, + "output": [], + } + continue + + if event_type in {"response.output_text.delta", "response.refusal.delta"}: + delta = event_dict.get("delta") or "" + if delta: + accumulated_text += delta + part = Part.from_text(text=delta) + part.thought = False + yield LlmResponse( + content=Content(parts=[part], role=const.MODEL), + partial=True, + response_id=response_id, + custom_metadata={const.CHUNK: event_dict}, + ) + continue + + if event_type in {"response.reasoning_summary_text.delta", "response.reasoning_text.delta"}: + delta = event_dict.get("delta") or "" + if delta: + accumulated_reasoning += delta + part = Part.from_text(text=delta) + part.thought = True + yield LlmResponse( + content=Content(parts=[part], role=const.MODEL), + partial=True, + response_id=response_id, + custom_metadata={const.CHUNK: event_dict}, + ) + continue + + if event_type in {"response.output_item.added", "response.output_item.done"}: + item = event_dict.get("item") or {} + if item.get("type") == "function_call": + upsert_function(item) + continue + + if event_type == "response.function_call_arguments.delta": + item_id = str(event_dict.get("item_id") or f"fc_{event_dict.get('output_index', 0)}") + function_call = function_calls.get(item_id) + if function_call is None: + continue + delta = event_dict.get("delta") or "" + function_call["arguments"] += delta + name = function_call.get("name", "") + if delta and name and name in streaming_tool_names: + part = Part.from_function_call(name=name, args={const.TOOL_STREAMING_ARGS: delta}) + part.function_call.id = function_call.get("call_id") # type: ignore + yield LlmResponse( + content=Content(parts=[part], role=const.MODEL), + partial=True, + response_id=response_id, + custom_metadata={ + const.CHUNK: event_dict, + const.TOOL_STREAMING: True + }, + ) + continue + + if event_type == "response.function_call_arguments.done": + item_id = str(event_dict.get("item_id") or f"fc_{event_dict.get('output_index', 0)}") + item = event_dict.get("item") or {} + if item.get("type") == "function_call": + upsert_function(item) + elif item_id in function_calls: + function_calls[item_id]["arguments"] = event_dict.get("arguments") or "" + continue + + if event_type in {"response.completed", "response.failed", "response.incomplete"}: + completed_response = response_data + + if completed_response is None: + output = [] + if accumulated_reasoning: + output.append({ + "type": "reasoning", + "summary": [{ + "type": "summary_text", + "text": accumulated_reasoning + }], + }) + if accumulated_text: + output.append({ + "type": "message", + "role": "assistant", + "content": [{ + "type": "output_text", + "text": accumulated_text + }], + }) + output.extend(function_calls[item_id] for item_id in function_order) + completed_response = {"id": response_id, "status": "completed", "output": output} + + final_response = self._create_responses_response(completed_response) + final_response.partial = False + final_response.custom_metadata = {"stream_complete": True} + yield final_response + finally: + if response is not None and hasattr(response, "aclose"): + await response.aclose() + try: + await self._http_client_provider.close_http_client(client) + except Exception: # pylint: disable=broad-except + logger.debug("Error closing HTTP client for Responses stream", exc_info=True) + + async def _generate_stream( + self, + api_params: Dict[str, Any], + request: LlmRequest, + http_options: Dict[str, Any] | None = None, + ctx: InvocationContext | None = None, + ) -> AsyncGenerator[LlmResponse, None]: """Generate streaming responses.""" if http_options is None: http_options = {} @@ -1611,7 +2208,7 @@ async def _generate_stream(self, response_id: str | None = None # Track response ID from API # For streaming tool call arguments - get the set of tool names that should stream - streaming_tool_names = getattr(request, 'streaming_tool_names', None) or set() + streaming_tool_names = getattr(request, "streaming_tool_names", None) or set() # Create tool prompt instance for streaming if needed tool_prompt = None @@ -1705,10 +2302,12 @@ async def _generate_stream(self, content_part.thought = True partial_content = Content(parts=[content_part], role=const.MODEL) - yield LlmResponse(content=partial_content, - partial=True, - response_id=response_id, - custom_metadata={const.CHUNK: chunk_dict}) + yield LlmResponse( + content=partial_content, + partial=True, + response_id=response_id, + custom_metadata={const.CHUNK: chunk_dict}, + ) # Handle regular content if delta.get(const.CONTENT): @@ -1733,10 +2332,12 @@ async def _generate_stream(self, content_part.thought = is_thinking partial_content = Content(parts=[content_part], role=const.MODEL) - yield LlmResponse(content=partial_content, - partial=True, - response_id=response_id, - custom_metadata={const.CHUNK: chunk_dict}) + yield LlmResponse( + content=partial_content, + partial=True, + response_id=response_id, + custom_metadata={const.CHUNK: chunk_dict}, + ) # Handle usage usage = self._process_usage(chunk_dict) @@ -1752,10 +2353,12 @@ async def _generate_stream(self, content_part = Part.from_text(text=flushed_reasoning_text) content_part.thought = True partial_content = Content(parts=[content_part], role=const.MODEL) - yield LlmResponse(content=partial_content, - partial=True, - response_id=response_id, - custom_metadata={"stream_filter_flushed": "reasoning"}) + yield LlmResponse( + content=partial_content, + partial=True, + response_id=response_id, + custom_metadata={"stream_filter_flushed": "reasoning"}, + ) flushed_content_text = self._adapter.flush_streaming_text(streaming_text_filter_state["content"]) if flushed_content_text: @@ -1764,10 +2367,12 @@ async def _generate_stream(self, content_part = Part.from_text(text=flushed_content_text) content_part.thought = is_thinking partial_content = Content(parts=[content_part], role=const.MODEL) - yield LlmResponse(content=partial_content, - partial=True, - response_id=response_id, - custom_metadata={"stream_filter_flushed": "content"}) + yield LlmResponse( + content=partial_content, + partial=True, + response_id=response_id, + custom_metadata={"stream_filter_flushed": "content"}, + ) # Yield final complete response final_content = None diff --git a/trpc_agent_sdk/server/ag_ui/_core/_agui_agent.py b/trpc_agent_sdk/server/ag_ui/_core/_agui_agent.py index 6478aa8a8..f2731c7cb 100644 --- a/trpc_agent_sdk/server/ag_ui/_core/_agui_agent.py +++ b/trpc_agent_sdk/server/ag_ui/_core/_agui_agent.py @@ -54,6 +54,7 @@ from trpc_agent_sdk.runners import Runner from trpc_agent_sdk.sessions import BaseSessionService from trpc_agent_sdk.sessions import InMemorySessionService +from trpc_agent_sdk.sessions import strip_framework_internal_state from trpc_agent_sdk.tools import LongRunningFunctionTool from trpc_agent_sdk.types import Content @@ -1103,11 +1104,19 @@ async def _run_trpc_in_background(self, # Ensure session exists await self._ensure_session_exists(app_name, user_id, input.thread_id, input.state) - # this will always update the backend states with the frontend states - # Recipe Demo Example: if there is a state "salt" in the ingredients state and in frontend user - # remove this salt state using UI from the ingredients list then our backend should also update - # these state changes as well to sync both the states - await self._session_manager.update_session_state(input.thread_id, app_name, user_id, input.state) + # Always synchronise the client-supplied state so UI-driven edits + # (e.g. the Recipe Demo removing an ingredient from the list) reach + # the backend for both normal turns and HITL tool-result rounds. + # Framework-internal keys (``_trpc_*``, e.g. the GraphAgent + # checkpoint / interrupt markers) are stripped first: they are never + # emitted to the client, so any occurrence here is a stale echo that + # must not overwrite the live checkpoint and restart the graph. + await self._session_manager.update_session_state( + input.thread_id, + app_name, + user_id, + strip_framework_internal_state(input.state), + ) # Convert messages # only use this new_message if there is no tool response from the user diff --git a/trpc_agent_sdk/server/ag_ui/_core/_event_translator.py b/trpc_agent_sdk/server/ag_ui/_core/_event_translator.py index 747fbab39..4c53b5a3d 100644 --- a/trpc_agent_sdk/server/ag_ui/_core/_event_translator.py +++ b/trpc_agent_sdk/server/ag_ui/_core/_event_translator.py @@ -51,6 +51,7 @@ from trpc_agent_sdk.events import LongRunningEvent from trpc_agent_sdk.log import logger from trpc_agent_sdk.models import TOOL_STREAMING_ARGS +from trpc_agent_sdk.sessions import is_framework_internal_state_key class EventTranslator: @@ -599,9 +600,15 @@ def _create_state_delta_event(self, state_delta: Dict[str, Any], timestamp: floa A StateDeltaEvent """ # Convert to JSON Patch format (RFC 6902) - # Use "add" operation which works for both new and existing paths + # Use "add" operation which works for both new and existing paths. + # Framework-internal keys (``_trpc_*``, e.g. GraphAgent checkpoint / + # interrupt markers) are never emitted to the client: they are backend + # continuation tokens, not client-facing state, and echoing them back + # would clobber the checkpoint. patches = [] for key, value in state_delta.items(): + if is_framework_internal_state_key(key): + continue patches.append({"op": "add", "path": f"/{key}", "value": value}) timestamp_ms = int(timestamp * 1000) @@ -621,8 +628,15 @@ def _create_state_snapshot_event( Returns: A StateSnapshotEvent """ + # Drop framework-internal keys (``_trpc_*``) so the client snapshot only + # carries business state. Keeping them would leak backend continuation + # tokens and let the client echo them back to overwrite the checkpoint. + safe_snapshot = { + key: value + for key, value in state_snapshot.items() if not is_framework_internal_state_key(key) + } timestamp_ms = int(timestamp * 1000) - return StateSnapshotEvent(type=EventType.STATE_SNAPSHOT, snapshot=state_snapshot, timestamp=timestamp_ms) + return StateSnapshotEvent(type=EventType.STATE_SNAPSHOT, snapshot=safe_snapshot, timestamp=timestamp_ms) async def force_close_streaming_message(self) -> AsyncGenerator[BaseEvent, None]: """Force close any open streaming message. diff --git a/trpc_agent_sdk/server/ag_ui/_core/_session_manager.py b/trpc_agent_sdk/server/ag_ui/_core/_session_manager.py index 848e7590f..2f18209c4 100644 --- a/trpc_agent_sdk/server/ag_ui/_core/_session_manager.py +++ b/trpc_agent_sdk/server/ag_ui/_core/_session_manager.py @@ -27,6 +27,7 @@ from typing import Set from typing import Union +from trpc_agent_sdk.dsl.graph import has_graph_resume_state from trpc_agent_sdk.log import logger @@ -92,9 +93,13 @@ def __init__( self._cleanup_task: Optional[asyncio.Task] = None self._initialized = True - logger.info("Initialized SessionManager - timeout: %ss, cleanup: %ss, max/user: %s, memory: %s", - session_timeout_seconds, cleanup_interval_seconds, max_sessions_per_user or 'unlimited', - 'enabled' if memory_service else 'disabled') + logger.info( + "Initialized SessionManager - timeout: %ss, cleanup: %ss, max/user: %s, memory: %s", + session_timeout_seconds, + cleanup_interval_seconds, + max_sessions_per_user or "unlimited", + "enabled" if memory_service else "disabled", + ) @classmethod def get_instance(cls, **kwargs): @@ -179,7 +184,9 @@ async def update_session_state(self, if not session: logger.debug( "Session not found for update: %s:%s - this may be normal if session is still being created", - app_name, session_id) + app_name, + session_id, + ) return False if not state_updates: @@ -575,11 +582,25 @@ async def _cleanup_expired_sessions(self): if session and hasattr(session, "last_update_time"): age = current_time - session.last_update_time if age > self._timeout: - # Check for pending tool calls before deletion (HITL scenarios) - pending_calls = session.state.get("pending_tool_calls", []) if session.state else [] + # Do not expire a session that is mid human-in-the-loop. + # Two independent HITL mechanisms can pause a session: + # - pending_tool_calls: classic long-running tool wait. + # - a GraphAgent interrupt that is still outstanding. + # Deleting either would lose the resume point. The graph + # check is scoped to the *outstanding interrupt* marker + # only (see has_graph_resume_state); a completed graph + # is eligible for normal cleanup. + state = session.state if session.state else {} + pending_calls = state.get("pending_tool_calls", []) if pending_calls: - logger.info("Preserving expired session %s - has %s pending tool calls (HITL)", session_key, - len(pending_calls)) + logger.info( + "Preserving expired session %s - has %s pending tool calls (HITL)", + session_key, + len(pending_calls), + ) + elif has_graph_resume_state(state): + logger.info("Preserving expired session %s - graph interrupt awaiting resume", + session_key) else: await self._delete_session(session) expired_count += 1 diff --git a/trpc_agent_sdk/sessions/__init__.py b/trpc_agent_sdk/sessions/__init__.py index 7f18a643c..bca44de22 100644 --- a/trpc_agent_sdk/sessions/__init__.py +++ b/trpc_agent_sdk/sessions/__init__.py @@ -40,13 +40,16 @@ from ._summarizer_checker import set_summarizer_token_threshold from ._summarizer_manager import SummarizerSessionManager from ._types import SessionServiceConfig +from ._utils import FRAMEWORK_INTERNAL_STATE_PREFIX from ._utils import StateStorageEntry from ._utils import app_state_key from ._utils import extract_state_delta from ._utils import find_events_for_summary +from ._utils import is_framework_internal_state_key from ._utils import is_summary_anchor from ._utils import merge_state from ._utils import session_key +from ._utils import strip_framework_internal_state from ._utils import user_state_key __all__ = [ @@ -79,11 +82,14 @@ "SummarizerSessionManager", "SessionServiceConfig", "StateStorageEntry", + "FRAMEWORK_INTERNAL_STATE_PREFIX", "app_state_key", "extract_state_delta", "find_events_for_summary", + "is_framework_internal_state_key", "merge_state", "is_summary_anchor", "session_key", + "strip_framework_internal_state", "user_state_key", ] diff --git a/trpc_agent_sdk/sessions/_utils.py b/trpc_agent_sdk/sessions/_utils.py index ff8420180..b118e961b 100644 --- a/trpc_agent_sdk/sessions/_utils.py +++ b/trpc_agent_sdk/sessions/_utils.py @@ -14,6 +14,40 @@ from trpc_agent_sdk.events import Event from trpc_agent_sdk.types import State +# Framework-internal Session.state keys use this reserved prefix. They are +# persisted to Session.state (e.g. the GraphAgent LangGraph checkpoint and +# interrupt markers stored under ``_trpc_graph_*``) but MUST NOT be exposed to +# external clients (AG-UI state snapshots / deltas) nor treated as business +# state when comparing backends: a client that echoes such a key back as an +# incoming state patch would overwrite the checkpoint and make +# ``Command(resume=...)`` restart the graph from START. +FRAMEWORK_INTERNAL_STATE_PREFIX = "_trpc_" + + +def is_framework_internal_state_key(key: Any) -> bool: + """Whether a Session.state key is framework-internal (not client-facing). + + Args: + key: A candidate state key. + + Returns: + True when the key uses the reserved framework-internal prefix and + therefore must not be surfaced to or accepted from external clients. + """ + return isinstance(key, str) and key.startswith(FRAMEWORK_INTERNAL_STATE_PREFIX) + + +def strip_framework_internal_state(state: Optional[dict[str, Any]]) -> dict[str, Any]: + """Return a shallow copy of ``state`` without framework-internal keys. + + Used at the client boundary (both outbound snapshots/deltas and inbound + state patches) so internal keys never leak out and can never be echoed + back to clobber internal storage. + """ + if not state: + return {} + return {key: value for key, value in state.items() if not is_framework_internal_state_key(key)} + @dataclass class StateStorageEntry: