Skip to content

ga: enforce long-term settlement at task completion for 15+ turn tasks - #821

Open
louisss1016 wants to merge 1 commit into
lsdefine:mainfrom
louisss1016:fix/issue-789-settlement-gate
Open

louisss1016 wants to merge 1 commit into
lsdefine:mainfrom
louisss1016:fix/issue-789-settlement-gate

Conversation

@louisss1016

Copy link
Copy Markdown

Addresses #789.

What

The schema only asks the model to call start_long_term_update when a 15+ turn task completes; the runtime never checks, and the issue's runs finished on turn 20/39 with zero settlement calls. This adds the suggested one-time completion gate.

Changes

ga.py

  • do_no_tool() final branch: at/above the threshold a normal completion now delegates to the existing do_start_long_term_update() (yield from, no duplicated prompt/SOP logic) and returns its settlement StepOutcome instead of next_prompt=None. The user's final answer is already streamed before tool dispatch, so nothing is hidden.
  • _needs_long_term_settlement() + _in_autonomous_flow(): gate only fires when current_turn >= 15, settlement hasn't started, --no-user-tools is absent (same argv detection as agentmain.BANNED_TOOLS), and the latest [USER]: message isn't an [AUTO] autonomous prompt.
  • do_start_long_term_update() sets _lt_started = True when it actually starts (after the < 10 refusal). This gives all three guarantees from the issue: an explicit prior call blocks the gate, a refused early call does not, and the settlement phase's own completion cannot re-trigger the gate (no recursion).

assets/tools_schema_cn.json

  • 超15轮 → 达到15轮(含), matching the EN schema (15+ turns) and the implemented turn >= 15 boundary — the issue's threshold-wording item.

Tests

frontends/tests/test_long_term_settlement_gate.py — 10 tests, one per case the issue lists: turn 14 exits normally; turn 15 / turn 40 enter settlement exactly once; settlement completion exits without recursion; a prior explicit call blocks the gate; a refused early call does not; no-op evaluation exits cleanly; --no-user-tools exempt; autonomous flow exempt (with latest-user-message detection). Tests use the real GenericAgentHandler — ga.py imports with stdlib only.

Full suite: 308 passed; the 3 failures (test_data_backup symlink ×2, test_release_qualification) are pre-existing Windows-env issues, identical before and after.

Scope notes

  • Generic agent loop / agent_loop.py untouched — the gate lives entirely in GenericAgentHandler, as the issue suggested.
  • Evaluation outcome is unchanged: the settlement prompt still allows a no-op conclusion; the gate forces evaluation, not memory modification.

lsdefine#789)

The tool schema only *asks* the model to call start_long_term_update when a
15+ turn task completes; nothing enforces it, and observed runs finished on
turn 20/39 with zero settlement calls. do_no_tool() now runs a one-time
completion gate: at/above the threshold a normal completion enters the
existing settlement phase (reusing do_start_long_term_update, no duplicated
prompt/SOP logic) instead of exiting.

Exemptions: settlement already started (explicit call included -- a refused
early call does not block the later gate), --no-user-tools, autonomous [AUTO]
flows, and sub-threshold tasks all exit normally. The flag is set inside
do_start_long_term_update, so the settlement phase itself cannot re-trigger
the gate. CN schema wording aligned to 达到15轮(含) to match the >= 15 gate.

10 new tests covering every case listed in the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant