Skip to content

fix(compression): add timeout to AI compression summary call - #2123

Open
xielixing wants to merge 1 commit into
GCWing:mainfrom
xielixing:fix/compression-async-timeout
Open

fix(compression): add timeout to AI compression summary call#2123
xielixing wants to merge 1 commit into
GCWing:mainfrom
xielixing:fix/compression-async-timeout

Conversation

@xielixing

Copy link
Copy Markdown

Summary

Fixes #1529

The context compression AI model call blocks the conversation round loop synchronously. When the model is slow or unresponsive, the entire conversation freezes with no fallback path.

Problem

In build_planned_compression_result, the call to generate_compression_model_summary (which internally calls request_compression_summary_with_retry with up to 2 retries and exponential backoff) runs without any time bound. If the AI provider is slow or hangs, the conversation is blocked for the full duration of all retry attempts with no escape.

The previous fix attempt (#2051) changed the safety reserve threshold, which the issue owner rejected because it doesn't address the core synchronous blocking problem.

Fix

Wrap generate_compression_model_summary in tokio::time::timeout(30s). On timeout:

  1. Return BitFunError::Timeout(...)
  2. This error does NOT match is_recoverable_context_overflow(), so it falls into the existing generic error fallback path (Err(err) => branch)
  3. The fallback keeps the selected compression plan but sets model_summary = None
  4. compress_plan_with_contract then uses local structured compression instead of the AI summary

No new code paths are introduced — the timeout simply activates an existing fallback that was already handling other non-overflow errors (network failures, auth errors, etc.).

Changes

  • Added COMPRESSION_MODEL_SUMMARY_TIMEOUT constant (30 seconds)
  • Wrapped the generate_compression_model_summary call in tokio::time::timeout with a warning log on timeout

Validation

  • cargo check -p bitfun-core — compiles cleanly (exit code 0, no new warnings)
  • The existing fallback path at the Err(err) => branch handles timeout errors identically to other non-overflow failures

Why 30 seconds?

  • A typical compression summary request completes within 5-15 seconds
  • 30 seconds allows for one retry within the existing retry logic (500ms base delay + 2 attempts)
  • If the call hasn't completed in 30s, the model is likely stuck, and local compression is a better user experience than indefinite blocking

Issue GCWing#1529: Context compression's AI model call blocks the conversation
round loop synchronously. When the model is slow or unresponsive, the
entire conversation freezes with no fallback.

Wrap generate_compression_model_summary in tokio::time::timeout(30s).
On timeout, return BitFunError::Timeout which falls into the existing
generic error fallback path in build_planned_compression_result, using
local structured compression instead of blocking indefinitely.

This directly addresses the synchronous blocking concern raised in GCWing#1529
without changing compression trigger thresholds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 上下文压缩触发过晚且同步阻塞,导致会话卡顿

1 participant