fix(compression): add timeout to AI compression summary call - #2123
Open
xielixing wants to merge 1 commit into
Open
fix(compression): add timeout to AI compression summary call#2123xielixing wants to merge 1 commit into
xielixing wants to merge 1 commit into
Conversation
Issue GCWing#1529: Context compression's AI model call blocks the conversation round loop synchronously. When the model is slow or unresponsive, the entire conversation freezes with no fallback. Wrap generate_compression_model_summary in tokio::time::timeout(30s). On timeout, return BitFunError::Timeout which falls into the existing generic error fallback path in build_planned_compression_result, using local structured compression instead of blocking indefinitely. This directly addresses the synchronous blocking concern raised in GCWing#1529 without changing compression trigger thresholds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1529
The context compression AI model call blocks the conversation round loop synchronously. When the model is slow or unresponsive, the entire conversation freezes with no fallback path.
Problem
In
build_planned_compression_result, the call togenerate_compression_model_summary(which internally callsrequest_compression_summary_with_retrywith up to 2 retries and exponential backoff) runs without any time bound. If the AI provider is slow or hangs, the conversation is blocked for the full duration of all retry attempts with no escape.The previous fix attempt (#2051) changed the safety reserve threshold, which the issue owner rejected because it doesn't address the core synchronous blocking problem.
Fix
Wrap
generate_compression_model_summaryintokio::time::timeout(30s). On timeout:BitFunError::Timeout(...)is_recoverable_context_overflow(), so it falls into the existing generic error fallback path (Err(err) =>branch)model_summary = Nonecompress_plan_with_contractthen uses local structured compression instead of the AI summaryNo new code paths are introduced — the timeout simply activates an existing fallback that was already handling other non-overflow errors (network failures, auth errors, etc.).
Changes
COMPRESSION_MODEL_SUMMARY_TIMEOUTconstant (30 seconds)generate_compression_model_summarycall intokio::time::timeoutwith a warning log on timeoutValidation
cargo check -p bitfun-core— compiles cleanly (exit code 0, no new warnings)Err(err) =>branch handles timeout errors identically to other non-overflow failuresWhy 30 seconds?