Repository navigation
Fix retrying interrupted HTTP/1.1 response cleanup - #1113
Open
FriendlyPasser wants to merge 2 commits into
Open
FriendlyPasser wants to merge 2 commits into
FriendlyPasser wants to merge 2 commits into
Conversation
code-crusher
added a commit
to code-crusher/httpx-trace
that referenced
this pull request
Nov 11, 2026
## Problem Cancelling a task that is streaming an `AsyncClient.stream(...)` response twice in quick succession (a second `task.cancel()` delivered while the first cancellation's cleanup is still in progress) leaves the underlying connection counted as active in the connection pool forever. Repeating the pattern exhausts `limits.max_connections`, after which the next request blocks until `PoolTimeout`. ## Root cause `httpx.Response.aclose()`/`close()` set `response.is_closed = True` *before* awaiting `stream.aclose()`. The close of the underlying stream contains suspension points (httpcore's `_response_closed` acquires an async state lock), so a second cancellation can land part-way through the close. When that happens the exception propagates but `is_closed` remains consumed, so any subsequent close attempt — including retries from httpx's own `except BaseException` handlers, or a user's `asyncio.shield(response.aclose())` — hits `if not self.is_closed` and silently no-ops. The connection is therefore never released back to the pool. Instrumented reproduction against httpcore 1.0.9 confirms the second cancellation lands inside `AsyncHTTP11Connection._response_closed`, and every later `aclose()` skips the cleanup. Note that httpcore's `HTTP11ConnectionByteStream.aclose` has the same consumed-flag shape; fixes for that side are pending upstream as encode/httpcore#1113 and encode/httpcore#1116. This change is the httpx half: without it, the fully-read-response variant of the leak persists even with those httpcore fixes applied, because httpx's flag prevents the retry from ever reaching httpcore. ## Fix If closing the stream raises, reset `is_closed` before re-raising, so that a subsequent close retries the real close instead of skipping it. Applied to both the sync and async `Response` close paths. ## Tests - `test_close_can_be_retried_after_interrupted_close` (sync) - `test_aclose_can_be_retried_after_interrupted_close` (async, both asyncio and trio backends) - `test_cancellation_during_response_close` (client-level, cancellation delivered part-way through `response.aclose()` via a task group) All fail against the unfixed code and pass with the fix. Co-Authored-By: Claude Code <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes encode/httpx#3782.
When an asyncio task is cancelled while consuming an HTTP/1.1 response, a second cancellation can interrupt response cleanup after
HTTP11ConnectionByteStreamsets_closed = True. The pool then attempts to close the stream again, but the flag makes that call return without finishing cleanup. The connection can remain active and eventually exhaust the pool.Reset
_closedif cleanup raises, then re-raise the original exception so the caller can retry. Successful close calls remain idempotent. The synchronous implementation is regenerated with the project's unasync script.Regression tests interrupt cleanup both while acquiring the connection state lock and while closing the network stream. They check that cancellation still propagates, the underlying stream closes, pool bookkeeping is cleared, and another request can acquire the pool's single connection slot. Additional tests cover retrying a failed close and repeated successful close calls for both fully read and unread responses.
This is a scoped HTTP/1.1 cleanup retry fix, not a general guarantee against arbitrary repeated cancellation elsewhere in client or pool shutdown.
The build configuration explicitly selects core metadata 2.4 for both distribution formats, keeping isolated Hatchling builds compatible with the existing distribution validator.
Validation
222 passed, 6 xpassedfor the full suite. The XPASS cases are existing HTTP/2 cancellation tests.scripts/checkpasses, including formatting, mypy, Ruff, and unasync consistency.scripts/buildpasses: wheel/sdist creation, Twine checks, and documentation generation.Checklist