fix(models): close the websocket event iterator on a non-streaming terminal failure - #4765
fix(models): close the websocket event iterator on a non-streaming terminal failure#4765ayaangazali wants to merge 1 commit into
Conversation
…rminal failure _fetch_response(stream=False) consumed _iter_websocket_response_events() with a bare async for and raised from inside the loop on response.failed or response.incomplete. async for does not close its iterator when the loop body raises, and that iterator owns _ws_request_lock and releases it in its finally, so the lock stayed held until generator finalization ran. A later request then blocked on a lock nothing was going to release promptly. Wrap the iteration in contextlib.aclosing so the release happens in the owning task, matching the cleanup ownership openai#4366 accepted for the streamed path. _iter_websocket_response_events is an async generator, so its return annotation is widened from AsyncIterator to AsyncGenerator, which is what aclosing needs.
linhongyu510
left a comment
There was a problem hiding this comment.
I independently exercised this head (58a91140) across the new terminal-failure regression and the surrounding websocket close/cancellation/connection-reuse tests. Excluding the two legacy-httpx compatibility tests whose optional module is absent from my environment, the selected suite is 59 passed; both response.failed and response.incomplete variants are included. The two excluded tests fail at import httpx before touching this change, not in websocket behavior.
The ownership shape looks correct: on a terminal event the consumer raises while the async generator is suspended at yield; aclosing.__aexit__() injects GeneratorExit, the generator recognizes that a terminal event was already yielded, preserves the reusable connection, and reaches its outer finally to release _ws_request_lock before _fetch_response() returns. The cancellation and non-terminal stream-close tests also remain green.
I also checked the annotation change: AsyncGenerator[ResponseStreamEvent, None] is the precise runtime shape of this function and remains an AsyncIterator for existing consumers while exposing aclose() to typing. I found no blocking issue. This is independent focused evidence only; GitHub currently publishes no check runs for the head, so it does not substitute for the repository verification gate.
Summary
OpenAIResponsesWSModel._fetch_response(stream=False)consumed_iter_websocket_response_events()with a bareasync forand raised from inside the loop body onresponse.failedorresponse.incomplete.async fordoes not close its iterator when the loop body raises. That iterator owns_ws_request_lockand releases it in itsfinally(openai_responses.py:1458), so a terminal failure left the generator suspended at its yield and the lock held until generator finalization ran. The next request then waits onrequest_lock.acquire()for a release nothing is scheduled to perform.Measured on
mainby driving the same consumption shape and checking both the iterator and the lock:Even an explicit
gc.collect()does not recover it. After the change both flip to finalized and released before control returns to the caller.This is the non-streaming sibling of #4366, which fixed the streamed runner path and described this same shape: "
async fordoes not close its iterator, so the generator chain stays suspended at its yield and the provider'sfinallynever runs." The fix reuses the ownership pattern accepted there,contextlib.aclosing, rather than introducing a different one.One type change comes with it:
_iter_websocket_response_eventsis an async generator but was annotatedAsyncIterator, which does not declareaclose(), soaclosingfails to typecheck against it. The annotation is widened toAsyncGenerator[ResponseStreamEvent, None], which is what the function already returns and remains compatible with thestream=Truebranch.Test plan
tests/models/test_openai_responses.py::test_fetch_response_non_streaming_terminal_failure_releases_request_lock, parametrized over both terminal event types. It replaces_iter_websocket_response_eventswith a retained async generator that acquires the real_get_ws_request_lock(), yields a terminal event, and records whether itsfinallyran. The test assertsModelBehaviorErrorpropagates and that both the iterator is closed and the lock is released before control returns, so it fails if the release is deferred to finalization rather than done in the owning task.Verified it fails without the source change by reverting
src/: both cases fail on the lock assertion..agents/skills/code-change-verification/scripts/run.shpasses end to end: format, lint, typecheck and the full suite.Issue number
Fixes #4762
Checks
.agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PR@sylvesterkaczmarek wrote up #4762 with the mechanism and the suggested
aclosingshape already worked out, so the analysis here is his and I only implemented and measured it. Happy to close this if he would rather carry it himself. The part I would most like checked is the annotation widening, since it is the one thing that reaches beyond the reported bug and I would rather narrow it than have it slip in unnoticed. I'm a freshman in college and this is the second lock-ownership bug I have looked at in this codebase, so please push back ifaclosingis not the pattern you want here.