Skip to content

fix(models): close the websocket event iterator on a non-streaming terminal failure - #4765

Open
ayaangazali wants to merge 1 commit into
openai:mainfrom
ayaangazali:fix/ws-nonstreaming-terminal-lock
Open

fix(models): close the websocket event iterator on a non-streaming terminal failure#4765
ayaangazali wants to merge 1 commit into
openai:mainfrom
ayaangazali:fix/ws-nonstreaming-terminal-lock

Conversation

@ayaangazali

Copy link
Copy Markdown
Contributor

Summary

OpenAIResponsesWSModel._fetch_response(stream=False) consumed _iter_websocket_response_events() with a bare async for and raised from inside the loop body on response.failed or response.incomplete.

async for does not close its iterator when the loop body raises. That iterator owns _ws_request_lock and releases it in its finally (openai_responses.py:1458), so a terminal failure left the generator suspended at its yield and the lock held until generator finalization ran. The next request then waits on request_lock.acquire() for a release nothing is scheduled to perform.

Measured on main by driving the same consumption shape and checking both the iterator and the lock:

iterator finalized right after raise: False
lock still held:                      True
after gc.collect():   finalized False, lock held True

Even an explicit gc.collect() does not recover it. After the change both flip to finalized and released before control returns to the caller.

This is the non-streaming sibling of #4366, which fixed the streamed runner path and described this same shape: "async for does not close its iterator, so the generator chain stays suspended at its yield and the provider's finally never runs." The fix reuses the ownership pattern accepted there, contextlib.aclosing, rather than introducing a different one.

One type change comes with it: _iter_websocket_response_events is an async generator but was annotated AsyncIterator, which does not declare aclose(), so aclosing fails to typecheck against it. The annotation is widened to AsyncGenerator[ResponseStreamEvent, None], which is what the function already returns and remains compatible with the stream=True branch.

Test plan

tests/models/test_openai_responses.py::test_fetch_response_non_streaming_terminal_failure_releases_request_lock, parametrized over both terminal event types. It replaces _iter_websocket_response_events with a retained async generator that acquires the real _get_ws_request_lock(), yields a terminal event, and records whether its finally ran. The test asserts ModelBehaviorError propagates and that both the iterator is closed and the lock is released before control returns, so it fails if the release is deferred to finalization rather than done in the owning task.

Verified it fails without the source change by reverting src/: both cases fail on the lock assertion.

.agents/skills/code-change-verification/scripts/run.sh passes end to end: format, lint, typecheck and the full suite.

Issue number

Fixes #4762

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

@sylvesterkaczmarek wrote up #4762 with the mechanism and the suggested aclosing shape already worked out, so the analysis here is his and I only implemented and measured it. Happy to close this if he would rather carry it himself. The part I would most like checked is the annotation widening, since it is the one thing that reaches beyond the reported bug and I would rather narrow it than have it slip in unnoticed. I'm a freshman in college and this is the second lock-ownership bug I have looked at in this codebase, so please push back if aclosing is not the pattern you want here.

…rminal failure

_fetch_response(stream=False) consumed _iter_websocket_response_events() with a
bare async for and raised from inside the loop on response.failed or
response.incomplete. async for does not close its iterator when the loop body
raises, and that iterator owns _ws_request_lock and releases it in its finally,
so the lock stayed held until generator finalization ran. A later request then
blocked on a lock nothing was going to release promptly.

Wrap the iteration in contextlib.aclosing so the release happens in the owning
task, matching the cleanup ownership openai#4366 accepted for the streamed path.

_iter_websocket_response_events is an async generator, so its return annotation
is widened from AsyncIterator to AsyncGenerator, which is what aclosing needs.
Copilot AI lite review requested due to automatic review settings August 29, 2026 17:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@linhongyu510 linhongyu510 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I independently exercised this head (58a91140) across the new terminal-failure regression and the surrounding websocket close/cancellation/connection-reuse tests. Excluding the two legacy-httpx compatibility tests whose optional module is absent from my environment, the selected suite is 59 passed; both response.failed and response.incomplete variants are included. The two excluded tests fail at import httpx before touching this change, not in websocket behavior.

The ownership shape looks correct: on a terminal event the consumer raises while the async generator is suspended at yield; aclosing.__aexit__() injects GeneratorExit, the generator recognizes that a terminal event was already yielded, preserves the reusable connection, and reaches its outer finally to release _ws_request_lock before _fetch_response() returns. The cancellation and non-terminal stream-close tests also remain green.

I also checked the annotation change: AsyncGenerator[ResponseStreamEvent, None] is the precise runtime shape of this function and remains an AsyncIterator for existing consumers while exposing aclose() to typing. I found no blocking issue. This is independent focused evidence only; GitHub currently publishes no check runs for the head, so it does not substitute for the repository verification gate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Non-streaming Responses WebSocket terminal failures can retain the request lock

3 participants