Description
OpenAIChatCompletionClient._inner_get_response's streaming generator iterates the OpenAI SDK's AsyncStream with a bare async for and no async with or finally. The SDK closes the HTTP response in AsyncStream.__stream__'s own finally, which only runs when that generator's frame unwinds — that is, when the stream is consumed to [DONE] or raises from inside itself. When the consumer stops early instead, nothing unwinds it: aclose() on the MAF generator raises GeneratorExit at its yield, the generator exits without closing anything it held, and the response stays open until the async generator is garbage-collected and asyncio's finalizer hook gets to it.
The same package already does this correctly on the Responses path. OpenAIChatClient wraps every one of its streaming branches in async with (async with _open_event_stream(raw_create_response) as stream_response:, async with client.responses.stream(**run_options) as response:). agent_framework_mistral does too (async with await self.client.chat.stream_async(**request) as response:). Chat Completions is the outlier.
Code Sample
`agent_framework_openai/_chat_completion_client.py`, in `_inner_get_response`:
async def _stream() -> AsyncIterable[ChatResponseUpdate]:
client = self.client
...
try:
async for chunk in await client.chat.completions.create(stream=True, **request_options):
...
yield update
except BadRequestError as ex:
...
except Exception as ex:
...
The `AsyncStream` returned by `create(stream=True)` is a temporary held only by the `async for`. It is an async context manager whose `__aexit__` calls `close()` → `response.aclose()`, and that is never used here.Error Messages / Stack Traces
Package Versions
agent-framework-core:1.18.0, agent-framework-openai: 1.14.4
Python Version
No response
Additional Context
Measurement
A real OpenAIChatCompletionClient over an httpx transport whose response body records its own aclose(). The body is still streaming when the consumer stops — roughly 2,000 SSE chunks are left unread — which is the condition that makes the effect visible at all; a probe that feeds a handful of chunks has already pulled the whole body into the SSE parser and every path below looks closed.
| How iteration ends |
Provider response closed at that moment? |
Consumer breaks out of the async for |
No |
| Consumer's stream transform hook raises |
No |
…followed by an explicit await stream._iterator.aclose() |
No |
A chunk the SDK cannot parse (error raised inside __stream__) |
Yes |
The last row is the contrast that identifies the cause: when the exception originates inside the SDK's own generator, its finally runs and the response closes. Nothing a consumer can do from outside reaches it.
Changing the generator to async with await client.chat.completions.create(stream=True, **request_options) as stream: with the async for moved inside is necessary but not sufficient, and the grid is the honest form of the claim. Same transport, same guarded abort, measured across both fixes — "core releases it" standing in for an error path that closes the iterator it was pulling, which ResponseStream does not do today on any path:
_stream() shape |
Core releases the iterator |
Response closed at the abort |
bare async for |
no |
No |
bare async for |
yes |
No |
async with |
no |
No |
async with |
yes |
Yes |
The two bottom rows are what this report is for: without the async with, releasing the iterator changes nothing, so no consumer and no core fix can ever release the response. With it, a break that is followed by nothing still waits for the collector — but the wait shortens from three collection rounds to one, because the chain of un-finalized generators is one link instead of three, and anything that does close the generator now reaches the body.
Why the consumer cannot work around it
The wrapping ResponseStream exposes no handle on the SDK stream, and closing what it does expose does not cascade: aclose() on the MAF generator does not close the AsyncStream (no finally to run), and AsyncStream.__aiter__ is itself a second async generator that is not closed either. Both are left to the garbage collector.
The release does eventually happen, through asyncio's async-generator finalizer hook: once the collector reaches the generator, the hook schedules its aclose(), and the SDK's finally closes the response when that task runs. Measured, with every reference dropped: open immediately after the abort, still open right after gc.collect(), closed two scheduler ticks later. Hold a reference — an exception traceback the application is still handling, say — and it stays open for as long as you hold it. So this is a response held open past its failure for a nondeterministic interval, not a permanent leak. On a long-lived server that interval is whatever the collector and the load make it, and the runs it applies to are the ones with the most body left unread.
Suggested direction
Use the SDK stream as the context manager it already is, in OpenAIChatCompletionClient._inner_get_response's _stream(), matching what OpenAIChatClient and the Mistral client do. That closes the response whenever the generator itself ends — normal completion, an error from either side, and any close from outside, including the one asyncio's finalizer hook eventually performs. What it does not do on its own is release the response at the moment a consumer stops early, because nothing closes the generator at that moment; it makes such a close possible, where today it is not. Deterministic release on an aborted stream needs the core half too: ResponseStream's terminal-failure path closing the iterator it was pulling. Worth deciding together rather than in sequence, since either alone measures as no change at the abort.
Two other clients at this revision have the same bare shape and are worth checking under the same test: agent_framework_ollama's _stream() (async for part in response_object: over the Ollama SDK's stream) and agent_framework_gemini's (async for chunk in await generate_content_stream(...)). Both are read from the source rather than measured here — the measurement above covers Chat Completions only — but neither closes what it iterates, and whether that matters depends on each SDK's own finalization, which is the part worth confirming rather than assuming.
Testing note
A test for this needs a body with unread remainder at the moment the consumer stops. Feeding a short scripted stream hides the defect completely, because the SSE parser has already drained the transport by the second or third chunk and the response is closed for a reason unrelated to the code under test. We got this wrong once ourselves and drew the opposite conclusion from it.
Description
OpenAIChatCompletionClient._inner_get_response's streaming generator iterates the OpenAI SDK'sAsyncStreamwith a bareasync forand noasync withorfinally. The SDK closes the HTTP response inAsyncStream.__stream__'s ownfinally, which only runs when that generator's frame unwinds — that is, when the stream is consumed to[DONE]or raises from inside itself. When the consumer stops early instead, nothing unwinds it:aclose()on the MAF generator raisesGeneratorExitat itsyield, the generator exits without closing anything it held, and the response stays open until the async generator is garbage-collected and asyncio's finalizer hook gets to it.The same package already does this correctly on the Responses path.
OpenAIChatClientwraps every one of its streaming branches inasync with(async with _open_event_stream(raw_create_response) as stream_response:,async with client.responses.stream(**run_options) as response:).agent_framework_mistraldoes too (async with await self.client.chat.stream_async(**request) as response:). Chat Completions is the outlier.Code Sample
Error Messages / Stack Traces
Package Versions
agent-framework-core:1.18.0, agent-framework-openai: 1.14.4
Python Version
No response
Additional Context
Measurement
A real
OpenAIChatCompletionClientover anhttpxtransport whose response body records its ownaclose(). The body is still streaming when the consumer stops — roughly 2,000 SSE chunks are left unread — which is the condition that makes the effect visible at all; a probe that feeds a handful of chunks has already pulled the whole body into the SSE parser and every path below looks closed.breaks out of theasync forawait stream._iterator.aclose()__stream__)The last row is the contrast that identifies the cause: when the exception originates inside the SDK's own generator, its
finallyruns and the response closes. Nothing a consumer can do from outside reaches it.Changing the generator to
async with await client.chat.completions.create(stream=True, **request_options) as stream:with theasync formoved inside is necessary but not sufficient, and the grid is the honest form of the claim. Same transport, same guarded abort, measured across both fixes — "core releases it" standing in for an error path that closes the iterator it was pulling, whichResponseStreamdoes not do today on any path:_stream()shapeasync forasync forasync withasync withThe two bottom rows are what this report is for: without the
async with, releasing the iterator changes nothing, so no consumer and no core fix can ever release the response. With it, abreakthat is followed by nothing still waits for the collector — but the wait shortens from three collection rounds to one, because the chain of un-finalized generators is one link instead of three, and anything that does close the generator now reaches the body.Why the consumer cannot work around it
The wrapping
ResponseStreamexposes no handle on the SDK stream, and closing what it does expose does not cascade:aclose()on the MAF generator does not close theAsyncStream(nofinallyto run), andAsyncStream.__aiter__is itself a second async generator that is not closed either. Both are left to the garbage collector.The release does eventually happen, through asyncio's async-generator finalizer hook: once the collector reaches the generator, the hook schedules its
aclose(), and the SDK'sfinallycloses the response when that task runs. Measured, with every reference dropped: open immediately after the abort, still open right aftergc.collect(), closed two scheduler ticks later. Hold a reference — an exception traceback the application is still handling, say — and it stays open for as long as you hold it. So this is a response held open past its failure for a nondeterministic interval, not a permanent leak. On a long-lived server that interval is whatever the collector and the load make it, and the runs it applies to are the ones with the most body left unread.Suggested direction
Use the SDK stream as the context manager it already is, in
OpenAIChatCompletionClient._inner_get_response's_stream(), matching whatOpenAIChatClientand the Mistral client do. That closes the response whenever the generator itself ends — normal completion, an error from either side, and any close from outside, including the one asyncio's finalizer hook eventually performs. What it does not do on its own is release the response at the moment a consumer stops early, because nothing closes the generator at that moment; it makes such a close possible, where today it is not. Deterministic release on an aborted stream needs the core half too:ResponseStream's terminal-failure path closing the iterator it was pulling. Worth deciding together rather than in sequence, since either alone measures as no change at the abort.Two other clients at this revision have the same bare shape and are worth checking under the same test:
agent_framework_ollama's_stream()(async for part in response_object:over the Ollama SDK's stream) andagent_framework_gemini's (async for chunk in await generate_content_stream(...)). Both are read from the source rather than measured here — the measurement above covers Chat Completions only — but neither closes what it iterates, and whether that matters depends on each SDK's own finalization, which is the part worth confirming rather than assuming.Testing note
A test for this needs a body with unread remainder at the moment the consumer stops. Feeding a short scripted stream hides the defect completely, because the SSE parser has already drained the transport by the second or third chunk and the response is closed for a reason unrelated to the code under test. We got this wrong once ourselves and drew the opposite conclusion from it.