Skip to content

fix: keep the last streamed result set readable after nextset - #560

Merged
pscls merged 2 commits into
mainfrom
fix/streaming-nextset-keeps-last-set
Oct 28, 2026
Merged

pscls merged 2 commits into
mainfrom
fix/streaming-nextset-keeps-last-set

Conversation

@pscls

@pscls pscls commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Since #557, execute_stream can return several result sets in one response,
and nextset() walks through them. To find out whether another set follows,
nextset() has to read past the remaining rows of the current set. It threw
those rows away as it went. On the last set this meant nextset() returned
False with the rows already gone, so the common "skip to the last result
set, then fetch" loop got nothing back. execute was not affected: buffered
results keep the current set when nextset() returns False.

cursor.execute_stream(
    "INSERT INTO t VALUES (1); SELECT 1 UNION ALL SELECT 2;",
    skip_parsing=True,
)
while cursor.nextset():
    pass
cursor.fetchall()  # before: []    after: [[1], [2]]
# cursor.execute(...) with the same loop already returned [[1], [2]]

Fix

  • nextset() now keeps the rows it reads past. If no next set follows, they
    stay readable from where reading stopped. Once a next set starts, they are
    dropped as before.
  • If a statement fails while its rows are being skipped, those rows are
    dropped as well, so a failed statement's partial rows can't be fetched
    afterwards.
  • row_count is unchanged: it still covers every row of the set, fetched or
    skipped.
  • Sync and async row sets get the same change. The nextset() docstrings on
    both cursors now describe what happens when there is no next set.

Trade-off: skipping a result set that hasn't been read now holds its
remaining rows in memory until the next set starts. There is no way around
this: the only way to know whether another set follows is to read past
every remaining row, so nextset() must either keep them or lose them.

Test plan

Unit tests for the sync and async streaming row sets:

  • the last set stays readable after nextset() returns False, with 0 to 3
    rows already fetched across several DATA records
  • repeated nextset() at the end neither loses nor counts rows twice
  • walking past empty result sets to the last one
  • skipped rows are dropped when the next set is in the next response
  • rows of a statement that errors during nextset() cannot be fetched

Cursor tests (sync and async) for execute_stream + while nextset() +
fetchall(). Two existing mock-based test_recovery_after_error tests had
their setup adjusted to a state the real stream can reach; none were removed.

🤖 Generated with Claude Code


Note

Medium Risk
Changes core streaming cursor iteration and multi-result-set navigation; behavior shift is intentional but could affect callers relying on rows being discarded after nextset.

Overview
Fixes execute_stream + while nextset() + fetchall() on the final result set: streaming nextset() used to read ahead to see if another set exists and threw away those rows, so the last set often came back empty even though buffered execute kept them.

Streaming row sets (sync and async) now buffer rows read during nextset() in _skipped_records. If no next set exists, iteration continues from that buffer; if a next set starts (or the statement errors), those rows are cleared as before. row_count still counts skipped rows. Cursor nextset() docstrings describe the new behavior.

Trade-off: skipping an unread result set may hold its remaining rows in memory until the next set begins.

Reviewed by Cursor Bugbot for commit f331710. Bugbot is set up for automated code reviews on this repo. Configure here.

When a streamed response carried several result sets, nextset() read and
dropped the remaining rows of the current set before it found out whether
another set followed. On the last set it returned False with those rows
already gone, so walking to the last set with `while cursor.nextset()` and
then fetching returned nothing. Buffered results kept them.

nextset() now keeps the rows it reads past and serves them if no next set
follows; they are dropped once the next set starts. Rows of a statement
that fails while they are skipped are dropped too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pscls
pscls requested a review from a team as a code owner October 28, 2026 13:07
@pscls
pscls requested a review from gm42 October 28, 2026 13:07
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@pscls
pscls merged commit 1895787 into main Oct 28, 2026
12 checks passed
@pscls
pscls deleted the fix/streaming-nextset-keeps-last-set branch October 28, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants