Repository navigation
Description
Summary
tests/test_004_cursor_arrow.py::test_arrow_reader_cancel_from_other_thread still fails after the HY008 handling added in microsoft/mssql-python#857, which closed microsoft/mssql-python#856.
The evidence points to another close/cancel interleaving: the test sleeps for 50 ms but does not establish that an ODBC fetch is active when reader.close() calls SQLCancel. When cancellation lands between calls, idle SQLCancel closes the cursor rather than being a harmless no-op. A subsequent metadata request can then raise ProgrammingError, which the test does not accept.
This should be investigated as a reader-close sequencing/test synchronization issue, not addressed by broadly swallowing ProgrammingError.
Observed CI failure
- Pipeline: mssql-rs build 191239.
- Job:
mssql-python suite on mssql-odbc driver (cross-repo). - Pinned mssql-python revision:
bd0aaf89c724a92c934b68bca41469655bfc733c(already includes the previous deflake fix). - Test:
tests/test_004_cursor_arrow.py::test_arrow_reader_cancel_from_other_thread. - Exception:
mssql_python.exceptions.ProgrammingError, SQLSTATEHY010,Function sequence error. - Stack:
_ArrowReader.__next__→ PyArrowread_next_batch→batch_generator→arrow_batch→DDBCSQLFetchArrowBatch→helpers.check_error. - Arrow test file: 1 failed, 49 passed, 1 skipped. Overall harness: 70/71 files passed, no crashes or timeouts.
- The job explicitly runs with
-m "not slow", so thisstress-marked test is included.
Source evidence
At the pinned revision:
_ArrowReader.__next__checks_closedbefore entering the inner reader; that check does not synchronize the full batch operation with concurrent close._ArrowReader.closemarks the reader closed and calls_cancel(). Its comment describes idle SQLCancel as a safe no-op, which does not match the measured drivers. Closing an already-executing generator may leave it alive for later cleanup.batch_generatorhas no close-request check before fetching another batch.FetchArrowBatch_wraprequests result metadata before fetching.SQLNumResultCols_wrappropagates errors throughCheckFetchError.
These paths expose a close-versus-batch window; the failing CI log does not contain a per-ODBC-call trace proving the exact scheduling of that particular run.
Controlled native reproduction
Using live local SQL Server on Windows, direct driver entry points, valid retained handles/buffers, and the same million-row query with a 64-row bound rowset:
- Execute
select top 1000000 1 a from sys.objects o1, sys.objects o2, sys.objects o3. - Fetch one 64-row batch.
- Call
SQLCancelfrom a second thread while no ODBC operation is active; wait for it to return. - Call
SQLNumResultColson the same statement. - Close/reset and execute
select 5to verify cursor reuse.
| Observation | Rust mssql-odbc | Microsoft ODBC Driver 18 |
|---|---|---|
| Idle SQLCancel | SQL_SUCCESS_WITH_INFO, 01S05 (Cancel treated as FreeStmt/Close) |
Same |
| Following SQLNumResultCols | SQL_ERROR, HY010, 20/20 runs |
SQL_ERROR, HY007, 20/20 runs |
| Cursor reusable afterward | Yes | Yes |
The Microsoft reference was retail 18.7.1.1, reporting SQL_DRIVER_VER=18.07.0001. The Rust driver was built from source matching the failing build's cancellation, close, fetch, operation-completion, statement-state, and result-column-count implementations.
Both HY007 and HY010 map to ProgrammingError in mssql-python. Consequently, merely changing the Rust diagnostic to HY007 would not make the test accept this ordering. The different SQLSTATE is a separate observed driver-parity discrepancy.
Separate active-execution and active-fetch cancellation probes produced HY008 diagnostics on both drivers and left the cursor reusable. This does not claim that every cancellation return code is identical between drivers.
Validation limit: this is a controlled native reproduction on Windows, not a replay of the full Linux/PyArrow stress test. The complete CI thread interleaving remains unproven.
Expected follow-up
Coordinate reader close with the complete batch operation so a close request cannot lead to another metadata/fetch sequence on an already-closed cursor. Make the test establish the intended in-flight cancellation point rather than relying on a 50 ms sleep, and cover the between-batch close case separately.
Retain the existing postconditions: the closer finishes, no unexpected close exception occurs, the reader is closed, and the parent cursor remains reusable.
Activity
github-actions commentedon Nov 12, 2026
Hi ttk (@Theekshna), thank you for opening this issue!
Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.
If you have additional information to share, please feel free to update the issue.
Thank you for your patience!
Theekshna commentedon Nov 12, 2026
Confirmed with the actual, unchanged Python test (not just a forced native call sequence).
Environment: Windows x64, local SQL Server 2022 over TCP, Python 3.13.17, PyArrow 26.0.1, pytest 8.3.4. Python source and native bindings were built from bd0aaf8. The test file, 50 ms sleep, assertions, and Python reader implementation were not modified. The same native bindings and package-loading path were used for both drivers, swapping only the packaged driver binary as the mssql-rs job does.
Command, once per fresh process:
Results without tracing or native instrumentation
SQL_DRIVER_VER=18.06.0002)--releasebuild from the failing pipeline's sourceThese are observed counts from one local workload, not stable failure-rate estimates.
Actual test failures captured with native ODBC call tracing
For a separate run, a session-local build of the bindings recorded native entry/exit timestamps, thread IDs, return codes, and SQLSTATEs returned by the existing diagnostic calls. Records were buffered in memory and written only at process exit; the observer added no sleeps or ODBC calls. No test or Python reader changes were made. Tracing can still perturb scheduling, so its counts are separate from the uninstrumented baseline.
Classic HY007 trace: the preceding SQLFetch has completed. Thread B enters SQLCancel between batches; thread A enters SQLNumResultCols 0.0204 ms later. SQLCancel returns SQL_SUCCESS_WITH_INFO after 5.2309 ms. SQLNumResultCols then returns SQL_ERROR, and the normal diagnostic read reports HY007. This is the idle-cancel-as-close ordering discussed in this issue.
Rust HY010 trace: the preceding SQLFetch has completed. Thread B enters SQLCancel; thread A enters SQLNumResultCols 0.0298 ms later and receives SQL_ERROR, then HY010 from its diagnostic read. SQLCancel later returns SQL_SUCCESS_WITH_INFO. This reproduces the CI exception with the actual reader/test, not an artificial post-cancel metadata call.
Second classic failure path: cancellation lands during SQLFetch. That fetch returns SQL_SUCCESS_WITH_INFO and its diagnostic contains HY008. The native Arrow wrapper continues with another SQLFetch, which returns SQL_ERROR / 24000. The test's HY008 OperationalError handler does not cover this subsequent ProgrammingError.
Conclusion and limits
The test/reader close race demonstrably occurs with both drivers. It is not evidence that idle SQLCancel must be changed to a no-op, or that all ProgrammingError exceptions should be ignored. Reader shutdown must account for the complete batch operation and partial-success cancellation, and tests should cover both in-flight and between-batch close.
The Rust HY010 versus classic HY007 diagnostic difference remains a separate parity issue tracked by https://sqlclientdrivers.visualstudio.com/mssql-rs/_workitems/edit/50265. Fixing that SQLSTATE alone cannot resolve the test because both map to ProgrammingError.
This reproduction is Windows, not the original Linux/Python 3.12 CI environment; the original failed CI run itself still has no per-call trace. However, the exact unchanged test now fails on both drivers locally, and a matching Rust HY010 failure has been traced.
FIX: Coordinate Arrow reader cross-thread shutdown (#892)