Skip to content

FEAT: Add table-valued parameter support - #789

Open
Gaurav Sharma (bewithgaurav) wants to merge 27 commits into
mainfrom
bewithgaurav/ab48177-tvp-support
Open

Gaurav Sharma (bewithgaurav) wants to merge 27 commits into
mainfrom
bewithgaurav/ab48177-tvp-support

Conversation

@bewithgaurav

@bewithgaurav Gaurav Sharma (bewithgaurav) commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Work Item / Issue Reference

ADO Work Item: Fixed AB#48177


Summary

Adds the shared TableValuedParameter(type_name, columns, rows, *, schema=None) definition and synchronous ODBC execution support.

  • Explicit table identity and ordered column definitions. Lists, tuples and synchronous iterators are accepted; ordinary nested sequences are not inferred as TVPs.
  • Streams one row at a time through ODBC data-at-execution within one SQL execution. Reuses scalar conversions, preserves generator exceptions and retains bound storage until native cleanup is safe.
  • Includes synchronous documentation and coverage for empty input, nullable and typed cells, multiple TVPs, named parameters, cursor reuse, cancellation and recovery. The large memory case is marked stress and excluded from normal PR CI.

Async TVP integration, TVP executemany(), TVP setinputsizes() overrides and the mssql-odbc provider are outside this PR.

Copilot AI added 2 commits September 15, 2026 16:37
Accept materialized nested row sequences for stored-procedure TVP parameters. Resolve the declared table type and child schema from prepared ODBC metadata so empty and typed-NULL rows bind correctly without a public wrapper.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the stale GitHub issue reference with the ADO task that tracks this feature.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 15, 2026 11:40
@github-actions github-actions Bot added the pr-size: large Substantial code update label Sep 15, 2026
@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

88%


🎯 Overall Coverage

86%


📈 Total Lines Covered: 11991 out of 13935
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql_python/init.py (100%)
  • mssql_python/cursor.py (91.7%): Missing lines 1959,1981
  • mssql_python/pybind/connection/connection.cpp (50.0%): Missing lines 178-179
  • mssql_python/pybind/ddbc_bindings.cpp (87.7%): Missing lines 550-551,968-969,1004-1005,1633-1636,1738-1739,1800,2330-2331,2334,2390,2455,2472
  • mssql_python/pybind/ddbc_bindings.h (80.0%): Missing lines 301
  • mssql_python/pybind/param_detect.hpp (100%)
  • mssql_python/pybind/tvp.hpp (89.4%): Missing lines 79,82,90-95,104,143-145,166-168
  • mssql_python/table_valued_parameter.py (89.5%): Missing lines 44,52,58,64,67,69,81,83,89,91,98

Summary

  • Total: 444 lines
  • Missing: 50 lines
  • Coverage: 88%

mssql_python/cursor.py

Lines 1955-1963

  1955                         isinstance(parameter, TableValuedParameter)
  1956                         for parameter in bound_parameters
  1957                     ):
  1958                         if ProviderManager.effective() == PROVIDER_MSSQL_ODBC:
! 1959                             raise NotSupportedError(
  1960                                 "Table-valued parameters are not supported by "
  1961                                 "the mssql-odbc provider",
  1962                                 "Use the default msodbcsql18 provider for TVP statements",
  1963                             )

Lines 1977-1985

  1977         except BaseException as e:  # Reset even when a TVP iterator raises KeyboardInterrupt.
  1978             logger.warning("Execute failed, resetting cursor: %s", e)
  1979             try:
  1980                 self._reset_cursor()
! 1981             except Exception as cleanup_error:
  1982                 # A callback may have closed the connection. Preserve its
  1983                 # original error instead of replacing it with an allocation error.
  1984                 logger.warning("Cursor recovery failed: %s", cleanup_error)
  1985             raise

mssql_python/pybind/connection/connection.cpp

Lines 174-183

  174                 for (const auto& handle : childHandles) {
  175                     handle->markImplicitlyFreed();
  176                 }
  177                 for (const auto& buffers : _cleanupState->failedParameterBuffers) {
! 178                     buffers->release();
! 179                 }
  180                 _cleanupState->failedParameterBuffers.clear();
  181                 _childStatementHandles.clear();
  182                 _allocationsSinceCompaction = 0;
  183             }

mssql_python/pybind/ddbc_bindings.cpp

Lines 546-555

  546 }
  547 
  548 static void PrepareParamBuffer(const py::handle& param, ParamInfo& paramInfo,
  549                                std::vector<std::shared_ptr<void>>& paramBuffers,
! 550                                const std::string& charEncoding, int paramIndex,
! 551                                void*& dataPtr, SQLLEN& bufferLength, SQLLEN*& strLenOrIndPtr,
  552                                bool tableCell = false) {
  553         // TODO: Add more data types like money, guid, interval, TVPs etc.
  554         switch (paramInfo.paramCType) {
  555             case SQL_C_CHAR: {

Lines 964-973

  964     describeRc = PreResolveUnknownNullTypes(handle, hStmt, paramInfos, &params);
  965     if (!SQL_SUCCEEDED(describeRc)) return describeRc;
  966 
  967     std::vector<ParamBindSignature> signature;
! 968     if (reuseMatchingBindings) {
! 969         signature.reserve(paramInfos.size());
  970         for (const ParamInfo& info : paramInfos) {
  971             signature.push_back({info.inputOutputType, info.paramCType, info.paramSQLType,
  972                                  info.columnSize, info.decimalDigits, info.isDAE});
  973         }

Lines 1000-1009

  1000             if (!SQL_SUCCEEDED(rc)) {
  1001                 return rc;
  1002             }
  1003             continue;
! 1004         }
! 1005 
  1006         PrepareParamBuffer(param, paramInfo, paramBuffers, charEncoding, paramIndex,
  1007                            dataPtr, bufferLength, strLenOrIndPtr);
  1008         assert(SQLBindParameter_ptr && SQLGetStmtAttr_ptr && SQLSetDescField_ptr);
  1009         RETCODE rc;

Lines 1629-1640

  1629 
  1630     SQLDescribeParam_ptr = GetFunctionPointer<SQLDescribeParamFunc>(handle, "SQLDescribeParam");
  1631 
  1632     bool success =
! 1633         SQLAllocHandle_ptr && SQLSetEnvAttr_ptr && SQLSetConnectAttr_ptr && SQLSetStmtAttr_ptr &&
! 1634         SQLGetConnectAttr_ptr && SQLDriverConnect_ptr && SQLExecDirect_ptr && SQLPrepare_ptr &&
! 1635         SQLBindParameter_ptr && SQLExecute_ptr && SQLRowCount_ptr && SQLGetStmtAttr_ptr &&
! 1636         SQLSetDescField_ptr && SQLFetch_ptr && SQLFetchScroll_ptr &&
  1637         SQLGetData_ptr && SQLNumResultCols_ptr && SQLBindCol_ptr && SQLDescribeCol_ptr &&
  1638         SQLMoreResults_ptr && SQLColAttribute_ptr && SQLEndTran_ptr && SQLDisconnect_ptr &&
  1639         SQLFreeHandle_ptr && SQLFreeStmt_ptr && SQLGetDiagRec_ptr && SQLGetInfo_ptr &&
  1640         SQLParamData_ptr && SQLPutData_ptr && SQLTables_ptr && SQLDescribeParam_ptr &&

Lines 1734-1743

  1734 
  1735 void SqlHandle::retainFailedParameters(std::vector<std::shared_ptr<void>> buffers) {
  1736     _failedParameterBuffers = std::make_shared<FailedParameterBuffers>(std::move(buffers));
  1737     py::gil_scoped_release release;
! 1738     auto cleanupLock = lockForCleanup();
! 1739     if (!_handle || _implicitly_freed || (_cleanupState && _cleanupState->disconnected)) {
  1740         _failedParameterBuffers->release();
  1741     } else if (_cleanupState) {
  1742         _cleanupState->failedParameterBuffers.push_back(_failedParameterBuffers);
  1743     }

Lines 1796-1804

  1796         auto cleanupLock = lockForCleanup();
  1797         if (!_handle) {
  1798             return SQL_INVALID_HANDLE;
  1799         }
! 1800         clearDescribeCache();
  1801         SQLRETURN ret = SQL_SUCCESS;
  1802         if (!_implicitly_freed && !(_cleanupState && _cleanupState->disconnected)) {
  1803             ret = SQLFreeHandle_ptr ? SQLFreeHandle_ptr(_type, _handle) : SQL_INVALID_HANDLE;
  1804         }

Lines 2326-2338

  2326             ThrowStdException("Cannot execute unprepared statement");
  2327         }
  2328     }
  2329 
! 2330     const bool hasTvp = std::any_of(paramInfos.begin(), paramInfos.end(),
! 2331                                    [](const ParamInfo& info) { return info.isTVP; });
  2332     std::vector<std::shared_ptr<void>> paramBuffers;
  2333     try {
! 2334     rc = BindParameters(statementHandle, hStmt, params, paramInfos, paramBuffers, charEncoding,
  2335                         /*reuseMatchingBindings=*/!hasTvp);
  2336     if (!SQL_SUCCEEDED(rc)) ThrowParameterError(statementHandle, rc, "Binding parameters");
  2337     // The APD now points only at paramBuffers. Keep them on the handle; the
  2338     // previous execution's buffers are no longer referenced and are freed on return.

Lines 2386-2394

  2386                 ThrowStdException("SQLExecute: unrecognized paramToken from SQLParamData");
  2387             }
  2388             PyObject* pyObj = matchedInfo->dataPtr.ptr();
  2389             if (!pyObj || pyObj == Py_None) {
! 2390                 putData(nullptr, 0);
  2391                 continue;
  2392             }
  2393 
  2394             if (PyUnicode_Check(pyObj)) {

Lines 2451-2459

  2451         rc = ResetParamBindings(*statementHandle, hStmt);
  2452         if (!SQL_SUCCEEDED(rc)) ThrowParameterError(statementHandle, rc, "Resetting parameters");
  2453     }
  2454     return exec_rc;
! 2455     } catch (...) {
  2456         // Convert diagnostics to the original Python exception before cleanup
  2457         // can overwrite them. Keep every buffer alive until NEED_DATA is left.
  2458         if (statementHandle->get() == hStmt && !statementHandle->isImplicitlyFreed()) {
  2459             SQLRETURN resetResult;

Lines 2468-2476

  2468                 auto& bound = statementHandle->boundParamBuffers;
  2469                 paramBuffers.insert(paramBuffers.end(), std::make_move_iterator(bound.begin()),
  2470                                     std::make_move_iterator(bound.end()));
  2471                 bound.clear();
! 2472                 paramBuffers.push_back(retainedInfos);
  2473                 statementHandle->retainFailedParameters(std::move(paramBuffers));
  2474                 // Bound tokens/indicators are native storage. Drop Python owners
  2475                 // with the GIL held, before retained storage can outlive Python.
  2476                 for (auto& info : *retainedInfos) {

mssql_python/pybind/ddbc_bindings.h

  297 class FailedParameterBuffers {
  298   public:
  299     explicit FailedParameterBuffers(std::vector<std::shared_ptr<void>> buffers);
  300     ~FailedParameterBuffers();
! 301     void release() { _buffers.reset(); }
  302 
  303   private:
  304     std::unique_ptr<std::vector<std::shared_ptr<void>>> _buffers;
  305 };

mssql_python/pybind/tvp.hpp

Lines 75-86

  75                     lengths[i] = sizeof(SQLCHAR); break;
  76                 case SQL_C_SHORT: lengths[i] = sizeof(SQLSMALLINT); break;
  77                 case SQL_C_LONG: lengths[i] = sizeof(SQLINTEGER); break;
  78                 case SQL_C_SBIGINT: lengths[i] = sizeof(int64_t); break;
! 79                 case SQL_C_FLOAT: lengths[i] = sizeof(float); break;
  80                 case SQL_C_DOUBLE: lengths[i] = sizeof(double); break;
  81                 case SQL_C_TYPE_DATE: lengths[i] = sizeof(SQL_DATE_STRUCT); break;
! 82                 case SQL_C_TYPE_TIME: lengths[i] = sizeof(SQL_TIME_STRUCT); break;
  83                 case SQL_C_TYPE_TIMESTAMP: lengths[i] = sizeof(SQL_TIMESTAMP_STRUCT); break;
  84             }
  85             indicators[i] = SQL_DATA_AT_EXEC;
  86         }

Lines 86-99

  86         }
  87         return true;
  88     }
  89 
! 90     void releasePythonState() {
! 91         value = py::object();
! 92         iterator = py::object();
! 93         sizes.release().dec_ref();
! 94         row.release().dec_ref();
! 95     }
  96 };
  97 
  98 static SQLRETURN BindTableValuedParameter(const SqlHandlePtr& handle, int paramIndex,
  99                                          const py::handle& value, ParamInfo& info) {

Lines 100-108

  100     info.table = std::make_shared<TableParamState>(value);
  101     TableParamState& table = *info.table;
  102     SQLHANDLE hStmt = handle->get();
  103     if (!hStmt || handle->isImplicitlyFreed())
! 104         throw std::runtime_error("Statement closed while initializing TVP rows");
  105     SQLSMALLINT record = static_cast<SQLSMALLINT>(paramIndex + 1);
  106     // msodbcsql18 enforces ColumnSize as a cumulative row ceiling even for
  107     // variable-row DAE (1 rejects the second row). This is a descriptor limit,
  108     // not an allocation: all columns are DAE tokens with one row of storage.

Lines 139-149

  139                 &table.columns[i], 0, &table.indicators[i]);
  140             if (!SQL_SUCCEEDED(rc)) ThrowParameterError(handle, rc, "Failed to bind TVP column");
  141         }
  142     } catch (...) {
! 143         SQLSetStmtAttr_ptr(hStmt, SQL_SOPT_SS_PARAM_FOCUS, nullptr, SQL_IS_INTEGER);
! 144         throw;
! 145     }
  146     return SQLSetStmtAttr_ptr(hStmt, SQL_SOPT_SS_PARAM_FOCUS, nullptr, SQL_IS_INTEGER);
  147 }
  148 
  149 template <typename PutData>

Lines 162-172

  162         }
  163         for (size_t i = 0; i < table.columns.size(); ++i) {
  164             if (token != &table.columns[i]) continue;
  165             if (table.indicators[i] == SQL_NULL_DATA) {
! 166                 rc = putData(nullptr, SQL_NULL_DATA);
! 167                 return true;
! 168             }
  169             const auto cType = table.columns[i].paramCType;
  170             if (cType == SQL_C_WCHAR || cType == SQL_C_CHAR || cType == SQL_C_BINARY) {
  171                 rc = stream_dae_chunks(table.data[i], static_cast<size_t>(table.lengths[i]), putData);
  172             } else {

mssql_python/table_valued_parameter.py

Lines 40-48

  40         *,
  41         schema: str | None = None,
  42     ) -> None:
  43         if not isinstance(type_name, str):
! 44             raise TypeError("TVP type_name must be a string")
  45         if "." in type_name:
  46             if schema is not None or type_name.count(".") != 1:
  47                 raise ValueError("Specify TVP schema once: schema.Type or schema=...")
  48             schema, type_name = type_name.split(".")

Lines 48-56

  48             schema, type_name = type_name.split(".")
  49         schema = "dbo" if schema is None else schema
  50         for name in (type_name, schema):
  51             if not isinstance(name, str):
! 52                 raise TypeError("TVP schema and type name must be strings")
  53             if (
  54                 not name.strip()
  55                 or any(char in name for char in '\x00.[]"')
  56                 or len(name.encode("utf-16le")) > 256

Lines 54-62

  54                 not name.strip()
  55                 or any(char in name for char in '\x00.[]"')
  56                 or len(name.encode("utf-16le")) > 256
  57             ):
! 58                 raise ValueError("TVP names must be unquoted identifiers of 1–128 UTF-16 units")
  59         if not isinstance(columns, Sequence) or not columns:
  60             raise TypeError("TVP columns must be a non-empty sequence")
  61         normalized: list[_NormalizedColumn] = []
  62         for column in columns:

Lines 60-73

  60             raise TypeError("TVP columns must be a non-empty sequence")
  61         normalized: list[_NormalizedColumn] = []
  62         for column in columns:
  63             if not isinstance(column, tuple) or not 2 <= len(column) <= 4:
! 64                 raise TypeError("TVP columns use (name, sql_type[, size_or_precision[, scale]])")
  65             name, sql_type = column[:2]
  66             if not isinstance(name, str) or not name:
! 67                 raise TypeError("TVP column names must be non-empty strings")
  68             if type(sql_type) is not int:
! 69                 raise TypeError(f"TVP column {name}: sql_type must be an SQL_* integer constant")
  70             sql_type = {9: 91, 10: 92, 11: 93}.get(sql_type, sql_type)
  71             if sql_type not in self._types:
  72                 raise ValueError(f"TVP column {name}: unsupported SQL type {sql_type}")
  73             variable = sql_type in self._variable

Lines 77-87

  77                 raise ValueError(f"TVP column {name}: explicit size or precision/scale is required")
  78             size = column[2] if len(column) >= 3 else self._sizes.get(sql_type, 0)
  79             scale = column[3] if len(column) == 4 else (7 if temporal else 0)
  80             if type(size) is not int or type(scale) is not int:
! 81                 raise TypeError(f"TVP column {name}: size and scale must be integers")
  82             if not 0 <= size <= 2147483647 or scale < 0:
! 83                 raise ValueError(f"TVP column {name}: invalid size or scale")
  84             if numeric and not (1 <= size <= 38 and 0 <= scale <= size):
  85                 raise ValueError(
  86                     f"TVP column {name}: require 1 <= precision <= 38 and scale <= precision"
  87                 )

Lines 85-95

  85                 raise ValueError(
  86                     f"TVP column {name}: require 1 <= precision <= 38 and scale <= precision"
  87                 )
  88             if temporal and scale > 7:
! 89                 raise ValueError(f"TVP column {name}: temporal scale must be between 0 and 7")
  90             if not numeric and not temporal and scale:
! 91                 raise ValueError(
  92                     f"TVP column {name}: scale is only valid for numeric/temporal types"
  93                 )
  94             if (
  95                 sql_type in (SQL.SQL_CHAR.value, SQL.SQL_WCHAR.value, SQL.SQL_BINARY.value)

Lines 94-102

   94             if (
   95                 sql_type in (SQL.SQL_CHAR.value, SQL.SQL_WCHAR.value, SQL.SQL_BINARY.value)
   96                 and not size
   97             ):
!  98                 raise ValueError(f"TVP column {name}: fixed-length columns require a positive size")
   99             normalized.append((name, sql_type, size, scale))
  100         if not isinstance(rows, (Sequence, Iterator)) or isinstance(rows, (str, bytes, bytearray)):
  101             raise TypeError("TVP rows must be a sequence or iterator; use [] for an empty TVP")
  102         self._type_name, self._schema = type_name, schema


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 64.8%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python._native_fetch.py: 77.7%
mssql_python.pybind.ddbc_bindings.cpp: 81.2%
mssql_python.pybind.row_mapping_cache.hpp: 81.5%
mssql_python.pybind.connection.connection_pool.cpp: 82.5%
mssql_python.pybind.connection.connection.cpp: 82.9%
mssql_python.logging.py: 86.1%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The TVP execution path has critical single-TVP argument handling and moderate metadata-timeout diagnostic defects.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds ODBC-backed table-valued parameter support for stored procedure execution using materialized rows.

Changes:

  • Adds TVP detection, metadata resolution, validation, and row binding.
  • Supports empty, NULL, typed, multiple, and reusable TVPs.
  • Adds integration/regression tests and changelog documentation.
File summaries
File Summary
tests/test_028_table_valued_parameters.py Adds comprehensive TVP integration coverage.
tests/test_004_cursor.py Adds embedded-NUL regression coverage.
mssql_python/pybind/param_detect.hpp Adds TVP parameter detection and binding metadata.
mssql_python/pybind/ddbc_bindings.h Defines TVP metadata structures and ODBC declarations.
mssql_python/pybind/ddbc_bindings.cpp Resolves TVP schemas and binds rows through ODBC.
mssql_python/cursor.py Manages TVP execution and provider handling; contains critical single-TVP normalization and moderate timeout-diagnostic issues.
CHANGELOG.md Documents TVP support and provider limitations.
Review details

Suppressed comments (1)

mssql_python/cursor.py:1014

  • When statement_handle is supplied for the TVP metadata HSTMT, the timeout failure check still passes self.hstmt, so diagnostics are read from the wrong statement and a failed metadata timeout can be silently ignored. Pass the selected handle to check_error as well.
                    statement_handle or self.hstmt,
                    ddbc_sql_const.SQL_ATTR_QUERY_TIMEOUT.value,
                    timeout_value,
  • Files reviewed: 7/7 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mssql_python/cursor.py Resolved
Reuse scalar type detection and parameter metadata for TVP columns, and remove redundant sizing and metadata ordering work.

AB#48177

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 12:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate issues remain in TVP binding and cursor handling.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/cursor.py:1012

  • When an alternate handle is supplied for the TVP metadata statement, the timeout is set on that handle but check_error still reads diagnostics from self.hstmt. If setting the metadata handle attribute fails, the wrong statement's diagnostics are inspected and the exception is swallowed by this try/except, so metadata discovery can run without the requested query timeout. Check the same target handle that was passed to DDBCSQLSetStmtAttr.

mssql_python/cursor.py:1740

  • The documented single-TVP call does not work for an empty row sequence. With execute("EXEC proc ?", ([],)) (or ((),)), the existing length-one unwrapping sets actual_params to the empty sequence and then converts it to parameters=[]; the new TVP path is never reached and SQL receives zero bound parameters. Preserve the outer parameter tuple for empty list/tuple TVPs, or add equivalent disambiguation, so an empty TVP is actually bound.
        # pass a tuple as a single scalar parameter value. A TVP remains one
        # parameter by wrapping its rows in the ordinary outer parameter tuple:
        # execute("EXEC dbo.proc ?", (rows,))

mssql_python/pybind/ddbc_bindings.cpp:914

  • The prepared IPD's catalog is read and cached above, but binding only writes SQL_CA_SS_SCHEMA_NAME; SQL_CA_SS_CATALOG_NAME is never propagated to the implementation descriptor. A call to a procedure in a non-current database can therefore resolve the TVP type against the wrong catalog even though metadata discovery succeeded. Set the catalog descriptor field when it is present (or pass a fully qualified type name).
        rc = SQLSetDescField_ptr(implementationDescriptor, recordNumber, SQL_CA_SS_SCHEMA_NAME,
                                 reinterpretU16stringAsSqlWChar(*schema),
                                 static_cast<SQLINTEGER>(schema->size() * sizeof(SQLWCHAR)));
  • Files reviewed: 7/7 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Preserve source metadata and exact Decimal values during TVP binding so SQL Server can apply destination conversions. Retain binding diagnostics during parameter-focus cleanup and route timeout diagnostics to the selected statement handle. Add regression coverage for conversions, buffer sizing, validation, and recovery.

AB#48177

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 14:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical TVP binding issues must be resolved before approval.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

mssql_python/pybind/ddbc_bindings.cpp:569

  • This condition rejects a precision above 38, but the message says the TVP column requires precision greater than 38, which tells the caller the opposite of the actual constraint. Please use wording consistent with the scalar validation (for example, that the maximum supported precision is 38) and update the new assertion that matches this text.
                throw py::value_error("TVP Decimal column requires precision greater than 38");
  • Files reviewed: 8/8 changed files
  • Comments generated: 3
  • Review effort level: Lite

Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Release the GIL during TVP metadata reads and cleanup so Python forwarding threads can make progress. Preserve shutdown-safe destructor behavior and add bounded regressions for normal execution and metadata-error recovery.

AB#48177

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 18:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native ODBC TVP and cursor-lifecycle changes warrant final human review, and two assessments require changes.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/pybind/ddbc_bindings.cpp:570

  • This branch rejects precision above 38, but the message says the TVP column “requires precision greater than 38,” which communicates the opposite condition and can mislead callers diagnosing valid Decimal inputs. Please state that precision cannot exceed 38.
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Choose long TVP SQL types from source length regardless of destination size. Preserve valid Unicode values in bounded varchar columns and let SQL Server enforce destination limits. Add boundary and cursor-recovery regressions.

AB#48177

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 19:24
@bewithgaurav
Gaurav Sharma (bewithgaurav) marked this pull request as ready for review September 15, 2026 19:24
@bewithgaurav
Gaurav Sharma (bewithgaurav) marked this pull request as draft September 15, 2026 19:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The changes span Python, native ODBC bindings, metadata handling, and multiple execution paths requiring final human review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/pybind/ddbc_bindings.cpp:570

  • This branch rejects a common source precision greater than 38, but the message says the column requires precision greater than 38, which reads as the opposite condition. Report that the TVP decimal column exceeds SQL Server's 38-digit precision limit so callers know what to change.
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 17, 2026 04:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The native ODBC TVP binding and metadata lifecycle require final human validation across supported platforms and drivers.

Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Integrate current main while retaining TVP metadata cache cleanup in the synchronized handle-release path.

AB#48177

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Verify SQL_CHAR and SQL_VARCHAR TVP declarations use the existing wide C binding, so requested narrow encodings do not alter non-ASCII cell conversion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 13, 2026 14:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the change contains a critical VARCHAR TVP encoding regression and an additional per-parameter performance concern.

No open issues 🎉

1 resolved since last review

Review effort: Lite

Copilot AI lite review requested due to automatic review settings November 14, 2026 06:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Approval should be deferred because the native ODBC TVP streaming, cancellation, and cleanup paths require supported builds plus live, fault-injected SQL Server/provider testing across platforms that is unavailable in this review environment.

No open issues 🎉

Review effort: Lite

Copilot AI lite review requested due to automatic review settings November 16, 2026 07:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two critical active issues remain: the reset path preserves stale prepared state, and narrow TVP SQL types use an incorrect character binding/encoding that contradicts the regression test.

2 open issues

Review effort: Lite

Comment thread mssql_python/pybind/ddbc_bindings.cpp Resolved
Comment thread mssql_python/table_valued_parameter.py Resolved
Document that SQL_RESET_PARAMS keeps the ODBC statement prepared and add live coverage alternating table types on the same SQL text.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Approval should wait for live SQL Server/provider/platform and fault-injected lifecycle validation of the new ODBC TVP descriptor, streaming, cancellation, and teardown paths.

No open issues 🎉

2 resolved since last review

Review effort: Lite

Copilot AI and others added 2 commits November 18, 2026 16:56
Integrate strict package typing while preserving the explicit TVP interface and runtime behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bring in release and workflow dependency updates that landed during conflict validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bring in the restored release package pipeline before publishing the conflict resolution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 18, 2026 12:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform native ODBC TVP streaming path combines driver-specific descriptor handling, callbacks, cancellation, and failure cleanup that require CI or human validation with supported providers and live SQL Server scenarios unavailable here.

No open issues 🎉

Review effort: Lite

gargsaumya added a commit that referenced this pull request Nov 18, 2026
Drop the PR-local TableValuedParameter and TVP binder so the shared TVP definition (#789) is the single public API; vector TVP columns will build on it. Restore main's array-binding buffer ownership.

OutputParameter now takes the setinputsizes (sql_type, size, scale) triple; only SQL_SS_VECTOR is supported and other types raise NotSupportedError. InputOutputParameter infers its type from the initial Vector.

Avoid per-cell ABC isinstance scans in executemany() and bulkcopy() by gating on exact Vector type. Build fetched and output vectors through a cached trusted constructor instead of revalidating driver-produced values. Add a bulk fast path for list/tuple float inputs with identical values and errors. Accept bracket-quoted callproc name parts and reject malformed brackets.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
gargsaumya added a commit that referenced this pull request Nov 18, 2026
Complete dropping the PR-local TVP API in favor of the shared definition (#789); nothing imports this module any more.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: large Substantial code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants