Skip to content

FIX: enforce POSIX native binary hardening - #810

Open
Sumit Sarabhai (sumitmsft) wants to merge 7 commits into
mainfrom
sumitsar/fix-cf041-native-hardening
Open

Sumit Sarabhai (sumitmsft) wants to merge 7 commits into
mainfrom
sumitsar/fix-cf041-native-hardening

Conversation

@sumitmsft

@sumitmsft Sumit Sarabhai (sumitmsft) commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • default single-config POSIX builds to Release when no build type is supplied
  • enable stack protection for the native extension on POSIX
  • select _FORTIFY_SOURCE=3 when supported, otherwise level 2, excluding Debug builds
  • enable full RELRO, immediate binding, and non-executable stack declarations on Linux
  • add source-contract, ELF parser, and loaded-extension regression tests

The hardening options are private to ddbc_bindings; simdutf compilation is unchanged. macOS retains its existing dynamic-link behavior, and Windows behavior is unchanged.

Validation

  • Windows x64 native build
  • 37 passed, 4 expected platform skips
  • import, formatting, and diff checks
  • hosted CI will validate manylinux, musllinux, ARM64, macOS, and Clang coverage

AB#48377

AB#48377

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 23, 2026 18:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

ELF linker flags must be restricted to Linux targets to avoid unsupported options on other UNIX platforms.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

This PR strengthens POSIX native-extension binary hardening and adds regression coverage.

Changes:

  • Defaults single-config builds to Release.
  • Adds compiler and linker hardening flags.
  • Adds source-contract and ELF validation tests.
File Summary
tests/​test_040_native_binary_hardening.py Verifies hardening configuration and Linux ELF properties.
mssql_python/​pybind/​CMakeLists.txt Configures build defaults and native hardening flags.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mssql_python/pybind/CMakeLists.txt Resolved
@github-actions github-actions Bot added the pr-size: medium Moderate update size label Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

86%


📈 Total Lines Covered: 11690 out of 13576
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 64.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python._native_fetch.py: 77.7%
mssql_python.__init__.py: 80.8%
mssql_python.pybind.ddbc_bindings.cpp: 81.4%
mssql_python.pybind.row_mapping_cache.hpp: 81.5%
mssql_python.pybind.connection.connection_pool.cpp: 82.5%
mssql_python.pybind.connection.connection.cpp: 83.1%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

Give each concurrent connect call its own configured native connection mock so MagicMock child creation cannot race during close.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 24, 2026 06:18
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task Before After Paired change Result
Connection opening 10.483 ms 10.471 ms +2.2% no signal
SELECT queries 0.989 ms 0.995 ms +0.1% no signal
Row insertion 30.570 ms 30.412 ms -1.5% no signal
Executemany inserts 164.251 ms 174.032 ms +2.0% no signal
Fetch-all queries 118.884 ms 120.390 ms +2.1% no signal
Row-by-row fetching 4.872 ms 4.847 ms +0.3% no signal
Batched row fetching 115.210 ms 117.092 ms +1.6% no signal
Transaction commit and rollback 105.208 ms 110.975 ms +4.7% no signal
Arrow row fetching 90.278 ms 88.631 ms -1.9% no signal
100,000-row insertion 357.812 ms 342.801 ms -2.3% no signal
Row fetching in batches of 100 122.856 ms 119.969 ms -2.9% no signal
Row fetching in batches of 10,000 132.649 ms 128.559 ms -0.1% no signal
Repeated positional queries 31.759 ms 31.777 ms +1.0% no signal
Repeated named-parameter queries 32.339 ms 32.159 ms +2.2% no signal
Legacy 100,000-row insertion 258.074 ms 258.201 ms +0.8% no signal
Insertion with explicit input sizes 396.146 ms 384.350 ms -3.0% no signal
Joined aggregation queries 194.543 ms 179.939 ms -4.5% no signal
Large joined-result fetching 201.358 ms 203.107 ms +4.1% no signal
1.2-million-row fetching 3488.803 ms 3472.881 ms -1.1% no signal
Common table expression queries 5.310 ms 5.324 ms -0.9% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.490 ms 1.425 ms +3.5% no signal
10,000 scalar values / fetchval() (debug disabled) 16.609 ms 16.658 ms +0.9% no signal
10,000 output-converted rows / fetchall() 19.214 ms 19.249 ms +0.3% no signal
10,000 output-converted rows / fetchmany(1000) 24.486 ms 22.950 ms -6.8% no signal
10,000 INT/NVARCHAR rows / fetchmany(1) 24.308 ms 24.277 ms -0.4% no signal
10,000 rows x 32 columns / dict(row._mapping) (warm) 31.446 ms 30.867 ms -1.8% no signal

Unix / SQL Server 2025

Database task Before After Paired change Result
Connection opening 81.414 ms 81.416 ms +0.5% no signal
SELECT queries 1.091 ms 1.042 ms -5.2% no signal
Row insertion 29.480 ms 29.289 ms -0.9% no signal
Executemany inserts 137.301 ms 136.979 ms -1.8% no signal
Fetch-all queries 119.752 ms 118.911 ms -1.1% no signal
Row-by-row fetching 4.762 ms 4.752 ms +0.1% no signal
Batched row fetching 111.402 ms 111.060 ms -0.4% no signal
Transaction commit and rollback 103.190 ms 101.443 ms -2.4% no signal
Arrow row fetching 85.978 ms 86.983 ms +2.5% no signal
100,000-row insertion 322.316 ms 341.089 ms +5.8% no signal
Row fetching in batches of 100 108.963 ms 109.352 ms +0.6% no signal
Row fetching in batches of 10,000 122.603 ms 124.289 ms -0.9% no signal
Repeated positional queries 30.107 ms 29.996 ms -0.4% no signal
Repeated named-parameter queries 30.527 ms 30.308 ms -0.6% no signal
Legacy 100,000-row insertion 230.751 ms 230.529 ms -0.3% no signal
Insertion with explicit input sizes 354.117 ms 371.278 ms +4.4% no signal
Joined aggregation queries 176.348 ms 191.684 ms +8.4% no signal
Large joined-result fetching 201.060 ms 191.754 ms -4.6% no signal
1.2-million-row fetching 3598.446 ms 3598.418 ms +0.3% no signal
Common table expression queries 5.304 ms 5.354 ms +0.2% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.439 ms 1.370 ms -4.0% no signal
10,000 scalar values / fetchval() (debug disabled) 16.401 ms 16.505 ms +0.7% no signal
10,000 output-converted rows / fetchall() 19.210 ms 19.319 ms +1.7% no signal
10,000 output-converted rows / fetchmany(1000) 24.083 ms 23.945 ms -0.6% no signal
10,000 INT/NVARCHAR rows / fetchmany(1) 24.683 ms 24.547 ms +0.8% no signal
10,000 rows x 32 columns / dict(row._mapping) (warm) 32.459 ms 32.390 ms -0.2% no signal
Build and measurement details

ADO build 193342

PR head: 47a96391ee405251a7b002f59e332e200b04dfdf
Base: d257fdd86dc0c8f09462981cde0cf76e38b7d0d7
Measured merge: 6785d2e276a72610b2d04e5a9b0ce095c4f2ca87

  • Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4325.4; 5 paired comparisons and 1 warmup.
  • Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5065.9; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Native build and ELF hardening changes span multiple platforms and warrant final human validation.

Review effort: Lite
Findings: 1 High severity

Open (1)

@sumitmsft
Sumit Sarabhai (sumitmsft) marked this pull request as ready for review September 24, 2026 07:54
Copilot AI review requested due to automatic review settings September 24, 2026 09:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were identified.

Review effort: Lite
Findings: None

Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The hardening test must skip when the native extension is unavailable in source-only checkouts.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 25, 2026 06:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved issues were identified.

Review effort: Lite
Findings: None

Gaurav Sharma (bewithgaurav) added a commit that referenced this pull request Oct 12, 2026
Remove the native-hardening and auth-test changes owned by PR #810, retaining only the Python 3.15 header-order compatibility flag and its focused contract.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Gaurav Sharma (bewithgaurav) added a commit that referenced this pull request Oct 12, 2026
### Summary

Adds Python 3.15 validation while keeping every existing PR-validation
leg enabled.

- Validates Python 3.15 GA on Windows x64 and macOS x86_64 through the
hosted Python toolcache.
- Keeps Linux x86_64 and Linux ARM64 on RC3 until the official
`python:3.15.0-bookworm` image is published.
- Uses PyArrow 25.0.1 below Python 3.15 and stable PyArrow 26.0.0 on
Python 3.15+.
- Handles Python 3.15 POSIX feature macros, Decimal conversion traceback
privacy, and cyclic-GC finalization ordering.
- Inherits the stable ABI `mssql-python-rs 0.3.0` installer contract
merged in #804.

### Validation

- Python 3.15.0 is marked stable in the hosted Python toolcache.
- Stable PyArrow 26.0.0 publishes `cp315` wheels for every target
platform.
- Focused pipeline and hardening contracts: 9 passed, 6 platform skips.
- Python 3.10 release locks continue to resolve PyArrow 25.0.1.

### Follow-up

Move both Linux legs from RC3 to GA once the official Bookworm image is
published. The native hardening changes overlap with #810; pull `main`
again after #810 merges.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Sumit Sarabhai <sumitsar@microsoft.com>
Resolve native hardening and Python 3.15 build configuration overlap; preserve isolated native auth mocks and thread-safe credential assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 18, 2026 14:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform compiler and linker hardening changes require hosted Linux, musl, ARM64, and macOS builds to verify configure compatibility and shipped binary properties beyond the available source-level evidence.

No open issues 🎉

Review effort: Lite

Keep POSIX compile hardening unchanged and assert the Linux-only linker scope in the existing source-contract regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 18, 2026 19:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Approval should wait for hosted Linux, ARM64, musl, and macOS builds plus ELF/Mach-O and extension-import checks, since the supplied validation covers only Windows.

No open issues 🎉

Review effort: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: medium Moderate update size

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants