fix: allow thrift 0.25.x and clear additional Apache Thrift CVEs - #970
Open
hannonpi1228 wants to merge 1 commit into
Open
hannonpi1228 wants to merge 1 commit into
hannonpi1228 wants to merge 1 commit into
Conversation
Widen the thrift constraint from ~=0.24.0 to >=0.24.0,<0.26.0 so 0.25.0 can be resolved. thrift 0.25.0 fixes 61 CVEs across language bindings, several affecting the Python binding this connector uses, including CVE-2026-66858 (skip() recursion-limit bypass in the Python accelerator) and CVE-2026-85494 (framed transport / binary protocol size a read buffer from a peer-declared length with no effective maximum). thrift 0.25.0 also changes TBinaryProtocol's default string_length_limit from unbounded (None) to ~15.6 MiB (DEFAULT_MAX_FRAME_SIZE) as part of that same CVE-2026-85494 fix. This connector already bounds its own result stream via the TLS-verified, server-negotiated buffer_size_bytes (default 100 MiB), and inline Arrow result batches in TFetchResultsResp routinely exceed thrift's new ~15.6 MiB cap, so a bare version bump would regress large-result-set reads. Construct TBinaryProtocol with explicit string_length_limit=None and container_length_limit=None to preserve pre-0.25 unbounded behavior for the connector's own already-bounded, trusted-server result stream. 0.25.0 ships the same prebuilt wheel matrix as 0.24.0 (manylinux2014/macOS/musl/Windows, cp310-cp314), so the DBR LTS build-time packaging risk that originally capped this dependency (databricks#798, databricks#840) does not apply; the DBR LTS Install CI gate remains the authoritative check. Closes databricks#969 Signed-off-by: Paddy Hannon <pih@ehukai.com> AOS-Session: 01a10c87-3242-7517-a60e-279091b796c3 AOS-Session: pi-1791211537-21291-e08bda26 AOS-Commit-Time: 2026-10-05T15:03:26Z
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #969
Summary
thriftconstraint from~=0.24.0to>=0.24.0,<0.26.0so0.25.0can be resolved.thriftto0.25.0.string_length_limit=None, container_length_limit=NonetoTBinaryProtocolto preserve pre-0.25 unbounded read behavior for the connector's own already-bounded, trusted-server result stream.Why this isn't a bare version bump
Apache Thrift 0.25.0 (released 2026-09-30) fixes 61 CVEs across language bindings (combined announcement); several affect the Python binding, including CVE-2026-66858 (
skip()recursion-limit bypass in the Python accelerator) and CVE-2026-85494 (framed transport / binary protocol size a read buffer from a peer-declared length with no effective maximum). Snyk currently reports both as firing against this connector via thethrift==0.24.0pin.Fixing CVE-2026-85494 also changed
TBinaryProtocol's defaultstring_length_limitfrom unbounded (None) toDEFAULT_STRING_LENGTH_LIMIT(DEFAULT_MAX_FRAME_SIZE= 16,384,000 bytes ≈ 15.6 MiB).thrift_backend.pypreviously constructed the protocol with no kwargs:so a bare version bump would make every
readBinary/readStringcall -- including the inline Arrow result batches inTFetchResultsResp-- inherit the new ~15.6 MiB cap. This connector's own result-size knob,buffer_size_bytes(DEFAULT_RESULT_BUFFER_SIZE_BYTES= 100 MiB inclient.py), is well above that, so this would be a real regression on any result batch between ~15.6 MiB and the connector's own 100 MiB default -- not a theoretical one. Confirmed empirically: a 20 MiB binary field reads fine under thrift 0.24.0, and under thrift 0.25.0's default construction raisesTTransportException: Length exceeded max allowed: 16384000; passingstring_length_limit=None(this PR) restores the read.This connector already bounds and governs its own result stream (TLS-verified connection to a single trusted Databricks backend, server-negotiated
maxBytes/buffer_size_bytes), so re-enabling the unbounded local read limit does not reintroduce the untrusted-peer DoS/amplification scenario the thrift default is meant to guard against for general-purpose Thrift servers.DBR LTS packaging risk (the reason this pin existed at all, see #798/#840)
0.25.0 ships the same wheel matrix as 0.24.0 --
manylinux2014/macOS/musl/Windows forcp310-cp314, confirmed on PyPI -- so pip resolves a wheel and never runssetup.pyon DBR LTS; the SEV0 build-time failure class that originally forced this pin does not apply here. TheDBR LTS InstallCI check remains the authoritative gate for this.Other CVEs in the 0.25.0 batch with Python impact
Per #969, not all are mapped in GitHub Advisory Database / OSV yet for the PyPI
thriftpackage, but Apache's announcement lists several more affecting Python bindings (e.g.TJSONProtocolsize-limit issues,TZlibTransportdecompressed-size enforcement). This connector doesn't useTJSONProtocolorTZlibTransport(thrift_backend.pyonly importsTHttpClient,TBinaryProtocol,TSocket,TTransport), so those are moot for this codebase, but the version bump clears them for any downstream code path.Validation
TMemoryBufferround-trip of a 20 MiB binary field: fails under thrift 0.25.0's defaultTBinaryProtocol()construction, passes with the explicitstring_length_limit=Nonethis PR adds (same behavior as thrift 0.24.0).test_binary_protocol_preserves_unbounded_string_and_container_limits(asserts the connector constructsTBinaryProtocolwithstring_length_limit=None, container_length_limit=None) andtest_binary_protocol_reads_result_larger_than_thrift_default_limit(round-trips a 20 MiB field through the realthriftpackage with the connector's exact kwargs) totests/unit/test_thrift_backend.py.poetry run python -m pytest tests/unitwiththrift 0.25.0actually installed: 1020 passed, 5 skipped.black --check/mypyclean on changed files.Signed-off-by: Paddy Hannon pih@ehukai.com