Skip to content

fix: allow thrift 0.25.x and clear additional Apache Thrift CVEs - #970

Open
hannonpi1228 wants to merge 1 commit into
databricks:mainfrom
hannonpi1228:fix/969-thrift-0.25-upgrade
Open

hannonpi1228 wants to merge 1 commit into
databricks:mainfrom
hannonpi1228:fix/969-thrift-0.25-upgrade

Conversation

@hannonpi1228

Copy link
Copy Markdown

Closes #969

Summary

  • Widen the thrift constraint from ~=0.24.0 to >=0.24.0,<0.26.0 so 0.25.0 can be resolved.
  • Bump the locked thrift to 0.25.0.
  • Pass explicit string_length_limit=None, container_length_limit=None to TBinaryProtocol to preserve pre-0.25 unbounded read behavior for the connector's own already-bounded, trusted-server result stream.

Why this isn't a bare version bump

Apache Thrift 0.25.0 (released 2026-09-30) fixes 61 CVEs across language bindings (combined announcement); several affect the Python binding, including CVE-2026-66858 (skip() recursion-limit bypass in the Python accelerator) and CVE-2026-85494 (framed transport / binary protocol size a read buffer from a peer-declared length with no effective maximum). Snyk currently reports both as firing against this connector via the thrift==0.24.0 pin.

Fixing CVE-2026-85494 also changed TBinaryProtocol's default string_length_limit from unbounded (None) to DEFAULT_STRING_LENGTH_LIMIT (DEFAULT_MAX_FRAME_SIZE = 16,384,000 bytes ≈ 15.6 MiB). thrift_backend.py previously constructed the protocol with no kwargs:

protocol = thrift.protocol.TBinaryProtocol.TBinaryProtocol(self._transport)

so a bare version bump would make every readBinary/readString call -- including the inline Arrow result batches in TFetchResultsResp -- inherit the new ~15.6 MiB cap. This connector's own result-size knob, buffer_size_bytes (DEFAULT_RESULT_BUFFER_SIZE_BYTES = 100 MiB in client.py), is well above that, so this would be a real regression on any result batch between ~15.6 MiB and the connector's own 100 MiB default -- not a theoretical one. Confirmed empirically: a 20 MiB binary field reads fine under thrift 0.24.0, and under thrift 0.25.0's default construction raises TTransportException: Length exceeded max allowed: 16384000; passing string_length_limit=None (this PR) restores the read.

This connector already bounds and governs its own result stream (TLS-verified connection to a single trusted Databricks backend, server-negotiated maxBytes/buffer_size_bytes), so re-enabling the unbounded local read limit does not reintroduce the untrusted-peer DoS/amplification scenario the thrift default is meant to guard against for general-purpose Thrift servers.

DBR LTS packaging risk (the reason this pin existed at all, see #798/#840)

0.25.0 ships the same wheel matrix as 0.24.0 -- manylinux2014/macOS/musl/Windows for cp310-cp314, confirmed on PyPI -- so pip resolves a wheel and never runs setup.py on DBR LTS; the SEV0 build-time failure class that originally forced this pin does not apply here. The DBR LTS Install CI check remains the authoritative gate for this.

Other CVEs in the 0.25.0 batch with Python impact

Per #969, not all are mapped in GitHub Advisory Database / OSV yet for the PyPI thrift package, but Apache's announcement lists several more affecting Python bindings (e.g. TJSONProtocol size-limit issues, TZlibTransport decompressed-size enforcement). This connector doesn't use TJSONProtocol or TZlibTransport (thrift_backend.py only imports THttpClient, TBinaryProtocol, TSocket, TTransport), so those are moot for this codebase, but the version bump clears them for any downstream code path.

Validation

  • Reproduced the regression directly against both versions with a standalone TMemoryBuffer round-trip of a 20 MiB binary field: fails under thrift 0.25.0's default TBinaryProtocol() construction, passes with the explicit string_length_limit=None this PR adds (same behavior as thrift 0.24.0).
  • Added test_binary_protocol_preserves_unbounded_string_and_container_limits (asserts the connector constructs TBinaryProtocol with string_length_limit=None, container_length_limit=None) and test_binary_protocol_reads_result_larger_than_thrift_default_limit (round-trips a 20 MiB field through the real thrift package with the connector's exact kwargs) to tests/unit/test_thrift_backend.py.
  • poetry run python -m pytest tests/unit with thrift 0.25.0 actually installed: 1020 passed, 5 skipped.
  • black --check / mypy clean on changed files.

Signed-off-by: Paddy Hannon pih@ehukai.com

Widen the thrift constraint from ~=0.24.0 to >=0.24.0,<0.26.0 so 0.25.0
can be resolved. thrift 0.25.0 fixes 61 CVEs across language bindings,
several affecting the Python binding this connector uses, including
CVE-2026-66858 (skip() recursion-limit bypass in the Python accelerator)
and CVE-2026-85494 (framed transport / binary protocol size a read
buffer from a peer-declared length with no effective maximum).

thrift 0.25.0 also changes TBinaryProtocol's default string_length_limit
from unbounded (None) to ~15.6 MiB (DEFAULT_MAX_FRAME_SIZE) as part of
that same CVE-2026-85494 fix. This connector already bounds its own
result stream via the TLS-verified, server-negotiated buffer_size_bytes
(default 100 MiB), and inline Arrow result batches in TFetchResultsResp
routinely exceed thrift's new ~15.6 MiB cap, so a bare version bump would
regress large-result-set reads. Construct TBinaryProtocol with explicit
string_length_limit=None and container_length_limit=None to preserve
pre-0.25 unbounded behavior for the connector's own already-bounded,
trusted-server result stream.

0.25.0 ships the same prebuilt wheel matrix as 0.24.0
(manylinux2014/macOS/musl/Windows, cp310-cp314), so the DBR LTS
build-time packaging risk that originally capped this dependency
(databricks#798, databricks#840) does not apply; the DBR LTS Install CI gate remains the
authoritative check.

Closes databricks#969

Signed-off-by: Paddy Hannon <pih@ehukai.com>
AOS-Session: 01a10c87-3242-7517-a60e-279091b796c3
AOS-Session: pi-1791211537-21291-e08bda26
AOS-Commit-Time: 2026-10-05T15:03:26Z

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

thrift pin (<0.25.0) blocks fix for additional Apache Thrift CVEs (CVE-2026-66858, CVE-2026-85494, and others fixed in 0.25.0)

1 participant