Skip to content

feat(pg-catalog): support DataFusion 54#3

Open
discord9 wants to merge 43 commits into
GreptimeTeam:masterfrom
discord9:upgrade/datafusion-54
Open

feat(pg-catalog): support DataFusion 54#3
discord9 wants to merge 43 commits into
GreptimeTeam:masterfrom
discord9:upgrade/datafusion-54

Conversation

@discord9

@discord9 discord9 commented Jul 16, 2026

Copy link
Copy Markdown

Summary

  • Pin datafusion-pg-catalog and its arrow-pg DataFusion feature closure to exact DataFusion 54.0.0.
  • Apply the minimal DF54 trait compatibility update and add catalog OID/coercion coverage without changing pg_catalog schemas or OID mappings.

Tests

  • CARGO_TARGET_DIR=/mnt/nvme_rust/rust-targets/datafusion-pg-catalog-54-port cargo test -p datafusion-pg-catalog — 24 passed
  • CARGO_TARGET_DIR=/mnt/nvme_rust/rust-targets/datafusion-pg-catalog-54-port cargo test -p arrow-pg --features datafusion — 60 passed
  • git diff --check

Formatting

  • cargo fmt --all --check still reports pre-existing rustfmt drift in unmodified workspace files. This PR intentionally does not add unrelated formatting churn.

sunng87 and others added 30 commits March 2, 2026 17:30
@sunng87 @MichaelScofield
x (datafusion-contrib#301)
da75e67 
Co-authored-by: luofucong <luofc@foxmail.com>
@sunng87
chore: Release
e957839
@sunng87
chore: Release
e27f450
@dependabot
chore(deps): bump tokio from 1.49.0 to 1.50.0 (datafusion-contrib#302)
8b3acd5 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.49.0 to 1.50.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.49.0...tokio-1.50.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mjgarton
Avoid creating query string when not needed (datafusion-contrib#303)
47bdf5e 
Since the query String is only used later, move its creation lower down,
so we avoid it entirely if query hooks process the query.
@dependabot
chore(deps): bump datafusion from 52.2.0 to 52.3.0 (datafusion-contri…
f7b01b8 
…b#306)

Bumps [datafusion](https://github.com/apache/datafusion) from 52.2.0 to 52.3.0.
- [Changelog](https://github.com/apache/datafusion/blob/main/CHANGELOG.md)
- [Commits](apache/datafusion@52.2.0...52.3.0)

---
updated-dependencies:
- dependency-name: datafusion
  dependency-version: 52.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mjgarton
Improve pg_catalog type mapping (datafusion-contrib#305)
a4758dd 
* Improve pg_catalog type mapping

Instead of PgAttributeTable deciding how to map from arrow types to
postgres types, delegate that decision to arrow-pg.  This avoids
duplicationm, brings consistency, and makes us more robust against
future changes and additions to the mapping in arrow-pg.

* Revert back to signed ints for iod

Revert back to signed ints for iod since postgresql does not have
support for unsigned ints.
@dependabot
chore(deps): bump pgwire from 0.38.0 to 0.38.1 (datafusion-contrib#307)
179e01f 
Bumps [pgwire](https://github.com/sunng87/pgwire) from 0.38.0 to 0.38.1.
- [Release notes](https://github.com/sunng87/pgwire/releases)
- [Changelog](https://github.com/sunng87/pgwire/blob/master/CHANGELOG.md)
- [Commits](sunng87/pgwire@v0.38.0...v0.38.1)

---
updated-dependencies:
- dependency-name: pgwire
  dependency-version: 0.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump datafusion from 52.3.0 to 52.4.0 (datafusion-contri…
23615fc 
…b#308)

Bumps [datafusion](https://github.com/apache/datafusion) from 52.3.0 to 52.4.0.
- [Changelog](https://github.com/apache/datafusion/blob/main/CHANGELOG.md)
- [Commits](apache/datafusion@52.3.0...52.4.0)

---
updated-dependencies:
- dependency-name: datafusion
  dependency-version: 52.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump env_logger from 0.11.9 to 0.11.10 (datafusion-contr…
fbca72d 
…ib#310)

Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.11.9 to 0.11.10.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](rust-cli/env_logger@v0.11.9...v0.11.10)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-version: 0.11.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
fix: use int2 for arrow 8-bit int (datafusion-contrib#311)
83d9fdf
@dependabot
chore(deps): bump pgwire from 0.38.1 to 0.38.2 (datafusion-contrib#314)
58ca297 
Bumps [pgwire](https://github.com/sunng87/pgwire) from 0.38.1 to 0.38.2.
- [Release notes](https://github.com/sunng87/pgwire/releases)
- [Changelog](https://github.com/sunng87/pgwire/blob/master/CHANGELOG.md)
- [Commits](sunng87/pgwire@v0.38.1...v0.38.2)

---
updated-dependencies:
- dependency-name: pgwire
  dependency-version: 0.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
chore: update readme
9006d82
@dependabot
chore(deps): bump rust_decimal from 1.40.0 to 1.41.0 (datafusion-cont…
00b4f48 
…rib#316)

Bumps [rust_decimal](https://github.com/paupino/rust-decimal) from 1.40.0 to 1.41.0.
- [Release notes](https://github.com/paupino/rust-decimal/releases)
- [Changelog](https://github.com/paupino/rust-decimal/blob/master/CHANGELOG.md)
- [Commits](paupino/rust-decimal@1.40.0...1.41.0)

---
updated-dependencies:
- dependency-name: rust_decimal
  dependency-version: 1.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump postgres-types from 0.2.12 to 0.2.13 (datafusion-co…
a301e5e 
…ntrib#319)

Bumps [postgres-types](https://github.com/rust-postgres/rust-postgres) from 0.2.12 to 0.2.13.
- [Release notes](https://github.com/rust-postgres/rust-postgres/releases)
- [Commits](rust-postgres/rust-postgres@postgres-types-v0.2.12...postgres-types-v0.2.13)

---
updated-dependencies:
- dependency-name: postgres-types
  dependency-version: 0.2.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mjgarton
Upgrade to datafusion 53 (datafusion-contrib#321)
764c147 
* Upgrade to datafusion 53

Upgrade to datafusion 53 and upgrade some other dependencies to match
the versions used by datafusion 53 (arrow libraries, etc)

Also make a few minor changes to fix new deprecation warnings.

* Fix tests
@sunng87
chore: Release
a958f1f
@dependabot
chore(deps): bump tokio from 1.50.0 to 1.51.0 (datafusion-contrib#322)
9442098 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.50.0 to 1.51.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.50.0...tokio-1.51.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.51.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
fix: use text format as default in describe statement (datafusion-con…
c2459b7 
…trib#323)
@dependabot
chore(deps): bump tokio from 1.51.0 to 1.51.1 (datafusion-contrib#324)
60b3dac 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.51.0 to 1.51.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.51.0...tokio-1.51.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.51.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump pgwire from 0.38.2 to 0.38.3 (datafusion-contrib#325)
4ba7615 
Bumps [pgwire](https://github.com/sunng87/pgwire) from 0.38.2 to 0.38.3.
- [Release notes](https://github.com/sunng87/pgwire/releases)
- [Changelog](https://github.com/sunng87/pgwire/blob/v0.38.3/CHANGELOG.md)
- [Commits](sunng87/pgwire@v0.38.2...v0.38.3)

---
updated-dependencies:
- dependency-name: pgwire
  dependency-version: 0.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87 @mjgarton
feat: improve query parameter parsing for datafusion (datafusion-cont…
d3de9f4 
…rib#326)

* feat: improve query parameter parsing for datafusion

* feat: add range check and tests

* chore: fmt

* Simplify float & int casting (datafusion-contrib#327)

Instead of reusing `checked_int_cast` also for the float code paths and
requiring a `source` parameter, split the int & float paths completely,
making the callsites simpler, and reducing the overall code length.

---------

Co-authored-by: Martin Garton <mjgarton@users.noreply.github.com>
@dependabot
chore(deps): bump tokio from 1.51.1 to 1.52.0 (datafusion-contrib#328)
764be81 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.51.1 to 1.52.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.51.1...tokio-1.52.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.52.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
fix: only return schema for query (datafusion-contrib#329)
fe83376
@sunng87
fix: lint for rust 1.95 (datafusion-contrib#334)
f27a050
@dependabot
chore(deps): bump datafusion from 53.0.0 to 53.1.0 (datafusion-contri…
66a3242 
…b#332)

Bumps [datafusion](https://github.com/apache/datafusion) from 53.0.0 to 53.1.0.
- [Changelog](https://github.com/apache/datafusion/blob/main/CHANGELOG.md)
- [Commits](apache/datafusion@53.0.0...53.1.0)

---
updated-dependencies:
- dependency-name: datafusion
  dependency-version: 53.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump tokio from 1.52.0 to 1.52.1 (datafusion-contrib#331)
466df24 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.0 to 1.52.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.52.0...tokio-1.52.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.52.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
feat: update pgwire to 0.39 (datafusion-contrib#330)
1b9c12c
@sunng87
refactor: update for rust 2024 edition (datafusion-contrib#335)
789dc23
@dependabot
chore(deps): bump rustls-pki-types from 1.14.0 to 1.14.1 (datafusion-…
77af31f 
…contrib#336)

Bumps [rustls-pki-types](https://github.com/rustls/pki-types) from 1.14.0 to 1.14.1.
- [Release notes](https://github.com/rustls/pki-types/releases)
- [Commits](rustls/pki-types@v/1.14.0...v/1.14.1)

---
updated-dependencies:
- dependency-name: rustls-pki-types
  dependency-version: 1.14.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dependabot Bot and others added 13 commits May 4, 2026 20:34
@dependabot
chore(deps): bump arrow-schema from 58.1.0 to 58.2.0 (datafusion-cont…
b14ca2f 
…rib#337)

Bumps [arrow-schema](https://github.com/apache/arrow-rs) from 58.1.0 to 58.2.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md)
- [Commits](apache/arrow-rs@58.1.0...58.2.0)

---
updated-dependencies:
- dependency-name: arrow-schema
  dependency-version: 58.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump tokio from 1.52.1 to 1.52.2 (datafusion-contrib#340)
f86dae3 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.1 to 1.52.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.52.1...tokio-1.52.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.52.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump rust_decimal from 1.41.0 to 1.42.0 (datafusion-cont…
fc02f8e 
…rib#341)

Bumps [rust_decimal](https://github.com/paupino/rust-decimal) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/paupino/rust-decimal/releases)
- [Changelog](https://github.com/paupino/rust-decimal/blob/master/CHANGELOG.md)
- [Commits](paupino/rust-decimal@1.41.0...1.42.0)

---
updated-dependencies:
- dependency-name: rust_decimal
  dependency-version: 1.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
chore(deps): bump tokio from 1.52.2 to 1.52.3 (datafusion-contrib#343)
6aaf9b3 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.2 to 1.52.3.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.52.2...tokio-1.52.3)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.52.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
feat: cursor and fetch support (datafusion-contrib#339)
646dde3 
* feat: cursor and fetch support

* feat: update cursor to support extended query

* feat: use pgwire 0.40 fetch api

* test: update tests for empty fetch

* test: add an integration test for foreign data wrapper

* ci: install podman

* test: refactor integration test to use flake environment

* ci: remove podman info

* chore: add back podman info
@dependabot
chore(deps): bump arrow-schema from 58.2.0 to 58.3.0 (datafusion-cont…
a5a1780 
…rib#344)

Bumps [arrow-schema](https://github.com/apache/arrow-rs) from 58.2.0 to 58.3.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/commits)

---
updated-dependencies:
- dependency-name: arrow-schema
  dependency-version: 58.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@sunng87
flake.lock: Update
e384928 
Flake lock file updates:

• Updated input 'fenix':
    'github:nix-community/fenix/f1cf6dcfc51a08e154ec753f11146dc1d95be1e5?narHash=sha256-EIP2/UXzbkb1ZWoa0pTwNv2ZivlfbRT%2B7j0fj36IhWg%3D' (2026-01-25)
  → 'github:nix-community/fenix/3c09e6c592c6ebfb9cde94bba990af23b2c06a87?narHash=sha256-AIbVmDAJFVRl%2BPUkgoOHVvzulVLQW0pcltKNrBVWxww%3D' (2026-05-17)
• Updated input 'fenix/rust-analyzer-src':
    'github:rust-lang/rust-analyzer/cc76a8a0c6039abb17e005aa5f96c9d13d4d9665?narHash=sha256-LoyIkk68hFEwE49o8rzYK50bqX8jWhjcxpYLz2lBCbI%3D' (2026-01-24)
  → 'github:rust-lang/rust-analyzer/ace47025ddfc7fec399e11594318e64443c0e1c2?narHash=sha256-bNCTsqjqpSVXDKsCKHTyNEaw6LpaiRUoUVKlVDSzjHk%3D' (2026-05-16)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/48698d12cc10555a4f3e3222d9c669b884a49dfe?narHash=sha256-yxgb4AmkVHY5OOBrC79Vv6EVd4QZEotqv%2B6jcvA212M%3D' (2026-01-25)
  → 'github:NixOS/nixpkgs/d233902339c02a9c334e7e593de68855ad26c4cb?narHash=sha256-30sZNZoA1cqF5JNO9fVX%2BwgiQYjB7HJqqJ4ztCDeBZE%3D' (2026-05-15)
@sunng87
chore: Release
84c43e9
@discord9 @sunng87
fix(pg-catalog): restrict unqualified pg table rewrite (datafusion-co…
f6f5f21 
…ntrib#364)
@sunng87
chore: Release
b0269e4
@sunng87
feat: add support for select current_catalog, select current_schema
911bf47
@sunng87
chore: Release
b1f5242
@discord9
feat: support DataFusion 54
f1bfdcb 
Signed-off-by: discord9 <discord9@163.com>
@discord9 discord9 mentioned this pull request Jul 17, 2026
Draft
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@discord9 @sunng87 @mjgarton