Skip to content

fix(polars): preserve rows in zero-column pandas inputs - #518

Merged
JohnnyWilson16 merged 2 commits into
FreshCode-Org:mainfrom
WilliamK112:codex/polars-zero-column-463
Oct 13, 2026
Merged

JohnnyWilson16 merged 2 commits into
FreshCode-Org:mainfrom
WilliamK112:codex/polars-zero-column-463

Conversation

@WilliamK112

@WilliamK112 WilliamK112 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Cleaning a three-row pandas frame with no columns through the native Polars path returns (0, 0) on Polars 1.36 without a fallback record. Route nonempty, zero-column pandas inputs through the existing recorded pandas fallback before ingestion, preserving their rows, index, and report counts with pandas output.

The existing allow/warn/error fallback policy applies. Native-handle requests receive the disclosed pandas result, as with other input-driven fallbacks; a zero-row input with a RangeIndex still stays native. Also skip duplicate handling once the native schema has no columns, matching pandas and preventing Polars 2.0 from reporting or collapsing those rows as duplicates. Update the fallback matrix and changelog.

Fixes #463.

Validation on Windows / Python 3.12:

  • Reproduced the row loss on Polars 1.36.1; five new regressions failed before the fix. All 92 focused Polars, DuckDB, ingestion/output, and fallback-policy tests pass after it.
  • After the matrix compatibility follow-up, all 52 Polars/engine-parity cases pass on both pandas 1.5.3 + Polars 1.36.1 and pandas 2.3.3 + Polars 2.0.0, including dedup enabled/disabled after dropping every column. The new empty-input test explicitly supplies a RangeIndex because pandas 1.5 otherwise constructs an unsupported generic index.
  • Full fast lane on Polars 2.0.0: 7,610 passed, 49 skipped, 5 failed, and 94.99% coverage. All five failures reproduce on clean main@ba3f01f in this environment (Windows quoting/newline/filename fixtures and the zero-time semantic budget test).
  • Full Ruff check passes. Mypy passes for all 205 source files with --platform linux and numpy<2.5, matching CI's typecheck dependency cap.

Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: df6e1bef-c948-4486-bf96-6cbbd22dc06e
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T07:42:38.499751Z 722183f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com>

@JohnnyWilson16 JohnnyWilson16 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Cleanly fixes #463. Fallback policies are strictly enforced, input row counts and indices are preserved, and empty-column duplicate dropping in Polars is properly guarded. Ready to merge.

@JohnnyWilson16
JohnnyWilson16 merged commit 5433544 into FreshCode-Org:main Oct 13, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

engine="polars" drops all rows of a zero-column frame

2 participants