Skip to content

GH-51046: [Python][Interchange] Preserve the categorical ordered flag in from_dataframe - #51045

Draft
Kayvan-Zahiri wants to merge 1 commit into
apache:mainfrom
Kayvan-Zahiri:interchange-categorical-ordered
Draft

GH-51046: [Python][Interchange] Preserve the categorical ordered flag in from_dataframe#51045
Kayvan-Zahiri wants to merge 1 commit into
apache:mainfrom
Kayvan-Zahiri:interchange-categorical-ordered

Conversation

@Kayvan-Zahiri

@Kayvan-Zahiri Kayvan-Zahiri commented Aug 30, 2026

Copy link
Copy Markdown

Rationale for this change

from_dataframe builds the dictionary array without the is_ordered flag the producer reports, so an ordered categorical column comes back unordered. A pyarrow table with dictionary(int32, string, ordered=True) does not survive its own round trip, and an ordered pandas Categorical loses its ordering on the way in. The pandas consumer passes the flag through (pd.Categorical(values, categories=categories, ordered=categorical["is_ordered"])), and our own producer reports it, so pyarrow is the only side dropping it.

What changes are included in this PR?

categorical_column_to_dictionary passes ordered=categorical["is_ordered"] to DictionaryArray.from_arrays. Indices, dictionary and null handling are untouched.

Are these changes tested?

Yes. test_pyarrow_roundtrip_categorical is now parametrized over ordered True and False, and the ordered cases fail without the change: assert table.equals(result) sees ordered=1 going in and ordered=0 coming out. I also fixed a copy-paste in test_pandas_roundtrip_categorical, which read describe_categorical from the result column twice, so its is_ordered assertion compared the result with itself and could never fail.

Are there any user-facing changes?

An ordered dictionary column stays ordered through from_dataframe. No API change.

I used an AI assistant while finding and writing this. I checked the behavior and the tests myself before opening the PR.

…gorical column

categorical_column_to_dictionary dropped the is_ordered flag the producer
reports, so an ordered categorical column came back unordered.

The pandas roundtrip test compared the result column's describe_categorical
against itself, so its is_ordered assertion never ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BCtY1NAjuJ1jRvq5P6vSED
@github-actions github-actions Bot added the awaiting review Awaiting review label Aug 30, 2026
@Kayvan-Zahiri Kayvan-Zahiri changed the title GH-XXXXX: [Python][Interchange] Preserve the categorical ordered flag in from_dataframe GH-51046: [Python][Interchange] Preserve the categorical ordered flag in from_dataframe Aug 30, 2026
@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

@github-actions
github-actions Bot marked this pull request as draft August 30, 2026 18:02
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #51046 has been automatically assigned in GitHub to PR creator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant