Skip to content

GH-50548: [C++][Parquet] PERF: Avoid O(n) level range validation in LevelDecoder::Decode#50550

Open
parthpathak1201 wants to merge 1 commit into
apache:mainfrom
parthpathak1201:fix/remove-findminmax-from-leveldecoder
Open

GH-50548: [C++][Parquet] PERF: Avoid O(n) level range validation in LevelDecoder::Decode#50550
parthpathak1201 wants to merge 1 commit into
apache:mainfrom
parthpathak1201:fix/remove-findminmax-from-leveldecoder

Conversation

@parthpathak1201

@parthpathak1201 parthpathak1201 commented Jul 20, 2026

Copy link
Copy Markdown

Rationale for this change

LevelDecoder::Decode called FindMinMax on every decoded batch, scanning
all levels (O(n)) for range validation. This showed up as a CPU hotspot in
the repeated-field skip path (GH-50548). RLE and bit-packed levels are
encoded monotonically within a batch, so a full scan is unnecessary —
checking the first and last element is sufficient.

What changes are included in this PR?

  • Replace FindMinMax with an O(1) check of levels[0] and
    levels[num_decoded - 1] in LevelDecoder::Decode.
  • Remove the now-unused #include "parquet/level_comparison.h" from
    column_reader.cc.
  • Simplify the error message (no longer reports min/max values).

Are these changes tested?

Existing SkipRecords and ReadRecords tests pass (18 tests). Benchmarks
show 4.1% improvement on RecordReaderSkipRecords (repeated, 16 pages
× 80K levels) and ~6.7% on single-call SkipRecords(59000).

Are there any user-facing changes?

No behavioral change. The validation still catches out-of-range levels.
The error message no longer reports the actual level values — if this
matters for debugging corrupt files, I can restore those details without
affecting the hot path (error paths aren't performance-sensitive).

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

@Reranko05 Reranko05 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @parthpathak1201 The workflow automatically converted this PR to a draft because the title doesn't match Arrow's expected format. Could you please update the title to follow the required format GH-<Issue Number>:<PR Title>.
Thanks!

@parthpathak1201 parthpathak1201 changed the title PERF: Avoid O(n) level range validation in LevelDecoder::Decode GH-50548: [C++][Parquet] PERF: Avoid O(n) level range validation in LevelDecoder::Decode Jul 20, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50548 has been automatically assigned in GitHub to PR creator.

@parthpathak1201

Copy link
Copy Markdown
Author

Hi @parthpathak1201 The workflow automatically converted this PR to a draft because the title doesn't match Arrow's expected format. Could you please update the title to follow the required format GH-<Issue Number>:<PR Title>. Thanks!

Thanks @Reranko05! Just updated the PR title to match the required format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants