Skip to content

GH-3967: Skip the page copy for decompressors with stable output - #3968

Open
NicolaeNMV wants to merge 1 commit into
apache:masterfrom
NicolaeNMV:gh3967-skip-stable-page-copy
Open

NicolaeNMV wants to merge 1 commit into
apache:masterfrom
NicolaeNMV:gh3967-skip-stable-page-copy

Conversation

@NicolaeNMV

@NicolaeNMV NicolaeNMV commented Nov 17, 2026 •

Copy link
Copy Markdown

Rationale for this change

Fixes the performance regression from #3801 described in #3967. On the default heap path the GH-3150 copy is redundant for UNCOMPRESSED and ZSTD/LZ4_RAW, and costs +28.5% / +9.6% page-read time plus one extra allocation per page.

What changes are included in this PR?

  • A package-private CodecFactory.BytesDecompressor.returnsStableBytes(), false by default and overridden by NO_OP and by HeapBytesDecompressor for ZSTD/LZ4_RAW.
  • ColumnChunkPageReadStore skips the copy (data pages V1/V2 and dictionary) only for those decompressors. DirectCodecFactory and third-party decompressors are still copied, so Reading fails when using DirectCodecFactory #3150 stays fixed without changing the BytesInputDecompressor contract.

Are these changes tested?

Are there any user-facing changes?

No.

Closes #3967

apacheGH-3150 (apache#3801) copies every decompressed page in ColumnChunkPageReadStore,
because the BytesInputDecompressor contract allows a decompressor to reuse
its output buffer. On the default heap path that copy is redundant for
UNCOMPRESSED (NO_OP returns the page as read) and for ZSTD/LZ4_RAW
(HeapBytesDecompressor already decompresses into a fresh array).

Add a package-private CodecFactory.BytesDecompressor.returnsStableBytes(),
false by default and overridden by NO_OP and by HeapBytesDecompressor for
eagerly materialized codecs. The reader skips the copy only for those;
DirectCodecFactory and third-party decompressors are still copied.
@NicolaeNMV
NicolaeNMV force-pushed the gh3967-skip-stable-page-copy branch from 91eebbf to 1ef9965 Compare November 17, 2026 21:37

@wgtmac wgtmac left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance regression from #3801 (unreleased): heap page reads copy every decompressed page

2 participants