Repository navigation
Compute legacy file stats from the aggregate stats accumulators - #10846
Conversation
The file writer accumulated every column's stats twice: once into the nested aggregate stats tree and again into separate legacy top-level accumulators. Derive the legacy top-level stats from the nested tree instead, adding a LegacyOnly node for the own stats of non-nullable top-level struct fields, which only the legacy layout records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LeUVVJGV8WkmDDtHukJFD Signed-off-by: Claude <noreply@anthropic.com>
Merging this PR will improve performance by 10.6%3 warnings, 1 note
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
The file writer accumulated every column's stats twice: once into the nested aggregate stats tree and again into separate legacy top-level accumulators. Derive the legacy top-level stats from the nested tree instead, adding a LegacyOnly node for the own stats of non-nullable top-level struct fields, which only the legacy layout records.