Repository navigation
[server] Support COUNT in the aggregation merge engine - #4998
platinumhamburg wants to merge 3 commits into
Conversation
|
LGTM, thanks! 👍 One note: COUNT turns This is not a blocker and we can merge once the CI passes and just have it a a follow-up if you think it makes sense. |
Leave untargeted columns null on the first partial write instead of initializing their aggregators. This allows a later partial delete to remove a row whose COUNT column was never written, while explicitly targeting COUNT with null still stores zero.
|
Thanks @polyzos, good catch! Addressed in f38fff0. First partial writes now respect the target columns, leaving an untouched COUNT column NULL so a later partial delete can remove the row. Explicitly targeting COUNT with a NULL input still stores 0. Added regression coverage for both KV formats and verified that the partial-delete case fails before the fix and passes afterward. The targeted Java 11 verification passed: 24 metadata tests and 119 aggregation tests, including 10 COUNT cases. |
Cover non-target values, schema evolution with reordered column IDs, and first partial writes without COUNT. Use the table.merge-engine option in the COUNT SQL example.
Purpose
Closes #4997.
Support native non-null input counting in the aggregation merge engine without requiring writers to generate a constant 1 and configure SUM.
Brief change log
AggFunctionType.COUNT,AggFunctions.COUNT(), andfields.<column>.agg = countsupport for BIGINT columns.Tests
Passed with Java 11:
Focused verification passed for metadata validation and the existing aggregation suite. The COUNT and row-merger verification passed all 30 cases, including 14 COUNT cases. The partial-delete regression failed on both KV formats before the fix and passed afterward. Checkstyle, Spotless, and RAT checks passed. The full repository test suite was not run.
API and Format
Adds the public evolving
AggFunctions.COUNT()factory and COUNT function type. Input and stored output use BIGINT. No RPC or storage format changes. COUNT follows the existing delete behavior and does not support retractions or COUNT DISTINCT.Documentation
Adds COUNT semantics, SQL and Java usage, the distinction between counting inputs and summing precomputed counts, and the behavior of omitted versus explicitly null COUNT columns on the first partial write to the aggregation merge engine documentation.