Support runend canonicalization for lists - #9708
Conversation
78849fb to
cec8d20
Compare
Merging this PR will degrade performance by 0.76%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | arrow_checked_add_u32_neon[16384] |
12.4 µs | 20.3 µs | -38.88% |
| ❌ | Simulation | cached_indices_i128[0.01] |
48 µs | 61.7 µs | -22.14% |
| ❌ | Simulation | compress_fsst[(500, 64, 8)] |
482.1 µs | 536.2 µs | -10.09% |
| ⚡ | Simulation | cached_indices_i32[0.01] |
51.6 µs | 37.5 µs | +37.36% |
| ⚡ | WallTime | arrow_checked_add_u32_avx512[16384] |
21.3 µs | 17.7 µs | +20.51% |
| ⚡ | WallTime | arrow_checked_add_u32_avx2[16384] |
21.3 µs | 17.7 µs | +20.13% |
| ⚡ | WallTime | mul_u32_nonnull_avx512 |
6.4 µs | 5.8 µs | +11.42% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing dk/runend-list-canonicalize (692c7e6) with develop (97953be)3
Footnotes
-
206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩
-
No successful run was found on
develop(a67cd1b) during the generation of this report, so 97953be was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Signed-off-by: Daniel King <dan@spiraldb.com>
cec8d20 to
692c7e6
Compare
| Validity::Array(validity) => Validity::Array(runend_decode_bools( | ||
| ends, | ||
| validity.execute_as::<BoolArray>("validity", ctx)?, | ||
| offset, | ||
| length, | ||
| ctx, | ||
| )?), | ||
| }; |
There was a problem hiding this comment.
i guess you could leave the validity as a run end array?
There was a problem hiding this comment.
@joseph-isaacs I actually wanted to ask you about that!
Would it be copacetic to simply recreate three RunEndArray's for the offsets, the sizes, and the validity? Or is that not what this method is supposed to do?
| let decoded_offsets = | ||
| runend_decode_primitive(ends.clone(), offsets, offset, length, ctx)?.into_array(); | ||
|
|
||
| let sizes = values.sizes().clone().execute_as("sizes", ctx)?; | ||
| let decoded_sizes = | ||
| runend_decode_primitive(ends.clone(), sizes, offset, length, ctx)?.into_array(); |
There was a problem hiding this comment.
I wonder if its worth fusing these two decode, seem like it will be much faster?
Summary
Tests