Add PiecewiseSequence index array#8799
Conversation
Polar Signals Profiling ResultsLatest Run
Previous Runs (7)
Powered by Polar Signals Cloud |
Merging this PR will improve performance by 11.17%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes
|
Benchmarks: Vortex queriesVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.080x ➖, 0↑ 0↓)
datafusion / parquet (1.018x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.069x ➖, 0↑ 0↓)
duckdb / parquet (1.022x ➖, 0↑ 0↓)
No file size changes detected. |
3edf6c8 to
1748e31
Compare
| ctx: &mut ExecutionCtx, | ||
| ) -> VortexResult<(PrimitiveArray, PrimitiveArray, PrimitiveArray)> { | ||
| let starts = array.starts().clone().execute::<PrimitiveArray>(ctx)?; | ||
| let lengths = array.lengths().clone().execute::<PrimitiveArray>(ctx)?; |
There was a problem hiding this comment.
I think while starts could be primitive we specifically want to handle cases where this is not primitive?
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
6eb0c92 to
16ca5dc
Compare
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Summary
Adds
PiecewiseSequenceArray, a serializable index encoding for piecewise arithmetic sequences:starts[i] + j * multipliers[i]forjin0..lengths[i].This PR only introduces the index array and its execution/scalar behavior. It intentionally does not specialize
take; dependent PRs consume this encoding for the constant-multiplier-1 contiguous case.Validation
cargo fmt -p vortex-arraycargo test -p vortex-array piecewise_sequence