Skip to content

perf[buffer]: iteration for fallible operations with validity#8120

Open
joseph-isaacs wants to merge 22 commits into
developfrom
ji/fast-iter-valid
Open

perf[buffer]: iteration for fallible operations with validity#8120
joseph-isaacs wants to merge 22 commits into
developfrom
ji/fast-iter-valid

Conversation

@joseph-isaacs
Copy link
Copy Markdown
Contributor

@joseph-isaacs joseph-isaacs commented May 27, 2026

Currently use (and arrow) handle fallible operations with scalar (non-SIMD) code.

This PR add a trait and methods to have fast SIMD checked operations (includes cast) but verified else where that checked_add benefits

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 27, 2026

Merging this PR will improve performance by 23.7%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 1 improved benchmark
✅ 1264 untouched benchmarks
🆕 3 new benchmarks
⏩ 1 skipped benchmark1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation cast_u32_to_u8[65536] N/A 250.6 µs N/A
🆕 Simulation cast_i32_to_u32[65536] N/A 832.7 µs N/A
🆕 Simulation cast_u16_to_u32[65536] N/A 210.7 µs N/A
Simulation bitwise_not_vortex_buffer_mut[128] 304.4 ns 246.1 ns +23.7%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/fast-iter-valid (2c314ab) with develop (126c431)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs changed the title faster iteration infra perf[buffer]: iteration for fallible operations with validity May 27, 2026
@joseph-isaacs joseph-isaacs marked this pull request as ready for review May 27, 2026 15:13
@joseph-isaacs
Copy link
Copy Markdown
Contributor Author

Open question is where to put this code?

f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the changelog/performance A performance improvement label May 27, 2026
@robert3005
Copy link
Copy Markdown
Contributor

Sounds like we want a crate in between the array and vortex-buffer or this could be a feature flag in vortex-buffer

@joseph-isaacs
Copy link
Copy Markdown
Contributor Author

I was thinking vortex-compute what only works with dtype, buffers and rust native types?

I cannot remember if this was a problem before?

f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs requested a review from robert3005 May 28, 2026 11:30
@robert3005
Copy link
Copy Markdown
Contributor

there was no problem with vortex-compute iirc. There were some constructs that made it hard in the past but I think we're fine now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants