Skip to content

Build precision 19 decimal scalars at i128 - #10903

Open
connortsui20 wants to merge 1 commit into
ct/decimal-width-ownerfrom
ct/decimal-scalar-factory-width
Open

connortsui20 wants to merge 1 commit into
ct/decimal-width-ownerfrom
ct/decimal-scalar-factory-width

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Summary

vx.scalar(decimal.Decimal("9999999999999999999")) fails with an overflow error, and so does a JNI decimal literal at precision 19 with a value above i64::MAX. Both factories picked the value width from DecimalDType::required_bit_width(), which counts magnitude bits without a sign bit. At precision 19 that is 64, so they chose i64, which can't hold the largest 19-digit values.

Changes

Both factories now match on DecimalDType::canonical_values_type() from #10897, which gives i128 at precision 19. Every other precision keeps its width, because the sign bit still fits there. The Python precision inference test and a new JNI unit test cover the largest and smallest values at precision 19.

This is stacked on #10897 and retargets to develop after that merges.

Comment on lines +614 to +615
/// The [`DecimalValue`] variant of the canonical width of `dtype`'s precision.
fn canonical_decimal_value(value: i256, dtype: &DecimalDType) -> PyResult<DecimalValue> {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might blow things up but I think that is fine because it is a scalar, not an array?

The Python and JNI decimal scalar factories picked the value width from
`DecimalDType::required_bit_width()`, which counts magnitude bits without a
sign bit. At precision 19 it returns 64, so both factories chose `i64`, and a
valid value above `i64::MAX` failed with an overflow error. For example,
`vx.scalar(decimal.Decimal("9999999999999999999"))` failed.

Both factories now match on `DecimalDType::canonical_values_type()`, which
gives `i128` at precision 19. Every other precision keeps its width.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/decimal-scalar-factory-width branch from 81cc7b1 to 41073b9 Compare October 27, 2026 22:18
@codspeed

codspeed Bot commented Oct 27, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 13.54%

3 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 3 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 3 regressed benchmarks
✅ 2200 untouched benchmarks
⏩ 370 skipped benchmarks1
🗄️ 30 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime threshold_neon[64] 2.6 µs 3.8 µs -31.52%
❌ Walltime threshold_neon[80] 3.3 µs 4.4 µs -24.01%
❌ Walltime filter_neon[80] 2.4 µs 2.8 µs -12.32%
⚡ CPU Simulation test_repeated_scan_scalar_at[int_str] 412.1 µs 336.5 µs +22.46%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/decimal-scalar-factory-width (41073b9) with ct/decimal-width-owner (9836fef)

Open in CodSpeed

Footnotes

  1. 370 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 30 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

@connortsui20
connortsui20 enabled stacked auto-merge (squash) October 28, 2026 12:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants