[SPARK-58217][SQL] CAST(DECIMAL AS TIMESTAMP) silently produces garbage timestamps on overflow

[jiangxt2]

What changes were proposed in this pull request?

Add Long range validation for the Decimal-to-Timestamp cast path in both interpreted and codegen execution.

When a Decimal value multiplied by MICROS_PER_SECOND (1,000,000) exceeds Long range, BigDecimal.longValue() silently truncates the low 64 bits per the Java specification. The fix detects this overflow and either throws CAST_OVERFLOW (ANSI mode) or returns NULL (non-ANSI mode).

The cast implementation now validates the multiplication result against Long range before calling longValue(). DecimalType → TimestampType is also registered in forceNullable since the cast can now return null.

Why are the changes needed?

CAST(DECIMAL(38,0) AS TIMESTAMP) silently produces garbage timestamps when the Decimal value overflows Long after scaling. This affects both ANSI and non-ANSI modes.

SET spark.sql.ansi.enabled=false;
SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
-- Returns: -265697-05-04 09:44:49.448384 (garbage)
-- Expected: NULL

SET spark.sql.ansi.enabled=true;
SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
-- Returns: -265697-05-04 09:44:49.448384 (garbage)
-- Expected: CAST_OVERFLOW error

Root cause: BigDecimal.longValue() silently returns the low 64 bits when the value exceeds Long range. The cast implementation had no range checking.

This is consistent with existing overflow protection:

• The double-to-timestamp path guards against NaN/Infinite by returning null
• SecondsToTimestamp uses DATETIME_OVERFLOW for similar overflow protection
• SPARK-45816 fixed the same class of bug for timestamp-to-integer overflow

Does this PR introduce any user-facing change?

Yes. Previously, CAST(large_decimal AS TIMESTAMP) returned a nonsensical timestamp value when the Decimal overflows Long after scaling. Now:

• Non-ANSI mode: returns NULL (consistent with other overflow casts)
• ANSI mode: throws CAST_OVERFLOW error

This is a behavior change for an edge case that previously produced silently wrong results.

How was this patch tested?

Added unit tests in CastWithAnsiOffSuite and CastWithAnsiOnSuite:

Non-ANSI mode (CastWithAnsiOffSuite):

• Positive overflow: Decimal(99999999999999999999, 38, 0) → null
• Negative overflow: Decimal(-99999999999999999999, 38, 0) → null
• Scaled overflow: Decimal(99999999999999999999.999999, 38, 6) → null
• Normal value: Decimal(1, 10, 0) → MICROS_PER_SECOND
• Zero value: Decimal(0, 10, 0) → 0L
• Boundary just under: Decimal(9223372036854, 20, 0) → passes through
• Boundary just over: Decimal(9223372036855, 20, 0) → null

ANSI mode (CastWithAnsiOnSuite):

• Positive overflow: throws CAST_OVERFLOW with correct error parameters
• Negative overflow: throws CAST_OVERFLOW with correct error parameters

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code with Claude Opus 4.8

[vboo123]

The null/CAST_OVERFLOW behavior for out-of-range decimal→timestamp is consistent with how existing overflow casts already handle this (e.g., SecondsToTimestamp, timestamp→byte/short/int). The approach here looks correct.

One thing I noticed: this changes user-visible behavior (4.2 returns a silently-wrong wrapping value from unguarded .longValue truncation, 4.3 returns null or raises CAST_OVERFLOW), but there's no entry in docs/sql-migration-guide.md. There's direct precedent for documenting this — under "Upgrading from Spark SQL 3.5 to 4.0":

Since Spark 4.0, when overflowing during casting timestamp to byte/short/int under non-ansi mode, Spark will return null instead a wrapping value.

Should we add a bullet to the existing "Upgrading from Spark SQL 4.2 to 4.3" section. Something like:

Since Spark 4.3, when casting a decimal to timestamp overflows Long after scaling, Spark returns null (non-ANSI mode) or raises CAST_OVERFLOW (ANSI mode) instead of a wrapping value.

WDYT?

[jiangxt2]

@vboo123 Agreed, this is a user-visible behavior change and should be documented. Added a migration guide entry in the latest commit — wording is close to what you suggested.