Skip to content

Make x^y with Dual arguments with zero partials consistent with Real arguments - #856

Open
devmotion wants to merge 5 commits into
masterfrom
dmw/pow-zero-partials
Open

devmotion wants to merge 5 commits into
masterfrom
dmw/pow-zero-partials

Conversation

@devmotion

@devmotion devmotion commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Mathematically, a dual number with zero partials is just a regular number: Dual(v, 0) and v represent the same quantity. Hence derivative rules should return the same result regardless of whether an unperturbed argument is a Real or a Dual with zero partials.

Currently, this is not the case for ^ and NaNMath.pow if both arguments are Duals with the same tag:

julia> using ForwardDiff: Dual

julia> Dual(0.0, 0.0)^Dual(0.5, 1.0)  # same as `0.0^Dual(0.5, 1.0) == Dual(0.0, 0.0)`
Dual{Nothing}(0.0,NaN)

julia> Dual(0.0, 1.0)^Dual(0.0, 0.0)  # same as `Dual(0.0, 1.0)^0.0 == Dual(1.0, 0.0)`
Dual{Nothing}(1.0,NaN)

The second example returns NaN also in NaN-safe mode: there the NaN is not caused by a zero partial but by powval = 0 * 0^(-1).

This PR fixes both cases: if one of the arguments has zero partials, the Dual^Dual branch now defers to the Dual^Real or Real^Dual implementation (moved to the helper functions _pow_dual_base and _pow_dual_exponent, which are called directly: calling ^ again with value(y) would mix up the nesting levels if value(y) is itself a Dual with the same tag). Hence a Dual with zero partials gives exactly the same result as a Real, including the sign of zero partials (_mul_partials turned -0.0 partials into 0.0 with NaN-safe mode disabled). For consistency, the check in the Dual^Real branch is now also written as iszero(y) || isconstant(x) instead of y == zero(y) || iszero(partials(x)) (which is equivalent).

Additionally, the Dual^Real branch now computes the derivative before scaling the partials, i.e., (y * x^(y - 1)) * partials(x) instead of partials(x) * y * x^(y - 1). This rounds the same way as the Dual^Dual branch (the old version differed in the last bit for, e.g., Dual(1.3, 0.7)^2.7), and it scales the partials once instead of twice. As a consequence, results of x^y with a Real exponent may change in the last bit.

The zero partials in the Dual^Real branch now have the same type as the non-zero partials. Otherwise deferring to it would make, e.g., Dual{T,Float64}^Dual{T,BigInt} type-unstable (Float64 vs BigFloat partials), which it is not on master. As a side effect, Dual{T,Float64}^BigInt is now type-stable as well.

The new tests check with isequal that replacing an argument with a zero-partials Dual does not change the result.

With this PR, the derivatives at the special points agree with the Real case, SymPy, and WolframAlpha (default / NaN-safe mode where they differ):

SymPy WolframAlpha master, Real master, zero-partials Dual this PR, zero-partials Dual
∂x^y/∂x at (0, 0) 0 0 0 NaN / NaN 0
∂x^y/∂x at (0, 0.5) ∞ ∞ Inf Inf Inf
∂x^y/∂x at (0, -0.5) -∞ -∞ -Inf -Inf -Inf
∂x^y/∂x at (2, 0) 0 0 0 0 0
∂x^y/∂y at (0, 0) -∞ -∞ -Inf NaN / -Inf -Inf
∂x^y/∂y at (0, 0.5) 0 0 0 NaN / 0 0
∂x^y/∂y at (0, 1.5) 0 0 0 0 0
∂x^y/∂y at (0, -0.5) -∞ -∞ -Inf NaN / -Inf -Inf

For x = 0, the SymPy and WolframAlpha values are the limits x -> 0+ of y * x^(y - 1) and x^y * log(x) (at (0, 0), ∂x^y/∂x is d/dx x^0 = 0). At (0, 0), ∂x^y/∂y = -∞ is also the limit of the one-sided difference quotients of y -> 0^y.

Note that this PR only makes the rules consistent for Duals whose partials are all zero. A Dual with only some zero partials still propagates NaNs in these directions in the default mode, so results depend on the chunk size, also with the existing check in the Dual^Real branch:

julia> f(v) = v[1]^0.5 + v[2];

julia> ForwardDiff.gradient(f, [0.0, 1.0], ForwardDiff.GradientConfig(f, [0.0, 1.0], ForwardDiff.Chunk{1}()))
2-element Vector{Float64}:
 Inf
   1.0

julia> ForwardDiff.gradient(f, [0.0, 1.0], ForwardDiff.GradientConfig(f, [0.0, 1.0], ForwardDiff.Chunk{2}()))
2-element Vector{Float64}:
 Inf
 NaN

That's not specific to ^ (e.g., sqrt(v[1]) + v[2] returns [Inf, NaN] for any chunk size), and only NaN-safe mode handles it consistently. See #857.

🤖 Generated with Claude Code

…Real` arguments

A `Dual` with zero partials represents the same number as a `Real`, so the
`Dual^Dual` branch of `^` and `NaNMath.pow` now uses the same check as the
`Dual^Real` branch (`iszero(vy) || isconstant(x)`) and no longer returns
`NaN` partials for, e.g., `Dual(0.0, 0.0)^Dual(0.5, 1.0)` and
`Dual(0.0, 1.0)^Dual(0.0, 0.0)` (the latter also in NaN-safe mode).

The check in the `Dual^Real` branch is rewritten equivalently as
`iszero(y) || isconstant(x)` for consistency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the fix, `∂x^y/∂x = 0` at `(0, 0)`: in NaN-safe mode the gradient is now
`[0.0, -Inf]`, and in the default mode `[NaN, -Inf]` (the remaining `NaN` is
caused by the zero partial of `y` in the first direction multiplied by
`log(0) = -Inf`, which only NaN-safe mode handles).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.23%. Comparing base (a3c0f4f) to head (571ca6a).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #856   +/-   ##
=======================================
  Coverage   91.23%   91.23%           
=======================================
  Files          11       11           
  Lines        1072     1072           
=======================================
  Hits          978      978           
  Misses         94       94           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

devmotion and others added 2 commits October 14, 2026 15:34
If one argument of `Dual^Dual` has zero partials, defer to the `Dual^Real`
or `Real^Dual` method instead of combining the partials with
`_mul_partials`, which turned `-0.0` partials into `0.0` with NaN-safe mode
disabled. In the `Dual^Real` method, compute the derivative before scaling
the partials: `partials(x) * y * f(v, y - 1)` scaled them twice and rounded
differently than the `Dual^Dual` method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calling `^` again with `value(y)` (or `value(x)`) in the `Dual^Dual`
branch is wrong if the value is itself a `Dual` with the same tag: it
dispatches back to the `Dual^Dual` branch and mixes up the nesting levels.
Call the `Dual^Real` and `Real^Dual` implementations directly instead.

Also make the zero partials in the `Dual^Real` branch have the same type
as the non-zero partials, so that a constant `BigInt` exponent does not
make `Dual^Dual` type-unstable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant