Repository navigation
Conversation
…Real` arguments A `Dual` with zero partials represents the same number as a `Real`, so the `Dual^Dual` branch of `^` and `NaNMath.pow` now uses the same check as the `Dual^Real` branch (`iszero(vy) || isconstant(x)`) and no longer returns `NaN` partials for, e.g., `Dual(0.0, 0.0)^Dual(0.5, 1.0)` and `Dual(0.0, 1.0)^Dual(0.0, 0.0)` (the latter also in NaN-safe mode). The check in the `Dual^Real` branch is rewritten equivalently as `iszero(y) || isconstant(x)` for consistency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the fix, `∂x^y/∂x = 0` at `(0, 0)`: in NaN-safe mode the gradient is now `[0.0, -Inf]`, and in the default mode `[NaN, -Inf]` (the remaining `NaN` is caused by the zero partial of `y` in the first direction multiplied by `log(0) = -Inf`, which only NaN-safe mode handles). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #856 +/- ##
=======================================
Coverage 91.23% 91.23%
=======================================
Files 11 11
Lines 1072 1072
=======================================
Hits 978 978
Misses 94 94 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
If one argument of `Dual^Dual` has zero partials, defer to the `Dual^Real` or `Real^Dual` method instead of combining the partials with `_mul_partials`, which turned `-0.0` partials into `0.0` with NaN-safe mode disabled. In the `Dual^Real` method, compute the derivative before scaling the partials: `partials(x) * y * f(v, y - 1)` scaled them twice and rounded differently than the `Dual^Dual` method. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calling `^` again with `value(y)` (or `value(x)`) in the `Dual^Dual` branch is wrong if the value is itself a `Dual` with the same tag: it dispatches back to the `Dual^Dual` branch and mixes up the nesting levels. Call the `Dual^Real` and `Real^Dual` implementations directly instead. Also make the zero partials in the `Dual^Real` branch have the same type as the non-zero partials, so that a constant `BigInt` exponent does not make `Dual^Dual` type-unstable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mathematically, a dual number with zero partials is just a regular number:
Dual(v, 0)andvrepresent the same quantity. Hence derivative rules should return the same result regardless of whether an unperturbed argument is aRealor aDualwith zero partials.Currently, this is not the case for
^andNaNMath.powif both arguments areDuals with the same tag:The second example returns
NaNalso in NaN-safe mode: there theNaNis not caused by a zero partial but bypowval = 0 * 0^(-1).This PR fixes both cases: if one of the arguments has zero partials, the
Dual^Dualbranch now defers to theDual^RealorReal^Dualimplementation (moved to the helper functions_pow_dual_baseand_pow_dual_exponent, which are called directly: calling^again withvalue(y)would mix up the nesting levels ifvalue(y)is itself aDualwith the same tag). Hence aDualwith zero partials gives exactly the same result as aReal, including the sign of zero partials (_mul_partialsturned-0.0partials into0.0with NaN-safe mode disabled). For consistency, the check in theDual^Realbranch is now also written asiszero(y) || isconstant(x)instead ofy == zero(y) || iszero(partials(x))(which is equivalent).Additionally, the
Dual^Realbranch now computes the derivative before scaling the partials, i.e.,(y * x^(y - 1)) * partials(x)instead ofpartials(x) * y * x^(y - 1). This rounds the same way as theDual^Dualbranch (the old version differed in the last bit for, e.g.,Dual(1.3, 0.7)^2.7), and it scales the partials once instead of twice. As a consequence, results ofx^ywith aRealexponent may change in the last bit.The zero partials in the
Dual^Realbranch now have the same type as the non-zero partials. Otherwise deferring to it would make, e.g.,Dual{T,Float64}^Dual{T,BigInt}type-unstable (Float64vsBigFloatpartials), which it is not on master. As a side effect,Dual{T,Float64}^BigIntis now type-stable as well.The new tests check with
isequalthat replacing an argument with a zero-partialsDualdoes not change the result.With this PR, the derivatives at the special points agree with the
Realcase, SymPy, and WolframAlpha (default / NaN-safe mode where they differ):RealDualDual∂x^y/∂xat(0, 0)000NaN/NaN0∂x^y/∂xat(0, 0.5)∞∞InfInfInf∂x^y/∂xat(0, -0.5)-∞-∞-Inf-Inf-Inf∂x^y/∂xat(2, 0)00000∂x^y/∂yat(0, 0)-∞-∞-InfNaN/-Inf-Inf∂x^y/∂yat(0, 0.5)000NaN/00∂x^y/∂yat(0, 1.5)00000∂x^y/∂yat(0, -0.5)-∞-∞-InfNaN/-Inf-InfFor
x = 0, the SymPy and WolframAlpha values are the limitsx -> 0+ofy * x^(y - 1)andx^y * log(x)(at(0, 0),∂x^y/∂xisd/dx x^0 = 0). At(0, 0),∂x^y/∂y = -∞is also the limit of the one-sided difference quotients ofy -> 0^y.Note that this PR only makes the rules consistent for
Duals whose partials are all zero. ADualwith only some zero partials still propagatesNaNs in these directions in the default mode, so results depend on the chunk size, also with the existing check in theDual^Realbranch:That's not specific to
^(e.g.,sqrt(v[1]) + v[2]returns[Inf, NaN]for any chunk size), and only NaN-safe mode handles it consistently. See #857.🤖 Generated with Claude Code