[release/11.0] Fix not-greater-than predicate when lowering SIMD selects

[github-actions]

Backport of #133589 to release/11.0

/cc @tannergooding

Customer Impact

• Customer reported
• Found internally

This fixes #133550, where AVX-512 lowering changes CompareNotGreaterThan to CompareGreaterThan when folding a comparison into a masked SIMD select. Applications can silently receive the opposite selected value for less-than, equal, greater-than, and unordered inputs. The repro fails on .NET 11 RC2 but behaves correctly on .NET 8, 9, and 10.

Regression

• Yes
• No

The incorrect predicate mapping was introduced during the .NET 11 development cycle; the exact introducing change was not identified.

Testing

The source PR added focused 128- and 256-bit float and double regressions covering less-than, equal, greater-than, and NaN inputs. All 10 cases failed before the fix on AVX-512 and passed afterward. They also passed with AVX-512 and AVX2 disabled, with the applicable 128-bit cases passing with AVX disabled. Existing packed and scalar AVX comparison tests passed, and disassembly confirmed only the intended predicate changed without increasing instruction count or code size.

Risk

Low. The product change corrects a single intrinsic mapping from CompareGreaterThanMask to CompareNotGreaterThanMask. Focused coverage exercises all relevant predicates, element types, vector widths, NaN handling, and non-AVX-512 fallbacks.

IMPORTANT: If this backport is for a servicing release, please verify that:

• For .NET 8 and .NET 9: The PR target branch is release/X.0-staging, not release/X.0.
• For .NET 10+: The PR target branch is release/X.0 (no -staging suffix).

Package authoring no longer needed in .NET 9

IMPORTANT: Starting with .NET 9, you no longer need to edit a NuGet package's csproj to enable building and bump the version.
Keep in mind that we still need package authoring in .NET 8 and older versions.

Note

This backport template was completed by GitHub Copilot at the user's direction.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[JulieLeeMSFT]

Approved. Please check test failures once CI is done.