[release/11.0] Fix NLS ordinal casing of supplementary characters to agree with OrdinalIgnoreCase - #134229
Open
github-actions[bot] wants to merge 1 commit into
Open
[release/11.0] Fix NLS ordinal casing of supplementary characters to agree with OrdinalIgnoreCase#134229github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
…nalIgnoreCase (#134149) ## Summary Under Windows NLS, ordinal casing and ordinal comparison are backed by two different Windows APIs that disagree for some supplementary scalars: - `ToUpperOrdinal` / `ToLowerOrdinal` map via `LCMapStringEx`, which casts the Deseret upper/lower pairs (for example `U+10428` -> `U+10400`). - `OrdinalIgnoreCase` comparison uses `CompareStringOrdinal`, which treats those same supplementary pairs as **not equal**. Because of this Windows-level incompatibility, the canonicalization contract was broken in NLS mode: `ToUpperOrdinal(a) == ToUpperOrdinal(b)` no longer matched `string.Equals(a, b, StringComparison.OrdinalIgnoreCase)`. Two strings that ordinal-ignore-case comparison considered different could be folded to the same value by ordinal casing (and vice versa). ## Fix The change is **NLS only** and **adds no casing tables** (no binary size impact). After the normal casing pass, any supplementary pair that casing changed is validated against `CompareStringOrdinal`; if the cased value fell outside its `OrdinalIgnoreCase` class, the original scalar is restored. This reconciles ordinal casing output with the comparison oracle without carrying static data. Applies to the `string`/`Span` `ToUpperOrdinal`/`ToLowerOrdinal` paths and to `Rune.ToUpperOrdinal`/`Rune.ToLowerOrdinal`. **ICU mode is not touched at all.** ## Performance (NLS mode only) Release benchmarks, matched hosts differing only by `System.Private.CoreLib`, `DOTNET_SYSTEM_GLOBALIZATION_USENLS=1`: | Scenario | Ratio | Notes | |---|---|---| | ASCII | ~1.0 | Fast path unchanged (ratios shown are measurement noise) | | BMP (no surrogates) | ~1.09 | One added vectorized surrogate scan | | Emoji (surrogate-dense) | ~1.24 | Unchanged surrogate runs are bulk-skipped | | Deseret (changed supplementary pairs) | higher | Rare correctness path; one native validation per genuinely changed pair | The added cost is confined to NLS mode. ASCII keeps its fast path, pure BMP adds a single vectorized scan, and unchanged surrogate runs are skipped in bulk so surrogate-dense text does not pay per-pair. The overhead is acceptable: this behavior ships as part of a new feature in .NET 11, the common ICU path is unaffected, and the alternative (embedding casing data) would increase binary size, which we want to avoid. ## Tests Added an NLS-specific test covering the full Deseret block `U+10400..U+1044F` that asserts `String`/`Span`/`Rune` ordinal casing agree with `OrdinalIgnoreCase` equality (canonicalization and collision), plus mixed-string and unpaired-high-surrogate cases. Focused `OrdinalCasingTests`: 91 passed, 0 failed. Resolves #133950
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @dotnet/area-system-globalization |
Member
|
CC @vcsjones |
artl93
approved these changes
Sep 18, 2026
artl93
left a comment
artl93
left a comment
Member
There was a problem hiding this comment.
Issue in newly introduced feature. Approved.
This was referenced Sep 19, 2026
Open
Member
|
/ba-g all failures are known tracked failures and not related. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #134149 to release/11.0
/cc @tarekgh
Customer Impact
Ordinal casing is a new feature introduced in .NET 11. Customers using the new APIs on Windows with NLS globalization can receive incorrect results for supplementary-plane characters, including scripts such as Deseret, Adlam, and Vithkuqi. Two strings considered distinct by OrdinalIgnoreCase can be converted to the same ordinal-cased value, violating the documented canonicalization contract and potentially causing incorrect matching, deduplication, hashing, or key normalization. The issue is difficult for applications to work around and creates behavior inconsistent with ICU-based platforms. The fix is narrowly scoped to NLS mode, introduces no additional API changes, and aligns casing with the existing CompareStringOrdinal behavior. The issue found by aspnet. Look at the comment for more info dotnet/aspnetcore#68912 (comment).
#133950
Regression
[If yes, specify when the regression was introduced. Provide the PR or commit if known.]
Testing
All regression tests are passed, additionally added more tests to cover the issue we are fixing
Risk
Low, this fixing issue in a new feature introduced in net11, additionally the change is scoped to NLS mode which is not the default mode (ICU mode is the default mode).