Repository navigation
Tracking Issue for explicit-endian String::from_utf16 #116258
Description
Activity
- changed the title
[-]Tracking Issue for endian specific String::from_utf16[/-][+]Tracking Issue for explicit-endian String::from_utf16[/+]on Sep 29, 2023 Perhaps as an unresolved question: with these added,
FromUtf16Error'sDisplayimpl is no longer always accurate; it says"invalid utf-16: lone surrogate found", but these functions introduce a new failure case: the&[u8]was of odd length. MakingFromUtf16Errorhold information about which kind of error occurred would require making it not a ZST anymore,which could degrade performance since currently(see below)Result<String, FromUtf16Error>is (non-guaranteed-ly) null-pointer-optimized to be the same size asString.Alternately, they could return some new
FromUtf16BytesErrortype which can represent both errors, so thatString::from_utf16can still return the null-pointer-optimizedResult<String, FromUtf16Error>.(Alternately,
FromUtf16Error's Display impl could be updated to say something like"invalid utf-16: lone surrogate found, or odd length byte string passed".)To note,
Result<String, enum { L, R }>is still niched. The data pointer is null and the other 2×usizeare available to carry theErrpayload. The only performance hit would be constructing or inspecting the error payload.But that said, I also think just rendering the error as
invalid utf-16would be sufficient. Adding a new variant to the existing enum is also fine, but I don't think making a new error type is particularly helpful.An alternative would be to panic if given an odd-length slice, since that's trivial to precheck. But not a particularly good alternative.
Reacted by zachs18, John Schug, Cole Tobin and Yiyu LinAny news on this tracking issue? I'd love to use it
Yeah, I have been using these for months without issues. This API is my last one that requires nightly version of Rust. I would love to move back to stable :)
Is there anything we can help to get this stabilized?
Reacted by Yiyu Lin and Oakchris1955I agree with the people above, I think that we should try to get this stabilized ASAP
Reacted by Eir NymWe discussed this in today's @rust-lang/libs-api meeting.
We'd like to see these stabilized. We're happy with the current proposed names.
@rfcbot merge
We do want to see the error improved, either by making the single error message more generic or by adding internal variants for multiple different errors.
@rfcbot concern fix-error-message
Reacted by Eir Nym and Sampo KivistöTeam member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:
Concerns:
fix-error-messageresolved by Tracking Issue for explicit-endian String::from_utf16 #116258 (comment)
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!
See this document for info about what commands tagged team members can give me.
Reacted by Eir Nym29 remaining items
- added a commit that references this issue
on Jun 7, 2026 - added a commit that references this issue
on Jun 8, 2026 The final comment period, with a disposition to merge, as per the review above, is now complete.
As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.
Reacted by Sampo Kivistö- added a commit that references this issue
on Jun 18, 2026 - added a commit that references this issue
on Jun 30, 2026 - added a commit that references this issue
on Aug 7, 2026 Hello, as of Rust 1.98.0-nightly
FromUtf16Errordoes not implementClonesimilar toFromUtf8Error.FromUtf16ErrorKinddoes implementCloneandFromUtf8Erroralso does with#[cfg_attr(not(no_global_oom_handling), derive(Clone))]@cooolbros those are small enough that if you are interested, you could submit a PR and nominate for T-libs: I expect it to be unobjectionable enough to FCP.
View all comments
Feature gate:
#![feature(str_from_utf16_endian)]This is a tracking issue for versions of
String::from_utf16which take&[u8]and use a specific endianness.Public API
Steps / History
Unresolved Questions
from_utf16le,from_utf16_le,from_le_utf16,from_le_utf16_bytes, and other such combinations.with_capacity+pushimplementation used forfrom_utf16whilecollectdoesn't reserve capacity? (Collecting into a Result<Vec<_>> doesn't reserve the capacity in advance #48994)FromUtf16Errorcurrently displays as"invalid utf-16: lone surrogate found"which isn't correct for an error due to odd byte length.from_utf16method takes&[u16]so can't have the odd-length problem that&[u8]can.Footnotes
https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩