We really ought to deprecate UTF-16 someday. The fact that it pretends to be a fixed-length encoding has caused all sorts of bugs over the years, with many people assuming n(UTF-16 codepoints) == n(characters) which breaks when the string contains non-BMP characters.
And also, for personal aesthetic reasons I hate that it limits the Unicode codepoint range to an awkward non-power-of-two number (now there are 0x110000 codepoints in total). UTF-8 and UTF-32's 2^31 feels much more natural.
I wonder what it would take. For things like Java, JSA, and c# there would have to be things like a parallel utf8 api (yes please), but the really hard parts is the things that are as old as time (windows).
I don't think it will ever happen, but one can dream.
sph · · focus · HN ↗
Not until they decide to expand the emoji range, allocate space for all past and future fictional languages, as well as birdsong and dog barks.
Someone at the consortium is rubbing their hands with glee with all the newfound space.
But honestly, cool hack! If you invent a method to encode large numbers into bytes, why limit yourself to 24-bit numbers?
throw0101a · · focus · HN ↗
For compatibility with UTF-16:
* <a href="https://datatracker.ietf.org/doc/html/rfc3629#section-12" rel="nofollow">https://datatracker.ietf.org/doc/html/rfc3629#section-12* <a href="https://en.wikipedia.org/wiki/UTF-16" rel="nofollow">https://en.wikipedia.org/wiki/UTF-16
The original spec had 31 bits (the UTF-32/UCS-4 range):
* <a href="https://datatracker.ietf.org/doc/html/rfc2279" rel="nofollow">https://datatracker.ietf.org/doc/html/rfc2279
* <a href="https://en.wikipedia.org/wiki/UTF-32" rel="nofollow">https://en.wikipedia.org/wiki/UTF-32
orangeboats · · focus · HN ↗
And also, for personal aesthetic reasons I hate that it limits the Unicode codepoint range to an awkward non-power-of-two number (now there are 0x110000 codepoints in total). UTF-8 and UTF-32's 2^31 feels much more natural.
bjoli · · focus · HN ↗
I don't think it will ever happen, but one can dream.