Comment by layer8

Comment by layer8 16 hours ago

It’s not a bug, it’s a feature. The reason is that ISO 8859-7 [0] used for Greek has a separate character code for Iota (for all greek letters, really), while ISO 8859-3 [1] and -9 [2] used for Turkish do not for the usual dotless uppercase I.

One important goal of Unicode is to be able to convert from existing character sets to Unicode (and back) without having to know the language of the text that is being converted. If they had invented a separate code point for I in Turkish, then when converting text from those existing ISO character encodings, you’d have to know whether the text is Turkish or English or something else, to know which Unicode code point to map the source “I” into. That’s exactly what Unicode was designed to avoid.

[0] https://en.wikipedia.org/wiki/ISO/IEC_8859-7

[1] https://en.wikipedia.org/wiki/ISO/IEC_8859-3

[2] https://en.wikipedia.org/wiki/ISO/IEC_8859-9

thaumasiotes 13 hours ago

I know that. That's why I mentioned

> in "defense" of Unicode, it has several principles that directly contradict each other

Unicode wants to do several things, and they aren't mutually compatible. It is premised on the idea that you can be all things to all people.

> It’s not a bug, it’s a feature.

It is a bug. It directly violates Unicode's stated principles. It's also a feature, but that won't make it not a bug.

Reply View 0 replies

newpavlov 4 hours ago

>If they had invented a separate code point for I in Turkish, then when converting text from those existing ISO character encodings, you’d have to know whether the text is Turkish or English or something else, to know which Unicode code point to map the source “I” into. That’s exactly what Unicode was designed to avoid.

Great. So now we have to know locale for handling case conversion for probably centuries to come, but it was totally worth to save a bit of effort in the relatively short transition phase. /s

Reply View 4 replies

JuniperMesos 10 minutes ago

You always have to know locale to handle case conversion - this is not actually defined the same way in different human languages and it is a mistake to pretend it is.

Reply View | 1 reply
- newpavlov 4 minutes ago
  
  In most cases locale is encoded in character itself, i.e. Latin "a" and Cyrillic "a" are two different characters, despite being visually indistinguishable in most cases.
  The "language-sensitive" section of the special casing document [0] is extremely small and contains only the cases of stupid reuse of Latin I.
  [0]: https://www.unicode.org/Public/UCD/latest/ucd/SpecialCasing....
  
  Reply View | 0 replies
fhars 4 hours ago

Without it, there would not have been a transition phase.

Reply View | 1 reply
- newpavlov 3 hours ago
  
  I call BS. Without a series of MAJOR blunders Unicode was destined to succeed. When the rest of the world has migrated to Unicode, I am more than certain that Turks would've migrated as well. Yes, they may have complained for several years and would've spent a minuscule amount of resources to adopt the conversion software, but that's it, a decade or two later everyone would've forgotten about it.
  I believe that even addition of emojis was completely unnecessary despite the pressure from Japanese telecoms. Today's landscape of messengers only confirms that.
  
  Reply View | 0 replies