Comment by degamad

Comment by degamad a day ago

4 replies

It's not just the variable byte length that causes an issue, in some ways that's the easiest part of the problem. You also have to deal with code points that modify other code points, rather than being characters themselves. That's a huge part of the problem.

amake a day ago

That has nothing to do with UTF-8; that's a Unicode issue, and one that's entirely unescapable if you are the Unicode Consortium and your goal is to be compatible with all legacy charsets.

  • degamad 14 hours ago

    Yep, that's the point I was making - that choosing fixed 4-byte code-points doesn't significantly reduce the complexity of capturing everything that Unicode does.

bawolff a day ago

That goes all the way back to the beginning

Even ascii used to use "overstriking" where the backspace character was treated as a joiner character to put accents above letters.

  • degamad 14 hours ago

    Agreed, we just conveniently forget about those when speaking about how complex Unicode is.