Comment by tomsmeding

Comment by tomsmeding a day ago

0 replies

It's not unlikely that what you would ideally use here is the number of grapheme clusters. What is the length of "ë"? Either 1 or 2 codepoints depending on the encoding (combining [1] or single codepoint [2]), and either 1 byte (Latin-1), 2 bytes (UTF-8 single-codepoint) or 3 bytes (UTF-8 combining).

The metrics you care about are likely number of letters from a human perspective (1) or the number of bytes of storage (depends), possibly both.

[1]: https://tomsmeding.com/unicode#U+65%20U+308 [2]: https://tomsmeding.com/unicode#U+EB