Why GNU moe does not use UTF-8 or other form of Unicode?(gnu.org)
gnu.org
Why GNU moe does not use UTF-8 or other form of Unicode?
https://www.gnu.org/software/moe/manual/moe_manual.html#Introduction
21 comments
> It’s clear though there is a little detachment from reality here, to put it gently.
Sounds like a perfect fit for the GNU organization
Sounds like a perfect fit for the GNU organization
I genuinely can't tell if this is an April Fool's joke that the gnu project forgot to take down, or a serious project.
I use non-iso8859 characters on a regular basis, so this editor is completely useless to me.
>We all have the right to be taught exactly one language that gives full access to the whole human culture and allows each of us to communicate with everybody else.
This makes even the most deranged conspiracy theorist sound lucid.
Edit: tried downloading it; the source tarball is compressed with lzip (instead of the more common gz/bzp2/xz) formats. Lzip is not installed by default on my machine, unlike literally every other decompressor.
This whole thing seems like an act of defiance purely for the sake of defiance.
I use non-iso8859 characters on a regular basis, so this editor is completely useless to me.
>We all have the right to be taught exactly one language that gives full access to the whole human culture and allows each of us to communicate with everybody else.
This makes even the most deranged conspiracy theorist sound lucid.
Edit: tried downloading it; the source tarball is compressed with lzip (instead of the more common gz/bzp2/xz) formats. Lzip is not installed by default on my machine, unlike literally every other decompressor.
This whole thing seems like an act of defiance purely for the sake of defiance.
Unicode?! YAGNI!
Advocating for a universal language (that must presumably be written in Latin letters) is an entertaining way to round off the manual to one's editor.
Advocating for a universal language (that must presumably be written in Latin letters) is an entertaining way to round off the manual to one's editor.
From the documentation:
> The lack of a common language is the most divisive border. The first step towards mutual comprehension is a common language. The first step towards a common language is a common alphabet.
> The lack of a common language is the most divisive border. The first step towards mutual comprehension is a common language. The first step towards a common language is a common alphabet.
There are some silly performance claims in there, but I think my favorite part is "An Unicode text console is both a danger and a waste. The right mode for presenting a large variety of complex glyphs is graphical mode using a specialized program."
That console's already using multiple megabytes just as a frame buffer. Some extra fonts are not going to be a problem.
(Also how exactly are they pronouncing "Unicode"?)
That console's already using multiple megabytes just as a frame buffer. Some extra fonts are not going to be a problem.
(Also how exactly are they pronouncing "Unicode"?)
"Oonicode" ?
If an Arduino can do it, how bad can it be (:
https://crawlingrobotfortress.blogspot.com/2021/10/semigraph...
(Blitting unicode glyphs to screen is not hard or slow; In this project, the glyph rendering code turned into spaghetti to save on space. Cleaner rendering code is possible if using uncompressed bitmap fonts. If there is a concern about size on disk, you could only install the codepages needed for a given locale? But then again, unless you include the various pages of Chinese characters, we're still talking no more than a few hundred K to store all of Unicode as a bitmap font. True, handling combining modifiers probably would increase binary size and maintenance costs, but again... if it fits on an Arduino it's probably OK for the desktop. And if you're already OK with not supporting unicode, you're probably OK with supporting most of unicode except combining modifiers.)
https://crawlingrobotfortress.blogspot.com/2021/10/semigraph...
(Blitting unicode glyphs to screen is not hard or slow; In this project, the glyph rendering code turned into spaghetti to save on space. Cleaner rendering code is possible if using uncompressed bitmap fonts. If there is a concern about size on disk, you could only install the codepages needed for a given locale? But then again, unless you include the various pages of Chinese characters, we're still talking no more than a few hundred K to store all of Unicode as a bitmap font. True, handling combining modifiers probably would increase binary size and maintenance costs, but again... if it fits on an Arduino it's probably OK for the desktop. And if you're already OK with not supporting unicode, you're probably OK with supporting most of unicode except combining modifiers.)
What about l-r vs r-l text? What about mixtures of l-r and r-l in the same document?
Supporting Unicode half-assed may be relatively easy. Supporting it properly is hard, and requires a team of experts. I doubt even John Cowan is enough an expert in all the fiddly little corner cases to do it all himself.
But you still have to do it.
Thankfully, there is libpango and libharfbuzz, which take care of many of these details for you.
Supporting Unicode half-assed may be relatively easy. Supporting it properly is hard, and requires a team of experts. I doubt even John Cowan is enough an expert in all the fiddly little corner cases to do it all himself.
But you still have to do it.
Thankfully, there is libpango and libharfbuzz, which take care of many of these details for you.
> There is no such thing as an 'ambiguous', 'invalid', or 'out of range' ISO-8859-15 character.
Er, okay, so, what does '\x00' (NUL), '\x05' (ENQ), '\x0e' (SO), '\x10' (DLE) etc. "mean"?
Er, okay, so, what does '\x00' (NUL), '\x05' (ENQ), '\x0e' (SO), '\x10' (DLE) etc. "mean"?
A quick look at the wiki page for 8859-15 reveals that there are in fact 33 bytes that are invalid/out of range for that character encoding: 00-1f, and 7f-9f inclusive.
For example, searching for a word like "file" in an Unicode document may fail if the document uses the presentation form (compound character) 'fi' instead of the two-character string "fi".
Not an issue in a collation-aware search. Test here by searching for "fi": https://en.wikipedia.org/wiki/Ligature_(writing)
Not an issue in a collation-aware search. Test here by searching for "fi": https://en.wikipedia.org/wiki/Ligature_(writing)
Just tried this on Firefox on Linux. It finds "fi" instances in a word like "first", since that's rendered as separate characters on my system, but not when wikipedia specifically uses the fi ligature.
<<The only sane approach to software internationalization is to adopt or develop a lingua franca. Such lingua franca should be fully phonetic and should fit in an 8-bit character set like ISO-8859-15 for ease of use by both humans and computers. Unicode is not a viable solution.>>
Clearly I will not use a defective and limited editor.
Probably the author has is mind limited because he is thinking with just 256 characters...
Clearly I will not use a defective and limited editor.
Probably the author has is mind limited because he is thinking with just 256 characters...
Parsing UTF-8 is not much harder than parsing ASCII unless you're doing something weird like relying on codepoint indices for absolute file locations, which is great for a toy and really weak design for a production parser.
So why nobody is doing it right then, still?
Everybody is prone to Unicode confusables attacks, and almost nobody checks for invalid chars. Almost nobody can search Unicode strings, almost nobody implemented identifiable identifiers.
Everybody is prone to Unicode confusables attacks, and almost nobody checks for invalid chars. Almost nobody can search Unicode strings, almost nobody implemented identifiable identifiers.
What could be a correct counter argument to this?
I don't know, but not going back to ISO-8859-15, that's for sure.
Fix the implementations.
Yikes, not a good look, my dude. Even Unicode is such a vadt oversimplification of the complexities of digitally encoding human writing. I say go the other way: make all text carry encoding info, including strings in programming languages (which would ideally incorporate encoding as part of their type, and forbid combining strings of different encodings without explicit conversion to a target encoding). Yes, it's more complicated. So is memory-safe static object lifetime management, and we considered that a W and worth the trouble.
Twitter discussion https://twitter.com/fish_cto/status/1482999487903662081?s=21
In my opinion, the greatest invention of humankind is the language, and the greatest error was inventing more than one. The language barrier is the ultimate obstacle to inclusiveness, and therefore linguistic diversity is a problem that needs to be fixed. We all have the right to be taught exactly one language that gives full access to the whole human culture and allows each of us to communicate with everybody else. Equal rights can't be achieved without equal language. Forcing someone to learn a second language in order to access knowledge is wasteful and ineffective: twice as many words need to be memorized and most people will never be so proficient in the second language as they are in their native language. Language is a tool, not an end in itself.
I mean… wowza.
I can see some sense in wanting to use a “safe”, well-understood character set. It’s clear though there is a little detachment from reality here, to put it gently.