From: Curt Sampson Date: 2002-08-05T14:58:20+09:00 Subject: Re: Unicode in Ruby now? On Mon, 5 Aug 2002, Clifford Heath wrote: > Curt Sampson wrote: > > These statements are both very wrong. > > I was deliberately being "reinterpretive", but what I said is the effective > truth. What part of "UTF-32 is restricted in values to the range 0..10FFFF16, which precisely matches the range of characters defined in the Unicode Standard (and other standards such as XML), and those representable by UTF-8 and UTF-16." (Unicode Standard Annex #19) don't you understand? Also note that UTF-32 is still "variable length" in some senses, in that it can still have combining characters that you need to interpret. > If you want to do Unicode-3 correctly and simply, then a 32 bit > internal representation is the right one... You have addressed none of the points I made in my previous when I said that one can do Unicode 3 correctly and simply in UTF-16. Please address them. > - surrogates simply recreate > exactly the same problems of variable-length encoding that plagued > earlier efforts at providing a simple way (for the programmer) to code > correctly. If you consider variable length a real problem; UTF-32 doesn't fix it since it still has combining characters. >Externally, a more compact encoding is needed (utf-8 or utf-16 > are valid choices), but internally, UTF-16 is bogus in the extreme. This is completely wrong. > You seem to be so wedded to the Java/Unicode model that you can't see > out of the hole into which you've dug yourself. No. I'm going by stuff out of the Unicode 3 standard here, not just the java model. If you'd actually work though some typical cases of string use and see what happens when they encounter surrogate pairs, you'd see that your analysis of the problem is not at all correct. cjs -- Curt Sampson +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC