From: Julian 'Julik' Tarkhanov <listbox@...>
Date: 2006-06-19T04:32:45+09:00
Subject: Re: Unicode roadmap?


On 18-jun-2006, at 21:17, Christian Neukirchen wrote:

> Tim Bray <tbray@textuality.com> writes:
>
>>> You need glyphs, and some glyphs can be
>>> produced with multiple code points (e.g., LOWERCASE A + COMBINING
>>> ACUTE
>>> ACCENT as opposed to A ACUTE).
>>
>> This is another thing you need your String class to be smart about.
>> You want an equality test between "m叩s" and "m叩s" to always be true
>> even their "叩" characters are encoded differently.  The right way to
>> solve this is called "Early Uniform Normalization" (see http://
>> www.w3.org/TR/2003/WD-charmod-20030822/#sec-Normalization); the idea
>> is you normalize the composed characters at the time you create the
>> string, then the internal equality test can be done with strcmp() or
>> equivalent.
>
> Does that mean that  binary.to_unicode.to_binary != binary  is  
> possible?
> That could turn out pretty bad, no?

And it does as long as you are not careful. One of the things I do is  
normalize all that come IN
into something that is suitable and predictable.

--
Julian 'Julik' Tarkhanov
please send all personal mail to
me at julik.nl