From: Julian 'Julik' Tarkhanov <listbox@...>
Date: 2006-08-01T02:15:03+09:00
Subject: Re: state of unicode support


On 31-jul-2006, at 18:51, Paul Battley wrote:

> On 31/07/06, Julian 'Julik' Tarkhanov <listbox@julik.nl> wrote:
>>
>> On 31-jul-2006, at 17:48, Paul Battley wrote:
>> > Whilst it's certainly useless for a lot of tasks, I'm not sure that
>> > Ruby is any worse than other languages in this regard. As far as  
>> I'm
>> > aware, most languages that 'support' Unicode don't handle grapheme
>> > clusters without using additional libraries.
>>
>> AFAIK Python regexps do that properly, and ICU does for sure (both as
>> free iterators and regexps).
>
> That's what I mean: ICU is a separate library, not part of a language
> core.

PHP took the best of both - they are integrating ICU into the core.  
Although I always hated
their tendency to bloat the core, this is one of the cases of bloat  
that I would want to applaud as a gesture
of sanity and common sense.

> We can use ICU in Ruby too - it's still pre-alpha and not
> seamless, but the possibility exists.

Except from the fact that the maintainer has abandoned it and nobody  
stepped in. I don't do C.

> From what I've read, Python
> doesn't do the heavyweight stuff natively, either. (Please tell me if
> I'm wrong - I don't use Python.)

It depends on what you call "heavyweight". For the purists out there,  
I gather, even including a complete Unicode table with
codepoint properties might be "heavyweight".

>>
>> To my knowledge you are intimately familiar with the subject so I
>> take it as sarcasm.
>
> I'm not being sarcastic at all, though perhaps I could have phrased it
> better. It's just that all Unicode discussions in Ruby end up going
> round and round in circles; if we as a community could identify some
> first-class examples of Doing It Right, I think we'd have some useful
> yardsticks.

The problem being, my "Right Examples" are nowhere near other's  
"Right Examples", which in turn supurs flamewars.
My "right example" is simple - Unicode on no terms, no encoding  
choice, characters only - but most already are dissatisfied with such
an attitude and the issue has been discussed in detail, with no  
solution satisfying all parties being devises. Too much compromise.

> You are someone with particularly high expectations
> (rightly so) of Unicode support in a language: have you found anything
> that really impressed you?

ICU in all it's incarnations (Java and C), compulsory character- 
oriented Strings without choice of encoding in Java and the upcoming  
Unicode support in Python (again - compulsory Unicode for all  
strings, byte arrays for everything else). Perl's regex support. I  
know everyone will disagree (how do I match a PNG header in a  
string???) but that's what I consider good.

As to localization - resource bundles are good, and of course I  
consider all languages that _did_ bother to print localized dates.  
Shame on Ruby.
>
>> But if you really feel like being constructive you can update the
>> Unicode gem (wich you promised about a month ago) :-))
>
> I promised I'd try :-) Thanks for the reminder, though! I'll get on  
> with it.

Gotcha :-)

-- 
Julian 'Julik' Tarkhanov
please send all personal mail to
me at julik.nl