From: MikkelFJ <mikkelfj-anti-spam@...>
Date: 2003-03-10T23:44:59+09:00
Subject: Re: What character sets are available in Ruby ?


"pj" <peterjohannsen@hotmail.com> wrote in message
news:127ce4a9.0303100359.3291cffb@posting.google.com...

> I think you're addressing identifier parsing -- am I getting that right ?

Yes I was mostly discussing identifier parsing, sorry if I wasn't clear on
that. However, the technique works generally - you can easily parse a string
in quotes the same way. This works because quotes are never part of other
characters than quotes themselves (because ASCII char always mean ASCII char
in UTF-8).

> I think UCS-2 was an unfortunate, short-sighted mistake that would
> probably be dead and gone, except that Microsoft invested heavily

There was a very long discussion in comp.lang.ruby a while back.

I think UCS-2 works fine in praxis because some it mostly works, and when it
doesn't, a character just takes up more space with the locations easily
identified if you are concerned with non-fixed length issues. However, given
that it isn't truly fixed length I find UTF-8 the best external
representation whereas UCS-2 or UCS-4 are sometimes easier to process
internally. UTF-8 is also a great external format because it is so easy to
parse - as I just explained.

Mikkel