From: Tobias Reif <tobiasreif@...>
Date: 2001-10-05T05:56:17+09:00
Subject: [ruby-talk:22099] Re: writing UTF-8  strings

from http://groups.yahoo.com/group/svg-developers:
robin@knowscape.com:

On Thursday 04 October 2001 18:23, Peter Sorotokin wrote:

 > I have not gotten to Ruby yet, but if it is like other scripting
 > languages, its strings are more like a chunks of (possibly
 > binary) data, so they do not have inherent notion of encoding.
 > They can hold whatever you put in them, and normally that would
 > be locale-encoded string (but that depends on what you
 > take as input).

That's still true, though it is quickly changing. In Perl for instance 
it is no longer the case as strings know whether they're UTF-8 or locale 
encoded, and where streams can be gotten from whatever encoding (the 
filtering of which is supported) and accessed as UTF-8 (or binary). I 
heard Python has that too (or will have soon).

Given it's relative newness I'm surprised that Ruby doesn't deal with
encodings gracefully. While it was a PITA to reverse-engineer it into 
"older" languages, integrating it from scratch shouldn't be too hard.

[...]

My personal advice to anyone that works with XML: store
*everything* as UTF-8 (or 16 if you prefer). Even if it doesn't look 
like it'll need it. Even if it's only English. Otherwise the day one 
foreign name, or even one (c) falls in, you won't get burnt. I must have 
gotten burnt a thousand times already, and I pay lots of attention to it.

Tobi

-- 
Tobias Reif
http://www.pinkjuice.com/myDigitalProfile.xhtml

go_to('www.ruby-lang.org').get(ruby).play.create.have_fun
http://www.pinkjuice.com/ruby/