From: "Bartosz Dziewoński" <matma.rex@...>
Date: 2012-04-18T17:17:03+09:00
Subject: Re: Upgrading to Ruby 1.9 - Should I? Is it painful?

W dniu 18 kwietnia 2012 00:46 użytkownik Brian Candler
<lists@ruby-forum.com> napisał:
> Bartosz Dziewoński wrote in post #1056782:
>> 2. Always immediately convert any incoming data to UTF-8. Sensible
>> frameworks should already do it for you, but mind it if rolling your
>> own. Use code like File.open('asd.txt', 'r:utf-8') if necessary. (E.g.
>> if it doesn't work without the utf-8 part.)
>
> Note: that doesn't actually "convert incoming data to UTF-8". It just
> *labels* incoming data as being UTF-8, whether it is or not. If you want
> to convert data, you have to provide both the target and original
> encodings, e.g.
>
> File.open('asd.txt', 'r:utf-8:iso-8859-1')

Yes, thank you for correction. My point still stands.


> The fundamental point with ruby 1.9 is that every String is a
> two-dimensional vector of bytes + encoding. There are some circumstances
> in which Strings which contain invalid characters (according to their
> declared encoding) work, and others in which they don't. There are some
> circumstances in which Strings with different encodings can be combined,
> and others when they can't.

That's unfortunately extremely true. Keeping *everything* UTF-8 makes
all of these problems disappear.

-- Matma Rex