From: Brian Candler <lists@...>
Date: 2012-04-18T07:46:49+09:00
Subject: Re: Upgrading to Ruby 1.9 - Should I? Is it painful?

Bartosz Dziewoński wrote in post #1056782:
> 2. Always immediately convert any incoming data to UTF-8. Sensible
> frameworks should already do it for you, but mind it if rolling your
> own. Use code like File.open('asd.txt', 'r:utf-8') if necessary. (E.g.
> if it doesn't work without the utf-8 part.)

Note: that doesn't actually "convert incoming data to UTF-8". It just 
*labels* incoming data as being UTF-8, whether it is or not. If you want 
to convert data, you have to provide both the target and original 
encodings, e.g.

File.open('asd.txt', 'r:utf-8:iso-8859-1')

The fundamental point with ruby 1.9 is that every String is a 
two-dimensional vector of bytes + encoding. There are some circumstances 
in which Strings which contain invalid characters (according to their 
declared encoding) work, and others in which they don't. There are some 
circumstances in which Strings with different encodings can be combined, 
and others when they can't.

Have fun trying to understand it.

-- 
Posted via http://www.ruby-forum.com/.