Re: Upgrading to Ruby 1.9 - Should I? Is it painful?

On Apr 18, 2012, at 12:46 AM, Brian Candler wrote:

> Bartosz Dziewo=C5=84ski wrote in post #1056782:
>> 2. Always immediately convert any incoming data to UTF-8. Sensible
>> frameworks should already do it for you, but mind it if rolling your
>> own. Use code like File.open('asd.txt', 'r:utf-8') if necessary. =
(E.g.
>> if it doesn't work without the utf-8 part.)
>=20
> Note: that doesn't actually "convert incoming data to UTF-8". It just=20=

> *labels* incoming data as being UTF-8, whether it is or not. If you =
want=20
> to convert data, you have to provide both the target and original=20
> encodings, e.g.
>=20
> File.open('asd.txt', 'r:utf-8:iso-8859-1')
>=20
> The fundamental point with ruby 1.9 is that every String is a=20
> two-dimensional vector of bytes + encoding. There are some =
circumstances=20
> in which Strings which contain invalid characters (according to their=20=

> declared encoding) work, and others in which they don't. There are =
some=20
> circumstances in which Strings with different encodings can be =
combined,=20
> and others when they can't.
>=20
> Have fun trying to understand it.

The fundamental problem is that you can only guess what the external =
world
looks like as input streams have no encoding. The problem exists in Ruby =
1.8=20
as well - its just not as visible.

In practice though, I rarely have any problems, only when dealing
with systems that are messy with respect to encoding in the first place.

Best regards,
Florian=