From: Brian Candler <b.candler@...>
Date: 2009-02-24T23:47:31+09:00
Subject: Re: encoding problem with tr() and hash keys

Florian Gilcher wrote:
> Then report the underlying problem.

IMO, the underlying problems are that ruby 1.9's concept of Strings is 
(a) not properly thought-out and (b) totally undocumented. I don't think 
a ticket on redmine would be appreciated on either count.

The idea that a string should carry along its encoding sounds great in 
principle, but dozens of questions come out from that: even simple ones 
like "what happens if I concatenate two strings with different 
encodings?" are not answered without experimentation. And then you start 
to uncover the rules about "compatible" encodings, automatic switching 
between some character sets and US-ASCII, which happens some times but 
not others, or is hidden away:

#encoding: UTF-8
a = "a"
p a.encoding       #<Encoding:UTF-8>
p "#{a}".encoding  #<Encoding:UTF-8>
p /#{a}/.encoding  #<Encoding:US-ASCII>

-- Ruby could have decided up-front that this string was US-ASCII, but 
didn't. Except that it magically carries along some hidden knowledge 
that this particular string, whilst declared to be UTF-8, is also 
'compatible' with US-ASCII. Presumably, if you mutate it to include 
special characters, this compatibility is lost, and it you mutate it 
again to remove them, it is restored.

But it's worse than that. With 1.8, someone could post a Ruby script on 
this mailing list, and I'd have high confidence that it would work 
exactly the same if I ran it on my machine against the same data. In 
1.9, all sorts of factors inherited from your environment may make the 
program either behave differently, or indeed crash, on one machine but 
not the other. Plenty of examples have been posted on this list. It 
makes test coverage very hard to achieve. You need to defend against it. 
In other words: it bites.
-- 
Posted via http://www.ruby-forum.com/.