From: Eric Hodel <drbrain@...7.net>
Date: 2009-08-07T09:39:47+09:00
Subject: Re: [ANN] 1.9 String and M17N documentation

On Aug 6, 2009, at 08:57, James Gray wrote:

> * You say that m17n's complexity can be avoided if we just used  
> UTF-8 everywhere and transcoded incoming and outgoing data.  I  
> agree.  If we do that in Ruby 1.9 though, transcode all data as it  
> comes in and just work with UTF-8 internally, doesn't all the  
> complexity of m17n go away?  Compatible encodings, the comparison  
> order of differing encodings, and the like will all be non-issues.   
> Thus it seems to me that m17n allows us to take this favored  
> approach or take harder roads, if we so choose.

I'm too lazy to dig this out of the archives, but there are some  
encodings that don't have a 1:1 mapping to Unicode thus the round-trip  
through UTF-8 (etc.) will destroy them.

In short, Ruby doesn't transcode everything to preserve the integrity  
of your data.