From: James Gray <james@...>
Date: 2008-12-19T00:26:55+09:00
Subject: [ruby-core:20653] Re: 1.9 character encoding (was: encoding of symbols)

On Dec 18, 2008, at 4:50 AM, Daniel Cavanagh wrote:

> On 18/12/2008, at 7:59 PM, Brian Candler wrote:
>
>> On Thu, Dec 18, 2008 at 06:22:28PM +0900, Daniel Cavanagh wrote:
>>> and you're against early conversion (ie. at I/O)
>>
>> That's what ruby-1.9 does out-of-the-box. e.g. if you set the  
>> external
>> encoding to ISO-8859-1, and the internal encoding to UTF-8, then your
>> program will see the source as if it were a stream of UTF-8.
>
> michael selig said "Ruby could have followed the Python route of  
> converting everything to Unicode, but that was rejected for various  
> good reasons. Also automatic transcoding to solve issues of  
> incompatible encodings was also rejected because it causes a number  
> of problems, in particular I believe that transcoding isn't  
> necessarilly accurate, because for example there may be multiple or  
> ambiguous representations of the same character."
>
> i was taking that to mean that ruby will not be doing automatic  
> conversion to one encoding. perhaps he just meant by default and  
> that the option is there if wanted

And his very next paragraph in the message you are quoting from was:

   What *was* introduced is the concept of a "default_internal"
   encoding, which, if used by the programmer, causes I/O and
   other interfaces to transcode to the internal encdoing on
   input & the opposite on output.

James Edward Gray II