From: James Gray <james@...>
Date: 2009-08-07T23:05:45+09:00
Subject: Re: R1.9 mixed encoding in file

On Aug 7, 2009, at 8:49 AM, Vít Ondruch wrote:

> Hello

Hello.

> I wonder if it is possible to enforce encoding of string in ruby 1.9.
> Let say I have following example:
>
> C:\enc>echo p 'test'.encoding > encoding.rb
> C:\enc>ruby encoding.rb
> #<Encoding:US-ASCII>
>
> Thats fine. But what if I like to have in single file ASCII, UTF-8 or
> strings with other encodings, i.e.
>
> C:\enc>echo p 'zufällige_žluťoučký'.encoding > encoding.rb
> C:\enc>ruby encoding.rb
> encoding.rb:1: invalid multibyte char (US-ASCII)
>
> I know that for this particular case I could use directive on top of  
> the
> file, but I would like to see something in following manner:
>
> String.new 'zufällige_žluťoučký', Encoding.CP852
>
> It means read the content in between quotes binary and interpret it
> according to specified encoding.

The problem with an idea like this is that before your String is ever  
created the code to create it must be read (correctly) by Ruby's  
parser and formed into a proper String literal.  That would be  
impossible to do if String literals could be in any random Encoding.

You have a couple of options though:

* Just set an Encoding like UTF-8 for the source code, enter  
everything in UTF-8, and transcode it into the needed Encoding.  This  
would make your example something like:

   # encoding: UTF-8
   cp852 = "zufällige_žluťoučký".encode("CP852")  # literal in  
UTF-8

* Have one or more data files the program reads needed String objects  
from.  Those files can be in any Encoding you need and you can specify  
it to IO operations, so your String objects are returned with that  
Encoding.

I hope that helps.

James Edward Gray II