From: Brian Candler <b.candler@...>
Date: 2010-09-27T17:40:06+09:00
Subject: Re: Encoding problems .. ruby 1.9.2

Bhay Zone wrote:
> I am pretty new to ruby and am trying to read text data coming from a
> backend which can only be queried using proprietary Command Line
> Interface commands.
> 
> The problem is that this text data contains non-ascii characters...I
> don't know what these characters are .. and nor do I know the encoding.

How are you interfacing with this interface - a TCP socket? IO.popen? 
Backticks? Something else? If you show the code which opens the 
connection, we can show how to fix it.

TCP sockets default to "ASCII-8BIT" encoding, but for other methods, 
unless you tell ruby what encoding to use, it will guess based on 
environment variables on your PC. That is, the same program may work 
fine on one PC but fail on another.

To avoid these problems, there are magic incantations you can add to 
force ruby not to guess. e.g.

IO.popen: add "b" to the mode string

Backticks or %x: res = `foo`; res.force_encoding("ASCII-8BIT")

Or try running ruby with -Kn flag.

> I just cannot figure how to fix this problem and any help would be
> greatly appreciated.

It's probably possible to fix your code, as above. However, sticking 
with ruby 1.8.7 is also a reasonable solution if you don't want to have 
to deal with this sort of nonsense.

I had a go at reverse-engineering the string encoding behaviour of ruby 
1.9. I gave up after documenting about 200 behaviours:
http://github.com/candlerb/string19/blob/master/string19.rb

I'm sticking with 1.8, because 1.9 makes my brain hurt.
-- 
Posted via http://www.ruby-forum.com/.