From: James Gray <james@...>
Date: 2009-07-09T01:47:57+09:00
Subject: Re: Ruby 1.8 - character encoding

On Jul 8, 2009, at 11:42 AM, Thomas Thomassen wrote:

> James Gray wrote:
>> On Jul 8, 2009, at 11:30 AM, Thomas Thomassen wrote:
>>
>>> I checked the $KCODE variable and it returns "UTF8".
>>>
>>> Now, what does that do to Ruby?
>>
>> I answer that question in detail in this article:
>>
>> http://blog.grayproductions.net/articles/the_kcode_variable_and_jcode_library
>>
>>> Why does File.exist?('c:\Test ���') fail if it's UTF-8  
>>> encoded?
>>
>> The IO methods are not $KCODE aware.  You will likely need to
>> transcode the Strings you pass them.
>>
>> James Edward Gray II
>
> What does the IO method require?

That's a good question.  I'm not sure what it does on Windows.

> Is it the Ruby IO methods or the system methods it calls that  
> doesn't handle UTF-8?

I assume it's the underlying Windows API, though I'm just guessing  
there.

> Windows' NTFS format supports UTF-16 encoding - would it work if I
> transcoded the strings from UTF-8 to UTF-16?

I think it depends on which API methods you call, so I'm guessing you  
cannot do this.  I think Ruby would need to be changed to use those  
methods first.

> I'm trying to avoid transcoding to a 8bit only encoding as that'll  
> just
> cause grief when I encounter characters outside the range.

Have you had a look at Ruby 1.9 yet?  I'm wondering if this issue has  
been improved there, using the new encoding support.  I don't know  
that it has.  I'm more just wondering out-loud…

James Edward Gray II