From: Thomas Thomassen <thomas@...>
Date: 2009-07-09T01:59:13+09:00
Subject: Re: Ruby 1.8 - character encoding

James Gray wrote:
> On Jul 8, 2009, at 11:42 AM, Thomas Thomassen wrote:
> 
>>>
>>>> Why does File.exist?('c:\Test ���') fail if it's UTF-8  
>>>> encoded?
>>>
>>> The IO methods are not $KCODE aware.  You will likely need to
>>> transcode the Strings you pass them.
>>>
>>> James Edward Gray II
>>
>> What does the IO method require?
> 
> That's a good question.  I'm not sure what it does on Windows.
Any clues what I does on OSX? The scripts will run on macs as well.

> 
>> Is it the Ruby IO methods or the system methods it calls that  
>> doesn't handle UTF-8?
> 
> I assume it's the underlying Windows API, though I'm just guessing
> there.
> 
>> Windows' NTFS format supports UTF-16 encoding - would it work if I
>> transcoded the strings from UTF-8 to UTF-16?
> 
> I think it depends on which API methods you call, so I'm guessing you
> cannot do this.  I think Ruby would need to be changed to use those
> methods first.
Since NTFS supports UTF, then I guess it's the Ruby API that calls the 
wrong WinAPIs?
Can I make my own API calls?


>> I'm trying to avoid transcoding to a 8bit only encoding as that'll  
>> just
>> cause grief when I encounter characters outside the range.
> 
> Have you had a look at Ruby 1.9 yet?  I'm wondering if this issue has
> been improved there, using the new encoding support.  I don't know
> that it has.  I'm more just wondering out-loud…
> 
> James Edward Gray II

The scripts I write is plugins for Google Sketchup - so the Ruby version 
I have at disposal is the one Sketchup bundles - a partial 1.8 version.
While I've been searching for solutions I've noticed that v1.9 have 
better support for various encoding, but unfortunately it's of no use 
for me.

So my problem is that I have to deal with string data that comes from 
Sketchup in UTF-8 format - might even have to deal with files and folder 
that include characters outside the Windows1252 or ISO8859 range 
(whatever the IO functions are using - I've not been able to pin-point 
this.). If I get characters outside that range it's impossible to 
transcode.
Andd, I also don't know what would happen for an eastern user. I'm 
wondering if the IO functions would assume a different 8bit encoding...
-- 
Posted via http://www.ruby-forum.com/.