From: Paul Battley <pbattley@...>
Date: 2006-06-14T17:54:33+09:00
Subject: Re: Unicode roadmap?

On 14/06/06, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:
> Windows 32 path encoding is a nightmare.  Our Win32 maintainers often
> troubled by unexpected OS behavior.  I am sure we _can_ handle Russian
> path names, but we need help from Russian people to improve.

str.sub!('32 path encoding ', '') # :-)

I don't use Windows much, but as I understand it, Ruby interacts with
most of the Win32 API using the 'legacy code page', which is only a
subset of what the filesystem can handle. (Windows NT and its
successors use Unicode internally, and the filesystem is UTF-16
KC-normalised IIRC). Windows does provide Unicode API functions, but
to use those, a layer of translation between UTF-16 and UTF-8 would be
needed, as Ruby can't do anything useful with UTF-16 at present. I
believe that Austin Ziegler was looking into this; I don't know if
he's made any progress.

Even if a Ruby program uses UTF-8 internally, it should be possible to
access the filesystem by Iconv'ing paths to the appropriate code page
- providing that they don't contain characters not in the code page.
It's far from ideal, though: the real solution is for Ruby to use the
Unicode functions (those suffixed with W) in the API. The upside is
that UTF-8/UTF-16 conversion should be less expensive than the code
page conversion that's inside each of Win32's non-Unicode functions.

On the other hand, plenty of Windows programs don't support Unicode
properly either.

Paul.