From: Ned Konz <ned@...>
Date: 2001-08-01T23:00:42+09:00
Subject: [ruby-talk:18957] Ruby and Unicode

I haven't dug into the source yet, but it seems that Ruby ignores the fact 
that the native storage of filenames under Windows/NT is in Unicode. That is, 
I don't see any definitions of UNICODE in the source that would otherwise be 
needed to deal with filenames as Unicode.

So how would you get the first character (as opposed to first byte) of a 
filename you read from a directory? I would hope that the fact that filenames 
are Unicode would be transparent, and you could just ask for the first 
character and you'd get what you expected.

I haven't seen (yet) any discussion of so-called "internationalization" 
issues in the Pickaxe book. And my English Ruby documentation doesn't mention 
anything in the documentation of String other than that they're arbitrary 
length byte strings.

I'm not sure what the -K command line option does. Does it enable Unicode 
behavior? (note that JIS, etc. are irrelevant from a Win32 API point of view).

This is more of an issue under WinCE, where you don't have a set of optional 
ASCII wrappers around the Unicode functionality like you do in Windows NT. In 
WinCE, _everything_ is in Unicode (filenames, registry keys and string 
values, error strings...).

Do Strings view themselves as sequences of bytes or as sequences of 
characters? Smalltalk makes a distinction between String/Character and 
ByteArray/byte.

Since Ruby is popular in Japan, I'm sure they don't just throw away 
information (what happens when you have filenames in Japanese?). But I don't 
see (yet) how this distinction is made.

I would think that you should be able to associate a particular 
coding with IO streams, as well; if you have a Unicode or Shift-JIS 
file, you should be able to read it a character (as opposed to a byte) at a 
time, right?

-- 
Ned Konz
currently: Stanwood, WA
email:     ned@bike-nomad.com
homepage:  http://bike-nomad.com