From: Austin Ziegler <halostatue@...>
Date: 2007-12-30T16:22:35+09:00
Subject: Re: [Spam] Re: Wide strings to ruby strings

On Dec 29, 2007 6:41 PM, Michal Suchanek <hramrach@centrum.cz> wrote:
> wchar_t is typically utf-16 or utf-32. However, as far as I know it is
> defined to be "implementation specific wide character representation"
> (that is, the encoding is undefined).

Um. I was going to disagree with you, but the following seems to
agree: http://www.delorie.com/gnu/docs/glibc/libc_87.html -- the
situation is both better and worse than I thought. So, Linux (GNU
libc), OS X, and *BSD seem to use a 4 byte wchar_t, meaning there's no
chance of surrogate chomping. You might still mess up composites
(remember that � can be represented as LOWERCASE E WITH ACUTE ACCENT
or COMBINING ACUTE ACCENT + LOWERCASE E). Windows uses a 16-bit
wchar_t, which means that it's UTF-16 and has the possibility of
munging surrogates if you're not careful.

Of course, it's definitely possible to have wchar_t that isn't in a
standard Unicode encoding, as you noted. In that case, it's almost
impossible to do anything useful.

A quick search of Google suggests that ISO C99 seemed to standardize
some of this, but we don't seem to see much ISO C99 yet.

-austin
-- 
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * http://www.halostatue.ca/feed/
               * austin@zieglers.ca