From: "Gonzalo Garramuño" <ggarra@...>
Date: 2007-12-30T02:29:35+09:00
Subject: Re: [Spam] Re: Wide strings to ruby strings

Austin Ziegler wrote:
> 
> If you treat it as UTF-16, IConv can handle it. The problem will be
> when it's *not* actually UTF-16.
> 
> -austin

Thanks, Austin and Tanaka.

I have no experience in multi-byte character handling, so bear with me.

Could you give me a simple example of a wchar_t not being utf-16 (or 
some function returning a wchar_t that's not utf16) or the composite 
breaking?  Can this be detected so that if the decoding fails, the 
string is treated just as an array of bytes?

Would there be any benefit between what Austin and Tanaka proposed (ie. 
iconv directly vs. going thru wcsrtombs first)?

Should I treat wchar_t by default as just an array of bytes then and let 
the user then handle the decoding himself?

The other final question is something that was already asked in a 
previous thread.  That is, who is responsible for setting the locale 
correctly?  The extension, iconv or ruby itself.

NOTE:  this is to handle swig's defaults, which is, hopefully, the most 
common case scenario.  Obviously if someone is using wchar_t with some 
non-standard date, she can still write special typemaps and conversion 
functions for those cases.

P.S.  As far as I can tell, python seems to treat wchar_t as utf-16. 
And other special functions are provided to handle mbs.


-- 
Gonzalo Garramu�o
ggarra@advancedsl.com.ar

AMD4400 - ASUS48N-E
GeForce7300GT
Xubuntu Gutsy