From: Martin Duerst <duerst@...>
Date: 2008-01-04T15:12:00+09:00
Subject: Re: multibyte strings & bucket-of-bytes efficiency under 1.9.0

At 22:53 08/01/03, Austin Ziegler wrote:
>On Jan 3, 2008 1:04 AM, Martin Duerst <duerst@it.aoyama.ac.jp> wrote:

>> - Internally, there is usually something called UTF-16 which has
>>   machine byte order. There should also be explicit endianness versions,
>>   UTF-16BE and UTF-16LE.
>
>Right, but there needs to be a way for a program to programmatically
>tell which of the two (BE or LE) is actually used based on machine
>byte order. I realize that when work is done in PDF::Writer (or
>something similar) to enable proper Unicode output, we'll still
>probably need to call #force_encoding(Encoding::UTF16BE), but it'd be
>nice to not have to do that:
>
>   foo.force_encoding(Encoding::UTF16BE) unless foo.encoding ==
>Encoding::UTF16BE

I'm a bit confused here. First, force_encoding is a cheap method,
just setting the encoding information, not doing any actual transcoding,
so the unless part looks like overkill.

Second, assuming you just want to output your data, force_encoding
isn't necessary at present (it might be possible that in the future,
if you have declared that a file you write to is in a certain encoding,
you get an error unless your strings are also in that encoding, but
I don't think I have heard about such plans yet).

Maybe what you mean is that you actually want to transcode to
UTF-16-BE. In that case, you would write
   foo.encode('UTF-16-BE')
Again, the unless part should be unnecessary.

Regards,    Martin.


#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:duerst@it.aoyama.ac.jp