From: darren kirby <bulliver@...>
Date: 2006-06-25T05:36:22+09:00
Subject: Re: ascii representation of unicode string?

--nextPart1548999.R6eACMbffL
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

quoth the Paul Battley:
> Sorry, I translated that code from somewhere else, but forgot that
> UTF-16 needs an even number of bytes. The fact that it worked as
> advertised was serendipity rather than good judgement! The trouble
> with Iconv's //IGNORE flag is that it doesn't ignore trailing errors;
> you can get around this by adding a valid codepoint at the end, and
> removing it after conversion. Adding a valid byte <128 gets around
> this for UTF-8 input, but only worked for your example as it had an
> odd number of input bytes. For UTF-16 (LE or BE) without surrogates,
> this will work:
>
> t =3D ic.iconv(s[0,s.length/2*2])
>
> although a more general solution that should also handle surrogates is
> this:
>
> t =3D ic.iconv(s[0,s.length/2*2]+"\000\000")[0..-2]

This  ^^^  is working perfect for all my test files now...thank you.

> Finally, your input string has a trailing null; a regexp-based
> solution is probably the most reliable way to remove this:
>
> t.sub!(/\x00$/, '')
>
> > I wonder though, the docs lead me to believe the iconv library is UNIX
> > only. Is this true? I really need a cross-platform solution, but don't
> > have a win32 box to try on...
>
> It's definitely possible to use iconv on Windows, but it wasn't in the
> one-click installer until 1.8.4, I believe.

Ok, good. I can live with that.

Thank you very much for the help,

> Paul.
=2Dd
=2D-=20
darren kirby :: Part of the problem since 1976 :: http://badcomputer.org
"...the number of UNIX installations has grown to 10, with more expected..."
=2D Dennis Ritchie and Ken Thompson, June 1972

--nextPart1548999.R6eACMbffL
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQBEnaIlwPD5Cr/3CJgRAtyMAKCvNgdTM60IZXK1fnWO7h4FwCUqDACbBRUY
HPQmU8lbvr/YXKPngfS8tbA=
=iVsM
-----END PGP SIGNATURE-----

--nextPart1548999.R6eACMbffL--