From: Stephen Boisvert <stephen.boisvert@...>
Date: 2008-08-29T02:19:19+09:00
Subject: Re: Decode/encode Unicode

On Thu, Aug 28, 2008 at 9:51 AM, Kless <jonas.esp@googlemail.com> wrote:
> How to decode  a String type to Unicode?
> And, to encode Unicode to String?
>
>

This is a trickier question than you probably realize.
You might need to know what encoding you want to transform your string
to and from.  Sometimes you may not know - mp3 ID3 tag info is
supposed to be UTF8 for example but often isn't.  If you don't specify
an encoding it may or may not end up being encoded based on your
locale settings.

Things I have seen mentioned:

iconv - a unix based library for translating between different
encodings.  Requires you tell it what you decoding from and encoding
to.
http://wiki.rubyonrails.org/rails/pages/iconv

unidecode  - I have used this translate Unicode strings to ASCII by
simple character mapping- it works about 99% of the time so you will
need error handling for when it goes bonk.

Setting KCODE:

$KCODE = 'UTF8'
or
$KCODE = 'u'
or
use -Ku at the command line.  This tells Ruby that you want to be
using UTF8 encoding

require 'jcode'

will give you access to some code developed to deal with Japanese
Unicode encoding that address some of the problems with the String
class struggles with Unicode chars like jlength. Always keep in mind
that a lot of the string methods in Ruby do not work properly with
Unicode because they count letters wrong.

Good luck.

Stephen Boisvert
http://blog.ennuyer.net