From: Hal Fulton <rubyhacker@...>
Date: 2012-10-28T03:40:57+09:00
Subject: #encode and (pre/de)composed characters

--f46d04182518c2bb7b04cd0ec6bb
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Why does encode to UTF-8 (in at least this one case) appear to favor a
decomposed character while encode to UTF-16LE favors a precomposed
character?

See below...

Thanks,
Hal


1.9.2p290 :009 > str =3D [233].pack("c*").force_encoding("ISO-8859-1")
 =3D> "\xE9"
1.9.2p290 :010 > s2 =3D str.encode("UTF-8")
 =3D> "=E9"
1.9.2p290 :012 > s2.bytes.to_a
 =3D> [195, 169]
1.9.2p290 :018 > s3 =3D str.encode("UTF-16LE")
 =3D> "\u00E9"
1.9.2p290 :019 > s3.bytes.to_a
 =3D> [233, 0]

--f46d04182518c2bb7b04cd0ec6bb
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Why does encode to UTF-8 (in at least this one case) appear to favor a<br>d=
ecomposed character while encode to UTF-16LE favors a precomposed<br>charac=
ter?<br><br>See below...<br><br>Thanks,<br>Hal<br><br><br>1.9.2p290 :009 &g=
t; str =3D [233].pack(&quot;c*&quot;).force_encoding(&quot;ISO-8859-1&quot;=
)<br>
=A0=3D&gt; &quot;\xE9&quot; <br>1.9.2p290 :010 &gt; s2 =3D str.encode(&quot=
;UTF-8&quot;)<br>=A0=3D&gt; &quot;=E9&quot; <br>1.9.2p290 :012 &gt; s2.byte=
s.to_a<br>=A0=3D&gt; [195, 169] <br>1.9.2p290 :018 &gt; s3 =3D str.encode(&=
quot;UTF-16LE&quot;)<br>
=A0=3D&gt; &quot;\u00E9&quot; <br>1.9.2p290 :019 &gt; s3.bytes.to_a<br>=A0=
=3D&gt; [233, 0] <br><br>

--f46d04182518c2bb7b04cd0ec6bb--