From: niklas | brueckenschlaeger <niklas@...>
Date: 2010-11-18T01:15:22+09:00
Subject: Re: Extended ASCII character handeling

--=-xrkJMnNHOl9MjBFua5ax
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2010-11-18 at 01:01 +0900, Don Norcott wrote:
> "200 Millionen Jahre sp=C3=A4ter                              # 17.39
> \n",
> "200 Millionen Jahre sp=C3=A4ter                              #  9.87
> 3404211707    \n",
> "A l'assaut de l'invisible                         1977 #  4.91
> \n",
> "A l'assaut de l'invisible                         1990 #  5.18
> 226603779      \n",
>=20
> The above 4 lines are data I was attempting to load into an array to
> test some code.  I was getting what I thought were strange results until
> I realized not all characters were being loaded into the element
> resulting in column alignment problems.
>=20
> The data above was cut from a file that had been manipulated a dozen
> times in ruby arrays before being written to a file.  So it appears the
> default way ruby handles extended ASCII(?) is fine.
>=20
> I have two questions
> 1) Should I ever have to worry about data being scraped from web pages
> not being handled correctly by ruby.

That would depend very much on how you scrape the data and if you handle
stuff like meta tags correcty.

>=20
> 2)How do I flag this data to allow me to manipulate it properly. That is
> load it into an array or write to a file.
>=20
> Tried playing with the following but even if the code below is correct
> the extended ascii characters are lost by the time it gets to IRB
>=20
> str =3D String.new
> str.encode(("US-ASCII")
> str =3D "Millionen Jahre sp=C3=A4ter"
>=20
> Any suggestions where I might find some insight.
>=20
> Thanks Don
>=20


--=-xrkJMnNHOl9MjBFua5ax
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQIcBAABAgAGBQJM4/+NAAoJEIQdQGZOjo82Us0P+wVSA2BG4MHFGiQOALe0yMe2
n36RLWuaqTyBS4du99X2N5MPYSkwfXhgYWQHF4O3SCuBlhnmuRJm+Sg4sp99tfL2
WnYuMlPCR7CnGujDQH0C2/4hbHG4p3jJ0sXEZVKf7GKUKZmuGXPAwop0tVW3MscH
NsSXBq3STj0OUSfaCuzBKr3QkEzs7xyT5FlHodxDIbTA7WosdUsFkjh74j8ulnmO
3P1AY8Dm2EepsXdTevGxL8FMtCaorCuoa9mlkNoFoXO3lMDFZqqCspPlTErnxwdN
4JKdaD6KrFuz8Y8ZHbawoCMorRbHJ63vCBaFogjc48VjRt+e9fJLSLYvp8dyv22K
Fe3mAGaAm1BFKvkqWSBqTbluU1Nny0uIu++kzDYy6d6Nejm/ZuCbnWLygYn2lTXH
sTN3c2rtr5CEu4Bp+v164/TgnJ7q7cK5hDe8kh3yMkfLfzQTS5JAATnIe3cvTR5F
eUEPh5jtoN3tWTzHZt6Sft7Od5H/cGFb/4Yo6ALaGxy7j2lGnOPJ1loO0F1sFQOK
On+whCpHv6vIB2Bmw5Mr8s3/PPC2hzkuc3DB+Fd7Ra0300iMvLJ/73RK9DEg02vj
OAuKr2QWB1NpYvrtxoWHAf17fTH4kYtifeMJ8dmtHbq4EXsZAHz2O8wFUMzgr9c0
zzcwL2H8/BJE9PkQ/Pbb
=tTcr
-----END PGP SIGNATURE-----

--=-xrkJMnNHOl9MjBFua5ax--