From: Richard Conroy <richard.conroy@...>
Date: 2006-11-15T21:40:20+09:00
Subject: Re: Charset detection

On 11/9/06, David Vallner <david@vallner.net> wrote:
> Miquel Oliete wrote:
> > Hi
> >
> > I'm coding a planet software in Ruby, like planet planet, but inserting
> > all rss data into a database (mysql now) and showing the entries from
> > database.
> >
> > I don't know how can I detect the rss charset. Can you help me?
> >
>
> Look for the XML header? That one should list encoding.

Shouldn't you be looking at the HTTP header instead/also?

Or just default to UTF-8 which *should* cover you anyway for 8859-1 loving
anglo-philes and most of the rest of the world. Though Japan can be a
bit native-charset-centric, especially the further you get from well-resourced
web sites (hobby sites etc.). There was a larger utf-8 burden-of-effort there,
whereas in the west being non-utf-8 is just pure laziness.