From: Mark Thomas <mark@...>
Date: 2008-09-17T21:42:52+09:00
Subject: Re: How to clean an xml files from non-utf-8 chars?

On Sep 17, 5:07 am, Krzysieq <krzys...@gazeta.pl> wrote:
> [Note:  parts of this message were removed to make it a legal post.]
>
> Hi,
>
> I have a problem. I'm trying to parse with ruby some test results from
> jmeter, that are stored in xml files. Unfortunately, while they should be
> utf-8, some of them aren't. Probably because some db data isn't. In any
> case, this makes other toys break down, like xslt transformation and
> anything else that relies on the xml files being utf-8.

Look at http://www.botvector.net/2007/11/encoding-problems.html,
particularly the "iconvert" method which attempts conversion to UTF-8,
but in the case where the string cannot be converted to UTF-8 (e.g.
double-byte chars) then it replaces the chars with "?".

-- Mark.