From: Brian Candler <b.candler@...>
Date: 2008-09-17T22:33:18+09:00
Subject: Re: How to clean an xml files from non-utf-8 chars?

Rob Biedenharn wrote:
> On Sep 17, 2008, at 5:15 AM, Brian Candler wrote:
> 
>> If you really don't care about the content:
>> str.gsub(/[\x80-\xff]/,'?')
>> --  
> 
> 
> You can have bytes in that range as the first byte of a well-formed
> UTF-8 Byte Sequence.  They just can't represent a single byte.  It's
> just not that simple.

That's why I said "if you really don't care" ... it strips all valid 
non-ASCII UTF8 as well as invalid.

There is a nice table at http://en.wikipedia.org/wiki/UTF-8 which would 
let you build something more accurate. Ruby quiz perhaps? :-)
-- 
Posted via http://www.ruby-forum.com/.