From: ctdev <ctdev421@...>
Date: 2011-03-31T01:15:27+09:00
Subject: Re: Encoding issues when parsing HTML in 1.9

> Okay, try specifying that encoding when you parse it with Nokogiri?

I resolved this problem by opening and rewriting the original files
with a specified mode as described in Overbryd's answer:

http://stackoverflow.com/questions/951891/how-can-i-convert-a-string-from-windows-1252-to-utf-8-in-ruby

So:

old = File.open("old", "r:windows-1252:utf-8")
new = File.open("new", "w+:utf-8") {|f| f.write(old.read)}

Everything works now. The characters were all converted and I was able
to remove the encoding directive and non-breaking space literals from
my script by using '\u00A0' in the regex I'm passing to the split
function.

Thanks for the help. :)