From: Mr Peepers <mrpeepers@...>
Date: 2010-09-27T04:08:22+09:00
Subject: Re: Ruby, Unicode, and HTML Entities Problem

Ok, get the Ruby and a sample of the data attempting to be loaded here, 
www.khourys.com/Archive.rar.

Regarding 150340.html, as just an example.


I'll walk you through my troubleshooting quickly.


1) If I replace this text (where it fails):
<em>Ministerio P炭blico de la Federaci坦n</em>

with this:
<em>Ministerio P炭blico de la Federaci坦nnn</em>

FILE LOADS!


2) So I'm thinking, did this one o-acute get corrupted?  Nope.  But, if 
you remove this sentence:
Whenever the Federal Public Ministry (<em>Ministerio P炭blico de la 
Federaci坦n</em>) investigates the activities of organized crime members 
that deal with goods of illicit origin, the investigation is carried out 
with the assistance of the Secretariat of Finance and Public Credit 
(<em>Secretar鱈a de la Hacienda y Cr辿dito P炭blico</em>).

FILE LOADS!


3) I replaced this:
<em>Ministerio P炭blico de la Federaci坦n</em>
with this
Ministerio P炭blico de la Federaci坦n
(removed the <em>'s)

FILE LOADS!


Now I'm thinking, hhhmmmm... (actually, WTF)  Could the unicode 
character so close to the end HTML tag be causing the issue, 
potentially?

4) To test my theory, I'll modify these lines together in the file:

Before:
<em>Ministerio P炭blico de la Federaci坦n</em>
<em>Secretar鱈a de la Hacienda y Cr辿dito P炭blico</em>

After
<em>Ministerio P炭blico de la Federaci坦nnnn</em>  <-- this to prove it's 
not JUST this line
<em>Secretar鱈a de la Hacienda y Cr辿dito P炭</em> <-- this to prove that 
when a unicode character is so close to an HTML entity - we're screwed.


FILE BOMBED!

Think we have the culprit?
-- 
Posted via http://www.ruby-forum.com/.