From: why the lucky stiff <ruby-talk@...>
Date: 2006-07-14T02:58:10+09:00
Subject: Re: Decent HTML Parser?

On Fri, Jul 14, 2006 at 12:30:04AM +0900, Kevin Weller wrote:
> Thanks Seth!  Hpricot looks a little young, but promising.  It 
> apparently parses incomplete HTML documents, don't know how it handles 
> imbalanced tags.  I'll check it out...thanks again...

  >> pp Hpricot("<p><b>A test of <i>unbalanced</b> tags.</i>")
  => #<Hpricot::Doc
      {elem
       <p>
       {elem <b> {text "A test of "} {elem <i> {text "unbalanced"}} </b>}
       {text " tags."}
       {bogusetag </i>}}>

Hpricot will attempt to match tags, keeping around the unmatched end tags as
BogusEtag objects (as HTree does.)  When you output XHTML, these bogus tags are
ignored and you'll get back XHTML (though some work still has to be done to 
be sure it's truly valid XHTML.)

Thankfully, I've been getting some really mangled HTML from interested people
sent to my inbox.  So today's Hpricot has become much better at fixing all
manner of nutty tags.  If you encounter anything Hpricot can't do, send me an
e-mail and with the URL you're trying to parse and the results you expect.

_why