From: Aaron Patterson <aaron@...>
Date: 2009-06-24T06:47:29+09:00
Subject: Re: HTML dom

On Wed, Jun 24, 2009 at 05:50:47AM +0900, Robert Klemme wrote:
> On 23.06.2009 19:24, Skye Shaw!@#$ wrote:
>> On Jun 23, 8:55 am, Victor Tanvuia <victor.tanv...@tantanprod.com>
>> wrote:
>>> Hi,
>>>
>>> I'm trying to build a HTML page indexer in ruby and I'd like to be able
>>> to use DOM and or XPath on a document. The application is currently
>>> using REXML
>>
>> Yes, REXML can be awkward if you're used to using the DOM. IMHO.
>
> Why do you say that?  REXML provides an XML DOM in similar ways as other  
> XML libs.  You can even use XPath queries.
>
>>> Is there a way to make REXML more permissive or is there another library
>>
>> There's libxml bindings for Ruby, but I recall that library missing
>> getElementsByTagName and getElementsById. Though it does have a method
>> to query the DOM via Xpath.
>
> libxml won't help as Victor is not processing XML.
>
>> Have you tried using REXML's SAX2 parser? I think it would be better
>> suited for your problem.
>
> No, his problem is that he used an XML tool to process HTML.  While many  
> web pages are valid XML not all are due to the history of browser  
> development.  Thus it's better to use a tool suited to the job, i.e.  
> capable of parsing HTML which is not valid XML.

The libxml2 c library has contained a correcting HTML processor since it's
first release in April 2000.  libxml2 is quite capable of processing
broken HTML.

libxml-ruby and nokogiri both provide a ruby API for libxml2.

-- 
Aaron Patterson
http://tenderlovemaking.com/