From: Peter Szinek <peter@...>
Date: 2008-11-21T17:42:05+09:00
Subject: Re: Hpricot scraping returns nil

--Apple-Mail-28--158139258
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit


> It should work if you take the tbody off the xpath. I have read
> somewhere that tbody does not work for hpricot , I dont know Y .
> Gudluck.
> xpath = "/html/body/table//tr[3]/td[2]/font/strong/small/font"
> --  
> Posted via http://www.ruby-forum.com/.

There is more to it than "tbody does not work for hpricot".

When a HTML parser (Firefox and Hpricot in this case) parses a HTML  
page, it has to build a tree from it (a.k.a. DOM).
The problem is that a lot (most?) of the HTML out there is badly  
formatted, so the process of DOM building is very ambiguous (what if  
tags are not nested properly? tags that are never closed? and a lot of  
other problems) so every parser approaches it a bit differently  
(that's one reason why you have the 'works in IE but not in FF' kind  
of problems), and e.g. Firefox even makes some efforts to make the  
parsed HTML standards compliant - for example inserting a tbody tag  
after a table tag if it's missing.

However, this is  but only very small difference between how Hpricot  
and Firefox parses the HTML/builds the DOM tree (on which XPaths are  
evaluated) - Hpricot tries to be as close to FF as possible, but this  
doesn't always happen (though _why said he considers these cases bugs).

Bottom line: you can't expect that XPath yanked from FireBug will work  
with Hpricot/Mechanize (though it mostly does, and adding a tbody  
increases your chances even further).

Cheers,
Peter
___
http://www.rubyrailways.com
http://scrubyt.org



--Apple-Mail-28--158139258--