From: Dan Diebolt <dandiebolt@...>
Date: 2008-03-28T19:42:00+09:00
Subject: Re: hrpicot - cant extract what i want from page

--0-733329157-1206700917=:85182
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Firebug puts in tbody's into xpath's that reach into tables even if the <tbody> tag is not in the html source. Try removing the tbody path and debug using shorter xpaths to initially address content further up in the hierarchy. 

You might have some success addressing text nodes combined with some subsequent regexp processing:

b = doc.search("//text()")

I think you might be more successful using a css selector instead of an xpath selector.  To overcome hpricot not supporting all xpath axes you can sometimes find a way to address the elements with a clever css selector.

It can be a challenge to use hpricot with malformed html or if there are no containers wrapping items that otherwise appear visually as a list or table. I haven't tried it yet, but running the html through something like tidy before parsing so might create some of the missing structure.



--0-733329157-1206700917=:85182--