From: Estanislau Trepat <estanis@...>
Date: 2011-02-18T00:23:20+09:00
Subject: Re: Ruby(and programming) beginners question regarding 'NoMethodError' while using Hpricot

--0016e646972297121e049c7bf952
Content-Type: text/plain; charset=UTF-8

Hi Sandeep.

The #next_sibling method was not working because you were using it on the
whole elements array (in fact, an Hpricot::Elems object) and not on each of
the elements inside. That's because we had to use elements.first to get the
first node which met our search criteria and then call #next_sibling on that
node. The #next_sibling method is only defined on each of those nodes not on
the array itself.

I apologize for the * characters, I think I was trying to put that part in
bold and got bad formatting out of my email client.

For the problem you expose, maybe you could try using Watir<http://watir.com/>.
It drives a real web browser, and can thus handle Javascript links.

If you allow me a suggestion: Taking a look at the page you're trying to
scrape and the structure of the query parameters, I'd suggest to extract the
total number of results from the bottom part which reads "Showing 1 - 200 of
3529". If you extract that last number (the total number of results) then
you could point your scraping script to:
http://money.rediff.com/companies/all/1-3529 without needing to follow
javascript links.

Hope it helps.

Regards.

--
Estanislau Trepat
http://etrepat.com
http://twitter.com/etrepat



2011/2/17 Sandeep Guria <sndpgr@hotmail.com>

> Thank You! very much Estanislau
>
> If I am not bothering you too much why wasn't it('next_sibling') working
> on my code??and
> what are those '*' for in here
> >cell = elements*.first.*next_sibling
>
> They were giving an error 'syntax error, unexpected '.''.
> I removed them and now it's working fine.
>
> One more thing I need to ask , if I could use this thread!
> I have this web page 'http://money.rediff.com/companies/all/1-200'
> at the bottom there is a link('next') to the next page of the list.
> Now this link is a java script .
> What I want to do is after finishing scraping this page I want to go to
> the next through the 'Next' link. Is there any way to do it???
>
> Note:- A cruder method will be to go to every page o the list by their
> web page and scraping from that page (total number of pages will be 17).
>
> Any suggestions are welcome!
> Thank you!
> Sandeep Guria
>
> --
> Posted via http://www.ruby-forum.com/.
>
>

--0016e646972297121e049c7bf952--