From: Karl The_Cop <kls4f345m_forum@...>
Date: 2007-08-15T03:53:19+09:00
Subject: How to parse the "next" button on yahoo resultpages?

Hi guys,
I'm new to ruby and I really like it. But I'm standing in front of a
little maybe not so much ruby but more logical? problem.

I did some little tool which searches yahoo and prints the results on
the screen:

#!/usr/bin/env ruby
require 'net/http'

$VERBOSE = true

    class Yahoo
  def initialize(searchterm)
          # builts the yahoo_url from the searchterm
          yahoo_url =
'http://search.yahoo.com/search?p='+searchterm.gsub!(' ','+')
          # downloads the yahoo_url
          req = Net::HTTP.get_response(URI.parse(yahoo_url))
          # extracts the result-links from yahoo page
          req.body.scan(/^.+yschttl href.+/) {|hits| puts
hits.sub(/^.+="/,'').sub(/" >.+$/,'')}
     end
    end

search=Yahoo.new("ruby practical")
#eof

This is working pretty cool but my problem is: what if there are more
results on more pages? I need to look if there is a next url and then
doing some kind of a loop? But how to built this in? Or should I first
download any page in one single string and scan over this one? It would
be really nice if someone could help me with this problem.

Ps: I read about hpricot and scrubbyt but wanna solve this without
external modules.

bye
-- 
Posted via http://www.ruby-forum.com/.