From: brabuhr@...
Date: 2010-12-11T10:28:33+09:00
Subject: Re: Help with net/http

You might also consider the mechanize library:
http://mechanize.rubyforge.org/mechanize/GUIDE_rdoc.html

e.g.

require 'rubygems'
require 'mechanize'

Mechanize.new.get("http://www.apartment-directory.info/alabama/0") do |page|
  page.search('//tr').each do |tr|
    tds = tr.search('./td')
    puts tds[0].text.chomp rescue nil
    puts tds[2].text.chomp rescue nil
    puts tds[3].text.chomp rescue nil
    puts
  end
end

This sample script as-is is too greedy; it loops over every row of
every table table instead of just the interesting one.

$ ruby i.rb
[some garbage from other tables]
...
Aquadome Apartment
1619 8th Street Southwest
Decatur,  AL  35601

Arbor Park Apartments
175 Sloan Avenue East
Talladega,  AL  35160

Arbor Place Apartments
515 Fox Run Parkway No. 9A
Opelika,  AL  36801

Arbor Pointe Apartments
100 Dairy Road
Mobile,  AL  36612

Arboretum Apartments
1800 Arboretum Circle
Birmingham,  AL  35216

Arbors On Taylor
485 Taylor Road
Montgomery,  AL  36117

Arrow Head Apartments
129 South Union Avenue
Ozark,  AL  36360
...