From: Aaron Patterson <aaron@...>
Date: 2008-11-23T06:08:43+09:00
Subject: Re: Using Hpricot with HTML's

Hi Rick,

On Sun, Nov 23, 2008 at 04:43:20AM +0900, Rick Rick wrote:
> I have been working on this program for class all day and am not getting
> anywhere, so I figured I would try to ask you guys on the forum. Here is
> my code so far:
> 
> 
> #!/usr/bin/ruby
> # Lab 19
> 
> require 'rubygems'
> require 'fileutils'
> require 'hpricot'
> require 'net/http'
> 
> tamper_args = {
>     'my_countries' => ',"Germany", "Japan", "United Kingdom", "United
> States"',
>     'my_vars' => ',+pop',
>     'my_yrs' => ',1950,1960,1970,1980,1990,2000',
>     'selected_vars' => 'POP',
>     'format_selected' => 'csv'
> }
> 
> 
> host_name =
> "http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php"
> 
> puts tamper_screen = Net::HTTP.post_form(URI.parse(host_name),
> tamper_args)
> puts page_parse = Hpricot(tamper_screen.to_s)
> 
> puts page_parse.search("pre")
> 
> 
> ***
> 
> It is designed to go through the host_name link and give the populations
> for the 4 countries listed, with years 1950, 1960.... through 2000, and
> I have to get the data in csv format. I am not sure why I am not able to
> do my search the pre tag, which is where the data is stored in the html
> doc.
> 
> I think it has to do with the .to_s for the page_parse variable, but my
> program gives me an error when tamper_screen is not a string.

Try using the body method.  post_form returns a response object, and you
need to grab the body of the response:

  require 'rubygems'
  require 'fileutils'
  require 'hpricot'
  require 'net/http'
  
  tamper_args = {
      'my_countries' => ',"Germany", "Japan", "United Kingdom", "United
  States"',
      'my_vars' => ',+pop',
      'my_yrs' => ',1950,1960,1970,1980,1990,2000',
      'selected_vars' => 'POP',
      'format_selected' => 'csv'
  }                                                                                            
  
  
  host_name = "http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php"
  
  tamper_screen = Net::HTTP.post_form(URI.parse(host_name), tamper_args)
  
  page_parse = Hpricot(tamper_screen.body)
  puts page_parse.search("pre").inner_text

-- 
Aaron Patterson
http://tenderlovemaking.com/