From: Dan Diebolt <dandiebolt@...>
Date: 2008-05-14T15:54:06+09:00
Subject: Re: hpricot and regexp?

--0-1107208710-1210748046=:63203
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Google cached pages have this structure:

<body>
  <table width="100%" border="1">
  </table>
  <hr/>
  <div style="position: relative;">
  </div>
</body>

where the first <table> contains boilerplate cache text and a copy of the page is in the <div>.

This is what I would use to clip out the date:

url="http://64.233.167.104/search?q=cache:hydO8fs-rmQJ:en.wikipedia.org/wiki/Court-martial+court+martial&hl=en&ct=clnk&cd=1&gl=us&client=firefox-a"
doc = Hpricot(open(url))
a=doc.search("/table").inner_text
a[/retrieved on (.*?) GMT/,1]
=>May 13, 2008 11:37:34


Feng Tien <pood.forums@gmail.com> wrote: Feng Tien wrote:
> Feng Tien wrote:
>> I'm trying to grab the "cache date" off of the google search.
>> 
>> using Mechanize (and built in hpricot)
>> 
>> 
>> agent = WWW::Mechanize.new
>>   agent.user_agent_alias = 'Mac Safari'
>>   page = agent.get("http://www.google.com/")
>>   search_form = page.forms.with.name("f").first
>>   search_form.q = "Hello"
>>   search_results = agent.submit(search_form)
>>   cache_date = agent.click search_results.links.text('Cached')
>> 
>>   date = cache_date.search('table table > td').inner_html
>> 
>> 
>> How do i grab the date like on this page:
>> http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a
>> 
>> the part that's right after "as retrieved on" (the date)
>> Is there a built in hpricot method that can search by rexep?
>> or will I have to use something like gsub?
> 
> 
> oops, I mean, grep.
> 
> oh, i got it down to this:
> 
> date = cache_date.search('table table > td').inner_text.grep(/retrieved 
> on (.+)./)
> 
> 
> which outputs:["This is G o o g l e's cache of http://www.hello.com/ as 
> retrieved on May 11, 2008 01:09:29 GMT.\n"]
> 
> How do I get rid of everything before the date?


Now I have this:

date = cache_date.search('table table > td').inner_text.grep(/retrieved 
on (.+)./).to_s.gsub(/.+as retrieved on /,"").gsub(/.\n/,"")

which gives me exactly what i need. is there a better way to doing this?
-- 
Posted via http://www.ruby-forum.com/.



--0-1107208710-1210748046=:63203--