From: SpringFlowers AutumnMoon <summercoolness@...>
Date: 2007-11-03T16:21:23+09:00
Subject: Re: HTML parser Hpricot? and how to get all text

mortee wrote:
> kendear wrote:
>>> irb(main):001:0> require 'hpricot'
>> yup, mine is
>> 
>> C:\>gem list hpricot
>> 
>> *** LOCAL GEMS ***
>> 
>> hpricot (0.4)
>>     a swift, liberal HTML parser with a fantastic library
>> 
>> and d.inner_text or d.text both won't work.
>> 
> 
> Does something prevent you from upgrading?


I finally got the time to upgrade to Hpricot 6.0
so now, the following


require 'net/http'
require 'hpricot'

r = ""

Net::HTTP.start("www.google.com") do |http|
  r = http.get("/")
end

c = Hpricot(r.body)
p c.to_plain_text


will work, and so will

p c.inner_text

as the last line.  however, the CSS and Javascript lines are not 
removed.  So I think I can gsub the CSS and Javascript blocks with the 
multiline regexp gsub.

I wonder though if there is a quick way, that will do what the lynx on 
UNIX does... just print out a plain and readable text page.

-- 
Posted via http://www.ruby-forum.com/.