From: Mat Schaffer <schapht@...>
Date: 2006-08-18T12:09:53+09:00
Subject: Re: reading website


On Aug 17, 2006, at 3:40 PM, fabsy wrote:

> Hey!
> I have been going through some basic IO stuff and want to get into
> something more "advanced".
> I would like to make a script who asks for a website and takes the
> information (the text) on the website and put it into a textfile..
>
> So now im wondering.. how do I tell the script to download/read
> text/content from a website.. How do I do it? ( of course i do not  
> want
> anyone to actually write this script for me, i want to write it myself
> but im a newbie and don't know which libs and etc to read in)
>
> Thanks!

I'm a little late to this party, and it might be a little advanced  
for you yet, but if you're dealing with scripting against websites  
you can do some pretty sick stuff using 'mechanize' (http:// 
mechanize.rubyforge.org/) and 'hpricot' (http:// 
code.whytheluckystiff.net/hpricot/).

Mechanize can click links and fill out forms.  Hpricot can look for  
text/tags in the resulting HTML.  And Mechanize supports pluggable  
parsers which lets you drop Hpricot right into Mechanize.  It's so  
buttery... :)
-Mat