From: John Labovitz <johnl@...>
Date: 2006-09-21T14:03:32+09:00
Subject: Re: Mechanize for BIG website scrapping...

On Sep 20, 2006, at 9:24pm, Horacio Sanson wrote:

> In my script I cannot remove the WWW::Mechanize object since this  
> page in
> particular is a form and requires cookies state information to be  
> able to
> access to the pages I need to download.

What if you save the cookies out to a file?   
WWW::Mechanize::CookieJar has a #save_as and #load method to save and  
restore cookies.

> Is there a way to tell the Mechanize Object to delete the pages  
> alreade downloaded??

I actually ran into a similar issue recently; your diagnosis explains  
why my program used too much memory.

You might try the following (assuming "browser" is your  
WWW::Mechanize object):

	browser.page.content.replace ""	# that's an empty string
	browser.page.root.children = []

That should clear both the original text, and the parsed HTML.  I'm  
not sure whether this would get rid of all the references, but at  
least it should help.

--John