From: Jano Svitok <jan.svitok@...>
Date: 2009-03-18T01:42:53+09:00
Subject: Re: free RAM problem

On Tue, Mar 17, 2009 at 16:23, Edouard Dantes <edrd.dantes@gmail.com> wrote:
>
>> Without seeing code it's hard to comment,
>
> Thanks for answer,
>
> well, the loops are too basic to create problems, it's fill array with
> values & write to file.
>
> RAM jumps are too erratic (5 or 10% for 2GB RAM)
>
> The only processor angry stuff (and possibly RAM too) is the following
> parsing line, imported from another file.
>
> Nokogiri::HTML(page).css('div a').map{|dts|
> dts.text.gsub(/\s+/,"").gsub(/[^A-Za-z0-9\s]/,'Z') }

Use /o (once) flag for regexen (gsub(/\s+/o,"")), or make them
constants out of the loop - you'll save two object creations for each
loop.

Try replacing the block body with

tmp = dts.text.gsub(/\s+/,"")  # creates a copy of the string, with
whitespace removed
tmp.gsub!(/[^A-Za-z0-9\s]/,'Z') # avoid creating another copy of the string
tmp # return the temporary string

Run your code under RubyMemoryValidator, or at least under ruby-prof
(or any other similar tool) to see what kind of objects get created
and where. If you post the code and some sample data, I might find
some time to run with MemoryValidator for you.

Jano