From: Aaron Patterson <aaron@...>
Date: 2009-03-19T07:08:38+09:00
Subject: Re: free RAM problem

On Thu, Mar 19, 2009 at 06:04:51AM +0900, Edouard Dantes wrote:
> Hi,
> 
> Well I replaced the Nokogiri::HTML with a Hpricot an now my Ram usage 
> remains below 2% during the whole process.

I ran this:

  require 'nokogiri'
  
  html = File.read(ARGV[0])
  while true
    Nokogiri::HTML(html).css('div a').map{ |dts|
      dts.text.gsub(/\s+/,"").gsub(/[^A-Za-z0-9\s]/,'Z')
    } 
  end

vs this:

  require 'hpricot'
  
  html = File.read(ARGV[0])
  while true
    Hpricot(html).search('div a').map{ |dts|
      next unless dts.respond_to?(:text)
      dts.text.gsub(/\s+/,"").gsub(/[^A-Za-z0-9\s]/,'Z')
    } 
  end

As I expected, both were stable and neither leaked memory.  What version of
libxml2 are you running?  Also, what version of ruby?  It could also be
the html I'm parsing...  :-\

--
Aaron Patterson
http://tenderlovemaking.com/