From: Rob Doug <broken.m@...>
Date: 2009-10-22T11:52:27+09:00
Subject: Re: memory leak

> You could print out object statistics to get an idea about the source of
> the leak.  Something along the lines
> 
> require 'pp'
> 
> def print_class_counts
>    c = Hash.new 0
>    ObjectSpace.each_object(Object) do |o|
>      c[o.class] += 1
>    end
>    pp c
> end
> 
> Cheers
> 
>   robert

I retest my script again on 10k+ database and seems it stops to grow 
when reach 550mb size... I'm running 50 threads, so 10mb per thread. As 
I understand this is still not good ?

Robert, I did what you propose and here is some data. I print all 
objects that have more than 200 copies.

Here object after 100 URLs:
"Class:   1039"
"Thread:   395"
"Gem::Version::Part:   343"
"Regexp:   1463"
"MatchData:   301"
"Hpricot::Text:   31848"
"Gem::Version:   424"
"Array:   24977"
"Nokogiri::XML::Element:   1636"
"WWW::Mechanize::Chain:   203"
"Gem::Requirement:   380"
"WWW::Mechanize::Form::Field:   627"
"String:   199615"
"Hpricot::Elem:   21211"
"Nokogiri::XML::SyntaxError:   1562"
"Hash:   15884"
"Proc:   949"
"WWW::Mechanize::Form::Option:   458"

Here after 400 URLs:
"Class:   890"
"Gem::Version::Part:   343"
"Regexp:   1089"
"MatchData:   264"
"Hpricot::BogusETag:   1573"
"Hpricot::Text:   82593"
"Gem::Version:   424"
"Array:   53259"
"Nokogiri::XML::Element:   2982"
"WWW::Mechanize::Form::Button:   259"
"WWW::Mechanize::Chain:   242"
"Gem::Requirement:   380"
"WWW::Mechanize::Form::Field:   1179"
"String:   426729"
"WWW::Mechanize::Form::RadioButton:   351"
"Nokogiri::XML::SyntaxError:   6464"
"Hash:   37025"
"Hpricot::Elem:   55426"
"Proc:   774"
"WWW::Mechanize::Form:   338"
"WWW::Mechanize::Form::Option:   601"

And here after 500 URLs:
"Class:   991"
"Thread:   354"
"Gem::Version::Part:   343"
"Regexp:   847"
"MatchData:   409"
"Hpricot::BogusETag:   915"
"Hpricot::Text:   31707"
"Gem::Version:   424"
"Array:   24143"
"Nokogiri::XML::Element:   2076"
"WWW::Mechanize::Form::Button:   217"
"WWW::Mechanize::Chain:   218"
"Gem::Requirement:   380"
"WWW::Mechanize::Form::Field:   874"
"String:   197836"
"Nokogiri::XML::SyntaxError:   2608"
"Hash:   14719"
"Hpricot::Elem:   21530"
"Proc:   925"
"WWW::Mechanize::Form:   266"
"WWW::Mechanize::Form::Option:   416"


As you can see there is no stable increasing amount of some object.

What can I do more?
-- 
Posted via http://www.ruby-forum.com/.