[ruby-talk:04431] Re: Real world performance problems

> A> I don't know any real solution to this right now. GC is 
> slow, so let's not
> A> do it, OTOH without GCing now and then we run out of memory *fast*.

Dave and Guy proposed the same thing with different words almost
simultaneously, so I guess we should assume they're just clones of each
other and got a communication problem who's doing what ;->. Thanks guys!

>  Have you tried something like this :
> 
>    GC.disable
>    24001.times do |i|
> #
>       if i%1000 == 0
>          GC.enable
>          GC.start
>          GC.disable
>       end
>    end

Actually I didn't try the simplest thing!

I tried

  module Kernel
    def timely_gc(seconds)
      # stop gc
      GC.disable
      # make thread which sleeps and does the janitor job, and repeats
      @timely_gc = Thread.start do
        loop do
          sleep seconds
          GC.start
          GC.disable
        end
      end
    end
  end

but I didn't get satisfactory results. (I guess I was generating core dump.)

Well, while the simplest thing seems to almost solve the problem (8000
items, runtime 10.8206 :-D ), there are problems with this solution

First, 'ps' tells during the runtime:
30044 99.9 13.7 72356 71328 pts/27  R    18:37   0:02 ruby hash_test.rb
30044 99.9 29.4 153484 152460 pts/27 R   18:37   0:07 ruby hash_test.rb
30044 99.9 39.4 205852 204044 pts/27 R   18:37   0:12 ruby hash_test.rb
30044 99.9 53.2 276552 275528 pts/27 R   18:37   0:18 ruby hash_test.rb
30044 99.9 63.5 330488 328680 pts/27 R   18:37   0:24 ruby hash_test.rb
30044 99.9 72.3 375008 373984 pts/27 R   18:37   0:29 ruby hash_test.rb

This is before I have to kill it, as very soon it'll dump the core as it
fails to allocate more memory. Well, you can see, the memory usage is
growing fast (I wonder why, is there a bug in GC?).

So I couldn't get running time for 24000 entries. This might be partly due
my hack to increase HEAPS constants in gc.c.

Even provided all the memory this part of program might use in current form,
this is quite hard way to go. I'm multi-threading, and the information
source could be blocked for long times, we might end up in the situation
where some other thread makes tons of objects, expecting normal GC. So when
we'd finally have processed 1000 entries the core has been dumped a long
ago.

Of course all parts could be coded with do-the-gc-by-yourself -methodology
but I doubt it's the right way to go. For example, you can't use any
standard libraries before first making sure they GC too when they think they
have to.

Therefore I did go with the timely_gc solution, but even so you have to put
the sleep quite small, and you can never know when the thread gets execution
turn. So I guess that way is not good either as we can't make sure we will
GC when needed, but not too often.

The problem remains... I guess we're expecting Matz says something
definitive here :).

	- Aleksi