[ruby-core:20517] Re: Promising C coding techniques to reduce MRI's memory use

> I implemented a scheme for recording the maximum depth of the C stack in
> xmalloc and during garbage collection itself.  However, I realized that
> there was no point in clearing the stack when it is near its maximum depth.
> Instead, stack clearing is deferred until CHECK_INTS, as this tends to
> happen
> between evaluation of nodes, when the stack is likely to be shallower.
>
> At this point
> a tight loop quickly zeros the region between the current top of stack, as
> returned by alloca(0), and the maximum recorded stack extent.  It also
> updates
> the stack extent so no memory is cleared repeatedly if the stack contracts
> further.

This is sweet.  I liked the idea so much I coded my own [perhaps much
smaller, definitely less effective] version.  It only includes the
stack clearing you referred to, and doesn't even monitor "exactly" the
stack size, but approximates it by metering it once every CHECK_INTS.
Ruby seems to run "as fast as normal" with it, and collect better.

In principle, you'd only have to clear the stack once "between each
GC" so if you kept track of which portions of it you'd been able to
clear, you could avoid a few stack clearings :)
I'm not sure exactly how much cpu that would save, though.

This patch also doesn't fix the
 loop {@x=callcc{|c|c}}
aspect [presumably because ruby's green threads copy chunks of the
stack to heap, so they aren't cleaned]--so I'd imagine it's less
effective in multi-threaded codes [but hopefully still helpful].

Look forward to the real patch when it comes in :)

Note that as it is currently, if you run GC.start it also calls
clean_stack, so if you run GC.start when your program is at it "inner
depth [most nested call]" it will notice exactly how deep it is, and
hopefully clean up the stack "all the way" when you ascend out of deep
calls.  I suppose creating a new call "GC.clear_stack" would be
useful.

i.e. GC.start -> GC.start + "clean stack/make a note of how deep the
stack is currently"

With [1] it successfully prevents the string 'a' from not being
garbage collected:
With [2] it successfully collects a few more objects than the unpatched does.
I'm not positive how well it works but I think it does.

Enjoy.

-=R

[0] patch: http://wilkboardonline.com/roger/clear_stack_only2.diff

[1] file.rb:

def does_nothing
end
def deep(how, gc = false)
 if(how == 175)
    'a'*1000
 end
 if how == 300
        print "222222deepest"
        GC.start
        print "222222deepest"
 return
 end
 deep(how+1)
 20.times {does_nothing}
end
puts
deep(0)
GC.start
deep(0, true)
count = 0
ObjectSpace.each_object(String) do |s| print s, ' '; count = count+1; end
print count

[2] file2.rb:

count = 0
ObjectSpace.each_object{|o| count += 1 }
print count
GC.disable
def go depth
 if depth == 50
   GC.enable
   GC.start
   return
  end
  if(rand(10) == 3)
    a = 'abcd'
    go(depth+1)
    go(depth+1)
  end
  if(rand(10) == 3)
    b = 'abcd'
  end
  go(depth+1)
end

go 0
count = 0
ObjectSpace.each_object{|o| count += 1 }
print count