Re: Memory usage with blocks

Runs heaps faster too.  Woosh!

On Wed, Dec 3, 2008 at 2:32 PM, John Ky <[e-mail]> wrote:

> Hi Sebastian,
>
> Why doesn't ruby detect that tmp isn't used in the block and not keep a
> reference to it?
>
> I found the offending code:
>
>          def play_back(&block)
>             $trace.method_block "PlaybackChain::play_back(&)" do
>                sub_block = Proc.new do
>                   @tail.play_back do |root|
>                      block.call(root)
>                   end
>                end
>                @block.call(sub_block)
>             end
>          end
>
>          def play_back(&block)
>             @heads.each do |head|
>                selector.select_each do |selection|
>                   block.call(selection)
>                end
>             end
>          end
>
> Replacing those to methods with
>
>          def play_back(&block)
>             $trace.method_block "PlaybackChain::play_back(&)" do
>                sub_block = Proc.new do
>                   @tail.play_back(&block)
>                end
>                @block.call(sub_block)
>             end
>          end
>
>          def play_back(&block)
>             @heads.each do |head|
>                head.play_back(&block)
>             end
>          end
>
> wiped over 100 MB off the memory usage.
>
> Thanks for your help,
>
> -John
>
>
> On Tue, Dec 2, 2008 at 11:21 PM, Sebastian Hungerecker <
> [e-mail]> wrote:
>
>> John Ky wrote:
>> > When I replace the following code:
>> >
>> >                 head.play_back do |root|
>> >                    block.call(root)
>> >                 end
>> >
>> > With this:
>> >
>> >                 head.play_back(&block)
>> >
>> > And make other similar changes.  The program behaves exactly the same
>> way,
>> > but now uses 200MB less.
>>
>> Consider the following:
>>
>> def x(&block)
>>  block
>> end
>>
>> def a(&block)
>>  tmp = create_a_two_hundred_mega_byte_object()
>>  x {block.call} # Create a new block which closes over the current scope
>> end
>>
>> def b(&block)
>>  tmp = create_a_two_hundred_mega_byte_object()
>>  x(&block) # Reuse the block that has been passed to this method.
>>            # Do not create a new block
>> end
>>
>> If I call b {puts "hello"}, the following will happen:
>> The block {puts "hello"} is created. This block holds a reference to any
>> local
>> variable that has been defined in the scope in which I call method b.
>> Now method b is called with that block as an argument.
>> Then a 200mb object is created and stored in tmp.
>> Now x is called. x returns a Proc representing the block passed to b.
>> The method b ends, tmp goes out of scope and the 200mb object is garbage
>> collected.
>>
>> If I call a {puts "hello"}, this happens:
>> The block {puts "hello"} is created. This block holds a reference to any
>> local
>> variable that has been defined in the scope in which I call method a.
>> Now method a is called with that block as an argument.
>> Then a 200mb object is created and stored in tmp.
>> Now another block is created, which invokes the block passed to a.
>> This new block holds a reference to any local variable tmp defined in the
>> method body of a. Specifically it holds a reference to tmp.
>> Now x is called. x returns a Proc representing this new block.
>> The method a ends, but the 200mb is still referenced by the Proc that's
>> returned from a. So as long as reference to that Proc is stored, the 200mb
>> object will not be garbage collected.
>>
>> I don't know whether that's what happens in your code, but it's certainly
>> worth looking into.
>>
>> HTH,
>> Sebastian
>> --
>> Jabber: [e-mail]
>> ICQ: 205544826
>>
>>
>