From: Eric Mahurin <eric_mahurin@...>
Date: 2005-06-21T01:24:15+09:00
Subject: Re: GC.disable not working?

--- Robert Klemme <bob.news@gmx.net> wrote:

> > Just catching a RangeError is not all you need.  You better
> > make sure the object is finalized and the finalizer removes
> its
> > oid from FOOS before the space is reclaimed.  Otherwise oid
> > could refer to a completely new object and you wouldn't
> detect
> > it.
> 
> Probably.  My assumption was, that oids are not reused.  But
> I may be
> wrong here.

Yep.  Other than immediates, I believe an object id is just a
memory location.  After an object is GCed, it will want to
reuse the space at some point.  The new object may start at
that same location (same object id) or that location may
correspond to the middle of an object.

> > On top of that, these finalizers can't be called while this
> > FOOS.each loop is going on.  Otherwise you'd get an error
> about
> > the hash being modified while your iterating over it.  It
> is
> > like the GC and finalizers are in another thread, but
> > unfortunately you can't control it like a thread (i.e.
> > Thread.critical=).  This is my primary dilemma.
> 
> Hm...  Did you try Mutex or Monitor?

I have tried Thread.critical= which is what Mutex and probably
other mutual exclusivity stuff are based on.  The problem is
that GC/finalizers is not considered to be in separate threads.

> > These issues may be very difficult to detect problems with
> and
> > you may need 1000's of tests to excite the GC differently
> to
> > detect the problem.  You must be prepared for this using
> > _id2ref.
> >
> >>> This approach works ok - you'll have to imagine that x
> contains
> >>> information needed for proper cleanup of a Foo instance,
> for
> >>> example, an open IO instance (although I'm sure that will
> do proper
> >>> cleanup on finalization):
> >>>
> >>> class Foo
> >>>  FOOS = {}
> >>>
> >>>  def initialize(x)
> >>>    self.x=x
> >>>
> >>>    ObjectSpace.define_finalizer(self) do |oid|
> >>>       puts "Cleanup of #{oid} with #{FOOS[oid]}"
> >>>    end
> >>>  end
> >
> > Won't work.  This is the exact mistake I made when first
> trying
> > to use a finalizer.  In the above, you gave
> define_finalizer a
> > Proc that has a Binding with direct access to self (a Foo).
> > This creates an unintended reference to the object in
> > ObjectSpace/GC.  It will never be GCed because of this.
> 
> Darn, yes you're right!
> 
> Two alternatives would be
> 
> class Foo
>   class<<self
>     alias :_new :new
>     def new(*a,&b)
>       obj = _new(*a,&b)
>       ObjectSpace.define_finalizer(obj) do |oid|
>         puts "Cleanup of #{oid} with #{FOOS[oid]}"
>       end
>       obj
>     end
>   end
> end
> 
> class Foo
>   def initialize(x)
>     self.x=x
> 
>     self.class.instance_eval do
>       ObjectSpace.define_finalizer(obj) do |oid|
>         puts "Cleanup of #{oid} with #{FOOS[oid]}"
>       end
>     end
>   end
> end

Nope.  Still doesn't work.  Try this:

100000.times { obj=Foo.new("hi") }

The memory size just keeps growing and the obj's are not GCed. 
In your second solution above obj isn't defined.  I'm not sure
what you intended.

The problem is that the Proc's you give to define_finalizer
still have access to the object you are putting the finalizer
on.  This time through a local variable (obj) instead of self.

You see why I say that the block form of define_finalizer isn't
useful?  And should be removed?

If you've been creating finalizers this way you've probably
never had issues with _id2ref because the object are never
GCed!

> What is the real world problem you are trying to solve?

In my cursor package, I create children cursors and need to
keep track of them.  But, I don't want to keep a normal ref on
them so that they can't be garbage collected.  For example:

child = parent.postion # child holds the current position
parent.position? # any positions/children outstanding?
parent.position?(child) # is this a valid position?
parent.position! # kill positions/children
parent.delete1Next # update all children after this point
child.succ # next position - use with Range

I don't want the user of this package to have to worry about
closing every single child.  It would be a pain to have to do
this especially when intermediate expressions can yield a
child.  I need to keep track of any outstanding, but I want GC
to get rid of any that aren't used anymore.


Here is some more code that I did some testing on:

#!/bin/env ruby

require 'set'
require 'weakref'

class WeakRefList
    def finalizer(id)
        __old_status = Thread.critical if @critical
        Thread.critical = true if @critical
        begin
            print("f")
            @ids.delete?(id) or raise
        ensure
          Thread.critical = __old_status if @critical
        end
    end
    def initialize(flags)
        @ids = Set.new
        @useWeakRef = flags[0].nonzero?
        @disable = flags[1].nonzero?
        @start = flags[2].nonzero?
        @critical = flags[3].nonzero?
        @dummy = flags[4].nonzero?
    end
    def << (obj)
        if @useWeakRef
            @ids << WeakRef.new(obj)
        else
            @ids << obj.object_id
            ObjectSpace.define_finalizer(obj,
                method(:finalizer))
        end
        dummy = WeakRef.new("") if @dummy
        self
    end
    def each(&block)
        GC.start if @start
        GC.disable if @disable
        __old_status = Thread.critical if @critical
        Thread.critical = true if @critical
        begin
            @ids.to_a.each { |id|
                begin
                    block.call(@useWeakRef ?
                        id.__getobj__ :
                        ObjectSpace._id2ref(id)
                    )
                rescue RangeError,WeakRef::RefError
                    print("e")
                    @ids.delete(id)
                end
            }
        ensure
          Thread.critical = __old_status if @critical
        end
        GC.enable if @disable
    end
end


if __FILE__==$0
    class MyString < String; end
    weakrefs = WeakRefList.new((ARGV[0]||0).to_i)
    $stdout.sync=true
    at_exit {puts}
    1000.times { |i|
        print(".")
        weakrefs << MyString.new("X"*i)
        weakrefs.each { |o|
            MyString==o.class or
                raise("not a MyString: #{o.inspect}")
        }
    }
end



I tried out a bunch of ways to make this WeakRefList.  You can
pass in a flags number ORing the options I provided for this
class:

1: use WeakRef instead of simply an object id
2: use GC.disable/GC.enable around code using _id2ref
4: use GC.start before trying _id2ref
8: use Thread.critical= to try to stop the finalizer
16: add a dummy allocation of a WeakRef

I found that only using WeakRef's, GC.start, and dummy
WeakRef's worked for me.  In other cases it looks like the
obect_id got reclaimed by another object before the finalizer
was run on the original object.  I don't understand why using
WeakRef worked.  Looking at the code it looks like an object id
could still get reused before the finalizer is called and it
would still look OK.  I think it is just luck because
allocating dummy WeakRef's also worked.  Currently, I trust
using GC.start the most.  But I've still seen cases where I
have to call GC.start multiple times back-to-back.  Anybody
have a better solution for this WeakRefList?  I'll want more
methods eventually, but << and each seem sufficient for
testing.




		
____________________________________________________ 
Yahoo! Sports 
Rekindle the Rivalries. Sign up for Fantasy Football 
http://football.fantasysports.yahoo.com