[ruby-core:34865] Re: [Ruby 1.9-Bug#4168] WeakRef is unsafe to use in Ruby 1.9

On Tue, Jan 25, 2011 at 6:52 PM, Kurt  Stephens <[e-mail]> wrote:
> * Each SoftReference#_mri_gc_left and SoftRefernce#_mri_gc_ttl are set to some arbitrary value: SoftReference._mri_gc_ttl (default = 10).
> * If SoftReference#object was traversed since last GC, SoftReference#_mri_gc_left is reset to SoftRereference#_mri_gc_ttl.
> * If SoftReference#object was not traversed since last GC, SoftReference#_mri_gc_left is decremented by 1.
> * If memory pressure occurred, SoftReference#_mri_gc_left is also decremented by 0.5 * average of all SoftReference#_mri_gc_left.
> * "Memory pressure" is whenever MRI allocates a new GC heap.
> * When SoftReference#_mri_gc_left reaches 0, SoftReference#object= nil.
> * When SoftReference#object == nil or #object is not reachable, ReferenceQueues are notified.
>
> The values of 10 and 0.5 above are off-the-cuff tuning parameters.  I have no empirical data to warrant the values, but the test shows that the algorithm works.
> If the test also passes under JRuby, awesome!  :)
>
> See the pseudo-code here: https://github.com/kstephens/ruby/blob/trunk-weakref-c/ext/reference/lib/reference/mri/soft_reference.rb
> I'm not sure I like the heuristics.  I think others could come up with something better in pure Ruby, thus the pseudo-code may become the implementation.

This sounds similar to the JRockit JVM, which will definitely clear
soft references when out of heap memory and "probably" clean them
after some number of GC runs ("probably" as in it's not configurable
and is based on a rough LRU system).

Hotspot (Sun/OpenJDK) uses absolute time, measured from the time of
the GC prior to the last traversal of the soft reference (in other
words, every GC run records a timestamp, and when you traverse a soft
reference its internal "clock" variable is set to the previous GC's
timestamp). The time-based clearing is configured to be some number of
milliseconds per MB of heap. If you have a 500MB heap and ms-per-mb is
set to 10, you expect untraversed soft references to survive for
roughly 5 seconds (depending on GC schedules, of course. Memory
pressure will also trigger a flush of soft references.

I couldn't find information on the J9 JVM's soft reference heuristic.
That's where I stopped :)

> The unit tests have MRI-specific assertions in them -- I'll abstract them out.

Looking forward to it. I should be able to try out the test cases the
day you tell me they're ready.

> Aside:
>
>  Q: How should HardReference behave when associated with a ReferenceQueue?

It should do nothing. References are added to their queue when the
object they reference has been collected (or when it's about to be
collected). A HardReference by definition prevents the referenced
object from being collected. So registering a HardReference with a
ReferenceQueue has no meaning.

I'd almost say it should be illegal to construct a HardReference with
a ReferenceQueue, so nobody's confused about expectations. Or perhaps
we should have no HardReference at all, since it doesn't fit the rest
of the namespace's utility...

- Charlie