[ruby-core:34847] Re: [Ruby 1.9-Bug#4168] WeakRef is unsafe to use in Ruby 1.9

On Mon, Jan 24, 2011 at 11:44 PM, Kurt  Stephens <[e-mail]> wr=
ote:
> * Support for multiple Reference subclass instance caches.
> * Working WeakReference, SoftReference and ReferenceQueue in ext/referenc=
e/reference.c
> * SoftReference responds to new GC heap allocations as "memory pressure" =
by globally reducing all SoftReference#_mri_gc_left counts.

Can you provide a short description of the complete "soft" heuristic
as of this commit? Even on JVM, it's unspecified, but each JVM does
spell out what constitutes a "soft" reference for them.

> * Sample WeakKeyHash class using WeakReference and ReferenceQueue.
> * Support for JRuby-compatible WeakReference.new(object, reference_queue)=
 protocol.
> * Removed ReferenceQueue#monitor method which cannot be supported in JRub=
y.
> * Renamed MRI-specific methods to _mri_*.
> * Optimized gc_api.c callback lists.
> * Clean up the C stack after gc_api.c callbacks to avoid conservative GC =
pinning (might be overkill).
> * Removed the ugly (and stupid) flags hacks.
> * Added some basic RDOC-style comments.

It's looking pretty clean. The look-alike API in JRuby will be a thin,
direct layer over Java's reference logic, similar to that found in
weakling.

> Charles:
>
> =C2=A0I'm caching live Reference instances to minimize the number of Refe=
rences objects that must be scanned before the sweep phase.

As far as I know, JVMs do not cache weak/soft references internally.
The GC impact of having many weak references around is a frequent but
accepted problem.

> =C2=A0I'd either need to keep a doubly-linked list of References or scan =
all the GC heap for References (which I did not want to expose from gc.c). =
=C2=A0Using a lookup table (st_table in MRI) to track of live References wa=
s not much more work than a doubly-linked list and it collapses Reference.n=
ew(object) requests into a single instance. =C2=A0This is beneficial for ot=
her reasons: it reduces the total number of References to common objects an=
d keeps the Reference#=3D=3D operator simple for live instances (it's simpl=
y Reference#object_id#=3D=3D).

In MRI, where an object can be uniquely identified, I suppose this is
fine. I'm not familiar with how other conservative GCs implement
efficient weak/soft reference logic, so I'll defer.

> =C2=A0 Once Reference#object is sweeped, all the References to #object ar=
e no longer "live"; and are removed from cache and scheduled for ReferenceQ=
ueues after GC. =C2=A0References to immediates are never cached. =C2=A0Dead=
 References will probably will be removed from collections or instance vari=
ables shortly thereafter (i.e.:
> =C2=A0via ReferenceQueues).
>
> =C2=A0Why would a user want to insure that Reference.new(object) always r=
eturns a new instance, since References are/should be immutable? =C2=A0JRub=
y and other Rubies do not need to implement the Reference cache.

Provided that there's no specified requirement that the same object
get the same Reference, and as many users as want to can have their
own separate reference queues receive a reference for the same object,
I have no objection.

> Aside:
>
> =C2=A0Q: Should we have a trivial HardReference class that always keeps i=
ts #object pinned, just for API consistency?

I always thought this was a gap in java.lang.ref; there's an abstract
Reference class, but anyone who wants to mix hard and soft references
in a collection must always write their own HardReference subclass. I
wanted several times for there to just be a built in HardReference I
could use as a one-element indirection mechanism alongside weak, soft,
and phantom references.

But that's Java with static types, where I wanted to commingle weak
and non-weak references in the same logic. Would this be a useful
thing to have in Ruby? I don't know.

When you think your tests are stabilizing, I will implement the same
API in JRuby and we'll see how it looks.

- Charlie