[ruby-core:26144] Re: A challenge: Enumerator#next in JRuby

On Sat, Sep 26, 2009 at 6:38 AM, Tanaka Akira <[e-mail]> wrote:
> In general, internal iterator is easier to implement. =C2=A0This
> is why Ruby uses internal iterator long time. =C2=A0Although
> Fiber based external iterator is slower than ideal, it makes
> many internal iterators usable as external iterator without
> effort.

An update on this!

I have posted more information in JRuby's bug here:
http://jira.codehaus.org/browse/JRUBY-3964

At the moment, we have been unable to find a way to make "nexting"
threads work properly without a hard reference to the original
Enumerator, which means the Enumerator lives forever. I have described
a possible solution in the bug, but we have not implemented it yet.

I also asked JVM experts about this, and there is no way in any JVM to
mark a thread as non-rooting. The good reason for this is that it
could cause objects the thread references to suddenly GC while the
thread is executing, which would be catastrophic.

I also described in the bug why even the best solution for JRuby may
be unsatisfactory. It is looking likely at this point that we may
support only core-class enumerables and enumerables that implement
Akira's protocol in JRuby 1.4, since we can't ship a version that
spins up immortal threads for every Enumerator#next.

I am also starting to suspect that a protocol like Akira's may be the
*ONLY* way to reliably implement external enumeration on runtimes that
don't provide coroutines and that do not allow non-rooting threads. I
also suspect there may be object reference issues even on MRI, since
marking a thread as non-rooting is a very coarse-grained solution,
especially if objects are created only in the enumerating thread/fiber
and you want them to stay alive.

Bottom line once again: It may not be possible to support
Enumerator#next without requiring a lightweight enumerator protocol in
JRuby.

- Charlie