[ruby-core:25775] Re: A challenge: Enumerator#next in JRuby

On Fri, Sep 25, 2009 at 4:01 PM, Marc-Andre Lafortune
<[e-mail]> wrote:
> I'm curious as to what are good examples of uses of #next. Personally,
> I don't mind the fact that its use comes with a big warning "might be
> slow or unsupported".

I don't dispute that it's a useful construct. Hell, Java developers
have gotten by with *just* external enumeration forever, and managed
to get by. It's also nice if you want to walk a collection in pairs or
walk a bit and then have some other piece of code walk the rest.
There's lots of uses for it.

And it's not really the "cursor" style enumeration (Enumerator#next)
that's a problem...it's the ability to maintain a cursor over
arbitrarily complex iteration logic. Another example to illustrate how
complex it can get:

class StreamGenerator
  def initialize(io); @io =3D io; end
  def each
    until @io.eof?
      yield @io.gets
    end
  end
end

In this case, any IO-like object with eof? and gets can be fed into
enumeration, and Enumerator#next over this "collection" must pause the
execution after each yield. But this is also easily turned into a
non-fiber-based enumeration by implementing to_enum yourself, since
the only state that *actually* needs to be saved between elements is
the position in the stream.

One of my concerns about supporting Enumerator#next on any object that
implements #each is that people will use this rather than implement a
lightweight enumerator of their own...resulting in lots of fibers or
generators on Ruby 1.8.7 or Ruby 1.9 and lots of threading hassles on
JRuby and IronRuby. But if Enumerator#next were predicated on a
collection implementing both each and to_enum.

It's also worth noting that using Enumerator#next is *really* slow
right now, very likely *because* it's continuation-based. Any use of
Enumerator#next is going to have similar performance characteristics
without custom enumerators (best) or threads (workable, but problems I
mentioned).

~/projects/jruby =E2=9E=94 time ruby1.9 -e "ary =3D Array.new(10000, 1); a =
=3D 0;
10000.times { ary.each {|i| a +=3D i} }"

real	0m13.100s
user	0m12.555s
sys	0m0.055s

~/projects/jruby =E2=9E=94 time ruby1.9 -e "ary =3D Array.new(10000, 1); a =
=3D 0;
10000.times { e =3D ary.each; 10000.times { a +=3D e.next }}"

real	2m50.966s
user	2m48.892s
sys	0m0.921s

- Charlie