From: "David A. Black" <dblack@...>
Date: 2008-11-03T01:58:15+09:00
Subject: Re: ruby1.9: lazy versions of Enumerator#select and friends?

HI --

On Mon, 3 Nov 2008, Brian Candler wrote:

> I've been having a play with Enumerators in ruby 1.9, in particular
> adding 'lazy' versions of those methods in Enumerable which normally
> return arrays. Here's a proof-of-concept implementation:
>
> module Enumerable
>  def lmap(&blk)
>    Enumerator.new do |y|
>      each do |e|
>        y << blk[e]
>      end
>    end
>  end
>
>  def lselect(&blk)
>    Enumerator.new do |y|
>      each do |e|
>        y << e if blk[e]
>      end
>    end
>  end
>
>  def ltake(n)
>    Enumerator.new do |y|
>      count = 0
>      each do |e|
>        break if n <= count
>        y << e
>        count += 1
>      end
>    end
>  end
>
>  def lskip(n)
>    Enumerator.new do |y|
>      count = 0
>      each do |e|
>        y << e unless count <= n
>        count += 1
>      end
>    end
>  end
> end
>
> if __FILE__ == $0
>  big = (1..1_000_000_000_000)
>  big.lselect { |i| i % 2 == 1 }.lmap { |i| i + 100 }.
>      lskip(5).ltake(10).each { |i| puts i }
> end
>
> So instead of generating an array containing all processed elements,
> they return an Enumerator which processes the elements on demand. As a
> result, you can easily chain them together, as shown in the example; you
> can handle large sequences without creating large intermediate arrays;
> and also handle infinite lists.
>
> Ruby hides the internals of this very nicely, I presume using Fibers to
> maintain the state inside each Enumerator.
>
> My first question is, does ruby 1.9 have methods like this already, and
> I've just overlooked them? If so, where should I be looking?

It did have something like this, but not any more. I'm not sure they
were identical to what you've written, but the basic idea of an
enumerator that bundled itself with a block did exist in 1.9 for a
while.

> If not, then is there value in adding something like this to the
> language? For example,

I think so, absolutely. I'm still struggling with trying to believe
that enumerators are more than marginally useful without the ability
to travel with knowledge of a block and still be lazy.

> 1. Put this stuff into an external library, Facets-like, with new
> methods in Enumerable as above (the method names given are just
> examples, there are probably better ones)
>
> 2. Redefine Enumerator#map, Enumerator#select etc, so that they return
> Enumerators instead of arrays. It seems reasonable to me that once you
> have an Enumerator at the base of the chain, you will likely want to
> keep chaining them together. You can always collapse the result to a
> real array using 'to_a'.

I'd strongly go for #1. #2 would require not only massive code
overhaul, but change of habits, and lots of to_a is unsightly (and you
wouldn't necessarily want to have to hard-code the class of what you
want back, anyway [i.e., the 'a' in 'to_a']).


David

-- 
Rails training from David A. Black and Ruby Power and Light:
   Intro to Ruby on Rails  January 12-15   Fort Lauderdale, FL
   Advancing with Rails    January 19-22   Fort Lauderdale, FL *
   * Co-taught with Patrick Ewing!
See http://www.rubypal.com for details and updates!