From: Colin Bartlett <colinb2r@...>
Date: 2010-04-17T06:20:20+09:00
Subject: Re: Is it possible, a fully general Enumerable#recursive ?

On Mon, Apr 12, 2010 at 6:00 PM, Caleb Clausen <vikkous@gmail.com> wrote:
>> On Apr 12, 11:08 am, Robert Klemme <shortcut...@googlemail.com> wrote:
>>> I don't really understand why this thread seems to be so complicated.
>>> I suspect that either I am missing something or not understanding
>>> properly, or other contributors to this thread are missing something.
>>> :-)
Colin's comment: Or possibly both apply? (I think that's true for me! :-))

> I find this subject extremely confusing.
> Colin's answer to my last email is still causing me puzzlement.
Two apologies: first, for not replying sooner to this.
(I was going to do a combined response to this and to what Tom and Robert
 have said, hence the delay: I then decided that it would be better
 to make a reply to you on this specific point, and follow up later
 with more general observations. I think I've nearly got a proof of concept
 thing working. That said, I - for now - still agree with Tom's comment:
   "After working on this for far too long, I have concluded that
    a highly generalized implementation of recursion for all
    (or at least most) enumerable methods not feasible.")

Second, sorry for causing puzzlement. I (mostly) try not to be
deliberately obfuscatory. Was it (1) or (2) (or both!)
of my reply to your post that was/is causing puzzlement?

(1) Using #map inside #visit combined with using Enumerable#map
    seems to (within #visit) produce the structure of the original array,
    but with nil values instead of the original values.
    (Changing #map inside #visit to #each gets back the original values.)

(2) Irrespective of the internal results from #visit, Enumerable#map
    collects the successive results of the block for Enumerable#map
    into a new (flat) array.

Does the following reduce the puzzlement?

Taking point (2) of my "puzzling" post first, I was trying to say
that I didn't think that there was a bug
for two - or rather one and a half - reasons.

The full reason, rewording (2): I think (at least with hindsight:
I'm not sure that I would think this if I was looking at
the documentation and code without this thread prompting it)
Tom's code was producing the behaviour expected according to
the Ruby 1.9 documentation: that is Enumerable:map returns
a "flat" array built up from successive elements yielded
from an iteration, and that the Enumerable:map returned array
will be "flat" (and is intended to be "flat")
even if the underlying iteration is recursive.

The half reason is that as I understand it, the basic behavior
of Enumerator and Enumerable assumes that the underlying
iteration stream is "flat".
You can set up a recursive iteration, and use that with Enumerator
and Enumerable, but the recursion is hidden from Enumerator/Enumerable.
So the "expected" behaviour from an Enumerator/Enumerable method
is similar to what you'd expect from running the method against
a flat array. (If anyone knows different, *please* correct that.)

So, turning to (1), I think the question is not:
  why is array.visit.map flattening the result?
but rather:
  why is array.visit{block} preserving the nesting in the array?

What I was trying to say was that because Tom (I think correctly
for what he was wanting to do) was using Array#map inside
his recursive iterator, then internally to the iterator
the nesting structure of the arrays was preserved (but that
for reasons I didn't understand with values changed to nil),
but that also internally a "flat" array of the iterated elements
(possibly with different values from the returned value from
the yield) was being built up, and that for Enumerable#map
it is that "flat" array that is returned.

Does that help in any way?

> I have often found that too much recursion is confusing.
Yes - me too! My test version of a "general" recursive enumerator
looks more like C (or Fortran!) code than Ruby.
I sort of instinctively think in terms of what's underlying the recursion,
which can get in the way of me just using recursion,
when just using recursion might well be easier!

> This whole problem is going to be a lot easier if it's structured
> as an external iterator first and foremost.
> I have an extern iterator library (sequence)
> maybe I can figure out how to do this within that.
> (I don't really have a good grip on how Enumerator works, myself...)

Provided I understand more or less correctly what you mean by
an "extern iterator library", then it might be something that
I could understand!