Re: return_value? [Was: Re: want_object? - possible?]

On 4/15/05, Eric Hodel <[e-mail]> wrote:
> On 14 Apr 2005, at 22:20, Mark Hubbart wrote:
> 
> > On 4/14/05, David A. Black <[e-mail]> wrote:
> >
> >> I still don't like the look-ahead aspect of this.
> >
> > I don't see it as a "what happens after" type thing; more of a "what's
> > my position" type thing. It would simply glance into the parse tree,
> > checking if this method call is in the outer ring, so to speak, of a
> > statement. If it is, the function can optimize itself not to bother
> > returning any expensive values, since they would just be GC fodder.
> 
> Sneaking a peak up the parse tree is quite difficult, because its
> linked down, not up.

I thought that might be a problem... perhaps the information could
come through the other way then, passed up as environment?

> Besides, if you have expensive operations your
> API should allow users to avoid them until necessary by not bundling
> them with other operations.

In Ruby, everything has a return value. I quite often hear people
saying "method/statement x should return a useful value", because
they've gotten used to getting useful return values. Almost everything
in Ruby gives one. Yes, you can come up with another (longer) method
name that does the same thing but returns the expensive value, but
istm that's how things like Java's obscenely long class and method
names happen; by trying to factor out all the possible different
situations.

> >> It is no step at all in kind, and only a small step in degree, from
> >> the question "Is what I return going to be used in a chain?" to the
> >> question "Is what I return going to be chained to the 'flatten'
> >> method?" (or any other specific method).  It strikes me as a whole
> >> type of knowledge that is not the business of the method that's
> >> currently executing.
> >
> > The example above, while being very funny, is nothing like what's
> > being suggested:
> >  - it's a custom method grabbing absurdly specific external context
> > information
> >  - it's using the requested information to do something entirely silly
> > in it own right
> > By using this example, it appears that you are trying to say that it's
> > a slippery slope; from a method telling you whether a return value is
> > in a position to be used, to a specialized method call which tells you
> > whether a specific method of a certain class will be called on the
> > result.
> 
> I think the (snipped) example was highly illustrative.

Maybe I'm wrong. The code seemed intended to be ridiculous, a parody
of "what might happen". I mean really, "subsequently_flattened?"? But
I suppose I might have misjudged.

> If you want to avoid doing work in the future depending on some
> condition, make it explicit!
> 
> Build your API to be flexible enough that developers do the work when
> it needs to be done.
> 
> > Since we haven't been given an example of how just knowing whether the
> > return value is used is harmful, I'll present a simpler example of its
> > possible usefulness. I'll even shorten the method name a bit :)
> >
> > class MarkovChain
> >   [...]
> >   def parse_text(text)
> >     words = text.scan(WORDS)
> >     links = link_words(words)
> >     add_to_library(links)
> >     links.compute_statistics if return_value?
> 
> This looks poorly factored and too highly coupled to me.  Instead you
> should have a method that updates the links/library (#add_text), and a
> separate method that computes statistics (#current_stats).  If
> computing statistics is not a critical part of parsing text, then it
> should not be in the text parsing method!

I agree, it is very poorly factored. I was trying to give a quick
example that was short, so I ended up with something fairly ugly and
very procedural.

As for your refactoring ideas; compute_statistics is called on the
linked words object, and gets the statistics only for that piece of
text. So parse_text(foo) will return only the statistics for the text
in foo. Also, the links object (I'm supposing) is discarded once it's
data is fed to the library. So the statistical information would then
be gone forever. The only way to refactor this and have the same
functionality available would be to have two methods, parse_text
(returns nil) and parse_text_and_return_statistics (returns the
statistics). But he two could be combined, transparently, using
return_value?, or something like it.

> Decoupling makes much more sense because I can stream hundreds of
> megabytes through #add_text (more than can fit in memory) and when I'm
> done streaming I can grab #current_stats.
> 
> To do that with your return_value? proposal I would have to stream all
> the values through #parse_text then call #parse_text with an empty
> string to get the statistics I want out.  This makes for very ugly code
> (magic value of '').  (And if you wish to respond by saying "oh, I
> could break that up anyhow" then what is the point of using
> return_value? if your API no longer needs it?)

As I said, parse_text(foo) returns the stats for foo, not the entire
library. Once the method finishes, the value is irretrievable. Markov
chaining is already extremely expensive for large amounts of text;
keeping those values around for later consumption would be terribly
expensive.

> > The more I think on this, the more it seems like a good feature. Sure
> > it's a little magic, but then, so is callcc :) And if it wasn't a
> > little magic, we would just be implementing it ourselves, in pure
> > Ruby.
> 
> callcc is not very magical, in fact, it is quite simple to implement.
> It only appears magical because you don't use it.

I use it. I still think it's a little bit magical :) As for it's being
easy to implement, I sincerely doubt it:
<http://www.rubytalk.org/99008>

> You can't easily, or even with a little work, or a lot of magic,
> implement a return_value? feature because you cannot reliably, or
> without great expense, obtain the information required to implement it.
> 
> Either you need to save extra state (which will be unused 99% of the
> time), or you need to figure out where you are in the parse tree by
> moving to the top of the method and re-tracing your steps (which may
> not work, because you may take a different path), or you need to add
> bidirectional links to the parse tree.  Any of these changes come with
> a severe performance penalty.

If it can't be done without a performance penalty, then it won't be
done, I'm sure. I can honestly say I trust Matz to make the call; he's
more conservative on these things than I am, but I think I like it :)

> The better solution is to implement a properly factored API.

My example was, perhaps, poorly factored. But I still think that
return_value? (or something like it) would be useful, and performance
penalties aside (it's unimplemented, so any penalties are just
conjecture as of yet) it has many nice features:
 - it's clean; it's fairly obvious what you are doing when you use it.
 - it's optional; you don't have to use it.
 - it allows simple, transparent optimization.

If my C skills were better, I'd see what was required to implement it.
Unfortunately, they are not, so if noone who has better skills than me
decides to try it, it won't be tested.

But in the end, all will be decided my Matz. So if he ignores this
thread, I guess it's not to be :)

cheers,
Mark