From: Tony Arcieri <tony@...>
Date: 2009-05-22T11:24:09+09:00
Subject: Re: Pythonic indentation (or: beating a dead horse)

--0016e64355de7856d1046a76f4c9
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Thu, May 21, 2009 at 5:20 PM, Tony Arcieri <tony@medioh.com> wrote:

> It would involve some ad hoc grammatical analysis by the lexer for the
> purposes of having a pushdown deeper in the stack terminate one which is
> higher up by sneakily injecting "end" tokens.  Aiee!
>

Perhaps a minimalist example of the "expression problem" is in order.

Take this example (in normal Ruby):

result = 2 * [1,2,3].inject(0) do |a, b|
  a + b
end
puts result

prints "12"

Here the 2 * takes precedence over the method invocation (with a block).
Furthermore, nothing precludes us from sticking additional expressions at
the end of the block:

result = 2 * [1,2,3].inject(0) do |a, b|
  a + b
end + 1
puts result

Here the 2 * [method_with_block] is lower precedence than the plus operand,
so we get a parse tree ala:

(+ (* 2 [method_with_block]) 1)

And that doesn't even include setting "result"!

But through funky lexer tricks, we want indent blocks to terminate the
entire expression stack somehow.  We want [method_with_block] when it hits a
dedent to pop all the way up the pushdown stack.  It also means that a
method with a block must always be the rightmost operand in any pushdown.

That is not an easy problem to solve, and afaict cannot be done in a context
free grammar.  It also limits some of the expressivity that Ruby normally
provides.  Quite frequently I do something like:

str = [a, b, c].map do |x|
  thing1 = do_something_to x
  thing2 = do_something_else_to x
  "#{thing1}:#{thing2}"
end.join(",")

An ident-sensitive grammar would preclude that sort of thing.

But hey, if anyone's still about trying to make an indent-sensitive Ruby,
here's a nice simple example based on the one above to cut your teeth on:

result = 2 * [1,2,3].inject(0) do |a, b|
  a + b
puts result

Parse that.  I dare ya.

-- 
Tony Arcieri
medioh.com

--0016e64355de7856d1046a76f4c9--