From: Yusuke Endoh <redmine@...>
Date: 2010-04-08T00:31:22+09:00
Subject: [ruby-core:29304] [Bug #1240] parser bug in 1.8.7 and 1.9.1p0

Issue #1240 has been updated by Yusuke Endoh.

Assigned to changed from Yukihiro Matsumoto to Yusuke Endoh

Hi,

2009/3/3 Thomer Gil <redmine@ruby-lang.org>:
> # ruby parser accepts first line, not the second.
> x y { "#{}".z { } }
> x y { "#{}".z do end }


Interesting.  I found similar bugs:

  while ("#{}".foo { }   ); end # ok
  while ("#{}".foo do end); end # parse error

  while ->{ 1.times { }    }.call; done # ok
  while ->{ 1.times do end }.call; done # parse error

The cause is mishandling of cond_stack and cmdarg_stack.
When "#{}" is parsed, the parser pushes a value into these stacks once
(at #{ (tSTRING_DBEG)), and pops twice (at '}' and action of the rule
`string_content'.)

In addition, the parser forgets to push at ->{ (tLAMBEG).

I think it is better for not parser but lexer to push into the stacks
at tSTRING_DBEG.  See my patch.


BTW, I also noticed that block call with `do' keyword does not work in
`until' condition:

  until begin 1.times { }    end do end # ok
  until begin 1.times do end end do end # parse error
                      ~~
  until if true then 1.times { }    end do end # ok
  until if true then 1.times do end end do end # parse error
                             ~~
  until until true do 1.times { }    end do end # ok
  until until true do 1.times do end end do end # parse error
                              ~~
  until class Foo; 1.times { }   ; end do end # ok
  until class Foo; 1.times do end; end do end # parse error
                           ~~
  until case; when true; 1.times { }   ; end do end # ok
  until case; when true; 1.times do end; end do end # parse error
                                 ~~

This is because the underlined `do's are not considered as block call
but beginning of `until' body.

Although this is confusing a little and can be actually fixed, the fix
needs many COND_PUSH(0)/COND_POP(), which may decrease performance and
code maintenability.  In addition, writing such a long and complex
condition directly is absolutely bad (even insane) style.
So, we should accept the above behaviors as spec, I think.


Here is a patch to fix the issue Thomer reported.
I'll commit it unless there is objection.

diff --git a/parse.y b/parse.y
index 340a825..f234ae3 100644
--- a/parse.y
+++ b/parse.y
@@ -4033,14 +4033,10 @@ string_content	: tSTRING_CONTENT
 			$<node>$ = lex_strterm;
 			lex_strterm = 0;
 			lex_state = EXPR_BEG;
-			COND_PUSH(0);
-			CMDARG_PUSH(0);
 		    }
 		  compstmt '}'
 		    {
 			lex_strterm = $<node>2;
-			COND_LEXPOP();
-			CMDARG_LEXPOP();
 		    /*%%%*/
 			if ($3) $3->flags &= ~NODE_FL_NEWLINE;
 			$$ = new_evstr($3);
@@ -5873,6 +5869,8 @@ parser_parse_string(struct parser_params *parser, NODE *quote)
 	    pushback(c);
 	    return tSTRING_DVAR;
 	  case '{':
+	    COND_PUSH(0);
+	    CMDARG_PUSH(0);
 	    return tSTRING_DBEG;
 	}
 	tokadd('#');
@@ -6070,6 +6068,8 @@ parser_here_document(struct parser_params *parser, NODE *here)
 		pushback(c);
 		return tSTRING_DVAR;
 	      case '{':
+		COND_PUSH(0);
+		CMDARG_PUSH(0);
 		return tSTRING_DBEG;
 	    }
 	    tokadd('#');
@@ -7314,6 +7314,8 @@ parser_yylex(struct parser_params *parser)
 	    lex_state = EXPR_BEG;
 	    lpar_beg = 0;
 	    --paren_nest;
+	    COND_PUSH(0);
+	    CMDARG_PUSH(0);
 	    return tLAMBEG;
 	}
 	if (IS_ARG() || lex_state == EXPR_END)

-- 
Yusuke ENDOH <mame@tsg.ne.jp>
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1240

----------------------------------------
http://redmine.ruby-lang.org