[ ruby-Bugs-1745 ] ruby-lex hangs if there is no newline at end of file

Bugs item #1745, was opened at 2005-04-06 22:43
You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=1745&group_id=426

Category: Standard Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gunther Schmidl (gschmidl)
Assigned to: Nobody (None)
Summary: ruby-lex hangs if there is no newline at end of file

Initial Comment:
When using RubyParser.parse to parse *.rb files for gettext strings, I noticed that ruby-lex would go into an endless loop trying to fetch the last token when there was no newline at the end of a file.

Output when hitting Ctrl-Break:

Processing app/controllers/account_controller.rb
C:/programme/ruby/lib/ruby/1.8/irb/ruby-lex.rb:847:in `identify_identifier': In
errupt
        from C:/programme/ruby/lib/ruby/1.8/irb/ruby-lex.rb:711:in `lex_int2'
        from C:/programme/ruby/lib/ruby/1.8/irb/ruby-lex.rb:705:in `call'
        from C:/programme/ruby/lib/ruby/1.8/irb/slex.rb:234:in `match_io'
        from C:/programme/ruby/lib/ruby/1.8/irb/slex.rb:73:in `match'
        from C:/programme/ruby/lib/ruby/1.8/irb/ruby-lex.rb:280:in `token'
        from C:/programme/ruby/lib/ruby/site_ruby/1.8/gettext/parser/ruby.rb:27
in `token'
        from C:/programme/ruby/lib/ruby/site_ruby/1.8/gettext/parser/ruby.rb:15
:in `parse_lines'
        from C:/programme/ruby/lib/ruby/site_ruby/1.8/gettext/parser/ruby.rb:14
:in `parse'
        from C:/programme/ruby/lib/ruby/site_ruby/1.8/gettext/parser/ruby.rb:13
:in `each'
        from C:/programme/ruby/lib/ruby/site_ruby/1.8/gettext/parser/ruby.rb:13
:in `parse'
        from ./script/rgettext.rb:63:in `start'
        from ./script/rgettext.rb:51:in `each'
        from ./script/rgettext.rb:51:in `start'
        from script/localize:31

Output from lexer via ruby -d:

Processing app/controllers/account_controller.rb
#<RubyToken::TkCLASS:0x2e05f28>
#<RubyToken::TkCONSTANT:0x2e04f98>
#<RubyToken::TkLT:0x2e047e8>
#<RubyToken::TkCONSTANT:0x2e03ae0>
#<RubyToken::TkNL:0x2e037b0>
#<RubyToken::TkEND:0x2e030f0>
#<RubyToken::TkIDENTIFIER:0x2e02b50>: d
#<RubyToken::TkIDENTIFIER:0x2e02580>: d
#<RubyToken::TkIDENTIFIER:0x2e01fe0>: d
#<RubyToken::TkIDENTIFIER:0x2e01a40>: d
#<RubyToken::TkIDENTIFIER:0x2e01470>: d
#<RubyToken::TkIDENTIFIER:0x2e00eb8>: d
#<RubyToken::TkIDENTIFIER:0x2e00948>: d
#<RubyToken::TkIDENTIFIER:0x2e003c0>: d
#<RubyToken::TkIDENTIFIER:0x2dffe38>: d
(continues in this vein forever, with the hex number changing)

Adding a newline at the end of the file fixes this problem.

----------------------------------------------------------------------

You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=1745&group_id=426