ruby-core

Nobuyoshi Nakada wrote:
> Hi,
> 
> At Fri, 26 Oct 2007 07:01:53 +0900,
> Robin Stocker wrote in [ruby-core:12946]:
>> - The patch used GNU make extensions which wasn't acceptable, so I gave
>> it another try with standard shell features:
>>
>> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/7695
>>
>> - There was no response, I got discouraged and gave up.
> 
> Sorry, I've missed it.
> 
>> Now that there is interest in it again, maybe it can be resolved once
>> and for all. Would it help if I updated my old patch?
> 
> In 1.9, bison is now required for ripper, so it won't be necessary to
> strip them.
> 

Well, then.  Attached is an updated version of Robin's patch.  I made 
some small changes to some of the token names Robin had picked.  In 
particular, I avoided ambiguity when two tokens represented different 
uses of the same keyword by adding spaces.  E.g. "if" is the if 
statement and "if " is the if modifier. bison won't let me use the same 
string for both tokens, but I suspect that simpler is better in error 
messages.  Also, added a new end_of_file token hardcoded as token 0 
which is what bison seems to use.  I didn't give this one a 
double-quoted name because the double quotes get included in the error 
messages (with my version of bison at least) and I felt that this:

    unexpected end_of_file

is clearer than:

    unexpected "end of file"

This seems to work okay.  make test works, except for tests in 
bootstraptest/test_syntax.rb which explicitly test the content of error 
messages.  If this patch is accepted, I'll patch the test cases to match.

	David

	David

Attachments (1)

patch (5.46 KB, text/x-diff)

Index: parse.y
===================================================================
--- parse.y	(revision 13782)
+++ parse.y	(working copy)
@@ -596,60 +596,77 @@
 /*%
 %token <val>
 %*/
-	keyword_class
-	keyword_module
-	keyword_def
-	keyword_undef
-	keyword_begin
-	keyword_rescue
-	keyword_ensure
-	keyword_end
-	keyword_if
-	keyword_unless
-	keyword_then
-	keyword_elsif
-	keyword_else
-	keyword_case
-	keyword_when
-	keyword_while
-	keyword_until
-	keyword_for
-	keyword_break
-	keyword_next
-	keyword_redo
-	keyword_retry
-	keyword_in
-	keyword_do
-	keyword_do_cond
-	keyword_do_block
-	keyword_do_LAMBDA
-	keyword_return
-	keyword_yield
-	keyword_super
-	keyword_self
-	keyword_nil
-	keyword_true
-	keyword_false
-	keyword_and
-	keyword_or
-	keyword_not
-	modifier_if
-	modifier_unless
-	modifier_while
-	modifier_until
-	modifier_rescue
-	keyword_alias
-	keyword_defined
-	keyword_BEGIN
-	keyword_END
-	keyword__LINE__
-	keyword__FILE__
+        end_of_file 0
+	keyword_class     "class"
+	keyword_module    "module"
+	keyword_def       "def"
+	keyword_undef     "undef"
+	keyword_begin     "begin"
+	keyword_rescue    "rescue"
+	keyword_ensure    "ensure"
+	keyword_end       "end"
+	keyword_if        "if"
+	keyword_unless    "unless"
+	keyword_then      "then"
+	keyword_elsif     "elsif"
+	keyword_else      "else"
+	keyword_case      "case"
+	keyword_when      "when"
+	keyword_while     "while"
+	keyword_until     "until"
+	keyword_for       "for"
+	keyword_break     "break"
+	keyword_next      "next"
+	keyword_redo      "redo"
+	keyword_retry     "retry"
+	keyword_in        "in"
+	keyword_do        "do"
+	keyword_do_cond   "do "
+	keyword_do_block  "do  "
+	keyword_do_LAMBDA "do   "
+	keyword_return    "return"
+	keyword_yield     "yield"
+	keyword_super     "super"
+	keyword_self      "self"
+	keyword_nil       "nil"
+	keyword_true      "true"
+	keyword_false     "false"
+	keyword_and       "and"
+	keyword_or        "or"
+	keyword_not       "not"
+	modifier_if       "if "
+	modifier_unless   "unless "
+	modifier_while    "while "
+        modifier_until    "until "
+	modifier_rescue   "rescue "
+	keyword_alias     "alias"
+	keyword_defined   "defined?"
+	keyword_BEGIN     "BEGIN"
+	keyword_END       "END"
+	keyword__LINE__   "__LINE__"
+	keyword__FILE__   "__FILE__"
 
-%token <id>   tIDENTIFIER tFID tGVAR tIVAR tCONSTANT tCVAR tLABEL
-%token <node> tINTEGER tFLOAT tSTRING_CONTENT tCHAR
-%token <node> tNTH_REF tBACK_REF
-%token <num>  tREGEXP_END
+%token <id>
+  tIDENTIFIER             "identifier"
+  tFID
+  tGVAR                   "global variable"
+  tIVAR                   "instance variable"
+  tCONSTANT               "constant"
+  tCVAR                   "class variable"
+  tLABEL                  "label"
 
+%token <node>
+  tINTEGER                "integer"
+  tFLOAT                  "float"
+  tSTRING_CONTENT         "string content"
+  tCHAR                   "character literal"
+  tNTH_REF                "$n"
+  tBACK_REF               "$&, &`, &\\, or $+"
+
+%token <num>
+  tREGEXP_END             "regexp end"
+
+
 %type <node> singleton strings string string1 xstring regexp
 %type <node> string_contents xstring_contents string_content
 %type <node> words qwords word_list qword_list word
@@ -674,36 +691,52 @@
 /*%
 %type <val> program reswords then do dot_or_colon
 %*/
-%token tUPLUS		/* unary+ */
-%token tUMINUS		/* unary- */
-%token tPOW		/* ** */
-%token tCMP		/* <=> */
-%token tEQ		/* == */
-%token tEQQ		/* === */
-%token tNEQ		/* != */
-%token tGEQ		/* >= */
-%token tLEQ		/* <= */
-%token tANDOP tOROP	/* && and || */
-%token tMATCH tNMATCH	/* =~ and !~ */
-%token tDOT2 tDOT3	/* .. and ... */
-%token tAREF tASET	/* [] and []= */
-%token tLSHFT tRSHFT	/* << and >> */
-%token tCOLON2		/* :: */
-%token tCOLON3		/* :: at EXPR_BEG */
-%token <id> tOP_ASGN	/* +=, -=  etc. */
-%token tASSOC		/* => */
-%token tLPAREN		/* ( */
-%token tLPAREN_ARG	/* ( */
-%token tRPAREN		/* ) */
-%token tLBRACK		/* [ */
-%token tLBRACE		/* { */
-%token tLBRACE_ARG	/* { */
-%token tSTAR		/* * */
-%token tAMPER		/* & */
-%token tLAMBDA		/* -> */
-%token tSYMBEG tSTRING_BEG tXSTRING_BEG tREGEXP_BEG tWORDS_BEG tQWORDS_BEG
-%token tSTRING_DBEG tSTRING_DVAR tSTRING_END tLAMBEG
+%token tUPLUS		"+"    /* unary+ */
+%token tUMINUS		"-"    /* unary- */  
+%token tPOW		"**" 
+%token tCMP		"<=>"
+%token tEQ		"==" 
+%token tEQQ		"==="
+%token tNEQ		"!="
+%token tGEQ		">="
+%token tLEQ		"<="
+%token tANDOP           "&&"
+%token tOROP            "||"
+%token tMATCH           "=~"
+%token tNMATCH          "!~"
+%token tDOT2            ".."
+%token tDOT3    	"..."
+%token tAREF            "[]"
+%token tASET	        "[]=" 
+%token tLSHFT           "<<"
+%token tRSHFT           ">>"
+%token tCOLON2		"::"
+%token tCOLON3		":: "                     /* at EXPR_BEG */
+%token <id> tOP_ASGN	"abbreviated assignment"  /* +=, -=  etc. */
+%token tASSOC		"=>"
+%token tLPAREN		"("
+%token tLPAREN_ARG	"( "
+%token tRPAREN		")"
+%token tLBRACK		"["
+%token tLBRACE		"{"
+%token tLBRACE_ARG	"{ "
+%token tSTAR		"*"
+%token tAMPER		"&"
+%token tLAMBDA		"->"
 
+%token
+  tSYMBEG               "beginning of symbol"
+  tSTRING_BEG           "beginning of string"
+  tXSTRING_BEG          "beginning of execution string"
+  tREGEXP_BEG           "beginning of regexp"
+  tWORDS_BEG            "%w"
+  tQWORDS_BEG           "%W"
+  tSTRING_DBEG          "string interpolation #{}"
+  tSTRING_DVAR          "string interpolation #"
+  tSTRING_END           "end of string"
+  tLAMBEG               "beginning of lambda"
+ 
+
 /*
  *	precedence table
  */

Thread

Prev Next

In This Thread

Prev Next

[#12312] Need Japanese Help - VRuby & new One-Click Ruby Installer with patch 110 — "Curt Hibbs" <curt.hibbs@...>

[#12328] Dir.chdir patch for MS Windows — "Berger, Daniel" <Daniel.Berger@...>

[#12336] Re: gc.c -- possible logic error? — Brent Roman <brent@...>

[#12338] gc.c refactoring — Hugh Sasse <hgs@...>

[#12344] patch to implement Array.permutation — David Flanagan <david@...>

[#12346] a patch for Array.permutation and a generalization of Array.product — David Flanagan <david@...>

[#12351] ruby19 omits program name for missing files — Eric Hodel <drbrain@...7.net>

[#12352] system and Errno::ENOENT in 1.9 — Eric Hodel <drbrain@...7.net>

[#12371] Bug 6400: Time#+ and Time#- not honoring subclass — "Ron Leisti" <ron.leisti@...>

[#12372] Release compatibility/train — Prashant Srinivasan <Prashant.Srinivasan@...>

[#12375] some typos in doc/comments in repos/ruby/trunk — Eugene Ossintsev <eugoss@...>

[#12379] discrepancy between String#oct and String#hex — Ryan Davis <ryand-ruby@...>

[#12383] Include Rake in Ruby 1.9 — "NAKAMURA, Hiroshi" <nakahiro@...>

[#12401] tIDENTIFIER and is_local_id() — Thomas Enebo <Thomas.Enebo@...>

[#12404] ruby-1.9 build Sun-sparc-solaris2.9 — Hugh Sasse <hgs@...>

[#12411] rdoc patch — David Flanagan <david@...>

[#12412] REXML + RSS — Sean Russell <ser@...>

[#12414] Performance Issues with nil, true, false as Hash Keys (1.8.6) — =?ISO-8859-15?Q?Matthias_W=E4chter?= <matthias@...>

[#12418] Patch for configure.in — "Richard Brown" <rbrown@...>

[#12529] Re: Compile error for HEAD — "NARUSE, Yui" <naruse@...>

[#12533] Quick Enumerable#sort_by internals question — Daniel Berger <Daniel.Berger@...>

[#12537] Re: Why does def return nil? — "Jonas Pfenniger" <zimbatm@...>

[#12539] Ordered Hashes in 1.9? — Michael Neumann <mneumann@...>

[#12540] Re: Why does def return nil? — "Jonas Pfenniger" <zimbatm@...>

[#12547] Bug in win32ole — "Waters, Chris" <cwaters@...>

[#12550] to let Ruby 1.9 to process signals while blocked in ext/readline — Brent Roman <brent@...>

[#12564] Re: net/http.rb, patch, possible bug, test problem — Hugh Sasse <hgs@...>

[#12566] Revised net/http patch — Hugh Sasse <hgs@...>

[#12568] $" and require — "Tim Morgan" <tmorgan99@...>

[#12573] File::Stat#grpowned? is wrong — Bertram Scharpf <lists@...>

[#12575] Marshal.dump doesn't dump string encoding — David Flanagan <david@...>

[#12578] Possible memory leak in ruby-1.8.6-p110?? — "M. Edward (Ed) Borasky" <znmeb@...>

[#12579] iconv enhancement in Ruby 1.9 — "Eugene Ossintsev" <eugoss@...>

[#12581] ruby.exe exit code 255 — "Drew Sears" <dsears@...>

[#12584] Broken Ruby/ODBC due rbconfig COMMON_MACROS — "Luis Lavena" <luislavena@...>

[#12587] Confusion about arities — Charles Oliver Nutter <charles.nutter@...>

[#12588] MatchData#select rdoc and arity incorrect — Charles Oliver Nutter <charles.nutter@...>

[#12598] adds __file__ and __line__ methods to Methods and Procs — Brent Roman <brent@...>

[#12599] private method dispatch changes dropped from 1.9? — David Flanagan <david@...>

[#12610] Re: Release compatibility/train — Yukihiro Matsumoto <matz@...>

[#12615] Re: Why does def return nil? — Brent Roman <brent@...>

[#12617] Question about heap_slots in gc.c — Hongli Lai <h.lai@...>

[#12618] StringIO is not IO? — Hongli Lai <h.lai@...>

[#12623] Behaviour of String Objects - Why Not "Each" for Characters and Enumerable? — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12629] file encoding comments and a patch to parse.y — David Flanagan <david@...>

[#12632] Defining unicode methods — "Daniel Berger" <djberg96@...>

[#12636] Proc.yield method — David Flanagan <david@...>

[#12638] New lambda -> syntax — David Flanagan <david@...>

[#12639] test problem after compiling — Martin Duerst <duerst@...>

[#12664] minor rdoc patch to encoding.c — David Flanagan <david@...>

[#12665] Another encoding.c patch — David Flanagan <david@...>

[#12666] encoding.c bug — David Flanagan <david@...>

[#12670] Bug in Numeric#divmod — "Dirk Traulsen" <dirk.traulsen@...>

[#12675] Re: Import gem to Ruby 1.9 — Brent Roman <brent@...>

[#12676] Minor Array.shuffle patch — David Flanagan <david@...>

[#12679] documentation patch for enum.c — David Flanagan <david@...>

[#12681] Unicode: Progress? — murphy <murphy@...>

[#12693] retry: revised 1.9 http patch — Hugh Sasse <hgs@...>

[#12695] New Rails plugin 'Liftoff' is available. — "Ben Tompkins" <nbitspoken@...>

[#12697] Range.first is incompatible with Enumerable.first — David Flanagan <david@...>

[#12703] Long encoding names with -K and bad error message — David Flanagan <david@...>

[#12705] A couple of bugs? — "John Lam (DLR)" <jflam@...>

[#12706] Re: A couple of bugs? — "Gavin Kistner" <gavin.kistner@...>

[#12710] enum.c patch: fixes Enumerable.cycle and rdoc bugs — David Flanagan <david@...>

[#12714] Re: A couple of bugs? — "Gavin Kistner" <gavin.kistner@...>

[#12750] Re: [OT] Re: Should *most* memory be release back to the system? — "M. Edward (Ed) Borasky" <znmeb@...>

[#12754] Improving 'syntax error, unexpected $end, expecting kEND'? — Hugh Sasse <hgs@...>

[#12757] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12758] Encoding::primary_encoding — David Flanagan <david@...>

[#12759] Reading UTF-8 Data contains BOM — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12764] SystemStackError 1.8.6 (stack only 25 frames) — Dan Janowski <danj@3skel.com>

[#12767] \u escapes in string literals: proof of concept implementation — David Flanagan <david@...>

[#12776] Clarification of Ruby release plans? 1.8.7? 1.9.1? 2.0? — Lucas Nussbaum <lucas@...>

[#12779] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12780] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12787] How to specify in Ruby 1.9 the expected file encoding — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12795] patch for String.concat — David Flanagan <david@...>

[#12825] clarification of ruby libraries installation paths? — Lucas Nussbaum <lucas@...>

[#12834] Bug Tracker Access Request — James Edward Gray II <james@...>

[#12847] make test-all — "Florian Ebeling" <florian.ebeling@...>

[#12598] adds file and line methods to Methods and Procs — Brent Roman <brent@...>

[#12750] Re: [OT] Re: Should most memory be release back to the system? — "M. Edward (Ed) Borasky" <znmeb@...>