ruby-core

David Flanagan wrote:

> Another possibility is to hack yyerror to clean up the error messages 
> for us, possibly stripping the quotation marks (so that we could have 
> "\"rescue\" modifier" with only the keyword in quotes and removing any 
> hacky spaces or non-printing characters we stuck into the token strings 
> to fool bison.  I haven't done any real C string manipulation complete 
> with memory allocation and freeing in 10 years, however, so I'm a little 
> reluctant to attempt this myself...

Okay, despite my protests of not knowing how to do this, I figured it 
out.  The yyerror function already uses ALLOCA_N once to allocate a 
string on the stack, so I don't feel bad about using it again.

Now, before printing the error message it strips all double quotes out 
of it.  So we can get error messages like "unexpected 'rescue' modifier"
And I can get "end of file" in an error message without quotes. 
Furthermore, we can solve the issue of having to have unique strings for 
each token by just inserting additional escaped double-quotes characters 
where I was using spaces before.  They'll be stripped out.

Its kind of hacky, but it makes for nice error messages. Updated patch 
is attached.

	David

Attachments (1)

patch (6.48 KB, text/x-diff)

Index: parse.y
===================================================================
--- parse.y	(revision 13782)
+++ parse.y	(working copy)
@@ -592,64 +592,87 @@
 }
 
 /*%%%*/
+/* 
+ * Token names in double quotes are just for use in error messages. 
+ * The escaped double-quotes within the names are stripped out in yyerror.
+ * They are required so that two distinct tokens don't have the same name
+ */
 %token
 /*%
 %token <val>
 %*/
-	keyword_class
-	keyword_module
-	keyword_def
-	keyword_undef
-	keyword_begin
-	keyword_rescue
-	keyword_ensure
-	keyword_end
-	keyword_if
-	keyword_unless
-	keyword_then
-	keyword_elsif
-	keyword_else
-	keyword_case
-	keyword_when
-	keyword_while
-	keyword_until
-	keyword_for
-	keyword_break
-	keyword_next
-	keyword_redo
-	keyword_retry
-	keyword_in
-	keyword_do
-	keyword_do_cond
-	keyword_do_block
-	keyword_do_LAMBDA
-	keyword_return
-	keyword_yield
-	keyword_super
-	keyword_self
-	keyword_nil
-	keyword_true
-	keyword_false
-	keyword_and
-	keyword_or
-	keyword_not
-	modifier_if
-	modifier_unless
-	modifier_while
-	modifier_until
-	modifier_rescue
-	keyword_alias
-	keyword_defined
-	keyword_BEGIN
-	keyword_END
-	keyword__LINE__
-	keyword__FILE__
 
-%token <id>   tIDENTIFIER tFID tGVAR tIVAR tCONSTANT tCVAR tLABEL
-%token <node> tINTEGER tFLOAT tSTRING_CONTENT tCHAR
-%token <node> tNTH_REF tBACK_REF
-%token <num>  tREGEXP_END
+        end_of_file 0     "end of file"
+	keyword_class     "'class'"
+	keyword_module    "'module'"
+	keyword_def       "'def'"
+	keyword_undef     "'undef'"
+	keyword_begin     "'begin'"
+	keyword_rescue    "'rescue'"
+	keyword_ensure    "'ensure'"
+	keyword_end       "'end'"
+	keyword_if        "'if'"
+	keyword_unless    "'unless'"
+	keyword_then      "'then'"
+	keyword_elsif     "'elsif'"
+	keyword_else      "'else'"
+	keyword_case      "'case'"
+	keyword_when      "'when'"
+	keyword_while     "'while'"
+	keyword_until     "'until'"
+	keyword_for       "'for'"
+	keyword_break     "'break'"
+	keyword_next      "'next'"
+	keyword_redo      "'redo'"
+	keyword_retry     "'retry'"
+	keyword_in        "'in'"
+	keyword_do        "'do'"
+	keyword_do_cond   "'do'\""
+	keyword_do_block  "'do'\"\""
+	keyword_do_LAMBDA "'do'\"\"\""
+	keyword_return    "'return'"
+	keyword_yield     "'yield'"
+	keyword_super     "'super'"
+	keyword_self      "'self'"
+	keyword_nil       "'nil'"
+	keyword_true      "'true'"
+	keyword_false     "'false'"
+	keyword_and       "'and'"
+	keyword_or        "'or'"
+	keyword_not       "'not'"
+	modifier_if       "'if' modifer"
+	modifier_unless   "'unless' modifier"
+	modifier_while    "'while' modifer"
+        modifier_until    "'until' modifer"
+	modifier_rescue   "'rescue' modifier"
+	keyword_alias     "'alias'"
+	keyword_defined   "'defined?'"
+	keyword_BEGIN     "'BEGIN'"
+	keyword_END       "'END'"
+	keyword__LINE__   "'__LINE__'"
+	keyword__FILE__   "'__FILE__'"
 
+%token <id>
+  tIDENTIFIER             "identifier"
+  tFID
+  tGVAR                   "global variable"
+  tIVAR                   "instance variable"
+  tCONSTANT               "constant"
+  tCVAR                   "class variable"
+  tLABEL                  "label"
+
+%token <node>
+  tINTEGER                "integer"
+  tFLOAT                  "float"
+  tSTRING_CONTENT         "string content"
+  tCHAR                   "character literal"
+  tNTH_REF                "$n"
+  tBACK_REF               "$&, &`, &\\, or $+"
+
+%token <num>
+  tREGEXP_END             "regexp end"
+
+
 %type <node> singleton strings string string1 xstring regexp
 %type <node> string_contents xstring_contents string_content
 %type <node> words qwords word_list qword_list word
@@ -674,36 +697,52 @@
 /*%
 %type <val> program reswords then do dot_or_colon
 %*/
-%token tUPLUS		/* unary+ */
-%token tUMINUS		/* unary- */
-%token tPOW		/* ** */
-%token tCMP		/* <=> */
-%token tEQ		/* == */
-%token tEQQ		/* === */
-%token tNEQ		/* != */
-%token tGEQ		/* >= */
-%token tLEQ		/* <= */
-%token tANDOP tOROP	/* && and || */
-%token tMATCH tNMATCH	/* =~ and !~ */
-%token tDOT2 tDOT3	/* .. and ... */
-%token tAREF tASET	/* [] and []= */
-%token tLSHFT tRSHFT	/* << and >> */
-%token tCOLON2		/* :: */
-%token tCOLON3		/* :: at EXPR_BEG */
-%token <id> tOP_ASGN	/* +=, -=  etc. */
-%token tASSOC		/* => */
-%token tLPAREN		/* ( */
-%token tLPAREN_ARG	/* ( */
-%token tRPAREN		/* ) */
-%token tLBRACK		/* [ */
-%token tLBRACE		/* { */
-%token tLBRACE_ARG	/* { */
-%token tSTAR		/* * */
-%token tAMPER		/* & */
-%token tLAMBDA		/* -> */
-%token tSYMBEG tSTRING_BEG tXSTRING_BEG tREGEXP_BEG tWORDS_BEG tQWORDS_BEG
-%token tSTRING_DBEG tSTRING_DVAR tSTRING_END tLAMBEG
+%token tUPLUS		"'+'"    /* unary+ */
+%token tUMINUS		"'-'"    /* unary- */  
+%token tPOW		"'**'" 
+%token tCMP		"'<=>'"
+%token tEQ		"'=='" 
+%token tEQQ		"'==='"
+%token tNEQ		"'!='"
+%token tGEQ		"'>='"
+%token tLEQ		"'<='"
+%token tANDOP           "'&&'"
+%token tOROP            "'||'"
+%token tMATCH           "'=~'"
+%token tNMATCH          "'!~'"
+%token tDOT2            "'..'"
+%token tDOT3    	"'...'"
+%token tAREF            "'[]'"
+%token tASET	        "'[]='" 
+%token tLSHFT           "'<<'"
+%token tRSHFT           "'>>'"
+%token tCOLON2		"'::'"
+%token tCOLON3		"'::'\""                  /* at EXPR_BEG */
+%token <id> tOP_ASGN	"abbreviated assignment"  /* +=, -=  etc. */
+%token tASSOC		"'=>'"
+%token tLPAREN		"'('"
+%token tLPAREN_ARG	"'('\""
+%token tRPAREN		"')'"
+%token tLBRACK		"'['"
+%token tLBRACE		"'{'"
+%token tLBRACE_ARG	"'{'\""
+%token tSTAR		"'*'"
+%token tAMPER		"'&'"
+%token tLAMBDA		"'->'"
 
+%token
+  tSYMBEG               "beginning of symbol"
+  tSTRING_BEG           "beginning of string"
+  tXSTRING_BEG          "beginning of execution string"
+  tREGEXP_BEG           "beginning of regexp"
+  tWORDS_BEG            "'%w'"
+  tQWORDS_BEG           "'%W'"
+  tSTRING_DBEG          "string interpolation"
+  tSTRING_DVAR          "string interpolation\""
+  tSTRING_END           "end of string"
+  tLAMBEG               "beginning of lambda"
+ 
+
 /*
  *	precedence table
  */
@@ -4586,7 +4625,21 @@
     const char *p, *pe;
     char *buf;
     int len, i;
+    char *c;
 
+    /*
+     * Strip double quotes from the error message
+     * This is a hack to clean up error messages generated by bison
+     * See the token strings declared with %token
+     */
+    len = strlen(msg);
+    buf = ALLOCA_N(char, len);
+    for(i = 0, c = buf; i < len; i++) {
+	if (msg[i] != '"') *c++ = msg[i];
+    }
+    *c = '\0';  /* NUL-terminate */
+    msg = buf;
+
     compile_error(PARSER_ARG "%s", msg);
     p = lex_p;
     while (lex_pbeg <= p) {

Thread

Prev Next

In This Thread

Prev Next

[#12312] Need Japanese Help - VRuby & new One-Click Ruby Installer with patch 110 — "Curt Hibbs" <curt.hibbs@...>

[#12328] Dir.chdir patch for MS Windows — "Berger, Daniel" <Daniel.Berger@...>

[#12336] Re: gc.c -- possible logic error? — Brent Roman <brent@...>

[#12338] gc.c refactoring — Hugh Sasse <hgs@...>

[#12344] patch to implement Array.permutation — David Flanagan <david@...>

[#12346] a patch for Array.permutation and a generalization of Array.product — David Flanagan <david@...>

[#12351] ruby19 omits program name for missing files — Eric Hodel <drbrain@...7.net>

[#12352] system and Errno::ENOENT in 1.9 — Eric Hodel <drbrain@...7.net>

[#12371] Bug 6400: Time#+ and Time#- not honoring subclass — "Ron Leisti" <ron.leisti@...>

[#12372] Release compatibility/train — Prashant Srinivasan <Prashant.Srinivasan@...>

[#12375] some typos in doc/comments in repos/ruby/trunk — Eugene Ossintsev <eugoss@...>

[#12379] discrepancy between String#oct and String#hex — Ryan Davis <ryand-ruby@...>

[#12383] Include Rake in Ruby 1.9 — "NAKAMURA, Hiroshi" <nakahiro@...>

[#12401] tIDENTIFIER and is_local_id() — Thomas Enebo <Thomas.Enebo@...>

[#12404] ruby-1.9 build Sun-sparc-solaris2.9 — Hugh Sasse <hgs@...>

[#12411] rdoc patch — David Flanagan <david@...>

[#12412] REXML + RSS — Sean Russell <ser@...>

[#12414] Performance Issues with nil, true, false as Hash Keys (1.8.6) — =?ISO-8859-15?Q?Matthias_W=E4chter?= <matthias@...>

[#12418] Patch for configure.in — "Richard Brown" <rbrown@...>

[#12529] Re: Compile error for HEAD — "NARUSE, Yui" <naruse@...>

[#12533] Quick Enumerable#sort_by internals question — Daniel Berger <Daniel.Berger@...>

[#12537] Re: Why does def return nil? — "Jonas Pfenniger" <zimbatm@...>

[#12539] Ordered Hashes in 1.9? — Michael Neumann <mneumann@...>

[#12540] Re: Why does def return nil? — "Jonas Pfenniger" <zimbatm@...>

[#12547] Bug in win32ole — "Waters, Chris" <cwaters@...>

[#12550] to let Ruby 1.9 to process signals while blocked in ext/readline — Brent Roman <brent@...>

[#12564] Re: net/http.rb, patch, possible bug, test problem — Hugh Sasse <hgs@...>

[#12566] Revised net/http patch — Hugh Sasse <hgs@...>

[#12568] $" and require — "Tim Morgan" <tmorgan99@...>

[#12573] File::Stat#grpowned? is wrong — Bertram Scharpf <lists@...>

[#12575] Marshal.dump doesn't dump string encoding — David Flanagan <david@...>

[#12578] Possible memory leak in ruby-1.8.6-p110?? — "M. Edward (Ed) Borasky" <znmeb@...>

[#12579] iconv enhancement in Ruby 1.9 — "Eugene Ossintsev" <eugoss@...>

[#12581] ruby.exe exit code 255 — "Drew Sears" <dsears@...>

[#12584] Broken Ruby/ODBC due rbconfig COMMON_MACROS — "Luis Lavena" <luislavena@...>

[#12587] Confusion about arities — Charles Oliver Nutter <charles.nutter@...>

[#12588] MatchData#select rdoc and arity incorrect — Charles Oliver Nutter <charles.nutter@...>

[#12598] adds __file__ and __line__ methods to Methods and Procs — Brent Roman <brent@...>

[#12599] private method dispatch changes dropped from 1.9? — David Flanagan <david@...>

[#12610] Re: Release compatibility/train — Yukihiro Matsumoto <matz@...>

[#12615] Re: Why does def return nil? — Brent Roman <brent@...>

[#12617] Question about heap_slots in gc.c — Hongli Lai <h.lai@...>

[#12618] StringIO is not IO? — Hongli Lai <h.lai@...>

[#12623] Behaviour of String Objects - Why Not "Each" for Characters and Enumerable? — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12629] file encoding comments and a patch to parse.y — David Flanagan <david@...>

[#12632] Defining unicode methods — "Daniel Berger" <djberg96@...>

[#12636] Proc.yield method — David Flanagan <david@...>

[#12638] New lambda -> syntax — David Flanagan <david@...>

[#12639] test problem after compiling — Martin Duerst <duerst@...>

[#12664] minor rdoc patch to encoding.c — David Flanagan <david@...>

[#12665] Another encoding.c patch — David Flanagan <david@...>

[#12666] encoding.c bug — David Flanagan <david@...>

[#12670] Bug in Numeric#divmod — "Dirk Traulsen" <dirk.traulsen@...>

[#12675] Re: Import gem to Ruby 1.9 — Brent Roman <brent@...>

[#12676] Minor Array.shuffle patch — David Flanagan <david@...>

[#12679] documentation patch for enum.c — David Flanagan <david@...>

[#12681] Unicode: Progress? — murphy <murphy@...>

[#12693] retry: revised 1.9 http patch — Hugh Sasse <hgs@...>

[#12695] New Rails plugin 'Liftoff' is available. — "Ben Tompkins" <nbitspoken@...>

[#12697] Range.first is incompatible with Enumerable.first — David Flanagan <david@...>

[#12703] Long encoding names with -K and bad error message — David Flanagan <david@...>

[#12705] A couple of bugs? — "John Lam (DLR)" <jflam@...>

[#12706] Re: A couple of bugs? — "Gavin Kistner" <gavin.kistner@...>

[#12710] enum.c patch: fixes Enumerable.cycle and rdoc bugs — David Flanagan <david@...>

[#12714] Re: A couple of bugs? — "Gavin Kistner" <gavin.kistner@...>

[#12750] Re: [OT] Re: Should *most* memory be release back to the system? — "M. Edward (Ed) Borasky" <znmeb@...>

[#12754] Improving 'syntax error, unexpected $end, expecting kEND'? — Hugh Sasse <hgs@...>

[#12757] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12758] Encoding::primary_encoding — David Flanagan <david@...>

[#12759] Reading UTF-8 Data contains BOM — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12764] SystemStackError 1.8.6 (stack only 25 frames) — Dan Janowski <danj@3skel.com>

[#12767] \u escapes in string literals: proof of concept implementation — David Flanagan <david@...>

[#12776] Clarification of Ruby release plans? 1.8.7? 1.9.1? 2.0? — Lucas Nussbaum <lucas@...>

[#12779] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12780] Re: Unicode: Progress?? — Brent Roman <brent@...>

[#12787] How to specify in Ruby 1.9 the expected file encoding — =?ISO-8859-15?Q?Wolfgang_N=E1dasi-Donner?= <ed.odanow@...>

[#12795] patch for String.concat — David Flanagan <david@...>

[#12825] clarification of ruby libraries installation paths? — Lucas Nussbaum <lucas@...>

[#12834] Bug Tracker Access Request — James Edward Gray II <james@...>

[#12847] make test-all — "Florian Ebeling" <florian.ebeling@...>

[#12598] adds file and line methods to Methods and Procs — Brent Roman <brent@...>

[#12750] Re: [OT] Re: Should most memory be release back to the system? — "M. Edward (Ed) Borasky" <znmeb@...>