From: Caleb Clausen <vikkous@...>
Date: 2010-03-05T15:49:48+09:00
Subject: [ruby-core:28498] Re: [Feature #2759] Regexp /g and /G options

On 3/4/10, NARUSE, Yui <naruse@airemix.jp> wrote:
> But it is not perfetct; those regexp can't be marshalized.
> % ./ruby
> -e'Marshal.load(Marshal.dump(Regexp.new(%q/(a)(?<n>n)\\1\\k<n>/,256)))'
> -e:1:in `load': numbered backref/call is not allowed. (use name):
> /(a)(?<n>n)\1\k<n>/ (RegexpError)
>         from -e:1:in `<main>'
>
> You still think fixing Marshal should be easy.
> But in marshal.c:
>   case T_REGEXP:
>     w_uclass(obj, rb_cRegexp, arg);
>     w_byte(TYPE_REGEXP, arg);
>     {
>       int opts = rb_reg_options(obj);
>       w_bytes(RREGEXP_SRC_PTR(obj), RREGEXP_SRC_LEN(obj), arg);
>       w_byte((char)opts, arg);
>     }
> You know the value of this option is 256,
> but current code marshalize the option as char...
> It is difficult to extend this with compatibility.

I'm glad you're being careful and thinking about stuff like this,
because I wouldn't have known it might be a problem.

Can't you rewrite that as something like this:

   /*untested, intended more as an example*/
   case T_REGEXP:
     w_uclass(obj, rb_cRegexp, arg);
     {
       int opts = rb_reg_options(obj);
       if (opts<=0xFF)
         w_byte(TYPE_REGEXP, arg);
       else
         w_byte(TYPE_WIDEREGEXP, arg);
       w_bytes(RREGEXP_SRC_PTR(obj), RREGEXP_SRC_LEN(obj), arg);
       if (opts<=0xFF)
         w_byte((char)opts, arg);
       else
         w_int(opts, arg);
     }

That is, use the old format if the regexp options will fit in a byte,
else use a new format with a wider field for options. There'd be
similar logic on the unmarshal side. This new regexp format would be
unreadable on older ruby versions, but then they won't know what to do
with 256 in the options field anyway, so there's no loss there.