From: Yui NARUSE <redmine@...>
Date: 2009-09-09T19:02:01+09:00
Subject: [ruby-core:25497] [Bug #2054] Onigurma Isn't Documented

Issue #2054 has been updated by Yui NARUSE.


> >> + *  Capture groups can be referred to by name when defined with the
> >> + *  <tt>(?<name>)</tt> construct.
> 
> > Don't mention an alias (?'name'subexp) ?
> 
> It didn't seem worth it. The material is already dense, without describing
> alternative syntax which appears to have little merit of its own.

I don't like (?'name'subexp) too,
but reference is read not only when people write code but when read code.

> >> + *      /���(?<pounds>\d+)\.(?<pence>\d+)/.match("���3.67")
> > Pound Sign (U+00A3) is non-ASCII.
> > Dollar Sign (U+0024) is in ASCII (but Cent Sign is non-ASCII).
> 
> We need to re-think the ASCII restriction. Documenting a Unicode regular
> expression library without being able to use Unicode in the examples is somewhat limiting.
> See below for an alternative.

Some editors still don't support UTF-8 or
some  environmets don't have fonts.

> Changed to:
> 
>   A regexp can be matched against a string when they either share an encoding, or the regexp's encoding
>   is  _US-ASCII_ and the string's encoding is ASCII-compatible.
> 
> I'm not keen on the phrasing, but hopefully it's more accurate.

good.

> >> + *  The <tt>Regexp#fixed_encoding?</tt> predicate indicates whether the regexp
> > Needs encoding modifiers set FIXED_ENCODING.
>
> I don't understand that part.

This means, /a/e equals Regexp.new("a".force_encoding("EUC-JP"), Regexp::FIXEDENCODING).

> Regexp::FIXED_ENCODING is a constant, so how can an encoding option "set" it?
> I think the current text makes clear that a non-ASCII encoding,
> i.e. one set by an encoding option, is "fixed".
> That is, the modifiers change the state to fixed, which the predicate queries.
> I'll play around with

Hm, ok.

> > Don't mention following?
>
> I didn't know I was writing a book. :-) It's already far more comprehensive that O'Reilly's Programming Ruby...

a reference shall be comprehensive :-(

> > * subexp call \g<..>
> 
> This seemed particularly arcane; even Friedl doesn't mention it in his tome. ;-)
> I'll try and come up with an example use.

exapmles are following:

  # match parenthes
  r = /\A(?<paren>\(\g<paren>*\))*\z/
  r =~ "()"       #=> 0
  r =~ "(()())()" #=> 0
  r =~ ")"        #=> nil

  # match json
  r = Regexp.compile(<<'__REGEXP__'strip, Regexp::EXTENDED)
  (?<json> \g<object> | \g<array> ){0}
  (?<begin-array>  \g<ws> \[ \g<ws> ){0}
  (?<begin-object> \g<ws> \{ \g<ws> ){0}
  (?<end-array>    \g<ws> \] \g<ws> ){0}
  (?<end-object>   \g<ws> \} \g<ws> ){0}
  (?<name-separator>  \g<ws> : \g<ws> ){0}
  (?<value-separator> \g<ws> , \g<ws> ){0}
  (?<ws> [\x20\t\n\r]* ){0}
  (?<value>  false | null | true | \g<object> | \g<array> | \g<number> | \g<string> ){0}
  (?<object> \g<begin-object> (?: \g<member> (?: \g<value-separator> \g<member> )* )? \g<end-object> ){0}
  (?<member> \g<string> \g<name-separator> \g<value> ){0}
  (?<array> \g<begin-array> (?: \g<value> (?: \g<value-separator> \g<value> )* )? \g<end-array> ){0}
  (?<number> \-? \g<int> \g<frac>? \g<exp>? ){0}
  (?<exp> [eE] [-+] [0-9]+ ){0}
  (?<frac> \. [0-9]+ ){0}
  (?<int> 0 | [1-9] [0-9]* ){0}
  (?<string> " \g<char>* " ){0}
  (?<char> [^\x00-\x1F"\\] | \\ (?: ["\\\/\b\f\n\r\t] | u [0-9a-fA-F]{4} ) ){0}
  \A\g<json>\z
  __REGEXP__
  r =~ '{"foo":1,"bar":null,"baz":true}' #=> 0

  # match simplified XML
  r = Regexp.compile(<<'__REGEXP__'strip, Regexp::EXTENDED)
  (?<element> \g<stag> \g<content>* \g<etag> ){0}
  (?<stag> < \g<name> \s* > ){0}
  (?<name> [a-zA-Z_:]+ ){0}
  (?<content> [^<&]+ (\g<element> | [^<&]+)* ){0}
  (?<etag> </ \k<name+1> >){0}
  \A\g<element>\z
  __REGEXP__

> > If you can describe about  /(a|b+)*/ , it will be helpful.
> > http://www.ruby-forum.com/topic/70726
> 
> Hmm... Constructing accurate, performant, and optimal regexps is a significant topic...
> I suppose I could add a "Top 10 Pitfalls" callout. 

This problem sometimes comes to ML.
But you can write, "there is a backtracking problem. see Mastering Regular Expression" :-)
http://swtch.com/~rsc/regexp/regexp1.html is also written in detail.

> Given the scope of this tutorial, the character set limitations, the degree to which it's
> bloating re.c, and the way this text obscures the existing API documentation for Regexp,
> how about we move it to its own file? This could, presumably, either be included in re.c
> with RDoc's _include_ option, or, better, live separately from the API documentation yet
> still be accessible from `ri`. `ri Regexp` could display the API documentation and
> mention the existence of this document; `ri regexp` could display the current document.
> I can commit at least the rest of this week to writing some initial content, if necessary.

I'm not familiar to rdoc but this seems good.
But distingish document by case seems dangerous for some case-insensitive FS, e.g. Windows.
Regexp-intro or some different name is good.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2054

----------------------------------------
http://redmine.ruby-lang.org