From: Robert Klemme <shortcutter@...>
Date: 2008-05-18T20:40:12+09:00
Subject: Re: sub/gsub bug?

On 18.05.2008 06:33, Phlip wrote:
> Matt wrote:
> 
>> Is this a bug?
>>
>> irb(main):032:0> "matt's test".sub(/'/, "\\'")
>> => "matts tests test"
>>
>> It's trying to replace a single quote with an escaped quote.
> 
> To understand the bug, try this:

There is no bug.

> irb(main):009:0> "matt's test".sub(/(')/, '\1')
> => "matt's test"
> 
> The 'single ticks' interpret \1 as a literal \ and a 1, not an escape. Then 
> .sub sees the \1, and replaces it with the 1nth (grouping) from the Regexp - 
> which is a literal tick '.
> 
> Now try it with "double quotes":
> 
> irb(main):005:0> "matt's test".sub(/(')/, "\\1")
> => "matt's test"
> 
> The " forces us to escape the \ as \\ to get a literal \.

And a literal \ in a replacement string is a meta character.  To get a 
literal in a replacement, you need \\ in the string and consequently 
\\\\ in a double quoted string.

> So your \\' indeed goes in as a literal \'. But the group replacer still 
> sees a \, and snarfs it, expecting a number after it. I would call that a 
> bug (essentially because I might personally be capable of correctly parsing 
> a \1!)

No, this is not a bug and it is not the "group replacer".  The 
replacement string is parsed by the regexp engine and since the 
backslash does not escape a character that has meta capabilities (such 
as & and 1..9) it is silently discarded.

> Untill someone dissects the source and finds (or adds) a better fix, here's 
> a workaround:
> 
> irb(main):012:0> "matt's test".sub(/'/){"\\'"}
> => "matt\\'s test"

No, the proper way to do it is this:

irb(main):002:0> "matt's test".sub(/'/, "\\\\'")
=> "matt\\'s test"
irb(main):003:0> puts "matt's test".sub(/'/, "\\\\'")
matt\'s test
=> nil

irb(main):006:0> "matt's test".sub(/'/, '\\\\\'')
=> "matt\\'s test"
irb(main):007:0> puts "matt's test".sub(/'/, '\\\\\'')
matt\'s test
=> nil

Blocks are only needed if you need to do some calculations based on the 
match, e.g.

irb(main):008:0> "There is 1 number in this string".gsub(/\d+/) {|m| 
m.to_i * 34}
=> "There is 34 number in this string"

> That surprised me too, because I thought that sub's block form also 
> interpolated \n group inserters, but I suppose that blocks have room for 
> true $1 match references.

Yes.

> \1 is often available in some regular expressions 
> that cram all the searchers and replacers into one huge line, so a $1 is 
> naturally not available inside its own source expression. (I also don't 
> happen to know if Ruby supports such super-regices!)

I am not sure what you mean here.  The reason $1 cannot be used in 
regular expressions and replacement strings is that it is a variable 
that gets its value before the regexp is created (of course you can use 
it but it cannot refer to matching of the regexp you put it into).

You can use groups inside the regular expression as well as in the 
replacement string if this is what you mean.

irb(main):017:0> "aba abc".gsub(/(.)(b)\1/, "[\\1]<\\2>[\\1]")
=> "[a]<b>[a] abc"

Kind regards

	robert