From: tamouse mailing lists <tamouse.lists@...>
Date: 2013-02-22T12:40:51+09:00
Subject: Re: backslash substitution

On Thu, Feb 21, 2013 at 5:32 AM, Matthew Kerwin <lists@ruby-forum.com> wrote:
> I think doing this in irb might help illuminate the issue.  Please read
> the whole thing, and then read it again, and then if you think you
> understand it, read it again and type along in your own ruby/irb.
>
>   irb(main):001:0> a=" xxxxxx\tttt\3.4.2"
>   => " xxxxxx\tttt\x03.4.2"
>
> I now have a string, called `a`.  It has some characters in.  You may
> notice that in my 1.9.2 Ruby the "\3" becomes "\x03" when inspected.
> Interesting.  (I suspect that in 1.8 it would have become "\03", but I
> can't remember that far back to be certain.)
>
>   irb(main):002:0> a.unpack('C*')
>   => [32, 120, 120, 120, 120, 120, 120, 9, 116, 116, 116, 3, 46, 52, 46,
> 50]
>
> All I'm doing here is extracting the integer values of each ASCII
> character in my `a` string.  Did you note that there are 16
> bytes/characters, even though I typed 18 characters between the quotes
> in my original statement?  Note also that the eighth byte, corresponding
> with my first backslash, has the value 9.  I happen to know that 9 is
> actually the ASCII TAB character, which is usually referred to as "\t".
> I hope some bells are ringing here.
>
> Odd, let me convert those integer byte values into strings, one by one,
> so we can see them in a different way.
>
>   irb(main):003:0> a.unpack('C*').map{|int| int.chr }
>   => [" ", "x", "x", "x", "x", "x", "x", "\t", "t", "t", "t", "\x03",
> ".", "4", ".", "2"]
>
> Each string contains a single character.  Notice how Ruby has
> represented 9 as "\t" and 3 as "\x03", which look to be more than one
> character each?  That representation goes both ways (i.e. you may
> require multiple characters of source code to represent a single
> character of actual data.)
>
>
>
> Now let's have a look at the gsub bit:
>
>   irb(main):004:0> '\\'
>   => "\\"
>   irb(main):005:0> '\\'.unpack('C*')
>   => [92]
>   irb(main):006:0> '\\'.unpack('C*').map{|int| int.chr }
>   => ["\\"]
>
> You can see quite plainly that ruby interprets the '\\' as a single
> byte, with value 92, that is the ASCII backslash character.
>
> Looking at the earlier code, there are no 92-bytes in the original
> string.  I must infer, then, that gsub is quite correctly replacing 100%
> of those 92-characters with a forward slash.  (In case you missed it,
> 100% of 0 is 0.)
>
>
>
> Here is a little further experimentation on my part:
>
>   irb(main):006:0> b=" xxxxxx\\tttt\\3.4.2"
>   => " xxxxxx\\tttt\\3.4.2"
>   irb(main):007:0> b.unpack('C*')
>   => [32, 120, 120, 120, 120, 120, 120, 92, 116, 116, 116, 116, 92, 51,
> 46, 52, 46, 50]
>
> Ahh, now `b` contains 18 characters, which is what we thought should
> have been in `a`.  To achieve this, I had to actually type _20_
> characters between the double-quotes (doubling up the backslashes.)
>
> See how the eighth byte is now 92?  And so...
>
>   irb(main):008:0> b.gsub('\\', '/')
>   => " xxxxxx/tttt/3.4.2"
>
> Aha!
>
> Therefore, since `a` is apparently "like it is" and can't be changed, I
> would first be asking: what is it actually like?  Does it contain
> backslash characters, or does it contain TAB characters?  You can find
> this out fairly easily by adding the equivalent of:
>
>   STDERR.puts a.unpack('C*').inspect
>
> ..immediately before your current `gsub`.  Then, if there are 92s in the
> array, your gsub should be working, and our debugging effort should move
> elsewhere.
>
> However, if there are 9s in the array, then you have some serious
> issues, because it means that someone who meant to put 92s into their
> string actually put 9s and 3s and whatever other rubbish in there.  It's
> rather tricky (or impossible) to get them back out.
>
> Let me leave you with a simple illustration:
>
>   irb(main):009:0> a="bbb\aaa\7.6.5"
>   => "bbb\aaa\a.6.5"
>   irb(main):010:0> a.unpack('C*')
>   => [98, 98, 98, 7, 97, 97, 7, 46, 54, 46, 53]
>
> Oh noes!  When I type "\a" and "\7" Ruby interprets them both as the
> same thing!  I have no way to "fix up" those integer 7s to either "\\a"
> or "\\7" because by the time it looks like a 7, I have no way of knowing
> which it was originally meant to be.
>
> If you're in this situation, you need to go smack whoever messed up the
> string in the first place.  IMO
>
> --
> Posted via http://www.ruby-forum.com/.
>

Try your experiment again, only instead of wrapping the string with
backslashes in double quotes, wrap it in single quotes. If the string
coming in is unchangeable, experimenting using double quotes initially
is incorrect, as that will interpolate.

irb(main):011:0> a = "xxxxxx\tttt\3.4.2\uuuu"
SyntaxError: (irb):11: invalid Unicode escape
a = "xxxxxx\tttt\3.4.2\uuuu"
                        ^
	from /home/tamara/.rvm/rubies/ruby-1.9.3-head/bin/irb:16:in `<main>'
irb(main):012:0> a = 'xxxxxx\tttt\3.4.2\uuuu'
"xxxxxx\\tttt\\3.4.2\\uuuu"
irb(main):013:0> a.gsub('\\','/')
"xxxxxx/tttt/3.4.2/uuuu"