From: Wayne Brisette <wbrisett@...>
Date: 2013-01-08T07:58:23+09:00
Subject: gsub with multibyte characters

I'm running into a problem trying to read a line of text that has some multibyte 
characters gsub is failing. Is there a trick to finding and replacing multibyte 
characters? 

The multibyte characters look like this in a text editor: 

When simple_soft_en = 1, this registerís contents are transmitted to the remote 
PHY in DME format.

When I put this into irb, this is what that text looks like: 

1.9.3-p194 :001 > theText = "When simple_soft_en = 1, this 
register\U+FFC3\U+FFADs contents are transmitted to the remote PHY in DME 
format."
 => "When simple_soft_en = 1, this registers contents are transmitted to the 
remote PHY in DME format." 

When I run gsub on it, I see this error: 

/Users/wayneb/LSI_Scripts/RPG Scripts/ruby/axx5500/LSI_RPG_parse_55.rb:117: 
invalid multibyte char (US-ASCII)

I'm using:  File.open(fileNameString,"rb"){|io|io.read}

to read the files. I found that when I used 

 File.open(fileNameString,"r:UTF-8"){|io|io.read}

I would get some failures opening up files, so I switched to ready the file in a 
binary form. 

I'm a bit stumped at this point. 

Wayne