From: olczyk@... (Thaddeus L. Olczyk)
Date: 2001-12-16T22:15:19+09:00
Subject: [ruby-talk:28676] How do you do "character filtering" of a string using each_byte.

I'm trying to do several things where I produce new strings from old
strings by manipulating characters, and am stuck.
As an example (please keep in mind this is an example and I am
interested in the general technique ), consider a method which takes
a string and produces a new string such that:
1) all digits are removed.
2) all lowercase are changed to uppercase.
3) multiple spaces are converted to one space
4) tab is converted to one space.

So a very bad version of this might look like:

@convert(string)
     retval=""	
    oldchar='a'	
    string.each_byte{ |c| # aside: is it legal to pt the brace on the 
                                     # next line?
         case c
	when [a-z] 
	     d=(c+'A'.ord-'a'.ord).chr
	    retval=retval+d
	end
	when  [0-9] then nil # do nothing 
             when  ' ' && oldchar=' ' then nil  #do nothing
             when ' ' && oldchar!=' ' then retval=retval+' '
	when \t then retval=retval+' '
	else
	    retval=c.chr
       end
      return retval
end

As anyone can see, the main parts that I am having problem with are:
1) Character matching.
2) Character manipulation. ( Look at the [a-z]. )

For now 1) is more iomportant to me. ( I am still interested in 2
though. )
 
One technique that I could use is a lookup table.
lookup=Array.new;
0.upto(255){ |c| lookup[c]=0]
'abcdefghijklmnopqrstuvwxyz'.each_byte{ |c| lookup[c]=1} # for the
paranoid test lookup[c] to make sure it's 0 before assigning.
'0123456789'.each_byte{|c| lookup[c]=2}
' '.each_byte{|c| lookup[c]=3}
'\t'.each_byte{|c| lookup[c]=4}

then in the case statement just
do
case lookup[c]
     when 0 then retval=retval+c.chr
     when 1 then retval=retval+(c+'A'.ord-'a'.ord).chr
     when 2 then nil
     when 3 
	if oldchar!=' ' then retval=retval+' '
      when 3 then retval=retval+' '
end

---------------
The problem I most have with this technique is the explicit
specification of each char along with creating one-byte strings
( allthough I might fix that with lookup[' '.ord]=3, if possible ).

An alternate apprach using lookup tables might be:

@convert(string)
     if lookup==nil then # lookup is a class variable
	lookup=Array.new	
             0.upto(255){ |c| lookup[c]={ |retval oldchar|
					return retval+c.chr}
             'abcdefghijklmnopqrstuvwxyz'.each_byte{ |c|
	     lookup[c] ={ | retval oldchar|  
                                       return
                                       retval+(c+'A'.ord-'a'.ord).chr
                                     }
            '0123456789'.each_byte{|c| lookup[c]={|retval oldchar| 
					            return retval}	
                                                    }
            ' '.each_byte{|c| lookup[c]={|retval oldchar| 
					if oldchar!=' ' 
					   return retval
					else
					   return retval+' '
					end
				     }
		       }	
           '\t'.each_byte{|c| lookup[c]={|retval oldchar| 
				      return retval+' '
                                                        }
                                }
     end		             
     retval=""	
    oldchar='a'	
    string.each_byte{ |c| 
	retval=lookup[c](retval,oldchar)
	oldchar=c
             }
    return retval
end


Comments? Suggestions?