From: Peter Vandenabeele <peter@...>
Date: 2010-12-03T07:37:22+09:00
Subject: Re: Text Parsing Help

On Thu, Dec 2, 2010 at 8:10 PM, Jester Mania <jester_b84@hotmail.com> wrote:
> Yes, but I tried the code and it is still not working.  I used a puts
> statement to output the results to see whether the "\n" text was truly
> being replaced by a tab.
>
> #!/usr/bin/ruby -w
>
> IO.readlines("input.txt").each do |line|
>  lineItem = line.gsub(/\\n/, "\t")
>  puts lineItem.split("\t")
> end
>
> However, the results were that the output still had \n text.

I hope my example below can explain what happens

$ ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [i486-linux]

I used this input.txt file for testing

<start of file>
car\nplane\ntrain \n boat

second line, first token \n second token
<end of file>

irb(main):013:0> IO.readlines("input.txt").each do |line|
irb(main):014:1*  lineItem = line.gsub(/\\n/, "\t")
irb(main):015:1>  puts lineItem.split("\t").inspect
irb(main):016:1> end
["car", "plane", "train ", " boat\n"]    # the first line is parsed
and split correctly into this array
["\n"]     # the second line only has a newline
["second line, first token ", " second token\n"] # correct too
=> ["car\\nplane\\ntrain \\n boat\n", "\n", "second line, first token
\\n second token\n"]

# this last line is the result IO.readlines("input.txt") because the
"each" method
eventually returns self after having iterated over all entities

irb(main):017:0> IO.readlines("input.txt").each do |line|
irb(main):018:1*  lineItem = line.gsub(/\\n/, "\t")
irb(main):019:1>  puts lineItem.split("\t")
irb(main):020:1> end
car
plane
train
 boat

second line, first token
 second token
=> ["car\\nplane\\ntrain \\n boat\n", "\n", "second line, first token
\\n second token\n"]


So, one trick is to use .inspect and .class in many cases to better
understand what is
the object you are looking at and what the content really is.

Also, you could use chomp to get rid of the newline at the end of the
last entry in your array of tokens.
So, a shorter piece of code that may be useful is:

irb(main):025:0> IO.readlines("input.txt").map do |line|
irb(main):026:1*  line.chomp.gsub(/\\n/, "\t")
irb(main):027:1> end
=> ["car\tplane\ttrain \t boat", "", "second line, first token \t second token"]

Now there are the <TAB> delimiters that you wanted between the tokens
in the resulting output.

HTH,

Peter