From: "Ralf Müller" <r_mueller@...>
Date: 2005-02-01T17:47:51+09:00
Subject: Re: Why csv file processing is so slow?

On Sun, 30 Jan 2005 17:35:49 +0900
timsuth@ihug.co.nz (Tim Sutherland) wrote:

> In article <1107042295.816570.61420@z14g2000cwz.googlegroups.com>, mepython
> wrote:
> [...]
> >How hard to do reverse: create csv string from list?
> >
> >Thanks. I just started Ruby couple of days ago, so I am learning
> >instead of implementing, Sorry.
> 
> This assumes the input is an array of lines. (Where each line is an array.)
> 
> class Array
>     def to_csv
>         map { |line|
>             line.map { |cell|
>                 '"' + cell.gsub(/"/, '""') + '"'
>             }.join(',')
>         }.join("\n")
>     end
> end
> 
> Note that literal quotes " are replaced with "".
> 
Found a Parser in a german ruby-Book by R�hrl,Schmiedl and Wyss. With a little improvement, it supports unqoted, '-quoted and "-quoted cells in any order:

#!/usr/bin/env ruby
class CSVParser
   include Enumerable
   
   QUOTED = /('|"){1,1}(.*?)\1{1,1}(,|\r?\n)/m
   UNQUOTED = /()(.*?)(,|\r?\n)/m
   
   def initialize(string)
      @string = string
   end

   # datafields of a line are provided as an array
   def each
      while @string != ''
         tokens = []
         while @string != ''
            case @string[0..0]
               # empty cell
               when ","
                  tokens << nil
                  @string.slice!(0..0)
                  next
               # last cell is empty
               when /\r?\n/
                  tokens << nil
                  @string.slice!(0..$&.size)
                  break
               # complex cell
               when /('|")/
                  pattern = QUOTED
                  dequote = true
               # simple cell
               else
                  pattern = UNQUOTED
                  dequote = false
            end
            # match the content
            md = pattern.match(@string)
            token = md[2]
#            token.gsub('""','"') if dequote
            tokens << token
            @string.slice!(0...md[0].size)
            # last cell 
            break if md[0][-1..-1] == "\n"
         end
         yield tokens
      end
   end
end


# ============================================================================= 
# MAIN ------------------------------------------------------------------------ 
cvs =CSVParser.new($stdin.read)
Start = "'"
End  = "'\n"
Sep = "','"
cvs.each{|row|
   puts Start + row[0].to_s + Sep + row.join(Sep) + End  if row[2].to_i <= 4
00000 and row.last != ''
}


regards
ralf