From: Robert Klemme <bob.news@...>
Date: 2005-01-29T02:00:52+09:00
Subject: Re: Why csv file processing is so slow?


"mepython" <a@agni.us> schrieb im Newsbeitrag
news:1106927942.910013.321430@z14g2000cwz.googlegroups.com...
> It is csv module: reading file seems like half the speed of python. So
> real slowness is coming from csv
>
> count = 0
> File.open('x.csv') do |reader|
> reader.each {|data| count += 1}
> end
> p count
>
>
> [root@taamportable GMS]# time ruby x1.rb
> 26908
>
> real    0m0.077s
> user    0m0.060s
> sys     0m0.016s
>
>
> [root@taamportable GMS]# time python x1.py
> 26908
>
> real    0m0.041s
> user    0m0.032s
> sys     0m0.010s

As a simple CSV replacement you could try this:

File.open('x.csv') do |reader|
  reader.each {|line|
    count += 1
    data = line.split(/,/)
  }
end
p count

Depending on your data that might or might not be sufficient.  Regexps can
be arbitrarily sophisticated.  Here's another one:

data = []
line.scan( %r{
  "((?:[^\\"]|\\")*)" |
  '((?:[^\\']|\\')*)' |
  ([^,]+)
}x ){|m| data << m.find {|x|x}}

:-))

    robert