From: William James <w_a_x_man@...>
Date: 2007-08-17T04:25:04+09:00
Subject: Re: grep a csv?

On Aug 16, 10:56 am, Alex Young <a...@blackkettle.org> wrote:
> Kaldrenon wrote:
> > On Aug 16, 3:04 am, Alex Young <a...@blackkettle.org> wrote:
> >> There's a problem with using File.readlines that I don't think anyone's
> >> mentioned yet.  I don't know if it's relevant to your dataset, but CSV
> >> fields are allowed to contain newlines if the field is quoted.  For
> >> example, this single CSV row will break your process:
>
> >> 1,2,"foo
> >> Blah1",bar
>
> >> The only way around that is to actually parse the file, so unless you
> >> *know* that's not going to be a problem, I'd still suggest FasterCSV...
>
> > I guess the following is slightly OT, since the OP is talking about
> > grabbing whole lines, but it's a CSV-relevant question:
>
> > Another danger with not using CSV packages (as I've learned from my
> > present non-use of said packages) is that quoted elements in a row can
> > contain commas. I'm a fairly inexperienced programmer, and not just in
> > Ruby, and I haven't yet figured out an elegant way to break this down.
>
> > For example:
>
> > foo,bar,"foo,bar" is a three-column row in a CSV file, but using
> > split(/,/) on it will, of course, return ["foo","bar","\"foo","bar
> > \""], an array of size four. What's an efficient, elegant way of
> > gathering quoted columns?
>
> If you ignore that the quote character can also appear inside column
> data, then this will work, ishkinda.
>
> 'foo,bar,"foo,bar"'.scan(/("[^"]+")|([^,]+)/).flatten.compact
> => ["foo", "bar", "\"foo,bar\""]
>
> That breaks at least for empty fields, fields with newlines, and fields
> with '"' in them.

I think that this will work correctly with any complete csv record.


class String
  def csv
    if include? '"'
      ary =
        "#{chomp},".scan( /\G"([^"]*(?:""[^"]*)*)",|\G([^,"]*),/ )
      raise "Bad csv record:\n#{self}"  if $' != ""
      ary.map{|a| a[1] || a[0].gsub(/""/,'"') }
    else
      ary = chomp.split( /,/, -1)
      ##   "".csv ought to be [""], not [], just as
      ##   ",".csv is ["",""].
      if [] == ary
        [""]
      else
        ary
      end
    end
  end
end