From: Alex Young <alex@...>
Date: 2007-08-17T00:56:42+09:00
Subject: Re: grep a csv?

Kaldrenon wrote:
> On Aug 16, 3:04 am, Alex Young <a...@blackkettle.org> wrote:
>> There's a problem with using File.readlines that I don't think anyone's
>> mentioned yet.  I don't know if it's relevant to your dataset, but CSV
>> fields are allowed to contain newlines if the field is quoted.  For
>> example, this single CSV row will break your process:
>>
>> 1,2,"foo
>> Blah1",bar
>>
>> The only way around that is to actually parse the file, so unless you
>> *know* that's not going to be a problem, I'd still suggest FasterCSV...
> 
> I guess the following is slightly OT, since the OP is talking about
> grabbing whole lines, but it's a CSV-relevant question:
> 
> Another danger with not using CSV packages (as I've learned from my
> present non-use of said packages) is that quoted elements in a row can
> contain commas. I'm a fairly inexperienced programmer, and not just in
> Ruby, and I haven't yet figured out an elegant way to break this down.
> 
> For example:
> 
> foo,bar,"foo,bar" is a three-column row in a CSV file, but using
> split(/,/) on it will, of course, return ["foo","bar","\"foo","bar
> \""], an array of size four. What's an efficient, elegant way of
> gathering quoted columns? 
If you ignore that the quote character can also appear inside column 
data, then this will work, ishkinda.

'foo,bar,"foo,bar"'.scan(/("[^"]+")|([^,]+)/).flatten.compact
=> ["foo", "bar", "\"foo,bar\""]

That breaks at least for empty fields, fields with newlines, and fields 
with '"' in them.

> I'm sure FasterCSV is capable of dealing
> with such things, but it's something I'd like to actually learn,
> rather than finding a way around it.
That's just it - the cleanest way to handle this problem is to just take 
advantage of someone else's work.  CSV looks just trivial enough to the 
human eye that you can convince yourself "Oh, it can't be *that* 
hard"...  but it's still complicated enough that you get tired of coding 
around the fiddly little details very, very quickly.  Besides, when a 
library is as easy to use as either CSV *or* FasterCSV (my preference is 
for the latter, but it's not in stdlib), what's the point in stressing?

-- 
Alex