From: Will Jessop <will@...>
Date: 2006-11-30T01:58:05+09:00
Subject: Re: Splitting a CSV file into 40,000 line chunks

Paul Lutus wrote:
>> CSV.open(infile, 'r') do |row|
> 
> Why are you using CSV for this? You aren't parsing the lines into fields, so
> the fact that they contain CSV content has no bearing on the present task.
> Your goal is to split the input file into groups of lines delimited by
> linefeeds, not fields delimited by commas.
> 
> Why not simply read lines from the input file and write them to a series of
> output files, until the input file is exhausted?

Because CSV understands csv data with embedded newlines:

[will@localhost ~]$ cat csvfile.csv
some, field,"new
line"
other,field,here
[will@localhost ~]$ cat test.rb
require 'csv'

CSV.open('csvfile.csv', 'r') do |row|
   p row
end
[will@localhost ~]$ ruby test.rb
["some", " field", "new\nline"]
["other", "field", "here"]

will.