From: Paul Lutus <nospam@...>
Date: 2006-12-01T18:30:05+09:00
Subject: Re: Splitting a CSV file into 40,000 line chunks

Edwin Fine wrote:

/ ...

> The bottom line is, although I am perfectly capable of doing so, I don't
> WANT to reinvent the wheel. I really like reusing good, dependable,
> predictable code. I haven't had time to look at FasterCSV yet, but I
> predict that I will like what I see, because to my mind, from the works
> of his I have seen, the author does his best to embody the "Tao" of Ruby
> (LOLA). (Although you can never accurately describe the Tao of anything,
> I know...)

Nice, informative post. There are a lot of issues here, primarily the fact
that the database under discussion is too big to hold in memory, and it is
also too big to fit into Excel in one chunk, which appears to be its
destination.

Most people have begin to drift toward suggesting a database approach,
rather than anything that involves direct manipulation of the database in
Ruby. Because of the size of the database and because sorting the records
is one goal, I have to agree.

-- 
Paul Lutus
http://www.arachnoid.com