From: James Edward Gray II <james@...>
Date: 2009-11-18T06:25:15+09:00
Subject: Re: using FasterCSV to clean CSV file

On Nov 17, 2009, at 3:12 PM, John Mcleod wrote:

> I was wondering, can FasterCSV be used to clean a csv file of unwanted
> characters, endline spaces, tab spaces and other characters?

I believe the answer to this question is yes.  I've already shown you how to modify headers as they are read.  There's a similar system for normal fields.

Now, this does not modify the file on disk, obviously.  If you want to do that, it would be best to write out a new and corrected file.  You could then rename the new file to replace the old, if needed.

> I have a csv file 6,000 lines long and it comes from another department.
> I have been working to read this file into a database via FasterCSV.
> Went I start to read the file in, I get the dreaded
> "FasterCSV::MalformedCSVError" error message.  I've worked with this in
> the past and the solution was to "clean" the data. EOL spaces and other
> undesired characters were scattered within the file.

FasterCSV does need valid CSV data, yes.  If you have non-valid CSV data, it will need to be cleaned before FasterCSV can read it.

> Is it possible to use FasterCSV for this task?

If your data is valid CSV, FasterCSV can read and transform it.  If your data is not valid CSV, you will need to fix it before FasterCSV can read it.

Hope that helps.

James Edward Gray II