From: Ammar Ali <ammarabuali@...>
Date: 2010-11-14T21:08:16+09:00
Subject: Re: import csv with Fields that contain double quote characters

On Sun, Nov 14, 2010 at 1:05 PM, Shir Shir <hadasagr@gmail.com> wrote:
> hi
>  i do it wuth :fasterCSV,
> and using- windows-1255, and convert it to-utf-8.
> thank you

Sorry, I couldn't run that code. Maybe pasting in the forum messed it
up. I tried a simple test with double quotes and FasterCSV always
throws:

  Illegal quoting on line 2. (FasterCSV::MalformedCSVError)

If that is what you are seeing too, then it's probably a formating
issue. The best solution is to export the csv file with legal
formatting, but if that's not possible, you can replace the " with
single quotes as a last resort. But be careful, this is brute force
and can break other things. For example:

# read the file and replace the " with '
data = File.read data_file
data.gsub!('"', "'")

# parse instead of open
row_index = 0
FasterCSV.parse(data) do |row|
  row_index +=1
  next if row_index == 1

  converted = row.map {|val| Iconv.iconv("utf8","Windows-1255",val)}
  puts converted.inspect
end

If you have a small 2 to 3 line sample of the input (not real data), I
could try to help more.

Hope that helps,
Ammar