From: "NAKAMURA, Hiroshi" <nahi@...>
Date: 2004-05-18T15:06:38+09:00
Subject: Re: CSV bug?

Hi,

Pe�a, Botp wrote:
>>CSV module (of mine) just does not expect the space just after field 
>>delimiter(,) and before quote marker(").
>>
>>0% ruby -rcsv -e 'CSV::Reader.parse(%Q("a","b")) { |row| p 
>>row }' ["a", "b"] 0% ruby -rcsv -e 'CSV::Reader.parse(%Q("a", 
>>"b")) { |row| p row }' /usr/local/lib/ruby/1.9/csv.rb:557:in 
>>`get_row': CSV::IllegalFormatError 
>>(CSV::IllegalFormatError)
>>         from /usr/local/lib/ruby/1.9/csv.rb:506:in `each'
>>         from /usr/local/lib/ruby/1.9/csv.rb:484:in `parse'
>>         from -e:1
>>
>>Does Excel generates such a line?  Hmm.  It must be supported 

> I am not SER, but I hope I can comment, too, pls.

Of course.

>>What do you think about 
>>'a,junk"b"' and 

> Yes, we need it here. Parse it like ['a','junk"b"'].

I understand the point.  And Excel parses it as ["a", "junk\"b\""].  So 
it must be a bug of my CSV module.

 > Eg, we have lines like
> 
> 
> 1;SMTP:"local;part"@test.com;x400:blahblahblah;smtp:test@test.com
> 2;SMTP:"local;part2"@test.com;x400:blahblahblah;smtp:test2@test.com

With the following patch, CSV module can handle non-quoted " field.  But...

--- csv.rb.dist 2004-04-20 23:17:20.000000000 +0900
+++ csv.rb      2004-05-18 14:15:13.698072000 +0900
@@ -398,5 +398,5 @@ class CSV
                state = :ST_QUOTE
              else
-              raise IllegalFormatError.new
+              cell.data << c.chr
              end
            elsif state.equal?(:ST_QUOTE)

> Though my sample is ";"-delimited, I would like to get the ff result in
> parsing line 1 as:
> 
> ['1','SMTP:"local;part"@test.com','x400:blahblahblah','smtp:test@test.com']
> 
> I really hope that is possible (I beg).

Unfortunately, it works as below...

0% cat pena.csv
1;SMTP:"local;part"@test.com;x400:blahblahblah;smtp:test@test.com
2;SMTP:"local;part2"@test.com;x400:blahblahblah;smtp:test2@test.com

0% ruby -rcsv -e 'CSV.parse("pena.csv", ?;) { |row| p row.to_a }'
["1", "SMTP:\"local", "part\"@test.com", "x400:blahblahblah", 
"smtp:test@test.com"]
["2", "SMTP:\"local", "part2\"@test.com", "x400:blahblahblah", 
"smtp:test2@test.com"]

Excel and OpenOffice seems to parse as the same (though Excel cannot 
handle ; separated value).  Does this enough for your purpose?

> As long as the delimiter is "," , then that should be parse as
> ['a','\t\r\n"b"']

Sure.

> Thank you for your csv module. It's very helpful. btw, Who does the
> documentation? The doc is very sparse. I hope I can help :-(

It's my fault.  Ruby has now great RDoc with it but I don't like 
comments in code... it confuses me, even though I set
  set foldexpr=getline(v:lnum)=~'^\\s*#\.*'
  set foldmethod=expr
in my .vimrc.

The newest version is at 
http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/lib/csv/ , and it will be 
merged to Ruby/1.9 soon.  I hope someone point out which API seems not 
enough to be documented and not easy to use.  It must be a bad interface 
design.

Regards,
// NaHi