From: Dave Burt <dave@...>
Date: 2006-04-28T00:28:34+09:00
Subject: Re: Q about the FasterCSV

Pe�a wrote:
> Dave, you're a cool rubyist. I think you are cooler than microsoft's.

Thanks, I think. (I don't know how many Rubyists Microsoft has - I don't
recall anyone on this list signing their email with an MS certification.)

> and mabye, fastercsv can be more "intelligent" than other csv by
> 
> 1) ignoring extra spaces in a captured separated value
> 
>   eg,
> 
>   test, "1" ==> ["test","1"]
> 
>   iow, quotes rule (as in shellwords)
> 
> 2) not ignore spaces yet escape the quotes
> 
>   eg,
> 
>   test, "1" ==> ["test"," \"1\""]
>   test, "1"111 ==> ["test"," \"1\"111"]
> 
> 3) or maybe, fastercsv can include an option/flag to allow the above

Let's choose option 1. Ruby lets you modify classes from libraries.
Let's call this "lenient_and_still_a_little_bit_faster_csv.rb":

require 'faster_csv'
class FasterCSV
  # Pre-compiles parsers and stores them by name for access during
  # reads, just like the official FasterCSV version, BUT the central
  # parser allows arbitrary whitespace before and after the column
  # separator.
  def init_parsers( options )
    # prebuild Regexps for faster parsing
    @parsers    = {
      :leading_fields =>
        /\A#{Regexp.escape(@col_sep)}+/,      # for empty leading fields
      :csv_row        =>
        ### The Primary Parser ###
        / \G(?:^|#{Regexp.escape(@col_sep)})     # anchor the match
          \s*                          # <----- # ignore some whitespace
          (?: "((?>[^"]*)(?>""[^"]*)*)"          # find quoted fields
              |                                  # ... or ...
              ([^"#{Regexp.escape(@col_sep)}]*)  # unquoted fields
              )/x,
        ### End Primary Parser ###
      :line_end       =>
        /#{Regexp.escape(@row_sep)}\Z/           # safer than chomp!()
    }
  end
end

All that code except for the line consisting entirely of "\s*" was taken
from FasterCSV 0.2.0, and I should have asked Gray Productions for
permission to republish it, but I don't think Mr. Gray will mind this
particular use of his excellent work.

Cheers,
Dave