From: William James <w_a_x_man@...>
Date: 2005-01-30T04:20:53+09:00
Subject: Re: Why csv file processing is so slow?

mepython wrote:
>
> How can we modify so that we can pass field seperator and record
> seperator as an argument?

This should do it.  I found that not rebuilding the regular-expression
every time parse_csv is called made it even faster.


% # Record separator.
% RS = "\n"
%
% # Set regexp for parse_csv.
% # fs is the field-separator
% def fs_is( fs )
%   $csv_re = \
%     %r{ "( (?: [^\\"] | \\")* )" |
%         '( (?: [^\\'] | \\')* )' |
%          ( [^#{fs}]+ )
%       }x
% end
%
% class String
%   def parse_csv
%     raise "Method fs_is() wasn't called." if $csv_re.nil?
%     a = self.scan( $csv_re ).flatten
%     a.delete(nil)
%     a
%   end
% end
%
% fs_is( ',' )
%
% # Set Ruby's input record-separator.
% $/ = RS
% 
% ARGF.each_line { | line |
%   p line.chomp.parse_csv
% }