From: James Edward Gray II <james@...>
Date: 2006-02-28T11:55:28+09:00
Subject: Re: [ANN] FasterCSV 0.1.6 -- With Header Support!

On Feb 27, 2006, at 6:43 PM, Sascha Ebach wrote:

> Hi James,

Hello.

> first, thanks for FasterCSV. It is very useful. I have been wildly  
> using it the last couple of days.

Always nice to here.  Thank you!

>> Sadly, FasterCSV required one more release to get it working  
>> everywhere:  0.1.9 is out now.  :(
>
> Could you please make a couple of small examples of how each of  
> those new
> features is supposed to be used?

I am working on adding examples to the project tarball, but here is  
one I sent to Michael Schoen earlier today:

Neo:~/Desktop$ ls
csv_filter.rb   purchase.csv
Neo:~/Desktop$ cat purchase.csv
Quantity,Product Description,Price
1,Text Editor,25.00
2,MacBook Pros,2499.00
Neo:~/Desktop$ ruby csv_filter.rb purchase.csv > invoice.csv
Neo:~/Desktop$ cat invoice.csv
Quantity,Product Description,Price,Running Total
1,Text Editor,25.0,25.0
2,MacBook Pros,2499.0,5023.0
Neo:~/Desktop$ cat csv_filter.rb
#!/usr/local/bin/ruby -w

require "rubygems"
require "faster_csv"

running_total = 0
FasterCSV.filter( :headers           => true,
                   :return_headers    => true,
                   :header_converters => :symbol,
                   :converters        => :numeric ) do |row|
   if row.header_row?
     row << "Running Total"
   else
     row << (running_total += row[:quantity] * row[:price])
   end
end

__END__

The above is using quite a few of the new features.  Data converters  
are used to switch the numbers to Integers and Floats and header  
converters are used to convert the headers to Symbols for easy  
access.  These are just built-ins, but you can supply lambdas for  
custom conversions.

Obviously, this also makes use of the new headers functionality.   
FasterCSV is told to convert the first row to headers and allow us to  
index columns by them.  You can see that at work when I calculate the  
price.  The advantage is that we didn't have to use any indices and  
if the column order changes, everything will still work fine.  I also  
ask for the headers to be returned to me as a row, so I can add the  
new one and print them out.  You can let FasterCSV skip them instead,  
if you are just mining data.

You can see the new FasterCSV::filter() method at work here too.   
This is just Unix filters for CSV streams.  You can alter the row  
after it is read and it will be sent back out after the block returns.

The other big feature at work here is automatic row separator  
detection, but I hope you never notice it.  In this case, it used $/  
for input and output because that makes the most sense for STDIN and  
STDOUT.  If actual files had been involved, it would have tried to  
auto-detect the separator.  This should make the code more portable.

Hope this helps.

> Another idea. I am no C expert. But maybe it is worth to do an  
> optional parser as a C module. Maybe with the help of RubyInline?  
> Especially the parse method would be a great target.

One of my design goals is keeping FasterCSV pure Ruby.  This makes it  
trivial to bundle with you app, if needed, and makes it more fun for  
me to maintain.

If you want to build a C version though, best of luck to you.

James Edward Gray II