From: "M. Edward (Ed) Borasky" <znmeb@...>
Date: 2007-07-25T10:50:48+09:00
Subject: Re: Preparing Data For Statistical Analysis and Reporting

Robert wrote:
> How much data munging is done by Ruby-ists?
> 
> If you need some tools for preparing data for analysis, transforming
> data, and cleaning data up, there are some tools to check out at:
> 
> http://directory.fsf.org/math/stats
> 
> including:
> 
> Vilno
> DAP
> PSPP
> 
> DAP is an open-source version of SAS
> PSPP is an open-source version of SPSS
> Vilno uses it's own language for data transformation, similar to SQL
> SELECT, SAS, and SPSS.
> 
> For statistics itself, I am rather curious about Apophenia
> ( apparently a C library for statistics that can be called by any
> statistics package).
> 
> 
> 
> Robert
> 
> 
> 

Thanks for the pointers! This is actually something I do a lot in my day
job. Most of the data extraction is done in Perl, mostly because Ruby
didn't exist when I wrote most of the code. If there were a couple other
Ruby programmers around, I'd port it to Ruby/Rails/Ruport.

Nearly all of the statistics I do is in R, also an open source project.
There is an R-Ruby bridge called "RSRuby", so that would be another way
to get into this.

Nearly everything I do these days is large enough that I use either SQL
Server or PostgreSQL to manage the datasets. By the end of the year I
expect it will all be in PostgreSQL -- the SQL Server licenses are too
expensive.