From: Drew Olson <olsonas@...>
Date: 2007-02-09T00:11:10+09:00
Subject: Re: General Approach to Data Validation

Pit Capitain wrote:
> Drew Olson schrieb:
>> datasets here and want to streamline the code as much as possible.
> Drew, I would try to perform the comparison inside the database,
> especially if your datasets are "very large". (How large is this
> actually?) Given that you are working with Oracle, I would use Oracle's
> database links to get access to both tables.
> 
> Regards,
> Pit

Pit -

This does make sense, but I'm using ruby in this situation for a reason. 
I had previously written some validation scripts that ran on .csv dumps 
from the database and I leveraged this previous work to get these 
scripts up and running quickly by introducing ActiveRecord. Also, I'm 
writing an error report to .csv using FasterCSV and doing quite a bit of 
data manipulation during these compares. In short, I'd really like to 
continue using ruby/ActiveRecord here. However, I want to make sure that 
the way I'm going about it is as "efficiently as possible". It's not a 
__huge__ deal, however if I'm make some massive error that would save 
50% when running my scripts, it would be nice to change them.

As far as record size, we're talking close to 1 million records, more in 
some cases.

-Drew

-- 
Posted via http://www.ruby-forum.com/.