From: Brian Candler <b.candler@...>
Date: 2010-08-07T06:16:44+09:00
Subject: Re: comparing two arrays, too slow

Another option: sort them by id, then walk through them together. If the 
current id on both list A and list B is the same, then skip forward on 
both. If the current id on list A is less than the current id on list B, 
then it exists only in B (so report this, and skip forward on A). And 
vice versa.

The advantage of this is that it works with huge files - you can read 
them one line at a time instead of reading them all into RAM. And there 
are tools which can do an external sort efficiently, if they're not 
already sorted that way.

To avoid coding this, just use the unix shell command 'join' to do the 
work for you (which you can invoke from IO.popen if you like). Just 
beware that it's very fussy about having the files correctly sorted 
first.

$ join -t, -j1 -v1 sunday.csv monday.csv
2,curly,tall
3,moe,meanie
$ join -t, -j1 -v2 sunday.csv monday.csv
4,shemp,greasy
-- 
Posted via http://www.ruby-forum.com/.