From: Robert Klemme <shortcutter@...>
Date: 2009-10-22T06:55:05+09:00
Subject: Re: merging partial, incomplete data files

On 10/21/2009 11:20 PM, Benjamin Thomas wrote:
> Hello all,
> 
> I write scripts for fun and am still a noob.
> 
> I'd like to use ruby to solve a problem for me but find it difficult
> this time so I thought I'd ask for help.
> 
> It's very basic in fact.
> 
> I have 2 csv files I'd like to merge such as :
> 
> File1               File2
> -----               -----
> ref;qty             ref;price
> A;10                A;100
> B;20                D;150
> C;30                C;200
> E;5                 B;75
> 
> outputs to File3 =>
> 
> File3
> -----
> ref;qty;price;total
> A;10;100;1000
> B;20;75;1500
> C;30;200;6000
> D;"missing_data";150;"missing_data"
> E;5;"missing_data";"missing_data"
> 
> Here is the code I have so far:
> 
> ###########################################################################
> def process_file(file)
>   processed = []
>   opened = File.open(file, "r").readlines
> 
>   opened.each {|line| processed << line.strip.split(";")}
>   return processed
> end
> 
> arr1 = process_file("file1")
> arr2 = process_file("file2")
> 
> p arr1, arr2
> ###########################################################################
> 
> ==> [["ref", "qty"], ["A", "10"], ["B", "20"], ["C", "30"], ["E", "5"]]
> ==> [["ref", "price"], ["A", "100"], ["D", "150"], ["C", "200"], ["B",
> "75"]]
> 
> ----
> So I have 2 arrays which is great but I'm not sure on how to go about
> merging all this since data is sometimes missing from file1, some other
> times from file2. Of course those 2 files are more complex (and
> bigger!), and I'd like to setup a mechanism that would allow me to
> select which columns to filter and merge, but that's the basic idea
> anyway.
> 
> Thanks for your input and direction.

I'd say, you should look into classes Hash and Set.  If you have both 
files in a Hash each you can do

keys = f1.keys.to_set | f2.keys

Then you can do

keys.each do |key|
   # fetch from f1 and f2
end

Kind regards

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/