From: Joel VanderWerf <joelvanderwerf@...>
Date: 2010-05-26T10:20:48+09:00
Subject: Re: In-code data

Jerome David Sallinger wrote:
> Hi,
> 
> What is the best way/practice of storing data that is to be used and
> accessed within your code. For example if I downloded a language
> dictionay in txt format. Would it make more sense to read it from a text
> file and parse it into a hash table variable. Or could I have it
> contained within a secondary source file and simple use 'require' to
> load it. I work with a lot of specifications which often contain lots of
> tables with values, I would like to simply cut and paste the tables into
> my code. This would make the code easier to update between versions of
> specs especially where the data in the tables has changed. I know this
> sounds like a general question, but I am all ears to any ideas of new
> ways of working.
> 
> TC

Since you are updating the data periodically, you might want to consider 
setting up some rake tasks to download and parse the data and store it 
in a binary format that can be loaded quickly. This would probably be 
faster (for you) than cut and paste and faster (for the computer) than 
parsing on each run.

file "download.txt" do
   # code to download file
   sh "wget http://vii.path.berkeley.edu/~vjoel/download.txt"
end

rule ".dat" => ".txt" do |t|
   h = {}

   File.open(t.prerequisites[0]) do |f|
     f.each do |line|
       key, value = line.split
       h[key] = value
     end
   end

   File.open(t.name, "wb") do |f|
     Marshal.dump(h,f)
   end
end

task :clean do
   rm "download.txt"
   rm "download.dat"
end

task :run => "download.dat" do
   h = File.open("download.dat", "rb") do |f|
     Marshal.load(f)
   end
   p h
end