From: gga <GGarramuno@...>
Date: 2007-03-09T19:30:05+09:00
Subject: Re: File open, read and store in Hash, efficient?


Well, your code is more or less okay.  It may be buggy in that you are
also storing the \n (end of line) character.  You probably need
something like:
 @@store[temp[0]] = temp[1].chomp
to remove the it.

You can avoid checking if the file exists (if it does not, an Errno
exception will be raised and propagated upstream).  Let the
application, instead of your class, deal with what's probably a user
error (providing a missing file).
You can also avoid the file close by doing it in a block (let ruby's C
code automatically do the file close) and you can use IO#foreach
(File#foreach) for iterating thru each line more easily.
If you know you won't have files that won't fit in memory, you can
read all your text into a string or array in a single go (this is
usually called slurping), which can also speed things up a little in
some cases.

Here are some examples of doing the same thing written in different
ways:


require 'yaml'

class ReaderYAML
  def initialize(file)
    # slurp the whole file into a string
    lines = File.read(file)
    # change commas to : (yaml hash representation)
    lines.gsub!(/,/, ':')
    # create the hash thru yaml
    @h = YAML::load(lines)
  end
end

require 'csv'

class ReaderCSV
  def initialize(file)
    # read the file as a CSV file, flatten the resulting array and
    # make it a hash
    @h = Hash[*(CSV.read(file).flatten)]
  end
end

class ReaderCommas
  def initialize(file)
    @h = {}
    # slurp the file into an array
    lines = File.readlines(file)
    # process each line
    lines.each { |line|
      key, value = line.chomp.split(',')
      @h[key] = value
    }
  end
end

class ReaderCommasBigFile
  def initialize(file)
    @h = {}
    File.foreach(file) do |line|
      key, val = line.chomp.split(',')
      @h[key] = val
    end
  end
end

h = ReaderYAML.new('csv.txt')
p h

h2 = ReaderCSV.new('csv.txt')
p h2

h3 = ReaderCommas.new('csv.txt')
p h3

h4 = ReaderCommasBigFile.new('csv.txt')
p h4


require 'benchmark'

n = 5000
Benchmark.bm(5) do |b|
  b.report('big') { n.times do ReaderCommasBigFile.new('csv.txt');
end }
  b.report('file') { n.times do ReaderCommas.new('csv.txt'); end }
  b.report('csv')  { n.times do ReaderCSV.new('csv.txt'); end }
  b.report('yaml') { n.times do ReaderYAML.new('csv.txt'); end }
end


The YAML version does not do exactly the same as the others, but
depending on your data, it might still be what you want.  It also
works for a very simple key/value pair per line.  Albeit YAML involves
a little bit more work, it is still pretty optimized and will turn
numeric data automatically into the appropriate ruby numeric class.
CSV automatically deals with comma separated files for you, albeit it
is somewhat slow.

Anyway, hope that gives you some ideas.  Overall, unless you are
dealing with huge files, you should not worry too much about speed
while writing your class.