From: ara.t.howard@...
Date: 2006-01-03T02:24:53+09:00
Subject: Re: Memoization, files and Marshal?

--1yeeQ81UyVL57Vl7
Content-Type: MULTIPART/MIXED; BOUNDARY=1yeeQ81UyVL57Vl7

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1yeeQ81UyVL57Vl7
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII; FORMAT=flowed
Content-ID: <Pine.LNX.4.62.0601020908102.24863@harp.ngdc.noaa.gov>
Content-Disposition: INLINE

On Mon, 2 Jan 2006, Mauricio Fernandez wrote:

>
> Dumping the cache on every miss seems expensive.
> I changed the code to save it once every N misses, and made the cache update
> atomic to prevent problems with:
> * concurrent executions of the memoized method (processes and threads)
> * no HD space being left (the previous cache state is preserved)
>
> I also applied those changes to memoize.rb 1.2.0 and generalized it to work
> with instance methods, see the patch after memo.rb.

nice idea.

seems like the perfect time to introduce a Memoize::Cache class wrapping
pstore though doesn't it?

cheers.

-a
-- 
===============================================================================
| ara [dot] t [dot] howard [at] noaa [dot] gov
| all happiness comes from the desire for others to be happy.  all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
===============================================================================
--1yeeQ81UyVL57Vl7
Content-Type: TEXT/PLAIN; CHARSET=us-ascii
Content-ID: <Pine.LNX.4.62.0601020908103.24863@harp.ngdc.noaa.gov>
Content-Description: 
Content-Disposition: ATTACHMENT; FILENAME=memoize.rb

module Memoize
   MEMOIZE_VERSION = "1.2.0"

   def self.memoize(recv, name, file=nil, period=-1)
      class << recv; self end.instance_eval{ instance_memoize(name, file, period) }
   end

   def memoize(name, file=nil, period=-1)
      Memoize.memoize(self, name, file, period)
   end
end

class Module
   # Memoize the method +name+.  If +file+ is provided, then the method results
   # are stored on disk in addition to the in-memory cache.
   def instance_memoize(name, file=nil, period=-1)
      meth = instance_method(name)

      if file
         cache = Hash.new.update(Marshal.load(File.read(file))) rescue {}
      else
         cache = {}
      end

      save_cache_proc = lambda do
         tmpfile = "#{file}.#{Process.pid}"
         begin
            File.open(tmpfile, "wb+"){|f| Marshal.dump(cache, f) }
            File.rename tmpfile, file
         rescue Exception
         end
      end

      at_exit { save_cache_proc.call } if file

      calls = period
      define_method(name) do |*args|
         unless cache.has_key?(args)
            cache[args] = meth.bind(self).call(*args)
            if file && period != -1 && (calls -= 1) <= 0
               calls = period
               save_cache_proc.call
            end
         end
         cache[args]
      end

      cache
   end
end

if __FILE__ == $0
   def fib(n)
      return n if n < 2
      fib(n-1) + fib(n-2)
   end

   def fib2(n)
      return n if n < 2
      fib2(n-1) + fib2(n-2)
   end

   def fib3(n)
      return n if n < 2
      fib3(n-1) + fib3(n-2)
   end

   include Memoize

   module DumbFib
      def fib(n)
         return n if n < 2
         fib(n-1) + fib(n-2)
      end
   end

   DumbFib2 = DumbFib.clone

   memoize :fib, "xfib.memoize.cache", 1
   memoize :fib2, "xfib2.memoize.cache"
   memoize :fib3, "xfib3.memoize.cache", 10

   class Dummy1
      include DumbFib
      instance_memoize :fib, "xfib.dumb1.cache"
   end
   class Dummy2
      include DumbFib2
   end
   module DumbFib2
      instance_memoize :fib, "xfib.dumb2.cache"
   end

   require 'benchmark'
   runs = 1
   Benchmark.bmbm(10) do |bm|
      dummy1 = Dummy1.new
      dummy2 = Dummy2.new
      bm.report("period 1") { runs.times{ fib  500  } }
      bm.report("period 10"){ runs.times{ fib3 500  } }
      bm.report("at_exit")  { runs.times{ fib2 500  } }
      bm.report("at_exit''")  { runs.times{ dummy1.fib 500  } }
      bm.report("at_exit'''")  { runs.times{ dummy2.fib 500  }; runs = 100000}
   end

end

--1yeeQ81UyVL57Vl7--
--1yeeQ81UyVL57Vl7--