From: James Edward Gray II <james@...>
Date: 2006-02-04T02:44:11+09:00
Subject: Re: memoize to a file

--Apple-Mail-2--379837572
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

On Feb 1, 2006, at 9:56 AM, ara.t.howard@noaa.gov wrote:

> On Thu, 2 Feb 2006, James Edward Gray II wrote:
>
>> On Feb 1, 2006, at 9:31 AM, ara.t.howard@noaa.gov wrote:
>>
>>> why not pstore - it's done all that already and is built-in?
>>
>> PStore is just a wrapper on top of Marshal for transactional file  
>> storage. If you need transactions, it's great.  Otherwise, you  
>> might as well just use Marshal.
>
> it's not quite only that.  it also
>
>   - does some simple checks when creating the file (readability, etc)
>   - allows db usage to be multi-processed
>   - supports deletion
>   - rolls backs writes on exceptions / commits using ensure to  
> avoid corrupt
>     data file
>   - handles read vs write actions using shared/excl locks to boost  
> concurrency
>   - uses  md5 check to avoid un-needed writes
>   - opens in correct modes for all platforms

I've made a file caching example using PSTore for my toy Memoizable  
library.  I just thought I would post it here, in case it helps/ 
inspires others.

#!/usr/local/bin/ruby -w

# pstore_caching.rb
#
#  Created by James Edward Gray II on 2006-02-03.
#  Copyright 2006 Gray Productions. All rights reserved.

require "memoizable"
require "pstore"

#
# A trivial implementation of a custom cache.  This cache uses PStore  
to provide
# a multi-processing safe disk cache.  The downside is that the  
entire cache
# must be loaded for a key check.  This can require significant  
memory for a
# large cache.
#
class PStoreCache
   def initialize( path )
     @cache = PStore.new(path)
   end

   def []( key )
     @cache.transaction(true) { @cache[key] }
   end

   def []=( key, value )
     @cache.transaction { @cache[key] = value }
   end
end

class Fibonacci
   extend Memoizable

   def fib( num )
     return num if num < 2
     fib(num - 1) + fib(num - 2)
   end
   memoize :fib, PStoreCache.new("fib_cache.pstore")
end

puts "This method is memoized using a file-based cache..."
start = Time.now
puts "fib(100):  #{Fibonacci.new.fib(100)}"
puts "Run time:  #{Time.now - start} seconds"

puts
puts "Run again to see the file cache at work."

__END__

James Edward Gray II


--Apple-Mail-2--379837572
Content-Transfer-Encoding: 7bit
Content-Type: text/x-ruby-script;
	x-unix-mode=0644;
	name="memoizable.rb"
Content-Disposition: attachment;
	filename=memoizable.rb

#!/usr/local/bin/ruby -w

# memoizable.rb
#
#  Created by James Edward Gray II on 2006-01-21.
#  Copyright 2006 Gray Productions. All rights reserved.

# 
# Have your class or module <tt>extend Memoizable</tt> to gain access to the 
# #memoize method.
# 
module Memoizable
  # 
  # This method is used to replace a computationally expensive method with an
  # equivalent method that will answer repeat calls for indentical arguments 
  # from a _cache_.  To use, make sure the current class extends Memoizable, 
  # then call by passing the _name_ of the method you wish to cache results for.
  # 
  # The _cache_ object can be any object supporting both #[] and #[]=.  The keys
  # used for the _cache_ are an Array of the arguments the method was called 
  # with and the values are just the returned results of the original method 
  # call.  The default _cache_ is a simple Hash, providing in-memory storage.
  # 
  def memoize( name, cache = Hash.new )
    original = "__unmemoized_#{name}__"

    # 
    # <tt>self.class</tt> is used for the top level, to modify Object, otherwise
    # we just modify the Class or Module directly
    # 
    ([Class, Module].include?(self.class) ? self : self.class).class_eval do
      alias_method original, name
      private      original
      define_method(name) { |*args| cache[args] ||= send(original, *args) }
    end
  end
end

--Apple-Mail-2--379837572--