From: David Balmain <dbalmain.ml@...>
Date: 2005-11-16T11:29:09+09:00
Subject: Re: Index and Query (#54)

On 11/16/05, Dale Martenson <dale.martenson@gmail.com> wrote:
> David,
>
> After reading your results I thought I would try and make a couple of
> simple changes. I attempted to cleanup the 'insert' routine since that
> is where most of the processing time seemed to be spent. I also added
> the ability to perform multi-term searching (individual terms or single
> string). This will worsen the look-up times, but it might be a good
> change.
>
> If possible, could you run this version through your test to see how it
> does?

Hi Dale,
I've updated the results. You may notice a number of the indexing
times have changed. I modified my test to make it a little fairer.
Anyway, you earned yourself some green realestate on the bottom chart.
Somethings up with the search results from your first bitmap index.
I'm just running that one again. By the way, I had to change the
second last line of your find method to;

  result.push( symbol ) if value & mask > 0

Cheers,
Dave

> class IndexHash
>         def initialize( documents=nil )
>                 @index = Hash.new( [] )
>                 input( documents ) if documents
>         end
>
>         def input( documents )
>                 documents.each_pair do |symbol, contents|
>                         contents.split.each { |word| insert( symbol, word) }
>                 end
>         end
>
>         def insert( document_symbol, word )
>                 w = word.downcase
>                 @index[w] += [ document_symbol ] unless @index[w].include?(
> document_symbol )
>         end
>
>         def find( *strings )
>                 result = []
>                 strings.each do |string|
>                         string.split.each do |word|
>                                 result += @index[ word.downcase ]
>                         end
>                 end
>                 result.uniq
>         end
>
>         def words
>                 @index.keys.sort
>         end
> end
>
> class IndexBitmap
>         def initialize( documents=nil )
>                 @index = []
>                 @documents = Hash.new( 0 )
>                 input( documents ) if documents
>         end
>
>         def input( documents )
>                 documents.each_pair do |symbol, contents|
>                         contents.split.each { |word| insert( symbol, word) }
>                 end
>         end
>
>         def insert( document_symbol, word )
>                 w = word.downcase
>                 @index.push( w ) unless @index.include?( w )
>                 @documents[ document_symbol ] |= (1<<@index.index( w ))
>         end
>
>         def find( *strings )
>                 result = []
>                 mask = 0
>
>                 strings.each do |string|
>                         string.split.each do |word|
>                                 w = word.downcase
>                                 mask |= (1<<@index.index(w)) if @index.index(w)
>                         end
>                 end
>
>                 @documents.each_pair do |symbol, value|
>                         result.push( symbol ) if value & mask
>                 end
>                 result
>         end
>
>         def words
>                 @index.sort
>         end
> end
>
>
>