From: "Ara.T.Howard" <Ara.T.Howard@...>
Date: 2005-05-28T04:42:28+09:00
Subject: Re: How to build an index of phrases in a phrase/sentence?

On Sat, 28 May 2005, Dan Fitzpatrick wrote:

> I am trying to build an indexing structure on some phrases. Most phrases will 
> have 2 - 5 parts (words). The resulting array will be dumped into an index to 
> find the matching phrases. I don't want to do wildcard searching on the 
> resulting array to find the phrase.
>
> I would like to turn "This is some text" into
>
> ["This",
> "This is",
> "This is some",
> "This is some text",
> "is",
> "is some",
> "is some text",
> "some",
> "some text",
> "text"]
>
> The order of the resulting array doesn't matter. When someone searches for 
> "is some" or "some text", I want it to find this phrase. I don't want a 
> search for "is text" to find this phrase though.
>
> My solution so far can find all but the middle elements. In this case, "is 
> some". But when the original phrase has more parts, then more middle parts 
> are not added to the array.
>
> text = "This is some text"
> #=> "This is some text"
> ws = ''; text.split(/\W/).collect{|w| ws = (ws+' '+w).strip; ws}
> #=> ["This", "This is", "This is some", "This is some text"]
> ws = ''; text.split(/\W/).reverse.collect{|w| ws = (w+' '+ws).strip; ws}
> #=> ["text", "some text", "is some text", "This is some text"]
> text.split(/\W/).collect{|w| w}
> => ["This", "is", "some", "text"]
>
> Is there an better Ruby way to do this? Or is there a better data structure 
> for retrieving a word or an exact phrase within a phrase/sentence without 
> wild-carding the search.

maybe

   harp:~ > cat a.rb
   class String
     def subphrases delim = %r/\s+/
       words = strip.split delim
       return [] if words.empty?
       words.inject([]){|a,w| a << [a[-1],w].compact.join(' ')} +
         words[1..-1].join(' ').subphrases
     end
   end

   p "This is some text".subphrases
   p "foo".subphrases
   p "".subphrases

   harp:~ > ruby a.rb
   ["This", "This is", "This is some", "This is some text", "is", "is some", "is some text", "some", "some text", "text"]
   ["foo"]
   []

you may want to check out glimpse.

cheers.

-a
-- 
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| My religion is very simple.  My religion is kindness.
| --Tenzin Gyatso
===============================================================================