From: Thomas Hurst <tom.hurst@...>
Date: 2007-04-30T08:31:33+09:00
Subject: Re: Infix to Postfix Method

* Dan Zwell (dzwell@gmail.com) wrote:

>  That's pretty neat. I hadn't thought of using this stuff to generate
>  SQL. I'll probably acually use that idea some day.

It can perform the match itself too:

 irb(main):025:0> query = NSearch::QueryParser.new.parse('foo bar NOT heh')
 #<NSearch::RootQuery:0x1715380
  @query=
   #<NSearch::BooleanAndQuery:0x1714fe8
    @criteria=
     [#<NSearch::SubStringQuery:0x1715240 @substr="bar">,
      #<NSearch::SubStringQuery:0x1715268 @substr="foo">,
      #<NSearch::BooleanNotQuery:0x1715038
       @criteria=#<NSearch::SubStringQuery:0x17151f0 @substr="heh">>]>>

I can then query.rewrite to have it normalize itself (sort, minor
optimization), have it turn itself into various forms (SQL, boolean),
and of course execute it:

 irb(main):029:0> query.match('foo bar')
 => true
 irb(main):030:0> query.match('foo bar heh')
 => false

I also have a C library and a Ruby extension which uses it which
accelerates matches.  We're currently using it in our new search engine.
Performance is on the order of 5 million matches/second on a single
2GHz Opteron.  In pure Ruby it's closer to 200,000/sec.

I'm planning to_ferret to produce an equivilent ferret query, and
extending it to provide tagged queries: "category:ruby lang:en OR
lang:de".  Obviously the trickiest bit here is making it work in C.

None of this is public, alas, but if there's interest..

> And thanks for the bit about Generator. I'm currently looking through
> a computational script (ported from c++)--I could never figure out why
> it used all my ram within a few seconds, but it used Generators
> several thousand times. I'm rewriting parts of it now, and we'll see
> what happens.

Yeah, that'll be the callcc.  I really hope it's improved for 1.9.

-- 
Thomas 'Freaky' Hurst
    http://hur.st/