From: "James Britt (rubydev)" <james@...>
Date: 2001-12-08T12:32:01+09:00
Subject: [ruby-talk:27890] Re: DBI and large result sets

> From: Todd Gillespie [mailto:toddg@linux128.ma.utexas.edu]
> "James Britt (rubydev)" <james@rubyxml.com> wrote:
> : I can live with the performance hit if, in exchange, I can process huge
> : record sets without needing gobs of memory.
> 
> The performance hit is mostly a matter of the DB doing really dumb things
> like scanning half the freakin' disk to find all the elements.  DB
> designers have spent 30 years building smart query systems.  I've seen my
> fill over the last few years of application servers that try to supersede
> all that work and redo the query elements in-application, to the great
> suffering of users and administrators alike.  So I'd advise you
> find another way.  DBSource would make young girls squeal if you wrote a
> system that didn't kneecap the database.

I much prefer to let the database do what it's best at. It may not be
what's best for my application, though.  I'll look at using a cursor, though.

> : A complex query or stored proc would be better for any specific database, but
> : I want the DBSource class to work with DBI to (ideally) make it (more) 
> : portable. I don't *think* all databases support syntax for "give me records
> : m through n" (such as 'LIMIT' in MySQL)
> 
> Unfortunately, databases are certainly an area where utility is inversely
> proportional to portability.  Any high perf DB-backed application tries to
> stay as close to the DB as possible.  The 'lowest-common-denominator-SQL'
> approach is generally only used successfully by students and overfed
> internet consultancies that will go out of business next year.  I highly
> recommend you look into the path started by the OpenACS4 fold with their
> DB abstraction layer; they have implemented a better approach.

OpenACS looks interesting.  Not sure how much is easily snarfable, but perhaps
I can steal some ideas.

> 
> Happy hacking!

Thanks!

James
> 
>