From: Robert Klemme <shortcutter@...>
Date: 2008-08-14T18:22:35+09:00
Subject: Re: Mutithreading to implement near 7000 to 10000 mssage per min

2008/8/14 Michael T. Richter <ttmrichter@gmail.com>:
> On Thu, 2008-08-14 at 14:22 +0900, Kaja Mohaideen wrote:
>
> we want to process 7000 to 10000 message and store it to the Database
> perminute. we are trying through threads. can you please help me how we
> do?
>
> Use a quick language that's designed for massively parallel operations, not
> one of the slower scripting languages without proper support for
> parallelism.

There may be a few ways to get this working with Ruby:

1. use Ruby to create a CSV file or similar which is then loaded into
the database via the database's bulk loader.

2. if there is support for batch operations in DBD/DBI these might work as well.

Since this task is mostly IO bound (unless of course there are
expensive operations to be done on those messages), Ruby threads are
not too unlikely to work in this scenario.

> I'm not trying to be glib here (although I may well be succeeding anyway).
> What you are saying here looks an awful lot to me like "I want to hammer in
> this screw here.  Which wrench is best for the job?"  Select your tools
> appropriately for the problem.  Don't try to reforge your wrenches into
> bizarre combination screwdriver-hammers.

Well, maybe Kaja is just experimenting and trying out to which limits
he / she can push Ruby. :-)

Kind regards

robert


-- 
use.inject do |as, often| as.you_can - without end