From: Ezra Zygmuntowicz <ezmobius@...>
Date: 2008-08-15T03:58:51+09:00
Subject: Re: Mutithreading to implement near 7000 to 10000 mssage per min


On Aug 14, 2008, at 6:28 AM, M. Edward (Ed) Borasky wrote:

> On Thu, 2008-08-14 at 16:52 +0900, Michael T. Richter wrote:
>> On Thu, 2008-08-14 at 14:22 +0900, Kaja Mohaideen wrote:
>>> we want to process 7000 to 10000 message and store it to the  
>>> Database
>>> perminute. we are trying through threads. can you please help me  
>>> how we
>>> do?
>>
>> Use a quick language that's designed for massively parallel
>> operations, not one of the slower scripting languages without proper
>> support for parallelism.
>>
>> I'm not trying to be glib here (although I may well be succeeding
>> anyway).  What you are saying here looks an awful lot to me like "I
>> want to hammer in this screw here.  Which wrench is best for the  
>> job?"
>> Select your tools appropriately for the problem.  Don't try to  
>> reforge
>> your wrenches into bizarre combination screwdriver-hammers.
>
> Yes ... Erlang / Mnesia should be able to handle this, and then you
> could write some Ruby code to extract the messages from Mnesia to a
> "regular" RDBMS if needed.


	I'd say use rabbitmq with the ruby amqp library, this will allow you  
to easily push many thousands of messages/sec into the rabbitmq  
message bus and then you can have a set of fanout workers consuming  
the queues on the other side and putting the items in the database.

-Ezra