From: Robert Klemme <shortcutter@...>
Date: 2011-01-25T19:11:46+09:00
Subject: Re: Worker pool - what to use?

On Mon, Jan 24, 2011 at 8:16 PM, Christoph B. <ch.blank@gmail.com> wrote:
> Hey,
>
> My requirement: A worker pool that basically renders templates to
> html/pdf and sends emails.
>
> The messages to process are currently stored in an OpenMQ queue
> (accessible via stomp for ruby).
>
> I am wondering whats the best way to create this worker pool. Initially
> I thought about using resque, as most worker "frameworks" come with
> their own queue... however a few concerns:
>
> - does putting msgs from one into another queue make sense?
> - how reliable is resque? I cannot lose mails
> - how to handle confirmation on the OpenMQ? if I insert the msgs into
> resque
>
> So I wondered how you would do this...
> Maybe just create a lightweight pool of workers? How? I only found
> existing ones having a queue.. which is not really needed in this
> scenario as I confirm messages on the OpenMQ..
>
> I would appreciate any suggestions!

<disclaimer>I don't know stomp and I have only rudimentary knowledge
of OpenMQ,</disclaimer>

It seems the simplest solution would be to create a number of Ruby
processes that each individually fetch data from your OpenMQ one after
another and process it.  You could add a controlling master process
that restarts child processes if they die.  Sample:

https://gist.github.com/794746

Cheers

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/