From: Kaspar Schiess <eule@...>
Date: 2009-12-23T16:14:13+09:00
Subject: Re: Suggestions for a distributed job queue

Hi,

I am using an AMQP compliant queue for this. With its permanent queuing 
and routing mechanisms it can be made to meet many if not all of your 
requirements, I believe.

As job items I am using thrift RPC method calls, which is very 
convenient on both sides (server, client). The library that allows you 
to do this is here: http://github.com/kschiess/thrift_amqp_transport 
(currently being redesigned).

There are many solutions in this space. Most recently, people have been 
using resque (github I believe) and AMQP-Queues. Other solutions can be 
found using the database or the filesystem, as you already know.

my 2 cents
kaspar