From: the.liberal.media@...
Date: 2006-03-23T07:13:52+09:00
Subject: Ruby IPC In an OpenMosix Cluster

I'm undertaking a project that will eventually become a processing
pipeline application of sorts.  It will receive data in a common
format, transform it into one of _many_ other formats, compile and send
it off to various endpoints (sounds spammish, but it's all solicited,
really :).

My team and I have tentatively decided on openMosix to provide an
easily scalable cluster, and Ruby for the application itself.  I'm very
new to Ruby, and fairly new to IPC concepts -- The Little Book of
Semaphores and the many threads I've read here have helped me out a
_lot_.

Our application will rely on 3 sets of consumer process pools; each
pool is spawned by a daemon responsible for each basic operation (think
MTA): receive, transform, compile/send.  By using processes instead of
threads we allow openMosix to migrate each process and make use of the
entire cluster.

So we have the model down, but I need a bit of advise on how to most
efficiently get these processes talking.  What is the best form of IPC
to use here?  It seems there are tons of Ruby examples on concurrency
and communication between threads, but I can't seem to find anything
definitive on IPC (to more than one child at least).  I tried the sysv
extension off RAA, but couldn't get it to compile -- though I didn't
try my best.

Things I'm considering:

 - DRb
 - UNIXSocket
 - mkfifo
 - SysV message queue (openMosix doesn't support shmem segments)
 - popen (though I can't see how to do it without round robin
producing)

If anyone has any advice, please shove me in the right direction.

Thanks in advance!

Also, thanks to matz for the great language (code blocks are uber
bueno)!

Best,
Dan