From: Shashank Date <sdate@...>
Date: 2004-08-17T11:41:10+09:00
Subject: Re: [ANN] ruby queue : rq-0.1.2

Hello Ara,

"Ara.T.Howard" <ahoward@noaa.gov> wrote in message
> you'll probably want to test you nfs setup first - i can give a little
script
> that should determine if mixed windows/linux nfs access works with
locking -
> it's pure ruby so it should just run.  do you have a few nodes you can
test
> on?

Not right away ... but I will try and get a 2-node configuration working
over
the (already crowded ;-) week-end. No promises though.

If I succeed, I will have Win XP (Home) on one node and SuSE 8.2 on the
other. But please go ahead and email the script which determines
if it works or not.

> >
> > If and when we start using it, I will be able to contribute towards:
>
> > 0. GUI Console,
>
> that'd be great - it's on the TODO list...

Good ... I will be using wxRuby if it comes to me.

> > 1. dynamic load-balancer
>
> it doesn't need one! ;-)  all nodes access the queue taking jobs in a
'highest
> priority oldest out first' fashion.  in otherwords, all nodes bail water
as
> fast as possible - if the boat is sinking the only answers are:
>
>    - write faster jobs
>    - make network faster (vsftp is great!)
>    - buy faster nodes
>    - buy more nodes
>
> this is one of the great things about the totally decentralized
architechture
> - there is no need for load balancing or scheduling!

Hmmm... then may be I have mis-understood the nature of your project.
I was thinking of using it for CPU intensive (eg. image analysis) jobs over
a
cluster of heterogenous (in terms of CPU power, O.S. and in terms
of functionality) nodes and wanted the ability to "farm" (push) work
requests to least busy CPUs (provided there is a way to determine that, of
course).  Phil Tomson's TaskMaster comes to mind. See:
http://raa.ruby-lang.org/project/taskmaster/

[Phil: if you are reading this, will love some feedback from you:
specifically
are you planning to work on it in near future? Or is the project closed?]

From your describtion above it appears to me that work will be
"fetched" (pulled) by least busy CPUs. Am I correct? (We can take this
discussion off line if it starts becoming [OT]).

> > 2. rudimentary work-flow engine
>
> you mean dependancies?  yes that would be nice...

Well, a little more than that. Right now, this is still "pie in the sky"
kind
of an idea... but I can see some of the patterns being implemented.

See: http://tmitwww.tm.tue.nl/research/patterns/patterns.htm
for some details.

This is very likely to be extremely specific to the problem we are trying
to solve and may also be proprietory. I hope your licensing terms
will permit me that.

> it all sounds great - i'll try to get some more docs out in the next few
weeks
> so you can read about it.

Fantastic ... I am all ears (ummm eyes ;-)!

> cheers.

Thanks,
-- shanko