From: Brian Candler Date: 2012-08-29T21:20:44+09:00 Subject: Re: working on multiple machines in a LAN ajay paswan wrote in post #1073756: > I have a program, now I want this to be run on multiple computers > individually. > Among them one computer (say boss) will distribute inputs to other > computers (say followers) and itself in a LAN. The boss will also > collect outputs from followers. Whenever a follower (including boss) > finishes its task, it acknowledges to the boss, and boss gives the next > input to that machine. > > How can it be done? If you want to do this entirely in ruby, have a look at resque. The boss can queue up jobs (they actually go in a redis database) and the workers can do work. They can post their results back into redis or into a shared filesystem or whatever you choose. https://github.com/defunkt/resque If you want something a little less closely coupled to ruby, look at condor. This allows you to submit arbitrary jobs (which could be shell scripts or ruby scripts or binary executables) with arguments, and runs them on available worker nodes. There's a lot of functionality there, including the ability to set up DAGs (Directed Acyclic Graphs) of jobs, so that jobs don't run until all their necessary precursors have run, and fitting jobs into slots of available CPU and RAM. Links: http://research.cs.wisc.edu/condor/CondorWeek2012/presentations/adesmet-admin.pdf http://research.microsoft.com/en-us/events/scidata04/todd_tannebaum.ppt https://condor-wiki.cs.wisc.edu/index.cgi/wiki?p=GettingStarted http://spinningmatt.wordpress.com/2011/06/12/getting-started-creating-a-multiple-node-condor-pool/ http://research.cs.wisc.edu/condor/manual/v7.8/ For some other related options see: http://serverfault.com/questions/79350/software-for-batch-process-management-in-unix -- Posted via http://www.ruby-forum.com/.