From: Robert Klemme <shortcutter@...>
Date: 2011-12-12T21:23:12+09:00
Subject: Re: [ANN] jobQueue 1.0.1 - Running stuff with a user defined number of threads

On Mon, Dec 12, 2011 at 11:25 AM, Ralf Mueller <ralf.mueller@zmaw.de> wrote:
> Hi!
> JobQueue is a ruby gem, that allows you to run Ruby methods and system
> commands in parallel. It comes with an executable prun.rb with takes two
> arguments: Number of worker threads and a shell scripts, from which line is
> executed in parallel wrt the given number of threads.
>
> Thanks to Robert Klemme, who provided some implementation ideas some years
> ago.

You're welcome!

> Hints for improvements and any type of criticism are very welcome! Hope
> you'll find it usefull.
>
>
> doc: http://rubydoc.info/gems/jobQueue/frames

I would change the API slightly to modify behavior of #push:

1. push(obj, method, *arguments)
2. push(&block)

Example

1. send method

jq.push($stdout, :puts, "hello world")

2. use block

jq.push do
  puts "hello world"
end

One could even extend behavior by providing a back channel for results:

jq.push do |back_channel|
  back_channel << (1 + complicated_calculation() * 123)
end

For that of course you must define how reply values are dealt with
(there could be a null back channel which just discards results if
configured that way).  Alternatively however just the result values of
method and block invocation could be used.  Maybe that's cleaner.


Also I would separate support for the call of system: Basically
invoking system is a special case which does not necessarily have
something to do with job queues in general.  So a better solution
would be to have a specialized job queue, e.g.

class SystemJobs
  attr_reader :jq

  def initialize(jq)
    @jq = jq
  end

  def push(*args)
    jq.push do |back_channel|
      back_channel << system(*args)
      # we could use a variant of IO.popen here as well which
      # captures output
    end
  end
end

You then could still do the pretty short

sj.jq.push($stdout, :puts, "hello world")

> dev: https://github.com/Try2Code/jobQueue
> tests:
> https://github.com/Try2Code/jobQueue/blob/master/test/test_jobqueue.rb

Kind regards

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/