Re: [ANN] jobQueue 1.0.1 - Running stuff with a user defined number of threads

On Mon, Dec 12, 2011 at 5:18 PM, Ralf Mueller <[e-mail]> wrote:
> On 12/12/2011 01:23 PM, Robert Klemme wrote:
>>
>> I would change the API slightly to modify behavior of #push:
>>
>> 1. push(obj, method, *arguments)
>
> This was my first interface idea, too. But I skipped it to be able to pus=
h a
> whole list of items at once. But it really lacks readability - that's for
> sure. I guess I will let the user do the iteration over the items to push=
 in
> favour of having a beautiful interface. I could keep the old push interfa=
ce
> under a different name.

Or you just add #push_all(enum).

>> One could even extend behavior by providing a back channel for results:
>>
>> jq.push do |back_channel|
>> =A0 back_channel<< =A0(1 + complicated_calculation() * 123)
>> end
>>
>> For that of course you must define how reply values are dealt with
>> (there could be a null back channel which just discards results if
>> configured that way). =A0Alternatively however just the result values of
>> method and block invocation could be used. =A0Maybe that's cleaner.
>
> Hm. It's definitely good for testing. Could you image a "real" use case f=
or
> this? =A0Maybe parallel image processing or database requests.

Well, any farmer worker scenario where you need a single instance
composing all results into something complete.  Also, finding out that
all workers are finished could be viewed as one way of evaluating
results, too.

>> Also I would separate support for the call of system: Basically
>> invoking system is a special case which does not necessarily have
>> something to do with job queues in general. =A0So a better solution
>> would be to have a specialized job queue, e.g.
>>
>> class SystemJobs
>> =A0 attr_reader :jq
>>
>> =A0 def initialize(jq)
>> =A0 =A0 @jq =3D jq
>> =A0 end
>>
>> =A0 def push(*args)
>> =A0 =A0 jq.push do |back_channel|
>> =A0 =A0 =A0 back_channel<< =A0system(*args)
>> =A0 =A0 =A0 # we could use a variant of IO.popen here as well which
>> =A0 =A0 =A0 # captures output
>> =A0 =A0 end
>> =A0 end
>> end
>>
>> You then could still do the pretty short
>>
>> sj.jq.push($stdout, :puts, "hello world")
>
> This would simplify the task handling in the JobQueue class. Running syst=
em
> commands and calling ruby methods in the same queue should not be the
> regular case.

My main argument would be separation of concerns.  Your basic JobQueue
is simply only responsible for executing tasks in concurrent threads.
Executing system commands is a special case which would be of no use
for someone who just needs to concurrent execution in the current
process.

> Many thanks!

You're welcome!

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/