From: David Masover Date: 2008-09-10T00:46:29+09:00 Subject: Re: Drake: Distributed Rake On Tuesday 09 September 2008 05:03:43 quixoticsycophant@gmail.com wrote: > On Sep 9, 4:52 am, David Masover wrote: > > > > So what do you do with things which aren't thread-save? Or "j-safe"? > > > > The same thing you do with a Makefile that isn't j-safe: (1) write the > dependencies correctly, which makes it j-safe, or (2) don't run it > with -j. Still going to be a fair number of cases of (3), I imagine: use locks to synchronize non-thread-safe libraries, for which there's still a benefit to running those tasks in parallel. > If 'task' became 'multitask', Rake would run all your tasks at once -- > all at the same time. That's probably not what you want :) Actually, no, I assumed that 'multitask' only ran that specific task in parallel. Actually, I hadn't thought about it thoroughly enough to realize that this wasn't what was happening: > The dependency graph tells us > what can be run in parallel and what can't. I understand make -j, and I think I understand the difference with multitask -- if I understand it: multitask :foo ... multitask :bar ... In the above example, will everything really run concurrently? I'd assumed that foo would run concurrently, and then bar would run concurrently. In either case, I see what Drake is doing (real make -j behavior). Thanks for explaining this -- it looks cool! One more thing: I'm not sure what the best way to do this is, but I think it would still be useful to have the task/multitask dichotomy, for legacy programs. Multitasks would operate as properly parallized Drake tasks. Plain old tasks would run in complete isolation, with the exception that if they invoke a multitask, that multitask (and all its remaining dependencies) run in j-parallized mode. That would certainly break the purity of it, and it would be a bit more work, but I think it could be made to work. The benefit is, you could translate an existing project iteratively, without having to verify that the whole thing is correct, first.