From: Phillip Gawlowski Date: 2011-05-27T10:05:51+09:00 Subject: Re: ruby fork (thread like) on windows On Fri, May 27, 2011 at 1:44 AM, Roger Pack wrote: > > yeah best bet is to spawn off multiple processes. Though, I wonder if this is an improvement over the serial uploading. Unless the upload rate is limited to a fraction of the total upload speed for each process wanting to upload data (either on the uploading or the receiving end), two files uploaded in parallel will use half* the bandwidth, and thus upload just as slow or fast as the two files in series. Additionally, the program complexity rises: instead of keeping tabs on successful uploads of one file, it's now n uploads that have to be monitored and redone on failure. But since engineering challenges are fun, I'd do it like this (if I had to): - Create an upload queue server containing all unfinished uploads. - Spawn several worker processes that check the queue for the next available upload (beware: race condition. A simple block-and-backoff strategy would be sufficient to prevent that, I think), and mark their assigned uploads. - Once the upload is finished, the worker processes send the all clear, and the upload gets removed from the queue. - If a worker process cannot finish its upload for whatever reason, the upload gets marked as unfinished again. - If a worker finishes its upload, it marks the queue item for deletion. Rails can use delayed_job that does something similar, so go ahead and steal what you can. * Actually a little less due to a touch more overhead, but I doubt that that is a significant factor for 99% of cases. -- Phillip Gawlowski A method of solution is perfect if we can forsee from the start, and even prove, that following that method we shall attain our aim.                -- Leibnitz