From: Robert Klemme <shortcutter@...>
Date: 2008-06-25T14:27:58+09:00
Subject: Re: Multiple HTTP Sockets or Threads for file uploads.

On 24.06.2008 22:56, gberz3 wrote:
> On Jun 24, 7:03 am, Robert Klemme <shortcut...@googlemail.com> wrote:
>> This is not necessarily true for IO bound tasks. Also, if curl is used
>> you just need multiple child processes but not multiple threads.
> 
> Alright, I'm currently using Threads and it has sped things up
> tremendously.  Thank you both for your suggestions.

But note that the speedup likely comes from using multiple curl 
processes - not from using multiple Ruby threads.

> One problem I'm having now is that sometimes a file will hang cURL,
> and while the other subprocesses finish there might be one or two that
> hang up.  In this case I'd like to have some kind of granular control
> over killing individual items without necessarily killing the whole
> thread.  I'd be afraid to kill the entire thread as it may not be
> finished processing legitimate cURL processes when you send the kill
> switch.
> 
> Thoughts?

Killing threads does nothing to your external processes.  You can either 
kill curl and force its termination or you need to investigate whether 
curl is built in a way that it responds to certain signals by dropping 
the current upload.

Kind regards

	robert