[ruby-core:21334] Re: [PATCH] drastically improve rb_waitpid for short-lived children

On Jan 13, 2009, at 8:19 PM, Brent Roman wrote:

>
> Evan,
>
> Have you thought about the impact on this patch on multi- 
> (green)threaded
> Ruby scripts that "run small, fast programs as children, in true unix
> fashion"?
>
> Won't all the other threads pause during your new "QUICKSLEEP"  
> polling loop?

Yes, they will. But thats why it's only a 5ms window. Ruby's threading  
is decent, but my hunch is that an extra 5ms window here won't even be  
noticed.

Again, it was a bit of a quick hack. It proved quite useful in my  
friends use case, so I thought I'd post it.

>
>
> How about this non-polling alternative, that should be portable  
> among POSIX
> systems:
>
> Install a signal handler on SIGCHLD that wakes the appropriate waiting
> thread.  That thread simply does the waitpid(..., WNOHANG) as before
> (to free process table resouces).

I've implemented this before, and it does work ok. It's a bit more  
work, thats why my patch didn't implement it. We'd need to check what  
the code path through rb_thread_schedule() is, mainly if it searches  
for a runnable thread if select() is interrupted due to a signal.

>
>
> Ultimately, it should be possible to remove these two nasty
> rb_thread_polling() calls
> in process.c via this technique.
>
> In the short term, one could probably prototype the idea with a Ruby  
> trap
> handler and
> a wrapper around Kernel.spawn()
>
> What do you think?
>
> - brent
>
>
> Evan Phoenix wrote:
>>
>> This evening I wrote this patch while helping a friend diagnose why,
>> when Threads are off, `system("date")` takes 2.2ms, and when Threads
>> are on, 60ms.
>>
>> The radical difference is a symptom of MRI's green threads and how  
>> you
>> wait for a child in unix. The current implementation of system
>> immediately calls rb_syswait(), which calls rb_waitpid() right after
>> forking the child. rb_waitpid() calls waitpid(..., WNOHANG), which
>> almost always returns nothing, since the child probably hasn't even
>> actually started yet.
>>
>> My patch improves upon this by adding a 5ms 'quicksleep' zone. At
>> intervals of 0.5ms for the first 5ms after rb_waitpid is called,
>> waitpid() is called in a loop. This allows short-lived, fast exiting
>> children to be caught and processed almost right away, without
>> incurring the 60ms minimum time we currently have.
>>
>> The patch also adds a poll time backoff for use after the first 5ms.
>> Rather than jumping right to backing off for 60ms, it backs off
>> increments up to 60ms. This allows for children children that take
>> just a little bit longer to be processed a bit quicker too.
>>
>> For long running children, the behavior is the same.
>>
>> This improves behavior a lot for users who run a lot of small, fast
>> programs as children, in true unix style.
>>
>>  - Evan Phoenix // [e-mail]
>>
>>
>>
>>
>>
>>
>
> -- 
> View this message in context: http://www.nabble.com/-ruby-core%3A21302---PATCH--drastically-improve-rb_waitpid-for-short-lived-children-tp21431928p21449840.html
> Sent from the ruby-core mailing list archive at Nabble.com.
>
>