From: Robert Klemme <shortcutter@...>
Date: 2011-10-25T06:44:42+09:00
Subject: Re: Creating threads can be slow?

On Mon, Oct 24, 2011 at 11:26 PM, wam r. <wamrewam@googlemail.com> wrote:
> Hi guys,
>
> It is difficult to describe the load: it is a Rails app with many moving
> parts. By "loading" I mean I am sending 10 to 20 requests at once (and
> repeat) just to see how it reacts.
>
> I have been able to "track" the key time consuming parts in my code.
> There are three of them:
> - one linked to DB requests
> - one linked to heavy computations (some optimization algos)
> - the third one: the thread creation above.
>
> The more I load, the more time each of these take - it is very
> correlated.
> I know how to optimize the first two, but not the last one. It doesn't
> seem there is anything I can do (besides switching to a different Ruby
> which I can't unfortunately).
>
> But Robert it seems you are right: it is just CPU intensive. I had a
> chance to add another box to the cluster this week-end and it makes a
> huge difference. I just did not realize that creating threads would be
> that CPU intensive.
>
> Anyway thanks a lot for your help on this.

You're welcome!  One other measure which may help you is to limit
concurrency by using a thread pool fed from a queue.  That way you
avoid thread creation overhead for every work item plus you limit
concurrency - usually it does not make much sense to go beyond a
certain point, with real native threads that's related to the number
of cores.

And you should definitively switch to 1.9.  If you wait a few days /
weeks you can directly jump on 1.9.3 which already has a release
candidate out.

Kind regards

robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/