From: John Carter <john.carter@...>
Date: 2008-02-08T05:52:23+09:00
Subject: Re: is lots of files with Threads faster?

On Fri, 8 Feb 2008, Chris Richards wrote:

> Im required to open 50+ files and parse the data in them.  WOuld using
> multiple threads give me the best performance?  or is it best just to do
> it sequentially?

Prefer processes to threads on unix.

Depends on whether you have multiple cores.

Depends on what the file devices are. I have one small app where the
fd's are sockets to machines that may or may not have a certain other
application up. (The app finds out)

I spin one thread per machine, and open all connections in
parallel. The time to completion is the time for a single connect
fail, which is about N times faster than testing each connection in
series.

Depends also of data locality. Cache is many times faster than
ram. If you can live in cache, you go much faster. If multiple threads
mean you spend less time in cache, you go much slower.


John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand