From: Philip Rhoades <phil@...>
Date: 2008-06-27T04:45:03+09:00
Subject: Re: Performance improvement possible?

Eleanor,


Eleanor McHugh wrote:
> On 26 Jun 2008, at 04:24, Rob Biedenharn wrote:
>> On Jun 25, 2008, at 8:44 PM, Philip Rhoades wrote:
>>> Chuck,
>>> Chuck Remes wrote:
>>>> On Jun 25, 2008, at 5:13 PM, Philip Rhoades wrote:
>>>>>
>>>>> The change to a single Ruby script simplified the previous setup 
>>>>> quite markedly but I'm not sure that much more can be done with the 
>>>>> relatively small script now.  I logged into "ruby Talk" on 
>>>>> http://codeforpeople.com/ but I couldn't see anywhere to post the 
>>>>> code.  I can mail the files to anyone who is interested in having a 
>>>>> look.
>>>> Phil,
>>>> use one of the pastie sites to paste up your code for us to see.
>>>> http://pastie.org
>>>> http://rafb.net
>>>
>>> Done - http://pastie.org/222306
>>>
>>> I am also rerunning with v1.9 and now that the code is all in one 
>>> program it seems to be faster (but still about 3x slower than the 
>>> equivalent shell scripts and C/C++ program).
>>>
>> OK, even knowing that it started as shell+C, it needs help ;-)
>>
>>  http://pastie.org/222375
>>
>> Except for noticing that the Dir.chdir is still commented out, I think 
>> that this is equivalent to your original.  Of course, all that really 
>> means it that I think it produces the same output.  I don't have your 
>> input files to check (and I'm not really sure I have a clue what it is 
>> really doing either).
>>
>> If you can directly try this new set, I'd be curious as to the 
>> relative speed.  A few important notes:
>> * the $darr was never referenced so why bother to initialize it
>> * the multiple loops in stats06 have been collapsed
>> * the various seed* loops now use strings directly
>> * the parameters to statz32000 are strings rather that do somewhat 
>> pointless conversions (and sample_sz of 999 would 'overflow' the %02d 
>> into "999" anyway).  Some potential values won't work as literals (try 
>> 08 as an example, or 012) since the leading 0 on a literal means octal.
>> * the more common boolean values are true (rather than TRUE) and false 
>> and parentheses are not generally needed around the conditional 
>> expression of an 'if'
> 
> 
> 
> A further modified version of statz which should changes the way IO is 
> handled:
> 
>     http://pastie.org/222765
> 
> Let me know if this makes any difference.


A few things:

- you left a line in the loop:

	File.open( output_filename, 'w' ) do |fout|

which should be deleted

- I originally used:

	stats = [] 
                  	lines = File.readlines(input_filename, 'r')

but found that reading the whole file (8871 lines) and then processing 
the array was inefficient so I got rid of the array

- using:

	stats << stats06

and the file writing output of:

         File.open(output_filename, "a") do |file| 

		file.sync = false
		file.puts *stats
		file.fsync
	end

looks interesting - why should that be faster?

Thanks again,

Regards,

Phil.
-- 
Philip Rhoades

Pricom Pty Limited  (ACN 003 252 275  ABN 91 003 252 275)
GPO Box 3411
Sydney NSW	2001
Australia
E-mail:  phil@pricom.com.au