From: Rob Biedenharn Date: 2008-06-27T01:52:37+09:00 Subject: Re: Performance improvement possible? On Jun 26, 2008, at 12:08 PM, Philip Rhoades wrote: > Rob, > Rob Biedenharn wrote: >> On Jun 25, 2008, at 8:44 PM, Philip Rhoades wrote: >>> Chuck, >>> Chuck Remes wrote: >>>> On Jun 25, 2008, at 5:13 PM, Philip Rhoades wrote: >>>>> The change to a single Ruby script simplified the previous >>>>> setup quite markedly but I'm not sure that much more can be >>>>> done with the relatively small script now. I logged into "ruby >>>>> Talk" on http://codeforpeople.com/ but I couldn't see anywhere >>>>> to post the code. I can mail the files to anyone who is >>>>> interested in having a look. >>>> Phil, use one of the pastie sites to paste up your code for us to >>>> see. http://pastie.org http://rafb.net >>> Done - http://pastie.org/222306 >>> I am also rerunning with v1.9 and now that the code is all in one >>> program it seems to be faster (but still about 3x slower than the >>> equivalent shell scripts and C/C++ program). >>> Thanks, >>> Phil. -- Philip Rhoades >>> Pricom Pty Limited (ACN 003 252 275 ABN 91 003 252 275) GPO Box >>> 3411 Sydney NSW 2001 Australia E-mail: phil@pricom.com.au >> OK, even knowing that it started as shell+C, it needs help ;-) > > That's why I'm here! > >> http://pastie.org/222375 >> Except for noticing that the Dir.chdir is still commented out, I >> think that this is equivalent to your original. Of course, all that >> really means it that I think it produces the same output. I don't >> have your input files to check (and I'm not really sure I have a clue >> what it is really doing either). > > > The output seems good. > > >> If you can directly try this new set, I'd be curious as to the >> relative speed. A few important notes: * the $darr was never >> referenced so why bother to initialize it > > > That is for future reference (other stats). > > >> * the multiple loops in stats06 have been collapsed > > > Thanks - nice! > > >> * the various seed* loops now use strings directly > > > Much nicer! > > >> * the parameters to statz32000 are strings rather that do somewhat >> pointless conversions (and sample_sz of 999 would 'overflow' the %02d >> into "999" anyway). > > > In the originals I needed the leading zeros as strings for input and > output file name consistency. > > >> Some potential values won't work as literals (try 08 as an example, >> or 012) since the leading 0 on a literal means octal. > > > That could be a problem . . > > >> * the more >> common boolean values are true (rather than TRUE) and false > > > OK. > > >> and >> parentheses are not generally needed around the conditional >> expression of an 'if' > > > I guess I use them so that they are there for future, more complex > expressions. > > The performance improvement has been fairly dramatic! - the previous > version using Ruby v1.9 took 3.44 times as long as the original sh/C > stuff - this version is only takes 1.85 times as long! - an > improvement of 46%! However I am not sure what the reason is for > this improvement - though the changes certainly improve the > readability and "Rubyness" of the code - most of the changes seem > cosmetic to me . . > > Here is the profile (using v1.8) for pre and post your changes: > > Ruby v1.8 > v1.5.1 > % cumulative self self total > time seconds seconds calls ms/call ms/call name > 59.94 8.44 8.44 1 8440 14080 Object#statz > 10.44 9.91 1.47 58318 0.03 0.03 Array#[] > 8.1 11.05 1.14 26630 0.04 0.04 String#split > 4.05 11.62 0.57 70 8.14 616.14 Range#each > 4.05 12.19 0.57 17737 0.03 0.03 String#== > 3.05 12.62 0.43 8872 0.05 0.05 IO#gets > 2.98 13.04 0.42 8866 0.05 0.05 Fixnum#- > 2.34 13.37 0.33 9737 0.03 0.03 Fixnum#+ > 2.06 13.66 0.29 8916 0.03 0.03 String#to_i > 1.92 13.93 0.27 9955 0.03 0.03 Array#[]= > 0.36 13.98 0.05 880 0.06 0.06 Fixnum#> > 0.21 14.01 0.03 22 1.36 42.27 Object#stats06 > 0.21 14.04 0.03 274 0.11 0.26 Array#initialize > 0.14 14.06 0.02 440 0.05 0.05 Fixnum#== > 0.07 14.07 0.01 274 0.04 0.29 Class#new > 0.07 14.08 0.01 22 0.45 0.45 IO#puts > > Ruby v1.8 > v1.5.2 > % cumulative self self total > time seconds seconds calls ms/call ms/call > 60.36 7.02 7.02 2 3510 11625 IO#open > 9.46 8.12 1.1 29128 0.04 0.04 Array#[] > 5.25 8.73 0.61 17737 0.03 0.03 String#== > 3.96 9.19 0.46 8871 0.05 0.05 String#split > 3.87 9.64 0.45 8872 0.05 0.05 IO#gets > 3.87 10.09 0.45 22 20.45 27.73 Integer#times > 3.35 10.48 0.39 8800 0.04 0.04 Fixnum#- > 3.18 10.85 0.37 9737 0.04 0.04 Fixnum#+ > 3.1 11.21 0.36 9071 0.04 0.04 Array#[]= > 2.75 11.53 0.32 8915 0.04 0.04 String#to_i > 0.34 11.57 0.04 880 0.05 0.05 Fixnum#> > 0.17 11.59 0.02 115 0.17 0.35 Array#initialize > 0.09 11.6 0.01 22 0.45 0.45 Kernel.=== > 0.09 11.61 0.01 44 0.23 0.23 Array#initialize_copy > 0.09 11.62 0.01 440 0.02 0.02 Fixnum#== > 0.09 11.63 0.01 44 0.23 0.45 Kernel.dup > > Any comments on reasons for the improvement? > > Many thanks, > > Regards, > > Phil. > -- > Philip Rhoades > E-mail: phil@pricom.com.au Well, let's rearrange that so the differences are more apparent: Ruby v1.8 v1.5.2 v1.5.1 self self calls name calls 2 IO#open 29128 Array#[] 58318 17737 String#== 17737 8871 String#split 26630 8872 IO#gets 8872 22 Integer#times 8800 Fixnum#- 8866 9737 Fixnum#+ 9737 9071 Array#[]= 9955 8915 String#to_i 8916 880 Fixnum#> 880 115 Array#initialize 274 22 Kernel.=== 44 Array#initialize_copy 440 Fixnum#== 440 44 Kernel.dup Object#statz 1 Range#each 70 Object#stats06 22 Class#new 274 IO#puts 22 Now it's easier to see that the big differences are the calls to String#split (3x) and Array#[] (2x) in the older version. Array#[]= has about 10% more, but we'll ignore that for now. I'd guess that the Kernel.dup and Array#initialize_copy are much more efficient at creating the alleles arrays than the former element-at-a-time method. -Rob Rob Biedenharn http://agileconsultingllc.com Rob@AgileConsultingLLC.com