From: Peter Hickman <peter@...>
Date: 2003-04-03T18:47:49+09:00
Subject: Re: Newbie question:read file speed

I would like to point out that removing the ...

  lc += 1
  if lc % 1000 == 0
   puts "Read #{lc} lines"
  end

in the ruby code made a significant difference.

1. Elapsed = 0.1539530754
2. Elapsed = 0.07133197784

You are not only testing file reading but a whole lot more, such as the 
maths and screen io. As profiling shows.

With line counting and reporting

Elapsed = 11.28818595
   %   cumulative   self              self     total
  time   seconds   seconds    calls  ms/call  ms/call  name
  67.34     7.05      7.05        1  7050.00 10470.00  IO#each_line
  13.47     8.46      1.41    45425     0.03     0.03  Fixnum#==
   9.65     9.47      1.01    45425     0.02     0.02  Fixnum#%
   9.36    10.45      0.98    45425     0.02     0.02  Fixnum#+
   0.19    10.47      0.02       46     0.43     0.43  Kernel.puts
   0.00    10.47      0.00        1     0.00     0.00  Float#-
   0.00    10.47      0.00       92     0.00     0.00  IO#write
   0.00    10.47      0.00        1     0.00 10470.00  #toplevel
   0.00    10.47      0.00       45     0.00     0.00  Fixnum#to_s
   0.00    10.47      0.00        1     0.00     0.00  Array#[]
   0.00    10.47      0.00        1     0.00     0.00  File#open
   0.00    10.47      0.00        2     0.00     0.00  Time#to_f
   0.00    10.47      0.00        1     0.00     0.00  String#+
   0.00    10.47      0.00        2     0.00     0.00  Time#now
   0.00    10.47      0.00        1     0.00     0.00  Float#to_s

Without line counting and reporting

Elapsed = 0.0706551075
   %   cumulative   self              self     total
  time   seconds   seconds    calls  ms/call  ms/call  name
100.00     0.07      0.07        1    70.00    70.00  IO#each_line
   0.00     0.07      0.00        1     0.00     0.00  Kernel.puts
   0.00     0.07      0.00        1     0.00    70.00  #toplevel
   0.00     0.07      0.00        1     0.00     0.00  Array#[]
   0.00     0.07      0.00        2     0.00     0.00  IO#write
   0.00     0.07      0.00        2     0.00     0.00  Time#now
   0.00     0.07      0.00        1     0.00     0.00  Float#to_s
   0.00     0.07      0.00        1     0.00     0.00  Float#-
   0.00     0.07      0.00        1     0.00     0.00  File#open
   0.00     0.07      0.00        2     0.00     0.00  Time#to_f
   0.00     0.07      0.00        1     0.00     0.00  String#+

IO from printing the counter on screen seems to be quite a significant 
drain on the process and just doing a Fixnum#== seems to be quite a load.