From: Brian Candler <B.Candler@...>
Date: 2003-04-03T07:35:25+09:00
Subject: Re: Newbie question:read file speed


--UlVJffcvxoiEqYs2
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Thu, Apr 03, 2003 at 12:51:28AM +0900, Greg Brondo wrote:
> Here they are:

Cheers. I've run them here under FreeBSD-4.7 running on a Sony Vaio laptop,
Pentium 266MMX. For the data file, I'm using:

$ wc /usr/share/dict/words 
  235881  235881 2493066 /usr/share/dict/words

(with an average line length of only 10.6 bytes it's perhaps not the most
realistic example, but it will do)

Running the code you gave, I get the following times averaged over three
runs:

Ruby   (1.6.8)         6.81 secs
Ruby   (1.8.0p2)       6.57 secs
Python (2.2.1)         6.72 secs *
Java (jdk-1.3.1p6_4)  16.76 secs **

* it printed "Elapsed = 6", so the time given is using the 'time' command;
this is slightly unfair to python as it includes the interpreter start-up
time which the Ruby time doesn't.

** printed "Elapsed = 16", ditto

Note however that I am running under X, and it seems quite a lot of time is
spent just writing the progress messages to the xterm. If I add '>/dev/null'
to the end of the command line and use unix 'time' then I get:

Ruby   (1.6.8)         3.8 s
Ruby   (1.8.0p2)       3.4 s
Python (2.2.1)         3.7 s
Java (jdk-1.3.1p6_4)   8.6 s

So on this basis Ruby stacks up pretty well against the languages you
mention, running under Unix. From your figures it seems that Ruby is slower
under Windows, and I would guess that this is somehow related to the way
Ruby performs I/O, or perhaps to the threading support (e.g. making extra
calls to select() before I/O operations). Maybe a Windows developer could
compare the Python and Ruby I/O source code and see if there are significant
differences in the way they talk to Windows.

Out of interest, I also ported your code to Perl (attached), and also
modified the Ruby code to use a more iterative style to be closer to Perl. I
get:

                   (to screen)  (to /dev/null)
Perl (5.005_03)      3.9 secs     2.3 secs
Ruby (1.6.8)         6.5 secs     3.6 secs

So it seems that Perl is significantly faster in this simple test, but also
the block/yield approach does not by itself add a great deal of overhead
(fortunately!)

Regards,

Brian.

--UlVJffcvxoiEqYs2
Content-Type: application/x-perl
Content-Disposition: attachment; filename="x.pl"
Content-Transfer-Encoding: quoted-printable

my $filename=3D$ARGV[0];=0Amy $lc =3D 0;=0Amy $starttime =3D time;=0Aopen F=
ILE,"<$filename";=0Awhile (<FILE>) {=0A $lc++;=0A if ($lc % 1000 =3D=3D 0) =
{=0A  print "Read $lc lines\n"=0A }=0A}=0Amy $stoptime =3D time;=0A  =0Apri=
nt "Elapsed =3D ", ($stoptime - $starttime), "\n"=0A
--UlVJffcvxoiEqYs2
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="x2.rb"

filename=ARGV[0]
lc = 0
starttime = Time.now.to_f
f = File.open(filename)
while f.gets
 lc += 1
 if lc % 1000 == 0
  puts "Read #{lc} lines"
 end
end
stoptime = Time.now.to_f
  
puts "Elapsed = " + (stoptime - starttime).to_s

--UlVJffcvxoiEqYs2--