From: Peter Hickman <peterhickman386@...>
Date: 2012-09-19T18:41:43+09:00
Subject: Re: Ruby - is it worth the effort?

On 19 September 2012 10:09, Carlos Agarie <carlos.agarie@gmail.com> wrote:
> I'd like to know, too. I stumbled upon a similar problem, but it was long
> ago.

Ok here is a quick test that I hacked up. The data is a 2,659,800 line
639Mb text file. Some lines contain the string "FRED", count them

Here are the scripts

-- Perl
#!/usr/bin/env perl

use strict;
use warnings;

my $logfile = 'results201101.dat';
my $counter = 0;
open FILE, "<$logfile" or die $!;
while(my $line = <FILE>) {
  if($line =~ /FRED/) {
    $counter++;
  }
}
close(FILE);
print "$counter\n";

-- Ruby 1
counter = 0
File.open("results201101.dat").each do |line|
  if line =~ /FRED/
    counter += 1
  end
end

puts counter

-- Ruby 2
r = Regexp.new('FRED')

counter = 0
File.open("results201101.dat").each do |line|
  if r.match(line)
    counter += 1
  end
end

puts counter

-- Ruby 3
counter = 0
File.open("results201101.dat").each do |line|
  if line.index("FRED")
    counter += 1
  end
end

puts counter

Perl 5.12.3 vs Ruby 1.8.7

Perl real	0m3.759s user	0m3.182s sys	0m0.535s
Ruby 1 real	0m6.216s user	0m4.939s sys	0m0.711s
Ruby 2 real	0m5.888s user	0m5.121s sys	0m0.721s
Ruby 3 real	0m6.187s user	0m5.512s sys	0m0.609s

So Perl is nearly twice as fast as Ruby.

Perl 5.12.3 vs Ruby 1.9.2

Perl real	0m3.778s user	0m3.172s sys	0m0.539s
Ruby 1 real	0m7.745s user	0m6.736s sys	0m0.661s
Ruby 2 real	0m7.775s user	0m7.102s sys	0m0.642s
Ruby 3 real	0m6.444s user	0m5.793s sys	0m0.513s

Ruby actually gets worse!

The issue here is that the test file is not large (think gigabytes for
the real thing) or long (think 10s of millions of lines). Those
differences start to add up.

To be honest I suspect that it is more an issue with the regexes than
file io and the real regexes are much more complicated than just match
a string. I was a bit surprised that the index() wasn't faster.

We have to parse the files in a very short timeframe, if Perl hadn't
been fast enough I had a horrible feeling that we would be looking at
C or something. Ooo that would be fun :)