From: Hugh Sasse <hgs@...>
Date: 2005-11-29T22:20:37+09:00
Subject: Re: building ruby for speed: wise or otherwise?

On Tue, 29 Nov 2005, Robert Klemme wrote:

> Hugh Sasse wrote:
> > On Tue, 29 Nov 2005, Robert Klemme wrote:
> >
> >> Hugh Sasse wrote:
> >>> At the moment my script to populate the tables is taking about an
> >>> hour.  Anyway it's mostly ruby I think, because it spends most of
> >>> the time setting up the arrays before it populates the db with them.
> >>
> >> How did you measure that?
> >
> > By eye! :-)  The code doesn't access the database at all until the
> > last part, and it doesn't get there till about 45 mins.  But to be
> > honest, this is so slow it isn't worth benchmarking to get the
> > milliseconds.
> 
> Wow!  In that case it certainly seems to make sense to optimize that.  Did
> you keep an eye on memory consumption and disk IO?  Could well be that the

Not really.  The machine is 5.5 years old.  Rough Moore's law calc
shows it would take about 5 mins on a modern machine.

> sheer amount of data (and thus memory) slows your script down.
> 
> >      555    1676   17179 /home/hgs/csestore_meta/populate_tables2.rb
> > I could post the script if you like. I've not profiled it to find
> > out where the slow bits are because it would take about 5 hours
> > going by previous slowdowns when profiling.
> 
> Unfortunately we're close to release and I don't really have much time to
> look into this deeper.  If anyone else volunteers...

I'm profiling it now, while I wait for GCC-4.0.2 to cook.  Already
been snookered by needing gmp and mpfr for the fortran, then I found
I need autogen to build without the fortran, which needs guile, so
I've got those last two.  [cf "The Gas Man Cometh"]
> 
> > MySQL.  Part of the problem is that this script is also for
> > updating, based on new data.  If the db is empty it just inserts,
> > else it updates.  Easy enought in ActiveRecord.
> 
> Ok, bad for bulk loading.

I think so.
> 
> Kind regards
> 
>     robert

        Thank you,
        Hugh