From: "Aleksi Niemelä" <aleksi.niemela@...>
Date: 2000-12-16T07:19:24+09:00
Subject: [ruby-talk:7356] RE:  Ruby performance question

Eric Crampton wrote:
> I'm parsing simple text lines which look like this:
> 
>   key_1=val_1,key_2=val_2,...key_n=val_n
> Running on a small dataset of 13 MB of data (around 81000 lines), the
> program takes about 27 seconds on my Linux machine. 

I was doing just about the same (I had '&' as a pair separator), and had to
do it a lot. The performance was miserable compared to Perl. There was some
discussion (I could try to find a thread later, if you're interested in),
and it was concluded the performance bottleneck came from GC, which took
more and more time when there was more objects in the object space to check.

A patch was proposed by Matthieu Bouchard (IIRC), and it flattened the worst
case scenario from N^2 to N. I can't remember if it was good enough to make
Ruby comparable.

Nevertheless, Ruby GC is going through a revolution in the following
development series, as it will be replaced with a generational one. We could
expect some performance gains from that.

For a short term help, consider keeping the original strings and parse when
needed or use some other space-time optimizations (which is actually
misleading name, as I'd expect the real data structure to take more space
than the plain string version :). The trick is to keep as few objects in the
memory simultaneously as possible.

Bear with us.

	- Aleksi