From: Ron Jeffries <ronjeffries@...>
Date: 2002-03-07T21:18:40+09:00
Subject: file reading impossibly slow?

So I'm doing this benchmark to work with my set program. Part of the problem is
to read a six million record file. So I built one, six million 64-byte lines
plus newline, didn't seem to take that long. Then I decided to time how long it
would take to read, with this program:

t1 = Time.now
count = 1
File.open("data.txt") { | file |
	while (!file.eof?)
		line = file.gets
	end
}
puts Time.now - t1

I let it run for 20 minutes with no output. (A tiny file works fine.) I tried
the same program using read(65000) instead of gets, just to see if the line size
was the problem. I stopped it after 20 minutes also.

So I tried it in Dolphin Smalltalk, with this program:

	start := Time totalSeconds.
	stream := FileStream read:  'c:/data/ruby/sets/data.txt'.
	[stream atEnd] whileFalse: [
		line = stream nextLine
	].
	end := Time totalSeconds.
	Transcript show: (end - start) printString; cr

It runs in 178 seconds. Three minutes. What's up with that? Am I missing
something? I'm going to go let Ruby finish this job no matter how long it takes.
I'll report the total time when I get it. 

Is it me? Or is it Ruby? Thanks!

Ronald E Jeffries
http://www.XProgramming.com
http://www.objectmentor.com
I'm giving the best advice I have. You get to decide whether it's true for you.