From: Notmy Realname <guyminuslife@...>
Date: 2010-02-21T16:16:32+09:00
Subject: Read efficiency?

I'm wondering if anyone knows much about Ruby's efficiency with IO#read.
Specifically, I'm wondering about libraries I might use to speed up disk
reads.

To see what I mean, here's some test code that iterates over an
11-megabyte file. All it does is call IO#read on a number of bytes (set
on the command-line) over the entire file, and times it.

#!/usr/bin/env ruby
# readspeed.rb

buf_size = ARGV[0].to_i
fd = File.open("some.txt")

start = Time.now
while (fd.read(buf_size))
end
stop = Time.now

puts (stop - start).to_s + " seconds"

#--- EOF

Running this on my system yields:

$ ruby readspeed.rb 4096
0.014 seconds

$ ruby readspeed.rb 1
7.547 seconds


Obviously a big difference! This is a simplified version of the test I
was actually running, which tried to account for the increased amount of
overhead when calling with 1 byte at a time. There's still an
order-of-magnitude difference between the two...reading one byte at a
time is *slow*, slow enough to bog down an entire program.

I know this is supposed to be the case with unbuffered input, such as
the C standard library "read", but isn't IO#read supposed to be
buffered? What's causing this slowdown? I'm writing a class that will
hopefully speed up smaller reads from binary files by explicitly caching
data in memory, but I'm wondering if there are any pre-built (i.e.,
tested) solutions that Ruby programmers might be using.
-- 
Posted via http://www.ruby-forum.com/.