From: Robert Klemme <shortcutter@...>
Date: 2010-02-22T06:40:55+09:00
Subject: Re: Read efficiency?

On 02/21/2010 10:26 PM, Notmy Realname wrote:
> Robert Klemme wrote:
>> On 02/21/2010 09:08 AM, Notmy Realname wrote:
>>> I hate it when I put my foot in my mouth. After further testing, I'm 
>>> almost entirely sure this is just due to overhead, not a problem with 
>>> disk access.
>> Good that you did not use your real name. ;-)
>>
>>> Who would have thought looping 11*2**20 times would incur a performance 
>>> hit?
>> The overhead of a simple method call is significant already:
>>
>> robert@fussel:~$ ruby19 -r benchmark <<XXX
>> def f;end # function without effect
>> R1=11*1024*1024
>> R2=R1/4096
>> Benchmark.bmbm 20 do |r|
>> r.report("rare") { R2.times { f } }
>> r.report("often") { R1.times { f } }
>> end
>> XXX
>>
>> Rehearsal -------------------------------------------------------
>> rare                  0.000000   0.000000   0.000000 (  0.000643)
>> often                 2.620000   0.000000   2.620000 (  2.663366)
>> ---------------------------------------------- total: 2.620000sec
>>
>>                            user     system      total        real
>> rare                  0.000000   0.000000   0.000000 (  0.000643)
>> often                 2.620000   0.000000   2.620000 (  2.665057)
>> robert@fussel:~$
>>
>> Kind regards
>>
>>   robert
> 
> I know, right? I'm embarrassed I use this username elsewhere. And I 
> spent like an hour trying to figure this out.
> 
> In my defense, I did do more or less exactly what you did and found the 
> function call overhead to be significant, but not enough to account for 
> the difference. Then I remembered that Ruby isn't just doing a disk 
> read---it's also doing stuff like packaging the bytes it reads into a 
> string. Once I accounted for *that*, the difference magically 
> disappeared. Note to self: Ruby is not C.

Just an additional hint: you can dramatically reduce the String creation 
overhead by using the second argument to IO#write:

robert@fussel:~$ dd bs=1024 count=10240 if=/dev/zero of=x
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 0.135645 s, 77.3 MB/s
robert@fussel:~$ ruby19 bm
Rehearsal -------------------------------------------------------
simple                0.210000   0.240000   0.450000 (  0.450689)
smart                 0.100000   0.160000   0.260000 (  0.262826)
---------------------------------------------- total: 0.710000sec

                           user     system      total        real
simple                0.210000   0.240000   0.450000 (  0.463716)
smart                 0.130000   0.130000   0.260000 (  0.260381)
robert@fussel:~$ cat bm
require 'benchmark'
REP = 10
BS = 1024
Benchmark.bmbm 20 do |r|
   r.report 'simple' do
     REP.times do
       File.open "x", "rb" do |io|
	while b = io.read(BS)
	end
       end
     end
   end
   r.report 'smart' do
     REP.times do
       File.open "x", "rb" do |io|
	b = ""
	while io.read(BS, b)
	end
       end
     end
   end
end
robert@fussel:~$

Note: Ruby is not C but sometimes there is room for improvement. :-)

Kind regards

	robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/