From: Robert Klemme <shortcutter@...>
Date: 2007-12-07T18:27:02+09:00
Subject: Re: Reasons to use a buffer in IO::read?

2007/12/7, Steve Midgley <midgley.steve@gmail.com>:
> On Dec 5, 5:56 pm, MonkeeSage <MonkeeS...@gmail.com> wrote:
> > On Dec 5, 6:59 pm, SteveMidgley<midgley.st...@gmail.com> wrote:
> >
> >
> >
> > > Hi Ruby people,
> >
> > > I'm wondering what the functional and performance differences might be
> > > between the two statements below? Assume 'io' is an IO instance with
> > > gobs of data in it. Assume 'file' is an open file instance with write
> > > access:
> >
> > > until io.eof? do
> > >   file.write(io.read(10485760))
> > > end
> >
> > > buffer = ''

This line above is completely superfluous.

> > > until io.eof? do
> > >   buffer = io.read(10485760)
> > >   file.write(buffer)
> > > end
> >
> > > I see that Ruby provides for a buffer and I'm wondering what the
> > > reason is? I read this article but am still not clear on the benefit
> > > of a buffer at all:
> >
> > >http://rcoder.net/content/fast-ruby-io
> >
> > > I'm wondering if providing a buffer might reduce malloc issues and
> > > speed things up? I can't see any other reason to use one..
> >
> > > Thanks in advance for any information!
> >
> > > Steve
> >
> > $ ri IO#buffer
> > ----------------------------------------------------------------
> > IO#read
> >      ios.read([length [, buffer]])    => string, buffer, or nil
> > ------------------------------------------------------------------------
> >      Reads at most _length_ bytes from the I/O stream, or to the end
> > of
> >      file if _length_ is omitted or is +nil+. _length_ must be a
> >      non-negative integer or nil. If the optional _buffer_ argument is
> >      present, it must reference a String, which will receive the data.
> >
> >      At end of file, it returns +nil+ or +""+ depend on _length_.
> >      +_ios_.read()+ and +_ios_.read(nil)+ returns +""+.
> >      +_ios_.read(_positive-integer_)+ returns nil.
> >
> >         f = File.new("testfile")
> >         f.read(16)   #=> "This is line one"
> >
> > So...
> >
> > buffer = ""
> > file.write(io.read(nil, buffer))
> > print "I read this stuff ", buffer, "\n"
> >
> > Regards,
> > Jordan
>
> Thanks Jordan. How is your code different (if at all) from:
>
> buffer = io.read
> file.write(buffer)
> print "I read this stuff ", buffer, "\n"
>
> Am I missing something? I just don't see why buffer is useful - is it
> a performance benefit or some kind of syntax improvement that I'm
> missing?

Yes, the string referenced by buffer is reused.  This leads to
improved performance for the typical application which is like this:

buffer = ""
while ( io.read(1024, buffer) )
  file.write buffer
end

> The only thing I can see is that it has some kind of low
> level malloc optimization if the same string size is passed in
> repeatedly during partial writes.

Exactly (see above).  Note that it is very inefficient to read with
such a large chunk size as you use in your original posting.  If you
want to read the whole file you can simply do io.read.

Kind regards

robert

-- 
use.inject do |as, often| as.you_can - without end