From: Jano Svitok <jan.svitok@...>
Date: 2007-12-07T18:29:20+09:00
Subject: Re: Reasons to use a buffer in IO::read?

On Dec 7, 2007 6:25 AM, MonkeeSage <MonkeeSage@gmail.com> wrote:
>
> On Dec 6, 10:31 pm, Steve Midgley <midgley.st...@gmail.com> wrote:
> > On Dec 5, 5:56 pm, MonkeeSage <MonkeeS...@gmail.com> wrote:
> >
> >
> >
> > > On Dec 5, 6:59 pm, SteveMidgley<midgley.st...@gmail.com> wrote:
> >
> > > > Hi Ruby people,
> >
> > > > I'm wondering what the functional and performance differences might be
> > > > between the two statements below? Assume 'io' is an IO instance with
> > > > gobs of data in it. Assume 'file' is an open file instance with write
> > > > access:
> >
> > > > until io.eof? do
> > > >   file.write(io.read(10485760))
> > > > end
> >
> > > > buffer = ''
> > > > until io.eof? do
> > > >   buffer = io.read(10485760)
> > > >   file.write(buffer)
> > > > end
> >
> > > > I see that Ruby provides for a buffer and I'm wondering what the
> > > > reason is? I read this article but am still not clear on the benefit
> > > > of a buffer at all:
> >
> > > >http://rcoder.net/content/fast-ruby-io
> >
> > > > I'm wondering if providing a buffer might reduce malloc issues and
> > > > speed things up? I can't see any other reason to use one..
> >
> > > > Thanks in advance for any information!
> >
> > > > Steve
> >
> > > $ ri IO#buffer
> > > ----------------------------------------------------------------
> > > IO#read
> > >      ios.read([length [, buffer]])    => string, buffer, or nil
> > > ------------------------------------------------------------------------
> > >      Reads at most _length_ bytes from the I/O stream, or to the end
> > > of
> > >      file if _length_ is omitted or is +nil+. _length_ must be a
> > >      non-negative integer or nil. If the optional _buffer_ argument is
> > >      present, it must reference a String, which will receive the data.
> >
> > >      At end of file, it returns +nil+ or +""+ depend on _length_.
> > >      +_ios_.read()+ and +_ios_.read(nil)+ returns +""+.
> > >      +_ios_.read(_positive-integer_)+ returns nil.
> >
> > >         f = File.new("testfile")
> > >         f.read(16)   #=> "This is line one"
> >
> > > So...
> >
> > > buffer = ""
> > > file.write(io.read(nil, buffer))
> > > print "I read this stuff ", buffer, "\n"
> >
> > > Regards,
> > > Jordan
> >
> > Thanks Jordan. How is your code different (if at all) from:
> >
> > buffer = io.read
> > file.write(buffer)
> > print "I read this stuff ", buffer, "\n"
> >
> > Am I missing something? I just don't see why buffer is useful - is it
> > a performance benefit or some kind of syntax improvement that I'm
> > missing? The only thing I can see is that it has some kind of low
> > level malloc optimization if the same string size is passed in
> > repeatedly during partial writes.
> >
> > Steve
>
> I don't know if there is any optimization is the back end, but it lets
> you pass the results of io.read to another method and also put them in
> buffer at the same time. But since you can do that with assignment, I
> don't really see any point to it (I was just trying to give an example
> as the docs describe). To me, unless as you say, there is some
> optimization going on in the backend, this code...
>
> buffer = ""
> file.write(io.read(nil, buffer))
> print "I read this stuff ", buffer, "\n"
>
> ...looks the same as this code...
>
> file.write(buffer = io.read)
>
> print "I read this stuff ", buffer, "\n"
>
> Regards,
> Jordan

I'd *assume* the former saves you a bunch of allocations when looping
through a file
(I assume the buffer is reused instead of allocating a new one for
each iteration).

i.e.
buffer = ""
File.open('xxx','r') do |f|
  while f.read(1024, buffer) do
     process(buffer)
  end
end

vs.

File.open('xxx','r') do |f|
  while true do
     buffer = f.read(1024)
     break if buffer.empty?
     process(buffer)
  end
end