From: Lars Haugseth <njus@...>
Date: 2008-10-07T06:33:10+09:00
Subject: curb (Was: Re: Mirroring large files over HTTP)

* Lars Haugseth <njus@larshaugseth.com> wrote:
> 
> * Eric Hodel <drbrain@segment7.net> wrote:
> > 
> > On Oct 2, 2008, at 01:03 AM, Lars Haugseth wrote:
> > > Turns out the OpenURI module is indeed fetching the remote resource
> > > in segments and storing to a temporary file. However, my code above
> > > will read the complete contents of that file into memory before
> > > writing it back out to another file.
> > >
> > > I'm a little surprised not to find any convenience method in the  
> > > standard
> > > libraries doing all this for me, though.
> > 
> > Why? It's all of one line:
> > 
> > output.write input.read(16384) until input.eof?
> 
> Nice enough, but one will need a bit more than that singke line to do the
> whole operation from start to finish.
> 
> I was thinking more of something like SomeClass.mirror(url, filename).

Today I came across the the curb� gem (Ruby bindings for libcurl) while
reading a blog posting� about net/http performance, and this gem provides
a convenient class method that does exactly what I want:

  require 'curb'
  Curl::Easy.download(url, filename)

It also provides lots of other nice stuff, so I will definitely look
into using this one for my future HTTP client needs.

[1] http://curb.rubyforge.org/
[2] http://apocryph.org/analysis_ruby_18x_http_client_performance

-- 
Lars Haugseth

"If anyone disagrees with anything I say, I am quite prepared not only to
 retract it, but also to deny under oath that I ever said it." -Tom Lehrer