From: thefed <fedzor@...>
Date: 2008-01-02T04:09:38+09:00
Subject: Re: Best way to download >1GB files


On Jan 1, 2008, at 1:56 PM, Tim Hunter wrote:

> thefed wrote:
>> On Dec 31, 2007, at 7:20 PM, Tim Hunter wrote:
>>> thefed wrote:
>>>> But doesn't open-uri download the whole thing to your compy? I  
>>>> was about to use it, but then I ran it in irb and saw it  
>>>> returned a file object.
>>>
>>> Isn't that what you want to happen? I thought your question was  
>>> about how to download it in small chunks so it's not all in  
>>> memory at the same time. This code downloads the whole file, but  
>>> 8kb at a time.
>> No, I thought when you use Kernel#open with open-uri, it FIRST  
>> downloads the entire 1GB file to your temp folder, and THEN runs  
>> your block on that file in temp
>
> Interesting. I just tried downloading a 6.1MB file with open-uri  
> and didn't see that behavior. I'm using Ruby 1.8.6 on OS X 10.5.

That's good then! I'll test it out myself juuuust to make sure. I  
don't to waste 4GB of space when i only need 2GB.

open-uri uses Net::HTTP, of course. Am I correct?

Net::HTTP wraps connections in a Timeout, which is REALLY screwing  
with me downloading large files.

Will probably get some monkeys to patch that for me.

- Ari