From: SonOfLilit <sonoflilit@...>
Date: 2007-12-02T21:10:28+09:00
Subject: Re: Trying to download first 10k

On Dec 2, 2007 10:22 AM, David Beckwith <dbitsolutions@gmail.com> wrote:
> Hi, I'm trying to download via http only the first, say, 10k of a file.
> I'm using open-uri but even when I try to break out of the core loop
> where it's reading the buffer, I can hear my hard-drive grinding away
> downloading the entire 3MB.  When I look at the file written to the disk
> it shows only 10k, but judging by the length of time it takes to execute
> the script and the grinding sound of my hard-drive, I think it's
> downloading the entire file.
>
> I read the other posts about a progress bar, but that's not really what
> I'm trying to do..... or is it?
>
> Is there a way to interrupt the download?  As you can see I tried to
> break out of the code blocks every chance I got.
>
> Here is my code:
>
> def download_first_n_kilobytes n, file_name
>   n = 1024 * n
>   uri = file_name
>   count = 0
>   open(uri) do |fin|
>      open(convert_to_file_name(uri), 'w') do |fout|
>        while(buf = fin.read(n))
>          count += 1
>          fout.write buf
>          if count > 0
>            break
>          end # if
>        end # while
>        break
>      end # open
>      break
>   end # open
> end # def
>
> def convert_to_file_name uri
>   "mp3/" + File.basename(uri).to_s
> end
>
> uri = "http://www.sapht.com/music/tmp/sapht_-_alvaros_messenger.mp3"
> download_first_n_kilobytes 10, uri


Looking at the documentation for open-uri at
http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/, it seems to be
that it is default open-uri behavior to download the whole file and
just then give you control. You might succeed if you give a
:progress_proc that read()s and close()s if size>n, but I doubt it
would work.

Otherwise, use a more powerful way of downloading files, like
Net::HTTP, with something like
  s = 0
  http.request_get('/index.html') {|res|
    res.read_body do |segment|
      s += segment.size
      print segment
      res.finish if s > n
    end
  }

See the docs at http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/index.html


Aur