From: Elliot Temple <curi@...>
Date: 2006-06-08T03:23:34+09:00
Subject: Re: open-uri error


On Jun 6, 2006, at 11:47 PM, Elliot Temple wrote:

> I am writing a script to download warcraft 3 replays for me. It got  
> a few (which work) then had an error:
>
> URI::InvalidURIError: bad URI(is not URI?): http://ftp.replays.net/ 
> w3g/060607/060606_mYm]Lucifer(UD)_vs_mTw-LasH(Hum) 
> _TwistedMeadows_RN.w3g

I fixed my problem. The key change is:

url = URI.escape(url)


Here's the current version of the code:


require "open-uri"
path = "/Applications/Warcraft\ III/Replay/auto/"
Dir.chdir path
files = Dir.glob "*"
count = 0

urls = `lynx -dump http://war3.replays.net/`.split "\n"
urls = urls.select {|url| url =~ %r-\d{1,3}\. http://ftp.replays.net/ 
w3g-}
urls = urls.collect {|url| url.sub(%r-\s*\d{1,3}\.\s*-, "")}.uniq

puts "I found #{urls.length} replays!"

urls.each do |url|
   filename = url.sub(%r-http://ftp.replays.net/w3g/\d*/-, "")
   url = URI.escape(url)
   if not files.include?(filename)
     puts "Count is #{count}. Getting #{url}"
     open(url) do |remote_file|
       File.open(path + filename, "w") do |local_file|
         local_file.write remote_file.read
         count += 1
       end
     end
   end
end

puts "I got #{count} files!"

-- Elliot Temple
http://www.curi.us/blog/