From: Ron Jeffries <ronjeffries@...>
Date: 2002-01-15T23:33:06+09:00
Subject: grabbing stuff from web pages

Part of my web site has recommended books. I use the cover jpegs from
Amazon as part of my page. The rule with Amazon is that you have to
download the picture and put it on your site, not link to theirs. 

So I was starting out with Dave and Andy's example:

require 'net/http'

h = Net::HTTP.new('www.xprogramming.com', 80)
resp, data = h.get('/index.htm', nil)
if resp.message == "OK"
	puts data
  data.scan(/<img src="(.*?)"/) { |x| puts x }
end

 figuring that I could then come up with some way to get and save the
image ... I remember doing it once before as an experiment, though I
can't find the code now.

But when you set up to hook to the amazon book page, which can be
accessed this way in a web browser

http://www.amazon.com/exec/obidos/ASIN/0201708426/

so that the code looks like this:

require 'net/http'

h = Net::HTTP.new('www.amazon.com', 80)
resp, data = h.get('/exec/obidos/ASIN/0201708426/', nil)
if resp.message == "OK"
	puts data
  data.scan(/<img src="(.*?)"/) { |x| puts x }
end

 You get an error 302. I suppose that it's all that magic redirection
that they're using to get you to the real page, but I don't know enough
to be sure.

So I'd appreciate help on

  a) getting the right page address given that I only know the number
that comes after ASIN, and
  b) saving the jpeg once I get its src address.

Thanks!

Ronald E Jeffries
http://www.XProgramming.com
http://www.objectmentor.com
I'm giving the best advice I have. You get to decide whether it's true for you.