From: Meihua Liang <mliang@...>
Date: 2004-01-30T14:14:53+09:00
Subject: Net::http.get has a 50K limit?

I'm trying to write a screenscraper and am getting a 50K limit on the data
returned

require "net/http"
begin
 Net::HTTP.start("www.washingtonpost.com", 80){ |http|
  response , = http.get("/wl/jobs/JS_JobSearch?TS=1012409733026")
  data=response.body
  puts data.length
 }
rescue => err
 puts "Error: #{err}"
 exit
end

The last line returns 52166 . (the file is considerable bigger) What did I
do wrong?