From: Vidar Hokstad <vidar.hokstad@...>
Date: 2006-11-03T08:00:12+09:00
Subject: Re: net/http and rexml


Louis J Scoras wrote:
> The script I'm working on is grabbing a giant xml file from a remote
> site, and I want to process it as it comes in.  I've already changed
> the code to use rexml's pull parser, so I'm guessing that now all I
> need to do is give it the correct IO handle and let it go.
>
> I don't see anything in the docs about exposing the socket, and I
> don't want to rip open the class if there's something obvious I'm
> missing.  Any ideas on the best way to go about this?

I fully sympathize... I went through the same mess a while back.
IO Iis one of the spots where the Ruby standard library is fairly
messy.

net/http is overly complicated for this kind of stuff. Look at openuri:
http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/

And you can't actually "expose the socket" for the HTTP stream and
expect things to work - the server might very well be using HTTP/1.1
chunked encoding, which means you'd get things interspersed bytes
indicating the length of the following chunk etc., or the connection
might be marked Keep-Alive and use the content-length to indicate
how far you should read, so to pass it to REXML you'd need a wrapper -
which is what openuri provides you with.

Vidar