From: Eric Wong <normalperson@...>
Date: 2009-12-31T09:43:55+09:00
Subject: Re: Parallel::ForkManager, Net::HTTP and catching Timeout::Error

nvp <ruby-lists@noopy.org> wrote:
> Hello,
> 
> I'm about to release Parallel::ForkManager 1.1 and while generating
> new examples (that use Net::HTTP) for PFM 1.1 features, I catch the
> following error *sometimes* (maybe 10-15% of the time) when I try to
> connect to a URL where the host part doesn't exist or isn't reachable:
> 
> /usr/lib/ruby/1.8/timeout.rb:60:in `rbuf_fill': execution expired
> (Timeout::Error)

<snip>
>         from /usr/lib/ruby/1.8/net/http.rb:772:in `get'
>         from ./parallel_http_get2.rb:36
>         from ./lib/parallel/forkmanager.rb:232:in `call'
>         from ./lib/parallel/forkmanager.rb:232:in `start'
>         from ./lib/parallel/forkmanager.rb:232:in `fork'
>         from ./lib/parallel/forkmanager.rb:232:in `start'
>         from ./parallel_http_get2.rb:30
>         from ./parallel_http_get2.rb:26:in `each'
>         from ./parallel_http_get2.rb:26
> 
> Based on my review of the error message, it would seem to be a simple
> case that I should catch (rescue) 'Exception' because Timeout::Error
> is sending to stderr or such, but when I try to catch 'Exception' in
> my test program, nothing happens!  I never catch the error!
> 
> Here's my test code:

<snip>

>     begin
>         pfm.start(my_url)  {
>             url = URI.parse(my_url)
>             out_file = save_dir + '/' + url.host + '.txt';
> 
>             http = Net::HTTP.new(url.host, url.port)
>             http.open_timeout = http.read_timeout = my_timeout
>             res = http.get(url.path)

<snip>

>         } # end pfm.start { ... }
>     rescue Exception => e
>         print "Arggh, exception: ", e, "\n"
>         exit 255
>     end
> }

> Would somebody be able to shed some light on why I'm unable to handle
> the exceptions that Net::HTTP is throwing?  Am I just trying to catch
> the wrong exception or is there something else?  Is it because I'm
> trying to handle the exception in the child?

It looks like the parent is trying to rescue exception in a child, which
isn't possible.  Exceptions are Ruby-level objects that normally don't
get transferred/communicated between Unix processes.  The parent can
only normally get the exit status of a child process, otherwise it needs
to resort to explicit IPC mechanisms (pipes/sockets/files/shared
memory/...).

-- 
Eric Wong