From: Matthew Bloch <mattbee@...>
Date: 2002-07-17T23:23:09+09:00
Subject: Re: Patch to stop TCPSocket.new blocking on DNS lookups

On Wednesday 17 July 2002 14:12, you wrote:
> Hi,
>
> In message "Patch to stop TCPSocket.new blocking on DNS lookups"
>
>     on 02/07/17, Matthew Bloch <mattbee@soup-kitchen.net> writes:
> |I'm using Ruby for a multi-threaded (50 or 60 threads at once) web spider
> | and found that every time I started a fetch in a thread, the whole
> | process got blocked on getaddrinfo, which of course knackers the
> | throughput since all the already-connected threads have to wait for a
> | single DNS lookup to happen.
>
> How about using "Socket.do_not_reverse_lookup = true"?

The delays I'm experiencing were due to (necessary) forward lookups as far as 
I'm aware: my test program looks like this:

-----------------------------------------------------------
require 'socket'

t = Thread.new {
  count = 0
  loop {
    print "count = #{count += 1}\n"
    sleep 0.05
  }
}
 
ARGV.each { |name|
  sock = TCPSocket.new(name, 80) # forward lookup here?
  print "connected to #{name}\n"
  sock.print "GET /\n\n"
  sock.each { |l| print "#{l}\n" }
  sock.close
}
 
t.join
-----------------------------------------------------------

With the unpatched Ruby, my counter thread paused before every connection.  
Patched, it carries on fine and still returns the results.  As I said this is 
the only testing I've done on it so far.

-- 
Matthew Bloch         Bytemark Computer Consulting Limited
                                http://www.bytemark.co.uk/
                                  tel. +44 (0) 8707 455026