From: Pat Maddox Date: 2005-04-03T09:21:16+09:00 Subject: Re: Handling Timeout::Error from TCPSocket I think that a lot of machines are just dropping packets now. I've seen that as the default option in a lot of example firewall rulesets across the net lately. On Apr 2, 2005 5:11 PM, Sam Roberts wrote: > Quoting pergesu@gmail.com, on Sun, Apr 03, 2005 at 06:51:09AM +0900: > > Alright, basically this is what I want to do. > > > > Try to make a connection to a port, with a maximum timeout value of 3 > > seconds. If it times out, try again two more times. If the port is > > closed, just give up. So the second guess you made is more accurate. > > Glenn explained this, but to restate: you didn't implement a timeout of > 3 seconds, you did a timout of 6 secs (2 secs times the 3 times you > try). So, give the stack 6 seconds to make its connection. > > If its test code, I can see why you want to shorten the connect > timeouts. I don't understand why it takes that long, though. That means > you have an IP in the DNS, and the IP isn't reachable, and none of the > routers on the way tell you its not reachable, they just drop the > packets. If your packets got to the destination IP, and the port wasn't > open, you should get a packet back right away saying so. > > I guess with so much NAT and firewalls, and defences against denial of > service attacks, maybe machines are just dropping packats rather than > responding to them. Or maybe I just don't understand. > > Cheers, > Sam > > > I realize that the TCP stack takes care of all that, but it's just > > that the timeout value is far too long. As I said above, I want it to > > try for a maximum of two seconds. One problem with the way I've done > > it is that if it takes two seconds to resolve the host, then that eats > > up most of the time. I just don't see a better way of specifying the > > maximum time it tries to connect. > > > > The TestResult objects are just for reporting, I have to generate > > reports from the results of the tests, stored in a db. I'm using > > test/unit to test all of this out. > > > > > > > > On Apr 2, 2005 2:29 PM, Sam Roberts wrote: > > > Quoting pergesu@gmail.com, on Sun, Apr 03, 2005 at 05:19:36AM +0900: > > > > All I'm trying to do is see if a connection can be made or not, in a > > > > specified amount of time. Here's the method I've written, would you > > > > mind telling me how I could improve upon it? > > > > > > The code doesn't make any sense to me, you'll have to give me > > > background. You said you wanted to shorten the connect with a timeout, > > > but then you call connect over and over again some number of tries. > > > > > > Why is that connect() can fail, but then work the next time you try? > > > > > > The only reasons I can think of are that you expect that the server may > > > be down, but that if it is down it will be rebooted in some reasonable > > > amount of time, so you want to retry every few minutes for some amount > > > of time to give it a chance to recover, like say 15 minutes, before > > > deciding to give up. Or maybe its not the whole server, but just the app > > > that might be down. Or maybe there's load limiting on the server side, > > > and you want to keep trying until you get a connection. > > > > > > Is it a scenario like these? > > > > > > If so, you shouldn't be killing your connect with a timeout, let it do > > > its best to connect, and then retry after some sleep only if it fails. > > > If your code is going to be waiting, why not wait for connect to finish > > > rather than wait for sleep to finish? > > > > > > def execute > > > begin > > > timout(@retries * @timeoutval) do > > > loop do > > > begin > > > TCPSocket.new(host,port).close > > > result = ... > > > return result > > > rescue > > > # ignore TCP socket failures, and try again > > > end > > > # sleep a few secs so we don't hammer the network trying to > > > # connect to a (temporarily?) unreachable server > > > sleep @retryDelay > > > end > > > end > > > rescue Timeout::Error > > > # After all this time, no success, so the server is definitely > > > # unreachable. > > > result = ... > > > return result > > > end > > > end > > > > > > That scenario aside, I have a second guess as to what you are doing. If > > > timeout is LESS than the TCP stacks timeout, then what you are doing > > > with this code is trying to be better than your own TCP stack at making > > > connections, because you have coded what the stack does. > > > > > > It sends a SYN packet to the peer, and waits for the SYN/ACK to come > > > back. If it doesn't get one after a reasonable period it keeps trying a > > > few times, then gives up. If it gets an error response back (port > > > unreachable, say), it gives up right away. The stack is good at this, > > > those TCP stack programmers know their business, let them do it. > > > > > > Are either of my guesses as to the purpose of your loop correct? If not, > > > I don't know what you are doing. > > > > > > Sam > > > > > > Btw, I don't know if you are aware of test/unit in the standard ruby > > > lib, but it looks a bit like you are rolling your own test structure, > > > so I thought I'd mention it. > > > > > > > > > > def execute > > > > numtries = 0 > > > > > > > > while numtries < @retries + 1 > > > > puts "Trying to connect" > > > > numtries += 1 > > > > timeout(@timeoutval, StandardError) { > > > > socket = TCPSocket.new(@host, @port) rescue false > > > > > > > > if socket > > > > socket.close > > > > result = TestResult.new > > > > result.testId = @id > > > > result.status = TestResult::STATUS_SUCCEEDED > > > > return result > > > > end > > > > } rescue StandardError > > > > puts "Unable to connect" > > > > sleep @retryDelay if numtries < @retries + 1 > > > > end > > > > > > > > result = TestResult.new > > > > result.testId = @id > > > > result.status = TestResult::STATUS_FAILED > > > > result.info = "Unable to connect" > > > > return result > > > > end > > > > > > > > > > > > On Apr 2, 2005 1:09 PM, Sam Roberts wrote: > > > > > Quoting pergesu@gmail.com, on Sun, Apr 03, 2005 at 03:33:47AM +0900: > > > > > > > > I don't want to override the protocol's idea of a timeout - I'd like > > > > > > > > some means of specifying a timeout for the connection though. In > > > > > > > > > > Be clear that you ARE overriding the protocols idea of a timeout by > > > > > terminating the connect() before the stack has decided the peer cannot > > > > > be connected. > > > > > > > > > > > In Java: > > > > > > > > > > > > Socket s = new Socket(); > > > > > > s.connect(new InetSocketAddress(getHost(), port), timeout); > > > > > > > > > > > > I can specify a timeout value for the connection, as you see above. > > > > > > > > > > True enough, I was looking at the TCPSocket constructors, which don't > > > > > allow this. Note this didn't exist prior to JDK 1.4, its a recent hack, > > > > > true to Java's "everything plus the sink" philosophy it looks like. > > > > > > > > > > http://java.sun.com/features/2002/08/j2se-network.html > > > > > > > > > > I still find your "all the other languages" statement a little over the > > > > > top, JDK1.4 isn't a very large "all". > > > > > > > > > > Overriding the TCP protocol's timeouts in the application is > > > > > questionable practice. I'm not telling you how to write your app, you > > > > > might have good reasons to do this, so do it. I am saying that overeager > > > > > timeouts and retransmissions have been known to cause very hard to find > > > > > and fix bugs when applications that work fine on a local network or > > > > > machine mysteriously start timing out over dialup or WiFi links. > > > > > > > > > > > I don't see a way of doing that with the Ruby classes. I'm new to > > > > > > Ruby, so I really don't know - is there a way to do it? > > > > > > > > > > Yes, as I showed you; > > > > > > > > > > begin > > > > > Timeout.timeout(18) do > > > > > TCPSocket.new(... > > > > > end > > > > > rescue Timeout::Error > > > > > ... > > > > > end > > > > > > > > > > You would wrap this into a nice function if you wanted to do it a lot. > > > > > > > > > > Also, you may find that doing your connects in a seperate thread allows > > > > > you to not fiddle with the timeout, and still avoid your app blocking > > > > > if the network is slow or flaky. > > > > > > > > > > Have fun. > > > > > Sam > > > > > > > > > > > > > > > > > On Apr 2, 2005 11:14 AM, Sam Roberts wrote: > > > > > > > Quoting pergesu@gmail.com, on Sun, Apr 03, 2005 at 02:03:51AM +0900: > > > > > > > > every other language I've used, I can set a timeout value on the > > > > > > > > socket itself. It doesn't look like that's the case in Ruby. Is > > > > > > > > > > > > > > The C, perl, and Java networking APIS don't support timeouts on TCP > > > > > > > connect(), and the C/BSD sockets API is what most (all?) other languages > > > > > > > use internally. > > > > > > > > > > > > > > Out of curiosity, what are these languages? > > > > > > > > > > > > > > > there a method somewhere in the Socket classe heirarchy that lets me > > > > > > > > specify a timeout? > > > > > > > > > > > > > > > > According to timeout, you can set an Exception that gets thrown when > > > > > > > > it times out. I set that to StandardError, and just handle > > > > > > > > > > > > > > Not sure what you mean. You can make timeout raise errors other than > > > > > > > Timeout::Error, and catch them, but you don't do so in the example you > > > > > > > posted. > > > > > > > > > > > > > > > StandardError at the end. According to the docs, Timeout::Error can't > > > > > > > > be used with rescue because it doesn't inherit from StandardError. > > > > > > > > > > > > > > I assure you that Timeout::Error can be caught by rescue, as I showed, I > > > > > > > do it quite often and it works fine. > > > > > > > > > > > > > > > I'd still like some means of setting a timeout value for a socket > > > > > > > > connection. That's a pretty basic aspect of the protocol, there's no > > > > > > > > reason it shouldn't be in there. > > > > > > > > > > > > > > I see no evidence that user configurable timeouts on connect() are not > > > > > > > part of the TCP protocol. > > > > > > > > > > > > > > Cheers, > > > > > > > Sam > > > > > > > > > > > > > > > On Apr 2, 2005 9:37 AM, Sam Roberts wrote: > > > > > > > > > Quoting pergesu@gmail.com, on Sat, Apr 02, 2005 at 08:05:15PM +0900: > > > > > > > > > > I'm writing a little method that just tries to open a tcp socket > > > > > > > > > > connection, then closes it off. I'm using timeout to limit the amount > > > > > > > > > > of time it tries to connect (is there a better way?). If it times > > > > > > > > > > > > > > > > > > I have to wonder why you want to override the TCP protocols ideas of a > > > > > > > > > timeout. The protocol designers and implementors have a pretty good idea > > > > > > > > > of what the timeouts should be. If its not fast enough for you, you > > > > > > > > > probably want too much. Doing this will make your code flaky and > > > > > > > > > unreliable when the network isn't as fast as your local network. A good > > > > > > > > > text on network programming should explain this. If you are finding your > > > > > > > > > app is blocked, you can use ruby threads to do connections in the > > > > > > > > > background. > > > > > > > > > > > > > > > > > > That said, you can't catch exceptions like that, try: > > > > > > > > > > > > > > > > > > begin > > > > > > > > > Timeout::timeout(3) do > > > > > > > > > long running op > > > > > > > > > end > > > > > > > > > rescue Timeout::Error > > > > > > > > > p "hi" > > > > > > > > > end > > > > > > > > > > > > > > > > > > Sam > > > > > > > > > > > > > > > > > > > out, I just want to say that the connect failed. Despite catching the > > > > > > > > > > Timeout::Error (I think, anyway), I always get the Exception output. > > > > > > > > > > First, here's the method: > > > > > > > > > > > > > > > > > > > > def execute > > > > > > > > > > status = timeout(@timeoutval) { > > > > > > > > > > socket = TCPSocket.new(@host, @port) rescue false > > > > > > > > > > > > > > > > > > > > socket.close if socket > > > > > > > > > > return socket != false > > > > > > > > > > } rescue Timeout::Error > > > > > > > > > > > > > > > > > > > > return false > > > > > > > > > > end > > > > > > > > > > > > > > > > > > > > And now the output from my unit test: > > > > > > > > > > > > > > > > > > > > Started > > > > > > > > > > /usr/local/lib/ruby/1.8/timeout.rb:42:in `new': execution expired > > > > > > > > > > (Timeout::Error) > > > > > > > > > > from ./PortHostTest.rb:16:in `execute' > > > > > > > > > > from ./PortHostTest.rb:15:in `timeout' > > > > > > > > > > from /usr/local/lib/ruby/1.8/timeout.rb:55:in `timeout' > > > > > > > > > > from ./PortHostTest.rb:15:in `execute' > > > > > > > > > > from ./HostTest.rb:10:in `runTest' > > > > > > > > > > from ../tests/test_HostTest.rb:14:in `test_simple' > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit/testcase.rb:70:in `__send__' > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit/testcase.rb:70:in `run' > > > > > > > > > > ... 10 levels... > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:194:in `run' > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:14:in `run' > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit.rb:285 > > > > > > > > > > from /usr/local/lib/ruby/1.8/test/unit.rb:283 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >