From: Robert Klemme <bob.news@...>
Date: 2005-10-21T22:26:59+09:00
Subject: Re: Threads, timing and HTTP

Robert Klemme wrote:
> Alexander Lamb wrote:
>> On Oct 21, 2005, at 3:01 PM, Robert Klemme wrote:
>>
>>> Alexander Lamb wrote:
>>>
>>>> Hello list,
>>>>
>>>> I am implementing a very simple script to ping web servers or
>>>> services (to monitor how our environment is functionning).
>>>>
>>>> Some production url's run on more than one host. Therefore, I start
>>>> a thread for each separate url.
>>>>
>>>> The function run in my threads is:
>>>>
>>>>      def doPing(uri_string, probe)
>>>>        s = uri_string
>>>>        while true
>>>>          begin
>>>>            timeout(@seconds_before_timeout) do |timeout_length|
>>>>              start = Time.new
>>>>              begin
>>>>                open(s) do |result|
>>>>                  if result.status[0] != "200"
>>>>                    probe.addToLogFile([s,'ERR',0,result.status[1]])
>>>>                  else
>>>>                    probe.addToLogFile([s,'OK',Time.new - start,''])
>>>>                  end
>>>>                end
>>>>              rescue Exception
>>>>                probe.addToLogFile([s,'ERR',0,$!])
>>>>              end
>>>>            end
>>>>          rescue Timeout::Error
>>>>            probe.addToLogFile([s,'ERR',0,'timeout'])
>>>>          end
>>>>          sleep(@seconds_between_ping)
>>>>        end
>>>>      end
>>>>
>>>> However, this is a problem. Indeed, I want also to measure the time
>>>> (round-trip) it takes for the ping (these are only simple pings for
>>>> the time being). As you can see I get the local time before and
>>>> after the call. But this doesn't work with threads. Indeed, since
>>>> the process is shared among threads, the time will be dependent on
>>>> the number of threads I am running and not a correct view of the
>>>> actual time it takes to ping.
>>>>
>>>> I can't define the piece of code between the two times as critical
>>>> and only for one thread because if the open-uri blocks, it will
>>>> prevent another thread to ping another url in the mean time.
>>>>
>>>> Any idea? Maybe use processes instead of threads?
>>>>
>>>
>>> Maybe you can exploit one of the result headers.  Chances are that
>>> there
>>> is a timestamp somewhere.  Then you *only* need to synchronize
>>> clocks on
>>> your machine and on servers...
>>>
>>> Or you switch to a single thread solution.  I don't know your ping
>>> interval but if you don't need to ping too often and don't have too
>>> many
>>> servers that should be ok.  You can create a simple scheduling that
>>> always
>>> picks the URL with the closest ping point...
>>>
>>>
>> I could go single thread (since indeed I am doing a ping per 30
>> seconds more or less). However, if the first one I try hangs (until a
>> timeout for example), I am pushing back the time at which I will ping
>> the second url. Logically I would need to do something like "ping
>> each url one after another unless one of them seems to take longer
>> and then detach a thread to wait for the answer".
>> For the time being I will test forking a process.
>
> You could also have a controller thread that watches your single
> testing thread.  If the testing takes longer than n seconds (where n
> << timeout) it sets a flag for the current testing thread (with a
> thread local variable for example) and starts a new tester thread.

Yet another idea: you make the testing semi critical.  When a thread
starts testing it stores a timestamp somewhere.  Every other thread checks
whether the timestamp is set and is only max n seconds away.  If it's
longer, replace the timestamp with it's own timestamp and go ahead.  If
we're still in the n seconds range, go on sleeping.

    robert