From: Mikel Lindsaar Date: 2008-09-10T22:37:43+09:00 Subject: Re: How to pick up dropped script automatically, active records. On Wed, Sep 10, 2008 at 2:50 PM, Marc Hoeppner wrote: > I am using an active records based script that queries a remote MySQL > database and retrieves data. It's quite an extensive search, meaning it > takes a long time to run. Unfortunately, the connection closes at random > (I suppose there is some sort of time-out if a connection has been > existing for a certain amount of time). I recently made a similar system. The best way i found to do it is to make a table at one end that holds all the rows "to be replicated" or in your case "has been replicated". This holds the ID of the field you have replicated Then you write your script to grab the id to replicate, replicate it down, and if all was successful, put an entry in your 'has been replicated' table with the ID. Then the next time your script runs it checks what the last ID was that was successfully replicated and picks up from there. Put your replication code at the local end in a transaction so that you only write into the has been replicated table IF no errors came up, and you don't replicate anything unless the whole thing completes successfully. This then survives restarting hung processes etc. On detecting health, I found the best way is to set a timeout on each replication check, and then nuke the process if it takes too long. If you have used the above pattern on your replication, this won't hurt anything. Warning though, Ruby's timeout will not handle this as the DB call is a system call and will not be killed by Ruby timeout. See my previous thread in Ruby-talk "Frustrated about system timeouts". The upshot is that Ara and I are wrapping up a new library called Terminator that will handle the timeout correctly. But you could adapt the code in that thread to your problem right now. Hope that helps. Mikel -- http://lindsaar.net/ Rails, RSpec and Life blog....