From: "Ara.T.Howard" Date: 2005-05-21T07:10:16+09:00 Subject: Re: Web services and Ruby On Fri, 20 May 2005, Luke Kanies wrote: > On Fri, 20 May 2005, Ara.T.Howard wrote: > >> if you are in *nix and have a central nfs filesystem all nodes can see >> check >> out rq (ruby queue) > > I'm planning on having this system be cross-platform, and it'll be used to > do things like set up NFS, so I can't depend on NFS's existence, > unfortunately. ah. oh well. > I'm always interested in more cluster software, though, especially if it's > written in Ruby; a lot of the people working on the same problem I'm > attacking (building software to maintain the computers for us) come from the > cluster world, because the problem is generally both exacerbated there > (because of node count) and easier to solve (because of node consistency). we manange this by mounting all binaries via nfs - make && make install then just goes cluster wide ;-) > When I have something that's somewhat more functional, I'd love to hear > whether you thought it would be something you'd be interested in using on > your compute clusters. please do. >> i also have a peice of software called 'dirwatch' (on raa too) that makes >> it trivial to setup 'watches' on directories to trigger actions when files >> are created, modified, deleted, etc. it's under revision as we speak and >> is undergoing major internal overhaul - but the basic funtionality an user >> interface won't change much. > > Does dirwatch use FAM, or is it its own distinct implementation? I am > planning on using something FAM-like eventually, and I notice there's a very > early release of a Ruby FAM library, but, well, 0.1.4 does not inspire > confidence. nope. it uses an sqlite db to maintain the state of the filesystem. every so often (5 min by default) in scan the dir and compares to it's db - spawning actions accordingly. actions can be normal unix filter programs and are passed the list of file (deleted files, created files, etc) in stdin - or can also be given more complete info in yaml format - also on stdin. the programs are run in a syncronous fashion by default - but can also be run async. the issue with async is nothing prevent 40,000 processes being spawned at once if 40,000 files arrive in a directory at once and you have a job configured to be launched when a file is created... say - that reminds me - let me tell you about this time i took down our entire cluster and three raids ;-) the update of dirwatch should be out next week sometime. cheers. -a -- =============================================================================== | email :: ara [dot] t [dot] howard [at] noaa [dot] gov | phone :: 303.497.6469 | My religion is very simple. My religion is kindness. | --Tenzin Gyatso ===============================================================================