From: saurabh purnaye Date: 2008-12-13T03:35:15+09:00 Subject: Re: How to run background processes (more than 1 worker) parallely. ------=_Part_6838_4678863.1229107340088 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline hi Deepak, You may go for Starfish or Skynet, which are based on google's map-reduce algorithm. Even you may get more information on the rails cats episodes 127 to 129. On Fri, Dec 12, 2008 at 7:28 PM, Deepak Gole wrote: > Hi > > My requirement is as follows > > 1) I have around 200 feeds in the database that I need to parse (fetch > contents) *parallely* after some interval and store those feed items in > database. > > 2) Now I am using backgroundrb with 10 workers each worker has assigned a > job to parse data from 20 feeds (e.g 1st worker will fecth data from > feeds(1..20), 2nd from feeds(21..30) ..etc..... > > 3) But backgroundrb is not reliable and it fails after some time. So I have > tried Starling & Workling but those worker doesn't run *parallely. > > ( I need to run **parallely because those feeds will increase say 1000 > feeds. So I can't run them sequentially. ) * > * > Pls I need a help on above problem.* > > > Thanks > Deepak > -- -- Thanks and Regards Saurabh Purnaye +91-9922071155 skype: sorab_pune yahoo & gtalk: saurabh.purnaye msn: psaurabh@live.com ------=_Part_6838_4678863.1229107340088--