From: thunk Date: 2010-03-20T21:42:42+09:00 Subject: Living with a Swarm of Boids - A report from the front Hi, Haven't checked in for awhile... thanks, guys, for the info from other threads. The "beef" in this post, if there is any, will probably be the "Advantages Seen" below.... as the place to start - if you think like me. So, where any other "flow" diagram for a project I have worked on for (too many years > 30) would have all circles and squares, with a database shown as a big stack of cool disks and such I have my "flow diagram now on my wall inspiring me on... there are the usual major classes but there is a "Swarm_Sender" rectangle, and a "Swarm_Receiver" rectangle and a bunch of lines representing the literally 100's (to be 1000's) of "Boids" / Agents that make the "flight" between these two major classes. It's been a real "trip" working with this layout day in and day and for awhile the maintenance and additional confusion was threatening to overwhelm me, but as usual when I keep my head down on this stuff solutions come and if nothing else, I things like this can just "grow on me". Now, I think, the project has grown enough, worthwhile things are being resolved, performance has been studied and found more than adequate (2,000 boids can be fired per second), and I have shook down the code/refactored things a few times now and its just "holding up nicely". I've been writing a lot of "self tests" that detect and report a lot of basic stuff before anything even starts... all based on a lot of "reflection". Turns out, I think, that this kind of system work is more critical than usual when working with these things. A FEW NOTES ON THE "LIFECYCLE" OF A BOID *Every boid is a single text file, which is a proper Class written with the key method "written" in (generally) 4 sections supported by 4 DSL_ classes. *Phase one of the program is a bunch of checks for data module integrity and such. The Boids are loaded from designated subdir(s) and sequenced and "readied for flight" so to say. (There are multiple flights...) *Phase two 'fires!' the Boids sequentially Each Boid is executed/supported by a sequence of DSL classes (one each section) Each Boid "rendezvous"/docks with a Helper Class where the heavy lifting calcs/logic is done. Each Boid makes assertions per the authors choices - and records the results are recorded in gory detail into a record the the Boid "births" - (rather like a black box for an airplane?) Each Boid is "Debriefed" in a "overview sort of way" - which is immediately reflected into a "dashboard" like thingy I have created (but which is disconnected for now to avoid graphics) The "black box" records are tucked away for debugging, or really geeky clients to examine via the "dashboard" features. ADVANTAGES SEEN TO THIS DESIGN: ONE The normal curve for software complexity versus task complexity would probably show a increasingly upward sloping curve for the software, right? Well, with this design that is probably still true, and it is steeper at the gitgo - that is for sure- BUT, and I believe it is starting to be born out that after a certain complexity has been handled (THANK YOU RUBY!!!) the curve is considerably less steep. Seems like a rpm "shifting pattern" kinda thing going on here.... (bet I'm in 3rd now). Is there a "special" exception to handle? This is the payload of this design technique - you write a Boid or adopt and existing one. NOTHING else has to change (!) (THANK YOU RUBY!!! ) Entire "alternate views" could be forged by controlling the name of a subdir assuming the format of the data can be shared. In our domain one Clinic or Hospital might keep a "local layer" on top of a national one. These issues become "subdir" name level stuff. (Firing Boid Swarms from multiple views could be fascinating too!) TWO The Boids have been kept really simple. Amazingly simple, I think. The guts of the things are the "Assertion" statements and really, this has been reduced to "building a natural language sentence" from 2 or 3 "picklists". The "messages" attached to the assertions are the most time consuming / require the most time and effort - and they are only really needed for examining the logic / attribs gathered to make the decisions. I had this all working on-line. Because the Boids can be authored on-line, and the nature of the Web, an expert in South Africa could document a problem (perhaps using a even simpler Boid type - the Review type - makes no assertions) and it would show up in Boston minutes later. Mind boggling stuff? SUMMARY Nothing is done until it is done, it is clear to me that what I am doing so far with only about 100 Boids could have been done much faster and simpler using any one of a number of programming techniques. It would also execute much faster if done in the good old 'c' language. However, I believe that it is not only a fascinating design objective, but also some large commercial potential to opening up the 'rules' of the system for outsiders to contribute to via the web. Where timeliness is of value, complexity is significant/or subject to (rapid)change, and a passive "broadcast" type system fits the use patterns this technique seems to fit very well. Sincerely, Thunk PS somebody on this forum discouraged me in a phone call on this project, saying "that it would be 'high maintenance'" and I knew he was right, but I think I'm going to be proven right also, and that the extra effort is going to be worth it.