Living with a Swarm of Boids - A report from the front

Hi,

Haven't checked in for awhile...  thanks, guys,  for the info from
other threads.

The "beef"  in this post,  if there is any, will probably be the
"Advantages Seen" below....  as the place to start - if you think like
me.

So, where any other "flow" diagram for a project I have worked on for
(too many years > 30) would have all circles and squares, with a
database shown as a big stack of cool disks and such I have my "flow
diagram now on my wall inspiring me on...  there are the usual major
classes but there is a "Swarm_Sender" rectangle, and a
"Swarm_Receiver" rectangle and a bunch of lines representing the
literally 100's (to be 1000's) of "Boids" / Agents  that make the
"flight" between these two major classes.

It's been a real "trip" working with this layout day in and day and
for awhile the maintenance and additional confusion was threatening to
overwhelm me, but as usual when I keep my head down on this stuff
solutions come and if nothing else, I things like this can just "grow
on me".  Now, I think, the project has grown enough, worthwhile things
are being resolved, performance has been studied and found more than
adequate (2,000 boids can be fired per second), and I have shook down
the code/refactored  things a few times now and its just "holding up
nicely".   I've been writing a lot of "self tests" that detect and
report a lot of basic stuff before anything even starts... all based
on a lot of "reflection".  Turns out, I think, that this kind of
system work is more critical than usual when working with these
things.

A FEW NOTES ON THE "LIFECYCLE" OF A BOID

*Every boid is a single text file, which is a proper Class written
with the key method "written" in (generally) 4 sections supported by 4
DSL_ classes.

*Phase one of the program is a bunch of checks for data module
integrity and such.  The Boids are loaded from designated subdir(s)
and sequenced and "readied for flight" so to say.  (There are multiple
flights...)

*Phase two 'fires!' the Boids sequentially
Each Boid is executed/supported by a sequence of DSL classes (one
     each section)
Each Boid "rendezvous"/docks with a Helper Class where the
     heavy lifting calcs/logic is done.
Each Boid makes assertions per the authors choices - and
     records the results are recorded in gory detail into a record
     the the Boid "births" - (rather like a black box for an
      airplane?)
Each Boid is "Debriefed" in a "overview sort of way" - which is
immediately reflected into a "dashboard" like thingy I have created
(but which is disconnected for now to avoid graphics)
The "black box" records are tucked away for debugging, or really geeky
clients to examine via the "dashboard" features.



ADVANTAGES SEEN TO THIS DESIGN:

ONE
The normal curve for software complexity versus task complexity would
probably show a increasingly upward sloping curve for the software,
right?

Well,  with this design that is probably still true, and it is steeper
at the gitgo - that is for sure-  BUT,  and I believe it is starting
to be born out that after a certain complexity has been handled (THANK
YOU RUBY!!!) the curve is considerably less steep.   Seems like a rpm
"shifting pattern" kinda thing going on here.... (bet I'm in 3rd
now).

Is there a "special" exception to handle?  This is the payload of this
design technique - you write a Boid or adopt and existing one. NOTHING
else has to change (!)   (THANK YOU RUBY!!! )

Entire "alternate views" could be forged by controlling the name of a
subdir assuming the format of the data can be shared.

In our domain one Clinic or Hospital might keep a "local layer" on top
of a national one.   These issues become "subdir" name level stuff.

(Firing Boid Swarms from multiple views could be fascinating too!)

TWO
The Boids have been kept really simple.  Amazingly simple, I think.
The guts of the things are the "Assertion" statements and really, this
has been reduced to "building a natural language sentence" from 2 or 3
"picklists".   The "messages" attached to the assertions are the most
time consuming / require the most time and effort - and they are only
really needed for examining the logic / attribs gathered to make the
decisions.   I had this all working on-line.

Because the Boids can be authored on-line, and the nature of the Web,
an expert in South Africa could document a problem (perhaps using a
even simpler Boid type - the Review type - makes no assertions) and it
would show up in Boston minutes later.   Mind boggling stuff?


SUMMARY

Nothing is done until it is done, it is clear to me that what I am
doing so far with only about 100 Boids could have been done much
faster and simpler using any one of a number of programming
techniques.
It would also execute much faster if done in the good old 'c'
language.

However, I believe that it is not only a fascinating design objective,
but also some large commercial potential to opening up the 'rules' of
the system for outsiders to contribute to via the web.

Where timeliness is of value, complexity is significant/or subject to
(rapid)change, and a passive "broadcast" type system fits the use
patterns this technique seems to fit very well.


Sincerely,

Thunk


PS somebody on this forum discouraged me in a phone call on this
project, saying "that it would be 'high maintenance'"  and I knew he
was right, but I think I'm going to be proven right also, and that the
extra effort is going to be worth it.