From: Mark Wilson <mwilson13@...>
Date: 2002-11-29T13:34:25+09:00
Subject: Re: A lesson learned

I really appreciated this lesson learned about the development process 
and thank Gavin Sinclair for posting it.  As a neophyte programmer, 
what I take from this is the following:

(1) Try to remember to follow general rules and principles in 
development;

(2) If you forget to follow (1) above and run into difficulties, see 
(1) above;

(3) Reflect on your programming habits every once in a while and see if 
any of them seem to recur in association with problems (I don't do this 
yet because I don't have enough experience -- all of my programming 
efforts are associated with problems to be solved);

(4) Step back every once in a while and ask yourself -- do I need to do 
this or can I do something else to solve a more general problem;

(5) Extensive experience and past success will reinforce both good and 
bad habits and approaches -- so everyone will have the opportunity to 
learn from experience on a regular basis.

I hope I'll see more stories like this, that I'll learn something from 
them and that I'll have my own stories to contribute over time.

On Thursday, November 28, 2002, at 10:05 AM, Gavin Sinclair wrote:

> Folks,
>
> Today I rediscovered the truth of that very important saying in 
> software
> developement: "Make it work, make it right, then make it fast."
>
> In my project at work, I spent too much time trying to perform file 
> scanning
> operation fast: by reading it backwards until it determined that it 
> didn't need
> any more information, then committing to the database.
>
> I've now done what I should have in the first place: read the file 
> forwards,
> grabbing *all* the information, and then updating the database as 
> necessary.
> This involves grabbing potentially 99% redundant information from a 
> HUGE file,
> then comparing against lots of info in the database before knowing 
> what to
> commit.  It's a dumb approach, but it's more reliable (no dependence 
> on smarts
> to determine when to stop scanning), and more flexible (can run it on 
> old files
> usefully and safely).  In the end, it was these features I needed, so I
> scrapped my "optimised" approach that I had spent perhaps a week on a 
> few weeks
> ago.
>
> Furthermore, the same degree of "optimisation" can be achieved by 
> controlling
> an external factor: restarting a certain process every night so the 
> log file
> doesn't grow too large.  Then the original "need" to read it backwards 
> (itself
> an expensive task) vanishes.
>
> Oh well, lesson learned.  And although this has nothing directly to do 
> with
> Ruby, I share it herre in the hope that by warning others, I can 
> balance my
> karma and not have it happen to me again :)
>
> Cheers,
> Gavin
>
>
>