From: Mark Thomas <ruby@...>
Date: 2008-08-26T03:50:44+09:00
Subject: Re: Open file, get first line, delete first line close file

> the thing is, if i didn't delete the lines i've already added, everytime
> i ran the task, (it has to be run multiple times due to timeouts) i
> would only re-add the same lines. The deletion acts as a place holder of
> sorts. I'll play around with your suggestion, and i'll let you know. If
> anyone else has an alternate method...i'm all ears

I suggest exploring a distributed worker system like Rinda,
Backgroundrb, AP4R, or Sparrow. You can prepare a master list of
dictionary words, and then worker processes can take one at a time and
add them to your database. Having a timeout won't slow things down,
nor will it cause you to have to re-read your wordlist.