From: James Edward Gray II <james@...>
Date: 2007-03-12T01:42:52+09:00
Subject: Re: Opening a large file many times / optimisation

On Mar 11, 2007, at 11:30 AM, Paul Nulty wrote:

> I have a method that basically searches through a largish (5mb) text
> file for a word. I need to call this method about 1400 times, and i
> care about speed.
>
> If i open the file at the start of the script and then pass the file
> object as a parameter to my method each time its called, the code runs
> quite a bit faster than if i open the file inside the method each
> time; but this seems ugly to me.
>
> Is there a standard way to do this in ruby? How much overhead is
> involved in opening a large text file?

Well, if you have enough RAM to support pulling it into memory,  
that's certainly going to be faster.  However, there are some  
techniques you could use to speed up and index and query operation.   
See this old Ruby Quiz for some ideas:

http://www.rubyquiz.com/quiz54.html

James Edward Gray II