From: Michael Fellinger <m.fellinger@...>
Date: 2010-07-01T19:40:43+09:00
Subject: Re: Fast searching of large files

On Thu, Jul 1, 2010 at 6:47 PM, Stuart Clarke
<stuart.clarke1986@gmail.com> wrote:
> Hey all,
>
> Could anyone advise me on a fast way to search a single, but very large
> file (1Gb) quickly for a string of text? Also, is there a library to
> identify the file offset this string was found within the file?

You can use IO#grep like this:
File.open('qimo-2.0-desktop.iso', 'r:BINARY'){|io|
io.grep(/apiKey/){|m| p io.pos => m } }

The pos is the position the match ended, so just substract the string length.
The above example was a file with 700mb, took around 40s the first
time, 2s subsequently, so disk I/O is the limiting factor in terms of
speed (as usual).
Oh, and also don't use binary encoding if you are dealing with another one ;)

-- 
Michael Fellinger
CTO, The Rubyists, LLC