From: Caleb Clausen <vikkous@...>
Date: 2009-12-02T10:33:26+09:00
Subject: Re: regexing a file's contents without reading the whole thing?

On 11/30/09, Roger Pack <rogerpack2005@gmail.com> wrote:
> I see that it is possible currently to parse through a file without
> reading the whole thing into RAM, a la
>
> a = File.open('a', 'r')
> a.lines{|line|
>   if line =~ /some regex/
>     ...
>   end
> }
>
> But what if I can to do something like
> a = File.read('a').scan /some regex/
>
> is that possible?

The library which makes this possible is sequence. I'm coding this
from memory, so I'm likely to get something wrong, but the equivalent
in sequence looks more or less like this:

require 'rubygems'
require 'sequence'
require 'sequence/file'

seq=Sequence.new(File.open('a'))
seq.scan_until(/some regex/)

Keep the following in mind:
1) Sequence#scan works like StringScanner#scan, not String#scan.
2) The pattern to be matched must have a max length (4k by default, I
think; it can be changed).
3) If your pattern is guaranteed to not contain a nl, you're better
off with readline, as robert said.