From: Joel VanderWerf <vjoel@...>
Date: 2007-06-07T02:51:22+09:00
Subject: Re: efficient regex scanning

Trochalakis Christos wrote:
> Hello there,
> 
> I wan't to extract all the words from a file and so i wrote the
> following code:
> 
> file = ARGV[0]
> File.open('output','w') {|f|
>         IO.read(file).scan(/\w+/).each{|w| f.print w}
> }
> 
> The problem with this code is that it stores all the words in an array
> which is not so good in terms of efficiency.
> Is there a better way to do it?
> Something like IO.read(file).each_scan { foo }

Here's a thought. Note that it doesn't handle //m regexen. Like David's 
and Robert's solutions, it doesn't read the whole at once. (I guess one 
could check for pat.options&Regexp::MULTILINE, and read the whole IO in 
that case.)

class IO
   def scan pat
     if block_given?
       each {|line| line.scan(pat) {|s| yield s} }
     else
       read.scan(pat)
     end
   end
end

File.open(filename) do |f|
   f.scan(/\w+/) {|word| puts word}
end


-- 
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407