From: "ara.t.howard" <ara.t.howard@...>
Date: 2007-10-15T02:29:01+09:00
Subject: Re: Is there a standard pattern for threaded access to a file?


On Oct 12, 2007, at 5:51 PM, Jon Handler wrote:

> I'm pretty new to ruby and this is one of those areas where I can't
> quite seem to turn my head inside out as the language requires :-)
>
> I have a log file that I want to process in parts, with multiple  
> threads
> working from the same file, each one getting a line at a time and  
> doing
> something with it.
>
> I'd like something like
>
> 1. Open the file
> 2. Create 5 threads
>
> Each thread should read a line of the file and process it, but no 2
> threads should get the same line.
>
> What I have looks like
>
> open (ARGV.flags.log) do |logfile|
>   logfile.each do |line|
>
>              blah blah blah...
>
>          end
> end
>
> but that's inside out! How do I rubify this code?
>
> Thanks,
>
> Jon

*finally* i remembered where this has been abstracted.  in my own  
lib : alib ;-)


cfp:~ > cat a.rb
require 'alib'  ### gem install alib

alib.util.threadify IO.readlines(__FILE__), n_threads=5 do |line,  
lineno|

   puts "#{ lineno }:#{ line }"

end


cfp:~ > ruby a.rb
0:require 'alib'
1:
2:alib.util.threadify IO.readlines(__FILE__), n_threads=5 do |line,  
lineno|
3:
4:  puts "#{ lineno }:#{ line }"
5:
6:end


this works for *any* enumerable thing you want to process with 'n'  
backend threads.

the current (0.5.0) alib version will blow up if you give it an IO  
object though, as it uses #size to calculate the return value.  i'll  
tweak it and release 0.5.1 today.

cheers.

a @ http://codeforpeople.com/
--
it is not enough to be compassionate.  you must act.
h.h. the 14th dalai lama