From: Nabs Kahn <nabusman@...>
Date: 2009-05-28T01:03:35+09:00
Subject: Re: Newbie on Threads

This is what I ended up doing, similar to what was suggested. 
(definition of screenScrape method not included)

bufferSize = 10
buffer = SizedQueue.new(bufferSize)
threads = []

producer = Thread.new do
  File.open("urls.txt").each do |url|
    buffer.enq url
  end
  bufferSize.times {buffer.enq(:END_OF_WORK)}
end

bufferSize.times do
  threads << Thread.new do
    url = nil
    while(url != :END_OF_WORK)
      url = buffer.deq
      screenScrape(url)
    end
  end
end

producer.join
threads.each do |thr|
  thr.join
end

Thomas B. wrote:
> Nabs Kahn wrote:
>> require 'thread'
>> 
>> buffer = SizedQueue.new(10)
>> 
>> producer = Thread.new do
>>   File.open("urls.txt").each do |url|
>>     buffer << url
>>   end
>> end
>> 
>> consumer = Thread.new do
>>   while buffer.num_waiting != 0
>>     url = buffer.pop
>>     #do screen scraping with url here
>>   end
>> end
>> 
>> consumer.join
>> 
> 
> Hello. You said that you want X threads, but in your example you have 
> only one scraping thread. I think this is more what you intended (not 
> tested):
> 
> require 'thread'
> 
> buffer = SizedQueue.new(10)
> 
> producer = Thread.new do
>   File.open("urls.txt").each do |url|
>     buffer << url
>   end
> end
> 
> consumers = Array::new(x){
> Thread.new do
>   while url=buffer.deq
>     #do screen scraping with url here
>   end
> end
> }
> 
> Now if you want the program to stop after the producer finishes, you 
> should add
> 
> producer.join
> consumers.each{|c| c.join}
> 
> to make sure all processing is finished.

-- 
Posted via http://www.ruby-forum.com/.