From: John Stoffel Date: 2009-02-19T02:39:49+09:00 Subject: Quieting DRb messages in fork() code Hi, I've got some code which seems to work well, but the warning messages are driving me insane! I can't use this code if it's always going to spew warnings like these on a CentOS 5.2 system with ruby 1.8.5 (2006-08-25) [i386-linux] as the environment. I'm also using slave-1.2.1 from http://www.codeforpeople.com as well. $ ./readdir-slave-only.rb --kids 4 tmp Starting Slave Counter Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_counter_537520870_11835_11836_0_0.697410256497375 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_counter_537520870_11835_11836_0_0.697410256497375 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_counter_537520870_11835_11836_0_0.697410256497375 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_counter_537520870_11835_11836_0_0.697410256497375 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_counter_537520870_11835_11836_0_0.697410256497375 counter.count=0 counter.max=4 @count incremented to 1 Threaded! Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_proc_537509010_11835_11839_0_0.449993964192234 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_proc_537509010_11835_11839_0_0.449993964192234 Exception `DRb::DRbBadScheme' at /usr/lib/ruby/1.8/drb/drb.rb:814 - drbunix:///tmp/slave_proc_537509010_11835_11839_0_0.449993964192234 #kids = 1 Reaped a kid... @count incremented to 2 I've tried to capture these exceptions using 'begin ... rescue ... end' blocks, but I'm obviously not doing something right. What's frustrating is that when I pull the code out to just use 'sleep' calls, it seems to work just fine, without all the warnings. I know my code is quite long, but even so it's not complete since I need to insert my results into a DB at some point as well, so just getting the basics working will be a big help! Here's my code: #!/usr/bin/ruby require 'getoptlong' #require 'thread' require 'slave' $VERSION = "v1.1"; $max_slaves = 1 $count = 50 MYURI = "drbunix:/tmp/stoffj-test_" + Process.pid.to_s opts = GetoptLong.new( [ "--help", "-h", GetoptLong::NO_ARGUMENT], [ "--kids", "-k", GetoptLong::REQUIRED_ARGUMENT] ) #--------------------------------------------------------------------- class Counter def initialize(max) @slaves = [] @max = max @count = 0 @count_mutex = Mutex.new end def count @count end def max @max end # Increment the count of slaves, returning 1 if incremented, 0 if not. def increment ok = nil @count_mutex.synchronize do if (@count < @max) then @count += 1 puts "@count incremented to #{@count}" ok = 1 end end ok end # Decrement the count of slaves def decrement @count_mutex.synchronize do if (@count > 1) then @count -= 1 puts "@count decremented to #{@count}" end end @count end end #--------------------------------------------------------------------- class ReadDir def initialize(server) @server = server end def readdir(dir) #puts "readdir(#{dir})" #sleep 1 size_file = {} size_dir = {} size_total = 0 kids = [] # Traverse the directory and collect the size of all files and # directories begin Dir.foreach(dir) do |f| #print " #{f}," if(f != "." && f != "..") then f_full = addpath(dir, f) stat = File.lstat(f_full) if(!stat.symlink?) then if(stat.file?) then #puts " File: #{f}" size = File.size(f_full) size_file[f] = size size_total += size end if(stat.directory?) then #puts "DIR= #{f}" if (@server.max < 2) then #puts " no threads." size = readdir(f_full) if (size > 0) then size_dir[f] = size size_total += size end else ok = @server.increment if (ok) puts " Threaded!" kids << Slave.object(:async => true, :threadsage => true) { size = readdir(f_full) @server.decrement #puts "size = #{size}" # return the size from the slave properly size } puts "#kids = #{kids.length}" else size = readdir(f_full) if(size > 0) then size_dir[f] = size size_total += size end end end end end end end end kids.each { |kid| puts "Reaped a kid..." size_total += kid.value @server.decrement } #Puts "Dir: #{dir} = #{size_total}" return size_total end end #--------------------------------------------------------------------- # Read a directory and add to the database; this function is recursive # for sub-directories #--------------------------------------------------------------------- def usage puts puts "usage: readdir-drb [--kids NUM] " puts " defaults to #{$max_kids} children" puts puts " version: #{$version}" puts end #--------------------------------------------------------------------- def addpath(a, b) return a + b if(a =~ /\/$/) return a + "/" + b end #--------------------------------------------------------------------- # Main #--------------------------------------------------------------------- $DEBUG = true opts.each do |opt,arg| case opt when "--kids" $max_slaves = arg.to_i else usage exit end end if ARGV.length != 1 puts "Missing dir argument (try --help)" exit 0 end dir = ARGV.shift # Start the Slave for Counting... puts "Starting Slave Counter" slave = Slave.new :object => Counter.new($max_slaves), :async => true counter = slave.object puts "counter.count=#{counter.count}" puts "counter.max=#{counter.max}" puts # Fire up a new Kid Class readdir. kid = ReadDir.new(counter) # Now let's try to do a recursive readdir() algorith with threads. size = kid.readdir(dir) sizekb = size / 1024; sizemb = sizekb / 1024; sizegb = sizemb / 1024; puts "" printf "Total size: %d Bytes\n", size printf "Total size: %.2f Kb\n", sizekb printf "Total size: %.2f Mb\n", sizemb if (sizegb > 1.5) printf "Total size: %.2f Gb\n", sizegb end -- Posted via http://www.ruby-forum.com/.