From: Masatoshi SEKI <m_seki@...>
Date: 2004-06-06T11:52:53+09:00
Subject: Re: Transfering files using drb (distributed Ruby)


On 2004/06/05, at 22:53, Francis Joanis wrote:

> On Sat, 5 Jun 2004 05:46:04 +0900, Joel VanderWerf  
> <vjoel@PATH.Berkeley.EDU> wrote:
>
>> Kirk Haines wrote:
>>> On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote
>>>
>>>> Hello everyone,
>>>>
>>>> I'd like to know if it is possible to use drb to transfer files  
>>>> from the server to the client (and vice versa)?
>>>>
>>>> For example, a remotely callable method could return a byte array  
>>>> then, client side, that array could be written back on disk.
>>>>
>>>> If this is possible, how would the drb infrastructure "behave" when  
>>>> transfering rather big arrays (say, 5~10 megabytes)?
>>>
>>>
>>> It's definitely possible.  As to have the infrastructure behaves,  
>>> you will have processing costs because the data has to be marshalled  
>>> for transfer and then unmarshalled at the other end.  I regularly  
>>> pump hundreds of k per second of marshalled data around though, with  
>>> no problems.
>>>
>>> I'd say just try it.  DRb is extremely simple to use.  Put a  
>>> prototype together and test it out!
>>
>> And if the file just gets slurped into a string, the marshalling cost  
>> will be next to nothing.
>>
>> irb(main):002:0> str = (1..9).entries.to_s*10
>> =>  
>> "123456789123456789123456789123456789123456789123456789123456789123456 
>> 789123456789123456789"
>> irb(main):003:0> Marshal.dump(str)
>> =>  
>> "\004\010\"_1234567891234567891234567891234567891234567891234567891234 
>> 56789123456789123456789123456789"
>>
>>
>
> Thanks guys, I'm going to try it out and give you some news.

I wrote a small sample code.

# setup server
host1% ruby dcp.rb
druby://host1:12345

# and client
host2% irb -r dcp.rb
irb(main):001:0> DRb.start_service
irb(main):002:0> there = DRbObject.new_with_uri('druby://host1:12345')
irb(main):003:0> here = DCP::Site.new
# transfer a file
irb(main):004:0> here.store_from('filename.jpg', there)
# with progress(?)
irb(main):005:0> here.store_from('filename.jpg', there) {|x| p x}

## dcp.rb
require 'drb/drb'

module DCP
   class Site
     include DRbUndumped

     def size(fname)
       File.lstat(fname).size
     end

     def fetch(fname)
       File.open(fname, 'rb') do |fp|
         while buf = fp.read(4096)
           yield(buf)
         end
       end
       nil
     end

     def store_from(fname, there)
       size = there.size(fname)
       wrote = 0

       File.rename(fname, fname + '.bak') if File.exists? fname
       File.open(fname, 'wb') do |fp|
         yield([wrote, size]) if block_given?
         there.fetch(fname) do |buf|
           wrote += fp.write(buf)
           yield([wrote, size]) if block_given?
           nil
         end
       end

       wrote
     end
   end
end

if __FILE__ == $0
   DRb.start_service(ARGV.shift, DCP::Site.new)
   puts DRb.uri
   DRb.thread.join
end