From: Robert Klemme <bob.news@...>
Date: 2006-03-19T21:38:48+09:00
Subject: Re: safe way to calc md5 on very large files

Andrew Johnson <ajohnson@cpan.org> wrote:
> On Sun, 19 Mar 2006 13:49:51 +0900, ara.t.howard@noaa.gov
> <ara.t.howard@noaa.gov> wrote:
>> On Sun, 19 Mar 2006, Stephen Waits wrote:
>>>
>>> Close.. try this..
>>>
>>>    require 'md5'
>>>    File.open(filename,'rb') { |f| MD5.hexdigest(f.read) }
>>>
>>> And yes, the file is closed with the block form of open.
>>>
>>> --Steve
>>
>> i think the OP has the right approach - note that an 'f.read' will
>> consume 2GB.  but the OP's code
>>
>>    harp:~ > cat a.rb
>>    require 'digest/md5'
>>    md5 = Digest::MD5.new() and open(ARGV.shift, 'rb').each{|line|
>>    md5 << line} p md5.hexdigest
>>
>> will not.
>
>
> In my reading of the OP, both the block-open and iteration are
> actually desired:
>
>  md5 = Digest::MD5.new
>  File.open(file,'rb') do |ios|
>    ios.each {|line| md5 << line }
>  end

IMHO it's a bad idea to use line oriented reading on a binary file because 
"lines" can be arbitrary long (i.e. the whole file in worst case).  Using 
IO#read is much better.

Kind regards

    robert