From: Xavier Noria <fxn@...>
Date: 2011-03-08T04:43:11+09:00
Subject: Re: Checking if two files are the same

On Mon, Mar 7, 2011 at 6:38 PM, Colin Bartlett <colinb2r@googlemail.com> wrote:

> I'm interested in any (Ruby) solutions (actual or ideas) for this, as
> I have needed to do it in the past, and want to do something similar
> in the very near future.
>
> For comparing directories where the file names might have changed what
> I've done in the past is to first match on file name, then for the
> unmatching files in each directory see if there are any matches on
> file size, and for those matches either make a direct File.compare (if
> only two files match on a size) or compute checksums and use those to
> exclude definitely unmatching files, and then use File.compare on what
> (if anything) remains matching for that file size and checksum.

I have played with this as an exercise. The idea is to filter
candidates iteratively applying different criteria, from cheap to
expensive, until you arrive at the solution.

It is just a proof of concept in pseudocode, I wrote it off the top of
my head, it does not even run:

    https://gist.github.com/859046

The code above assumes a generic scenario with m-n possible
duplicates, if a particular situation has details that can speed up
the process they should be taken into account of course.