From: Tom Copeland <tom@...>
Date: 2006-04-28T23:37:51+09:00
Subject: Re: Ruby Drops

On Fri, 2006-04-28 at 23:30 +0900, Jake McArthur wrote:
> > How will it find similar code? One simple issue is that people will  
> > name their variables and methods differently, so you'll want to  
> > somehow see the structure of a section of code and ignore a lot of  
> > details. But you can't ignore the details too much. Maybe (trivial  
> > example) someone wrote a "max" function and someone else in-lined  
> > it, and otherwise their code blocks are the same.
> 
> I've already been working on this. Right now, I'm making a simple  
> algorithm that works on arbitrary text and returns a number  
> reflecting how similar two strings are. Even this alone has been  
> giving fairly good results on code, even code that was written rather  
> differently, but my plan is to use this algorithm to compare symbols  
> and literals. A similar algorithm, working on a slightly larger  
> scale, would compare entire lines of code for similar syntax,  
> augmented by data from the first algorithm.

CPD uses the Burrows-Wheeler transform to find exact matches.  It has
some options to ignore identifiers and literals, although that results
in false positives sometimes...

Yours,

Tom