From: Philipp Huber <huber.philipp@...>
Date: 2005-09-07T00:26:50+09:00
Subject: Re: Sorted arrays

should your list be sorted in the end? if not, you could simply use
Hash instead of Array.

On 9/6/05, ruby@danb64.com <ruby@danb64.com> wrote:
> I'm a relative newcomer to Ruby.  Most of my experience is in Delphi.  And in Delphi one of the most commonly-used classes is TStringList, which is sort of analogous to ruby's Array (Delphi also has dynamic arrays and static arrays).  TStringList has a property called Sorted, which if set to True makes it possible to insert strings into the list and have it maintain them as a sorted list (without having to re-sort it each time).  Then you can use the IndexOf method (or the Find method) to do a binary search on the list, so you can quickly find the element you're looking for.  My question is whether Ruby has anything like this.  It seems like one could create a descendant of Array that does this.
> 
> What I'm trying to accomplish is this:  I am processing a large number of items (almost 100,000 rows) of data and trying to find the duplicate items.  I create an MD5 hash based on all of the elements within each row.  Then I check to see if the MD5 value already exists in the list, and if it does, I know the item is a duplicate.  If not, then I add it to the list.  Very few of them items are duplicated, so most of the time it will be trying to locate the value in the list, and then immediately after that inserting the same value into the list.  So there will be a lot of searching and a lot of inserting going on.  For that reason, it won't be acceptable to re-sort it each time I insert an item.
> 
> Thanks in advance,
> 
> Dan
> 
>