From: ara.t.howard@...
Date: 2006-07-10T23:10:22+09:00
Subject: Re: fastest way of looping through directories

On Mon, 10 Jul 2006, Thomas Coopman wrote:

> Hi,
>
> I'm looking for the fastest way for looping through directories.
>
> at the moment i'm using Find but when I test this, the first time it runs
> really slow (like 20 seconds) the second time it runs much much quicker
> (less than a second).  The problem is that I don't need to run this twice
> but only once.  Is this because I use the unit tests that it runs slow or is
> it something else.

it's because the underlying filesystem does agressive caching - i'm assuming
you're on linux?  regardless, it has nothing to do with ruby at all.

> I only need the path of all the files in the directory + subdirectories.
> What would be the fastest for this?  using Dir.foreach or Find or something
> else

this will be the fastest way

   Dir.glob('**/**') do |entry|
     p entry
   end

it use it on directories with 200k files all the time.  basically you want to
avoid

   list = Dir.glob '**/**'  # creates huge array first

   list = Find.find d       # creates huge array first

etc.  eg.  avoid methods which scour the whole directory first and use methods
which provide at iterator to handle each file as it's encountered.  probably
you'll find this much faster or, at least, more responsive.

regards.

-a
-- 
suffering increases your inner strength.  also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama