Re: implementing python's os.walk

On Tue, 16 Dec 2008, Brad Volz wrote:

> Hello,
> 
> I seem to be having some difficulty creating a version of python's os.walk()
> for ruby, and I was hoping for some pointers.
> 
> As some background, python's os.walk() [1] is a generator function.  It is

You can have generators in Ruby 
  ri Generator 
furnishes you with the details...

> passed the top of a directory tree and it returns the following for each
> subdirectory that it encounters:
>  . the working directory
>  . an Array of subdirectories
>  . an Array of non-directory files
> 
> Here is some truncated output for my use case:
> 
> > > > import os
> > > > repo = '/usr/local/nfsen/profiles-data/live/lax1er1'
> > > > for root, dirs, files in os.walk(repo):
> ...     if len(files) == 288:
> ...         print root
> ...
> /usr/local/nfsen/profiles-data/live/lax1er1/2008/11/11
> ..
> /usr/local/nfsen/profiles-data/live/lax1er1/2008/12/13
> 
> Essentially, when there are exactly 288 files in a subdirectory, I want to
> print or otherwise do something with the working directory.

OK.
> 
> Here is my attempt at simply translating this library function to ruby:
> 
> #! /usr/bin/env ruby
> 
> require 'pp'
> 
> dirs = [ '/usr/local/nfsen/profiles-data/live/lax1er1' ]
> 
> def find_dirs(top)
>  dirs = []
>  nondirs = []
>  Dir.entries(top).each do |f|
>    next if f =~ /(\.$|\.\.$)/

or maybe 
     next if f =~ /^\.\.?$/
or
     next if f =~ /^\.{1,2}$/
     
>    full_path = [top, f].join('/')

     full_path = File.join(tmp,f)  # separator agnostic

>    if File.directory?(full_path)
>      dirs.push(full_path)
>    else
>      nondirs.push(full_path)
>    end
>  end
> 
>  yield top, dirs, nondirs

yielding to a proc with arity 3....

> 
>  dirs.each do |d|
>    if File.directory?(d)
>      for o in find_dirs(d) { |a,b,c| puts "#{a} #{b} #{c}"}
>        yield o

yielding to a proc with arity 1

That may be one problem

>      end
>    end
>  end
> end
> 
> find_dirs(dirs[0]) do |top,dirs,nondirs|
>  if nondirs.length == 288
>    puts "#{top}"
>  end
> end
> 
> There are some things that I know are wrong or missing, but that is due to
> trying to get something to run at all without throwing an exception.

ri Find is short enough to quote:

------------------------------------------------------------ Class: Find
     The +Find+ module supports the top-down traversal of a set of file
     paths.

     For example, to total the size of all files under your home
     directory, ignoring anything in a "dot" directory (e.g.
     $HOME/.ssh):

       require 'find'
     
       total_size = 0
     
       Find.find(ENV["HOME"]) do |path|
         if FileTest.directory?(path)
           if File.basename(path)[0] == ?.
             Find.prune       # Don't look any further into this directory.
           else
             next
           end
         else
           total_size += FileTest.size(path)
         end
       end

------------------------------------------------------------------------


Instance methods:
-----------------
     find, prune


That will do most of the lifting for you...
        [...] 
        Hugh