Re: making FileUtils.rm_rf robust: is anyone interested?

"Nobuyoshi Nakada" <[e-mail]> wrote:
> At Thu, 5 Oct 2006 01:25:00 +0900,
> Jim Meyering wrote in [ruby-core:08999]:
>> It is not at all trivial to fix this "properly".
>> By "properly," I mean in a way that rm_rf can remove an arbitrarily
>> deep hierarchy securely while remaining efficient and thread safe.
>> Modulo hard-coded diagnostics, the C implementation in the GNU coreutils
>> package (src/remove.c) should be appropriate.
>
> It doesn't feel appropriate to chdir inside a library since it affects
> whole process.

Hello,

You are right that calling chdir (or fchdir) is not appropriate in
a library:  it would render the caller thread-*un*safe.

However, given sufficient O/S support, the implementation in
coreutils/src/remove.c is indeed robust and thread-safe.  As of
coreutils-6.0 (the latest is coreutils-6.3), "rm -r" can remove an
arbitrarily deep hierarchy in a thread-safe manner on a system with
support for openat-like functions (Linux-2.6.16 and newer and Solaris 10).
I have taken great pains to ensure that the code degrades gracefully,
so that it works as well (but sacrifices thread safety) on systems that
have neither openat nor sufficient /proc support.  If you require
thread safety even without openat support, then currently you must
compromise on robustness: i.e., the code must once again be subject
to the PATH_MAX limitation.

However a robust, efficient, *and* always-thread-safe implementation
is possible: if the PATH_MAX limitation is encountered, incur the cost
of a single fork and then perform the remaining operations (including
f/chdir calls) from a separate process.

If you are interested, a viable alternative may involve using the fts
implementation from the coreutils (the same one that's in gnulib).
Then, once that version of fts has the proposed additional feature,
ruby's rm_rf will "just work".

FYI, fts is the file system traversing tool that is used by chmod,
chgrp, chown, and du.  It too takes advantage of openat, when possible,
and degrades gracefully.  However, it also has an option to make it use
the existing approach of accessing each operand via its full, relative
file name.  The version if coreutils/gnulib was initially based on
the one from *BSD and glibc, but I have changed its ABI slightly in order
to make it work for arbitrarily deep hierarchies.  For example, those
programs can now process hierarchies a million levels deep or more.

Jim