From: esparta@... Date: 2017-11-15T06:32:50+00:00 Subject: [ruby-core:83777] [Ruby trunk Feature#14109] FileUtils: Use Dir.children instead of Dir.entries Issue #14109 has been reported by esparta (Espartaco Palma). ---------------------------------------- Feature #14109: FileUtils: Use Dir.children instead of Dir.entries https://bugs.ruby-lang.org/issues/14109 * Author: esparta (Espartaco Palma) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- Dir.children is available since Feature #11302. FileUtils uses Dir.each on an internal method encapsulated on a private class Entry_#entry, having no '.' neither '..' entries would make now superfluous a chained reject filtering. This change can improve the performance of these FileUtils methods when the provided path covers thousands of files or directories: * chmod_R * chown_R * remove_entry * remove_entry_secure * rm_r * remove_dir * copy_entry Related: Feature #13896 Some profiling I did using 50,000 files on a given folder, using this code: ~~~ ruby require_relative 'fileutils' FileUtils.chmod_R 0777, 'benchmark' ~~~ Before the patch: ~~~ text 28.69 15.05 15.05 200004 0.08 0.10 FileUtils::Entry_#join 10.68 20.65 5.60 100005 0.06 0.26 FileUtils::Entry_#entries 6.75 24.19 3.54 100004 0.04 0.11 FileUtils::Entry_#lstat 5.68 27.17 2.98 150007 0.02 0.13 FileUtils::Entry_#path 4.97 29.78 2.61 50002 0.05 0.44 FileUtils::Entry_#chmod 4.97 29.78 2.61 50002 0.05 0.44 FileUtils::Entry_#chmod 4.80 32.29 2.52 50004 0.05 2.73 FileUtils.chmod_R 4.33 34.56 2.27 1 2268.74 52382.08 FileUtils::Entry_#preorder_traverse 4.18 36.76 2.19 450012 0.00 0.00 String#== 3.73 38.71 1.96 100004 0.02 0.13 FileUtils::Entry_#lstat! 3.56 40.58 1.87 400008 0.00 0.00 BasicObject#! 3.35 42.33 1.76 50002 0.04 0.08 FileUtils::Entry_#directory? 3.27 44.05 1.72 50002 0.03 0.25 FileUtils::Entry_#symlink? 2.14 45.17 1.12 150003 0.01 0.01 File.join 2.01 46.23 1.05 50002 0.02 0.03 Class#new 1.89 47.22 0.99 50002 0.02 0.02 FileUtils.fu_mode 1.47 47.99 0.77 4 192.12 2755.69 Array#map 1.40 48.72 0.73 1 733.87 2821.06 Array#reject 0.92 49.20 0.48 50002 0.01 0.01 File.lstat 0.85 49.65 0.44 100004 0.00 0.00 FileUtils::Entry_#dereference? 0.82 50.07 0.43 50002 0.01 0.01 File.chmod 0.57 50.37 0.30 50006 0.01 0.01 File.path 0.56 50.67 0.29 50002 0.01 0.01 FileUtils::Entry_#initialize 0.49 50.92 0.26 50002 0.01 0.01 File::Stat#directory? 0.49 51.18 0.25 50002 0.01 0.01 File::Stat#symlink? 0.47 51.43 0.25 50003 0.00 0.00 Array#pop 0.47 51.67 0.24 50001 0.00 0.00 FileUtils::Entry_#rel 0.46 51.91 0.24 50001 0.00 0.00 Kernel#untaint 0.43 52.14 0.23 50002 0.00 0.00 Kernel#is_a? 0.43 52.36 0.23 50001 0.00 0.00 FileUtils::Entry_#prefix 0.04 52.38 0.02 1 18.88 18.98 Dir.entries 0.03 52.40 0.01 1 13.28 59.69 Kernel#require_relative 0.02 52.41 0.01 320 0.04 0.09 nil# 0.01 52.41 0.01 86 0.08 0.08 Module#module_eval 0.01 52.42 0.01 180 0.03 0.10 FileUtils.collect_method 0.01 52.42 0.00 4 0.67 2.51 Array#select 0.01 52.42 0.00 44 0.06 0.15 Array#map! 0.01 52.43 0.00 3 0.88 1.08 Module#public 0.00 52.43 0.00 2 1.12 2.61 Kernel#require 0.00 52.43 0.00 5 0.39 10481.37 Array#each 0.00 52.43 0.00 262 0.01 0.01 Module#method_added 0.00 52.43 0.00 61 0.02 0.03 Module#module_function 0.00 52.44 0.00 226 0.00 0.00 BasicObject#singleton_method_added 0.00 52.44 0.00 176 0.00 0.00 Hash#[] 0.00 52.44 0.00 176 0.00 0.00 Array#include? 0.00 52.44 0.00 168 0.00 0.00 Symbol#== 0.00 52.44 0.00 17 0.03 0.06 FileUtils.private_module_function 0.00 52.44 0.00 22 0.02 0.03 Module#alias_method 0.00 52.44 0.00 86 0.00 0.00 Integer#+ 0.00 52.44 0.00 1 0.33 0.33 Array#reverse 0.00 52.44 0.00 44 0.01 0.01 UnboundMethod#parameters 0.00 52.44 0.00 44 0.01 0.01 Hash#[]= 0.00 52.44 0.00 7 0.03 0.05 Module#include 0.00 52.44 0.00 44 0.01 0.01 Module#instance_method 0.00 52.44 0.00 44 0.00 0.00 Array#compact! 0.00 52.44 0.00 1 0.19 0.19 Array#concat 0.00 52.44 0.00 44 0.00 0.00 Symbol#to_s 0.00 52.44 0.00 4 0.04 0.07 Kernel#extend 0.00 52.44 0.00 28 0.00 0.00 String#intern 0.00 52.44 0.00 1 0.10 0.10 Dir.open 0.00 52.44 0.00 17 0.00 0.00 Module#private_class_method 0.00 52.44 0.00 4 0.02 0.02 Module#extend_object 0.00 52.44 0.00 7 0.01 0.01 Module#append_features 0.00 52.44 0.00 4 0.01 0.01 Hash#keys 0.00 52.44 0.00 1 0.05 0.07 MonitorMixin#mon_enter 0.00 52.44 0.00 2 0.03 0.06 FileUtils.fu_list 0.00 52.44 0.00 7 0.01 0.01 Module#private 0.00 52.44 0.00 2 0.03 0.03 Kernel#singleton_methods 0.00 52.44 0.00 1 0.05 0.05 Numeric#zero? 0.00 52.44 0.00 1 0.04 12.84 Enumerable#inject 0.00 52.44 0.00 1 0.04 0.04 Module#private_instance_methods 0.00 52.44 0.00 4 0.01 0.01 IO#set_encoding 0.00 52.44 0.00 1 0.03 0.06 MonitorMixin#mon_exit 0.00 52.44 0.00 7 0.00 0.00 Module#included 0.00 52.44 0.00 2 0.01 0.01 Array#- 0.00 52.44 0.00 1 0.03 0.07 Numeric#nonzero? 0.00 52.44 0.00 1 0.02 0.02 Kernel#methods 0.00 52.44 0.00 1 0.02 0.03 FileUtils::StreamUtils_#fu_windows? 0.00 52.44 0.00 4 0.01 0.01 Module#extended 0.00 52.44 0.00 1 0.02 0.02 MonitorMixin#mon_check_owner 0.00 52.44 0.00 1 0.02 0.02 Array#& 0.00 52.44 0.00 3 0.00 0.00 Class#inherited 0.00 52.44 0.00 3 0.00 0.00 Thread.current 0.00 52.44 0.00 1 0.01 0.01 Regexp#=~ 0.00 52.44 0.00 1 0.01 0.01 TracePoint#enable 0.00 52.44 0.00 1 0.01 0.01 Gem::Specification.unresolved_deps 0.00 52.44 0.00 1 0.01 0.01 Array#flatten 0.00 52.44 0.00 1 0.01 0.01 Gem.find_unresolved_default_spec 0.00 52.44 0.00 1 0.01 0.01 Kernel#respond_to? 0.00 52.44 0.00 1 0.01 0.01 TracePoint#disable 0.00 52.44 0.00 1 0.01 0.01 Thread::Mutex#unlock 0.00 52.44 0.00 1 0.00 0.00 Thread::Mutex#lock 0.00 52.44 0.00 1 0.00 52442.37 #toplevel ~~~ After the patch ~~~ text 30.07 14.76 14.76 200004 0.07 0.10 FileUtils::Entry_#join 7.87 18.62 3.86 50002 0.08 0.41 FileUtils::Entry_#entries 7.20 22.15 3.53 100004 0.04 0.11 FileUtils::Entry_#lstat 6.09 25.14 2.99 150007 0.02 0.12 FileUtils::Entry_#path 5.34 27.77 2.62 50002 0.05 0.43 FileUtils::Entry_#chmod 5.12 30.28 2.51 50004 0.05 2.59 FileUtils.chmod_R 4.64 32.56 2.28 1 2279.40 49019.52 FileUtils::Entry_#preorder_traverse 3.98 34.51 1.95 100004 0.02 0.13 FileUtils::Entry_#lstat! 3.72 36.34 1.83 400008 0.00 0.00 BasicObject#! 3.56 38.09 1.75 50002 0.03 0.08 FileUtils::Entry_#directory? 3.54 39.82 1.74 350007 0.00 0.00 String#== 3.47 41.53 1.70 50002 0.03 0.25 FileUtils::Entry_#symlink? 2.29 42.65 1.13 150003 0.01 0.01 File.join 2.05 43.66 1.01 50002 0.02 0.03 Class#new 2.01 44.65 0.99 50002 0.02 0.02 FileUtils.fu_mode 1.51 45.39 0.74 4 184.89 2664.27 Array#map 1.01 45.88 0.50 50002 0.01 0.01 File.lstat 0.90 46.32 0.44 50002 0.01 0.01 File.chmod 0.88 46.75 0.43 100004 0.00 0.00 FileUtils::Entry_#dereference? 0.60 47.05 0.29 50006 0.01 0.01 File.path 0.57 47.33 0.28 50002 0.01 0.01 FileUtils::Entry_#initialize 0.53 47.58 0.26 50002 0.01 0.01 File::Stat#symlink? 0.53 47.84 0.26 50003 0.01 0.01 Array#pop 0.51 48.09 0.25 50002 0.01 0.01 File::Stat#directory? 0.49 48.33 0.24 50001 0.00 0.00 FileUtils::Entry_#rel 0.48 48.57 0.24 50001 0.00 0.00 Kernel#untaint 0.45 48.79 0.22 50002 0.00 0.00 Kernel#is_a? 0.43 49.00 0.21 50001 0.00 0.00 FileUtils::Entry_#prefix 0.04 49.02 0.02 1 18.47 18.49 Dir.children 0.03 49.03 0.01 1 13.31 59.77 Kernel#require_relative 0.02 49.04 0.01 320 0.03 0.08 nil# 0.01 49.05 0.01 86 0.08 0.08 Module#module_eval 0.01 49.06 0.01 180 0.03 0.10 FileUtils.collect_method 0.01 49.06 0.00 2 1.29 3.07 Kernel#require 0.01 49.06 0.00 4 0.64 2.52 Array#select 0.01 49.06 0.00 44 0.06 0.14 Array#map! 0.00 49.07 0.00 3 0.70 1.10 Module#public 0.00 49.07 0.00 5 0.41 9808.82 Array#each 0.00 49.07 0.00 226 0.01 0.01 BasicObject#singleton_method_added 0.00 49.07 0.00 262 0.01 0.01 Module#method_added 0.00 49.07 0.00 61 0.02 0.02 Module#module_function 0.00 49.07 0.00 44 0.02 0.02 Module#instance_method 0.00 49.07 0.00 176 0.00 0.00 Array#include? 0.00 49.07 0.00 176 0.00 0.00 Hash#[] 0.00 49.08 0.00 168 0.00 0.00 Symbol#== 0.00 49.08 0.00 17 0.03 0.06 FileUtils.private_module_function 0.00 49.08 0.00 22 0.02 0.03 Module#alias_method 0.00 49.08 0.00 86 0.00 0.00 Integer#+ 0.00 49.08 0.00 44 0.01 0.01 UnboundMethod#parameters 0.00 49.08 0.00 7 0.03 0.04 Module#include 0.00 49.08 0.00 44 0.01 0.01 Hash#[]= 0.00 49.08 0.00 1 0.20 0.20 Array#reverse 0.00 49.08 0.00 44 0.00 0.00 Array#compact! 0.00 49.08 0.00 44 0.00 0.00 Symbol#to_s 0.00 49.08 0.00 1 0.16 0.16 Array#concat 0.00 49.08 0.00 4 0.04 0.06 Kernel#extend 0.00 49.08 0.00 28 0.00 0.00 String#intern 0.00 49.08 0.00 17 0.00 0.00 Module#private_class_method 0.00 49.08 0.00 2 0.03 0.03 Kernel#singleton_methods 0.00 49.08 0.00 7 0.01 0.01 Module#private 0.00 49.08 0.00 1 0.05 0.07 MonitorMixin#mon_enter 0.00 49.08 0.00 2 0.03 0.05 FileUtils.fu_list 0.00 49.08 0.00 7 0.01 0.01 Module#append_features 0.00 49.08 0.00 4 0.01 0.01 Module#extend_object 0.00 49.08 0.00 1 0.04 0.04 Module#private_instance_methods 0.00 49.08 0.00 4 0.01 0.01 IO#set_encoding 0.00 49.08 0.00 7 0.00 0.00 Module#included 0.00 49.08 0.00 1 0.03 0.06 MonitorMixin#mon_exit 0.00 49.08 0.00 2 0.01 0.01 Array#- 0.00 49.08 0.00 1 0.03 0.03 Kernel#methods 0.00 49.08 0.00 1 0.02 0.03 Numeric#nonzero? 0.00 49.08 0.00 1 0.02 0.03 FileUtils::StreamUtils_#fu_windows? 0.00 49.08 0.00 1 0.02 12.57 Enumerable#inject 0.00 49.08 0.00 1 0.02 0.02 Dir.open 0.00 49.08 0.00 4 0.00 0.00 Module#extended 0.00 49.08 0.00 1 0.02 0.02 MonitorMixin#mon_check_owner 0.00 49.08 0.00 4 0.00 0.00 Hash#keys 0.00 49.08 0.00 1 0.02 0.02 Array#& 0.00 49.08 0.00 3 0.01 0.01 Class#inherited 0.00 49.08 0.00 3 0.00 0.00 Thread.current 0.00 49.08 0.00 1 0.01 0.01 TracePoint#enable 0.00 49.08 0.00 1 0.01 0.01 Regexp#=~ 0.00 49.08 0.00 1 0.01 0.01 TracePoint#disable 0.00 49.08 0.00 1 0.01 0.01 Gem.find_unresolved_default_spec 0.00 49.08 0.00 1 0.01 0.01 Array#flatten 0.00 49.08 0.00 1 0.01 0.01 Gem::Specification.unresolved_deps 0.00 49.08 0.00 1 0.01 0.01 Numeric#zero? 0.00 49.08 0.00 1 0.01 0.01 Thread::Mutex#lock 0.00 49.08 0.00 1 0.01 0.01 Thread::Mutex#unlock 0.00 49.08 0.00 1 0.01 0.01 Kernel#respond_to? 0.00 49.08 0.00 1 0.00 49079.91 #toplevel ~~~ -- https://bugs.ruby-lang.org/ Unsubscribe: