From: pdahorek@... Date: 2018-08-05T17:40:43+00:00 Subject: [ruby-core:88298] [Ruby trunk Bug#13167] Dir.glob is 25x slower since Ruby 2.2 Issue #13167 has been updated by ahorek (Pavel Rosick��). @h.shirosaki, thanks for your work on this. I tested your patch 0001-dir.c-performance-fix-with-braces.patch (ruby head + braces) based on the current trunk https://github.com/ruby/ruby/pull/1864 environment: ``` Samsung 850 Pro 250GB AMD 8350FX 8C Windows 10 and Ubuntu 16GB DDR3 ruby 2.6.0dev (2018-08-05 trunk 64192) [x86_64-linux] jruby 9.2.1.0-SNAPSHOT (2.5.0) 2018-08-02 5aa064b Java HotSpot(TM) 64-Bit Server VM 10.0.1+10 on 10.0.1+10 +jit [linux-x86_64] ``` ratio (faster than trunk) ``` linux braces 1.26x linux recursive 0.99x windows braces 10.75x windows recursive 1.66x ``` I think the patch fixes the main problem I originaly reported. Especially "windows braces" is almost 11-times faster, almost as fast as ruby 2.1.9 was. I also tested it with my rspec suite and it runs 2.14x faster, this is a huge perf difference. It passes all tests. ``` ruby trunk 22 minutes 46 seconds ruby trunk + patch 10 minutes 5 seconds ``` cc @nobu if you have time, could you please review it? ``` Linux ruby 2.1.9 list 12.627k (�� 1.6%) i/s - 63.232k in 5.008885s braces 4.332k (�� 1.9%) i/s - 21.889k in 5.054435s recursive 81.603 (�� 1.2%) i/s - 413.000 in 5.062313s ruby 2.5.0 list 11.752k (�� 1.3%) i/s - 59.176k in 5.036229s braces 4.305k (�� 2.0%) i/s - 21.600k in 5.019530s recursive 248.731 (�� 1.6%) i/s - 1.248k in 5.018503s ruby head list 12.128k (�� 2.4%) i/s - 60.840k in 5.019484s braces 4.667k (�� 3.1%) i/s - 23.613k in 5.064703s recursive 254.704 (�� 2.0%) i/s - 1.275k in 5.007455s ruby head + braces list 12.123k (�� 3.3%) i/s - 61.048k in 5.041848s braces 5.885k (�� 2.2%) i/s - 29.784k in 5.063815s recursive 251.895 (�� 2.0%) i/s - 1.275k in 5.063459s jruby-head list 9.931k (�� 2.4%) i/s - 49.764k in 5.014070s braces 4.758k (�� 1.7%) i/s - 23.940k in 5.032956s recursive 35.933 (�� 5.6%) i/s - 180.000 in 5.022796s Windows ruby 2.1.9 list 2.683k (�� 5.9%) i/s - 13.566k in 5.077196s braces 1.200k (�� 3.2%) i/s - 6.000k in 5.005971s recursive 111.844 (�� 0.9%) i/s - 561.000 in 5.016557s ruby 2.5.0 list 945.309 (�� 3.0%) i/s - 4.794k in 5.076069s braces 67.879 (�� 2.9%) i/s - 342.000 in 5.041694s recursive 33.314 (�� 3.0%) i/s - 168.000 in 5.046526s ruby head list 1.001k (�� 1.8%) i/s - 5.049k in 5.047494s braces 72.145 (�� 1.4%) i/s - 364.000 in 5.046341s recursive 34.943 (�� 2.9%) i/s - 177.000 in 5.068275s ruby head + braces list 1.001k (�� 1.3%) i/s - 5.049k in 5.044865s braces 773.822 (�� 0.9%) i/s - 3.927k in 5.075205s recursive 58.596 (�� 1.7%) i/s - 295.000 in 5.034900s jruby-head list 5.121k (? 1.3%) i/s - 25.935k in 5.064926s braces 1.308k (? 2.1%) i/s - 6.625k in 5.066130s recursive 9.987 (? 0.0%) i/s - 50.000 in 5.008338s ``` ![](linux_braces.png) ![](windows_braces.png) ![](linux_recursive.png) ![](windows_recursive.png) ---------------------------------------- Bug #13167: Dir.glob is 25x slower since Ruby 2.2 https://bugs.ruby-lang.org/issues/13167#change-73326 * Author: ahorek (Pavel Rosick��) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: 2.4.0 * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- Hello, we've found a huge speed regression in our Rails app. After some digging the reason is in `Dir.glob` method which is much slower since Ruby 2.2.6. This is probably Windows only! This code is used heavily in Rails for partial lookups: ```ruby Dir.glob('c:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}') ``` Comparsion (x64): jruby 9.1.7.0 2540 i/s ruby 2.1.5 2568 i/s ruby 2.1.9 2569 i/s **ruby 2.2.6 99 i/s 25 times slower! ruby 2.3.3 102 i/s ruby 2.4.0 103 i/s** I would like to help, but I don't know much about Ruby C internals. Please let me know if you need any additional info. Now we're stuck at 2.1.9 because this issue makes the development on more recent versions unusable. ---Files-------------------------------- logruby24.txt (484 KB) logruby21.txt (10.8 KB) bench_dir_glob.rb (880 Bytes) 0001-dir.c-performance-fix-with-braces-using-cache.patch (5.84 KB) 0001-dir.c-performance-fix-with-braces.patch (8.64 KB) linux_braces.png (24.1 KB) linux_list.png (23.5 KB) linux_recursive.png (26.6 KB) windows_braces.png (23.6 KB) windows_list.png (23.1 KB) windows_recursive.png (28 KB) bench_dir_glob2.rb (982 Bytes) -- https://bugs.ruby-lang.org/ Unsubscribe: