[ruby-core:88298] [Ruby trunk Bug#13167] Dir.glob is 25x slower since Ruby 2.2

Issue #13167 has been updated by ahorek (Pavel Rosick箪).


@h.shirosaki, thanks for your work on this. I tested your patch 0001-dir.c-performance-fix-with-braces.patch (ruby head + braces) based on the current trunk https://github.com/ruby/ruby/pull/1864

environment:
```
Samsung 850 Pro 250GB
AMD 8350FX 8C
Windows 10 and Ubuntu
16GB DDR3

ruby 2.6.0dev (2018-08-05 trunk 64192) [x86_64-linux]
jruby 9.2.1.0-SNAPSHOT (2.5.0) 2018-08-02 5aa064b Java HotSpot(TM) 64-Bit Server VM 10.0.1+10 on 10.0.1+10 +jit [linux-x86_64]
```

ratio (faster than trunk)
```
linux braces 1.26x
linux recursive 0.99x
windows braces 10.75x
windows recursive 1.66x
```
I think the patch fixes the main problem I originaly reported. Especially "windows braces" is almost 11-times faster, almost as fast as ruby 2.1.9 was.

I also tested it with my rspec suite and it runs 2.14x faster, this is a huge perf difference. It passes all tests.
```
ruby trunk
22 minutes 46 seconds
ruby trunk + patch
10 minutes 5 seconds
```

cc @nobu if you have time, could you please review it?

```
Linux
ruby 2.1.9
                list     12.627k (賊 1.6%) i/s -     63.232k in   5.008885s
              braces      4.332k (賊 1.9%) i/s -     21.889k in   5.054435s
           recursive     81.603  (賊 1.2%) i/s -    413.000  in   5.062313s

ruby 2.5.0
                list     11.752k (賊 1.3%) i/s -     59.176k in   5.036229s
              braces      4.305k (賊 2.0%) i/s -     21.600k in   5.019530s
           recursive    248.731  (賊 1.6%) i/s -      1.248k in   5.018503s

ruby head
                list     12.128k (賊 2.4%) i/s -     60.840k in   5.019484s
              braces      4.667k (賊 3.1%) i/s -     23.613k in   5.064703s
           recursive    254.704  (賊 2.0%) i/s -      1.275k in   5.007455s

ruby head + braces
                list     12.123k (賊 3.3%) i/s -     61.048k in   5.041848s
              braces      5.885k (賊 2.2%) i/s -     29.784k in   5.063815s
           recursive    251.895  (賊 2.0%) i/s -      1.275k in   5.063459s

jruby-head
                list      9.931k (賊 2.4%) i/s -     49.764k in   5.014070s
              braces      4.758k (賊 1.7%) i/s -     23.940k in   5.032956s
           recursive     35.933  (賊 5.6%) i/s -    180.000  in   5.022796s



Windows
ruby 2.1.9
                list      2.683k (賊 5.9%) i/s -     13.566k in   5.077196s
              braces      1.200k (賊 3.2%) i/s -      6.000k in   5.005971s
           recursive    111.844  (賊 0.9%) i/s -    561.000  in   5.016557s

ruby 2.5.0
                list    945.309  (賊 3.0%) i/s -      4.794k in   5.076069s
              braces     67.879  (賊 2.9%) i/s -    342.000  in   5.041694s
           recursive     33.314  (賊 3.0%) i/s -    168.000  in   5.046526s

ruby head
                list      1.001k (賊 1.8%) i/s -      5.049k in   5.047494s
              braces     72.145  (賊 1.4%) i/s -    364.000  in   5.046341s
           recursive     34.943  (賊 2.9%) i/s -    177.000  in   5.068275s

ruby head + braces
                list      1.001k (賊 1.3%) i/s -      5.049k in   5.044865s
              braces    773.822  (賊 0.9%) i/s -      3.927k in   5.075205s
           recursive     58.596  (賊 1.7%) i/s -    295.000  in   5.034900s

jruby-head
                list      5.121k (? 1.3%) i/s -     25.935k in   5.064926s
              braces      1.308k (? 2.1%) i/s -      6.625k in   5.066130s
           recursive      9.987  (? 0.0%) i/s -     50.000  in   5.008338s
```


![](linux_braces.png)
![](windows_braces.png)
![](linux_recursive.png)
![](windows_recursive.png)

----------------------------------------
Bug #13167: Dir.glob is 25x slower since Ruby 2.2
https://bugs.ruby-lang.org/issues/13167#change-73326

* Author: ahorek (Pavel Rosick箪)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.4.0
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
Hello,
we've found a huge speed regression in our Rails app. After some digging the reason is in `Dir.glob` method which is much slower since Ruby 2.2.6. This is probably Windows only!

This code is used heavily in Rails for partial lookups:

```ruby
Dir.glob('c:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}')
```

Comparsion (x64):
jruby 9.1.7.0 2540 i/s
ruby 2.1.5 2568 i/s
ruby 2.1.9 2569 i/s
**ruby 2.2.6 99 i/s 25 times slower!
ruby 2.3.3 102 i/s
ruby 2.4.0 103 i/s**

I would like to help, but I don't know much about Ruby C internals. Please let me know if you need any additional info. Now we're stuck at 2.1.9 because this issue makes the development on more recent versions unusable.

---Files--------------------------------
logruby24.txt (484 KB)
logruby21.txt (10.8 KB)
bench_dir_glob.rb (880 Bytes)
0001-dir.c-performance-fix-with-braces-using-cache.patch (5.84 KB)
0001-dir.c-performance-fix-with-braces.patch (8.64 KB)
linux_braces.png (24.1 KB)
linux_list.png (23.5 KB)
linux_recursive.png (26.6 KB)
windows_braces.png (23.6 KB)
windows_list.png (23.1 KB)
windows_recursive.png (28 KB)
bench_dir_glob2.rb (982 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>