[ruby-core:82749] [Ruby trunk Bug#13167] Dir.glob is 25x slower since Ruby 2.2

Issue #13167 has been updated by h.shirosaki (Hiroshi Shirosaki).

File 0001-dir.c-performance-fix-with-braces.patch added
File bench_dir_glob.rb added
File 0001-dir.c-performance-fix-with-braces-using-cache.patch added

replace_real_basename() is called for same head plain paths because braces are expanded early before ruby_glob0().

Moving braces expansion to later phase in glob_helper() is a way to reduce replace_real_basename().
The idea is same as #13873.

Another idea is caching real name of each directory and use the cache.

I attached a patch and benchmark script.

Here is my benchmark result.

\+ patch: 0001-dir.c-performance-fix-with-braces.patch
\+ cache: 0001-dir.c-performance-fix-with-braces-using-cache.patch

braces:
Dir["v:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}"]

recursive:
Dir["v:/test/myapp/app/views/**/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}"]

On Windows 10
ruby 2.5.0dev (2017-09-11 trunk 59831) [x64-mingw32]

~~~
              braces    148.111  (賊 3.4%) i/s -    742.000  in   5.015963s
+ patch       braces      1.809k (賊 3.8%) i/s -      9.078k in   5.027256s  => 12x faster
+ cache       braces    480.215  (賊 5.4%) i/s -      2.397k in   5.005954s  =>  3x faster

           recursive     71.280  (賊 4.2%) i/s -    357.000  in   5.014841s
+ patch    recursive    111.464  (賊 2.7%) i/s -    561.000  in   5.037387s
+ cache    recursive     94.775  (賊 3.2%) i/s -    477.000  in   5.037445s
~~~

On Linux(Ubuntu 16.04)
ruby 2.5.0dev (2017-09-11 trunk 59831) [x86_64-linux]

~~~
              braces      6.171k (賊 1.0%) i/s -     31.408k in   5.090401s
+ patch       braces     11.241k (賊 0.6%) i/s -     57.252k in   5.093467s

           recursive    720.448  (賊 1.4%) i/s -      3.640k in   5.053382s
+ patch    recursive    730.159  (賊 0.5%) i/s -      3.723k in   5.099068s
~~~

----------------------------------------
Bug #13167: Dir.glob is 25x slower since Ruby 2.2
https://bugs.ruby-lang.org/issues/13167#change-66606

* Author: ahorek (Pavel Rosick箪)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.4.0
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
Hello,
we've found a huge speed regression in our Rails app. After some digging the reason is in `Dir.glob` method which is much slower since Ruby 2.2.6. This is probably Windows only!

This code is used heavily in Rails for partial lookups:

```ruby
Dir.glob('c:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}')
```

Comparsion (x64):
jruby 9.1.7.0 2540 i/s
ruby 2.1.5 2568 i/s
ruby 2.1.9 2569 i/s
**ruby 2.2.6 99 i/s 25 times slower!
ruby 2.3.3 102 i/s
ruby 2.4.0 103 i/s**

I would like to help, but I don't know much about Ruby C internals. Please let me know if you need any additional info. Now we're stuck at 2.1.9 because this issue makes the development on more recent versions unusable.

---Files--------------------------------
logruby24.txt (484 KB)
logruby21.txt (10.8 KB)
0001-dir.c-performance-fix-with-braces.patch (8.64 KB)
bench_dir_glob.rb (880 Bytes)
0001-dir.c-performance-fix-with-braces-using-cache.patch (5.84 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>