From: h.shirosaki@... Date: 2017-09-11T14:28:13+00:00 Subject: [ruby-core:82749] [Ruby trunk Bug#13167] Dir.glob is 25x slower since Ruby 2.2 Issue #13167 has been updated by h.shirosaki (Hiroshi Shirosaki). File 0001-dir.c-performance-fix-with-braces.patch added File bench_dir_glob.rb added File 0001-dir.c-performance-fix-with-braces-using-cache.patch added replace_real_basename() is called for same head plain paths because braces are expanded early before ruby_glob0(). Moving braces expansion to later phase in glob_helper() is a way to reduce replace_real_basename(). The idea is same as #13873. Another idea is caching real name of each directory and use the cache. I attached a patch and benchmark script. Here is my benchmark result. \+ patch: 0001-dir.c-performance-fix-with-braces.patch \+ cache: 0001-dir.c-performance-fix-with-braces-using-cache.patch braces: Dir["v:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}"] recursive: Dir["v:/test/myapp/app/views/**/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}"] On Windows 10 ruby 2.5.0dev (2017-09-11 trunk 59831) [x64-mingw32] ~~~ braces 148.111 (�� 3.4%) i/s - 742.000 in 5.015963s + patch braces 1.809k (�� 3.8%) i/s - 9.078k in 5.027256s => 12x faster + cache braces 480.215 (�� 5.4%) i/s - 2.397k in 5.005954s => 3x faster recursive 71.280 (�� 4.2%) i/s - 357.000 in 5.014841s + patch recursive 111.464 (�� 2.7%) i/s - 561.000 in 5.037387s + cache recursive 94.775 (�� 3.2%) i/s - 477.000 in 5.037445s ~~~ On Linux(Ubuntu 16.04) ruby 2.5.0dev (2017-09-11 trunk 59831) [x86_64-linux] ~~~ braces 6.171k (�� 1.0%) i/s - 31.408k in 5.090401s + patch braces 11.241k (�� 0.6%) i/s - 57.252k in 5.093467s recursive 720.448 (�� 1.4%) i/s - 3.640k in 5.053382s + patch recursive 730.159 (�� 0.5%) i/s - 3.723k in 5.099068s ~~~ ---------------------------------------- Bug #13167: Dir.glob is 25x slower since Ruby 2.2 https://bugs.ruby-lang.org/issues/13167#change-66606 * Author: ahorek (Pavel Rosick��) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: 2.4.0 * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- Hello, we've found a huge speed regression in our Rails app. After some digging the reason is in `Dir.glob` method which is much slower since Ruby 2.2.6. This is probably Windows only! This code is used heavily in Rails for partial lookups: ```ruby Dir.glob('c:/test/myapp/app/views/common/_menu_stats{.en,}{.html,}{}{.erb,.builder,.raw,.ruby,.jbuilder,.coffee,}') ``` Comparsion (x64): jruby 9.1.7.0 2540 i/s ruby 2.1.5 2568 i/s ruby 2.1.9 2569 i/s **ruby 2.2.6 99 i/s 25 times slower! ruby 2.3.3 102 i/s ruby 2.4.0 103 i/s** I would like to help, but I don't know much about Ruby C internals. Please let me know if you need any additional info. Now we're stuck at 2.1.9 because this issue makes the development on more recent versions unusable. ---Files-------------------------------- logruby24.txt (484 KB) logruby21.txt (10.8 KB) 0001-dir.c-performance-fix-with-braces.patch (8.64 KB) bench_dir_glob.rb (880 Bytes) 0001-dir.c-performance-fix-with-braces-using-cache.patch (5.84 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: