From: tenderlove@... Date: 2020-12-07T23:39:52+00:00 Subject: [ruby-core:101299] [Ruby master Bug#17373] Ruby 3.0 is slower at Discourse bench than Ruby 2.7 Issue #17373 has been updated by tenderlovemaking (Aaron Patterson). It looks like [the global method cache was removed](https://github.com/ruby/ruby/commit/b9007b6c548f91e88fd3f2ffa23de740431fa969#diff-884a5a8a369ef1b4c7597e00aa65974cec8c5f54f25f03ad5d24848f64892869L9-L27). On f2286925f0 the debug counters looked like this: ``` [RUBY_DEBUG_COUNTER] mc_inline_hit 54,445,333 [RUBY_DEBUG_COUNTER] mc_inline_miss 1,800,036 [RUBY_DEBUG_COUNTER] mc_global_hit 14,489,677 [RUBY_DEBUG_COUNTER] mc_global_miss 790,656 ``` `mc_global_hit` is the number of times the global method cache was hit. I think for Rails, the problem is with respond_to. It looks like `Object#respond_to?` used to use the global method cache: ![callers](https://user-images.githubusercontent.com/3124/101416878-8f4bd800-389f-11eb-9263-44e3f0538172.png) Here is a benchmark that demonstrates the impact of removing the global method cache: ```ruby require "benchmark/ips" class Base def bar; end end class Foo 30.times do |i| class_eval "module M#{i}; end; include M#{i}" end end foo = Foo.new Benchmark.ips do |x| x.report("respond to") { foo.respond_to?(:bar) } end ``` On master: ``` aaron@whiteclaw ~/git> ruby/ruby -v test.rb ruby 3.0.0dev (2020-12-07T17:10:03Z master 146b3695c0) [x86_64-linux] Warming up -------------------------------------- respond to 585.997k i/100ms Calculating ------------------------------------- respond to 5.869M (�� 0.9%) i/s - 29.886M in 5.092708s ``` On f2286925f0: ``` aaron@whiteclaw ~/git> ruby/ruby -v test.rb ruby 2.8.0dev (2020-02-22T00:58:59Z before-imc f2286925f0) [x86_64-linux] Warming up -------------------------------------- respond to 1.686M i/100ms Calculating ------------------------------------- respond to 16.888M (�� 0.6%) i/s - 85.971M in 5.090695s ``` f2286925f0 is about 3x faster than master. It seems that the global method cache was important for Rails benchmarks. I guess we can't have a global method cache because of Ractors? Would it be possible to make a per-Ractor "global" cache? ---------------------------------------- Bug #17373: Ruby 3.0 is slower at Discourse bench than Ruby 2.7 https://bugs.ruby-lang.org/issues/17373#change-88985 * Author: sam.saffron (Sam Saffron) * Status: Open * Priority: Normal * ruby -v: 3.0 * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- We have a continuous effort to keep https://rubybench.org/ up to date. Unfortunately we have lapsed a bit on updates so I decided to run Discourse bench by hand. On my local system I am noticing the timing are much slower on Discourse bench than they were on Ruby 2.7: ``` 98c9120cc00ba691b4abcc13a49a30fa54638535 --- categories: 50: 34 75: 35 90: 36 99: 40 home: 50: 47 75: 50 90: 56 99: 92 topic: 50: 53 75: 55 90: 58 99: 92 categories_admin: 50: 33 75: 34 90: 38 99: 68 home_admin: 50: 45 75: 47 90: 51 99: 95 topic_admin: 50: 51 75: 52 90: 54 99: 59 timings: load_rails: 1449 ruby-version: 3.0.0-p-1 rss_kb: 293808 pss_kb: 284244 2.7.2 --- categories: 50: 29 75: 30 90: 33 99: 43 home: 50: 42 75: 43 90: 47 99: 73 topic: 50: 47 75: 47 90: 52 99: 54 categories_admin: 50: 30 75: 31 90: 36 99: 56 home_admin: 50: 41 75: 43 90: 48 99: 71 topic_admin: 50: 47 75: 48 90: 52 99: 55 timings: load_rails: 1339 ruby-version: 2.7.2-p137 rss_kb: 296836 pss_kb: 287310 2.6.6 --- categories: 50: 30 75: 30 90: 31 99: 51 home: 50: 40 75: 42 90: 60 99: 79 topic: 50: 47 75: 48 90: 53 99: 71 categories_admin: 50: 30 75: 31 90: 35 99: 50 home_admin: 50: 41 75: 43 90: 62 99: 100 topic_admin: 50: 48 75: 49 90: 65 99: 93 timings: load_rails: 1464 ruby-version: 2.6.6-p146 rss_kb: 332268 pss_kb: 322865 ``` Concretely this means a typical "homepage" view is taking 47 milliseconds now, when in the past it was taking 42 or even 40 ms. There may be some issues with the bench, you can see it here: https://github.com/discourse/discourse/tree/ruby-3 The script to run the bench is `ruby script/bench.rb` I re-tested against a cleanly built Ruby 2.7 direct from the Git branch to ensure compilation was not at fault. This is pretty concerning, Discourse bench usually tracks Rails performance pretty accurately. ---Files-------------------------------- aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 175��54 2020-12-07 12-41-12.png (842 KB) aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 175��54 2020-12-07 12-41-58.png (874 KB) aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 299��65 2020-12-07 13-19-20.png (142 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe> <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>