From: tenderlove@...
Date: 2020-12-07T23:39:52+00:00
Subject: [ruby-core:101299] [Ruby master Bug#17373] Ruby 3.0 is slower at Discourse bench than Ruby 2.7

Issue #17373 has been updated by tenderlovemaking (Aaron Patterson).


It looks like [the global method cache was removed](https://github.com/ruby/ruby/commit/b9007b6c548f91e88fd3f2ffa23de740431fa969#diff-884a5a8a369ef1b4c7597e00aa65974cec8c5f54f25f03ad5d24848f64892869L9-L27).

On f2286925f0 the debug counters looked like this:

```
[RUBY_DEBUG_COUNTER]	mc_inline_hit                 	    54,445,333
[RUBY_DEBUG_COUNTER]	mc_inline_miss                	     1,800,036
[RUBY_DEBUG_COUNTER]	mc_global_hit                 	    14,489,677
[RUBY_DEBUG_COUNTER]	mc_global_miss                	       790,656
```

`mc_global_hit` is the number of times the global method cache was hit.  I think for Rails, the problem is with respond_to.  It looks like `Object#respond_to?` used to use the global method cache:

![callers](https://user-images.githubusercontent.com/3124/101416878-8f4bd800-389f-11eb-9263-44e3f0538172.png)

Here is a benchmark that demonstrates the impact of removing the global method cache:

```ruby
require "benchmark/ips"

class Base
  def bar; end
end

class Foo
  30.times do |i|
    class_eval "module M#{i}; end; include M#{i}"
  end
end

foo = Foo.new

Benchmark.ips do |x|
  x.report("respond to") { foo.respond_to?(:bar) }
end
```

On master:

```
aaron@whiteclaw ~/git> ruby/ruby -v test.rb
ruby 3.0.0dev (2020-12-07T17:10:03Z master 146b3695c0) [x86_64-linux]
Warming up --------------------------------------
          respond to   585.997k i/100ms
Calculating -------------------------------------
          respond to      5.869M (�� 0.9%) i/s -     29.886M in   5.092708s
```

On f2286925f0:

```
aaron@whiteclaw ~/git> ruby/ruby -v test.rb
ruby 2.8.0dev (2020-02-22T00:58:59Z before-imc f2286925f0) [x86_64-linux]
Warming up --------------------------------------
          respond to     1.686M i/100ms
Calculating -------------------------------------
          respond to     16.888M (�� 0.6%) i/s -     85.971M in   5.090695s
```

f2286925f0 is about 3x faster than master.

It seems that the global method cache was important for Rails benchmarks.  I guess we can't have a global method cache because of Ractors?  Would it be possible to make a per-Ractor "global" cache?

----------------------------------------
Bug #17373: Ruby 3.0 is slower at Discourse bench than Ruby 2.7
https://bugs.ruby-lang.org/issues/17373#change-88985

* Author: sam.saffron (Sam Saffron)
* Status: Open
* Priority: Normal
* ruby -v: 3.0
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
We have a continuous effort to keep https://rubybench.org/ up to date. Unfortunately we have lapsed a bit on updates so I decided to run Discourse bench by hand.

On my local system I am noticing the timing are much slower on Discourse bench than they were on Ruby 2.7: 

```
98c9120cc00ba691b4abcc13a49a30fa54638535

---
categories:
  50: 34
  75: 35
  90: 36
  99: 40
home:
  50: 47
  75: 50
  90: 56
  99: 92
topic:
  50: 53
  75: 55
  90: 58
  99: 92
categories_admin:
  50: 33
  75: 34
  90: 38
  99: 68
home_admin:
  50: 45
  75: 47
  90: 51
  99: 95
topic_admin:
  50: 51
  75: 52
  90: 54
  99: 59
timings:
  load_rails: 1449
ruby-version: 3.0.0-p-1
rss_kb: 293808
pss_kb: 284244

2.7.2


---
categories:
  50: 29
  75: 30
  90: 33
  99: 43
home:
  50: 42
  75: 43
  90: 47
  99: 73
topic:
  50: 47
  75: 47
  90: 52
  99: 54
categories_admin:
  50: 30
  75: 31
  90: 36
  99: 56
home_admin:
  50: 41
  75: 43
  90: 48
  99: 71
topic_admin:
  50: 47
  75: 48
  90: 52
  99: 55
timings:
  load_rails: 1339
ruby-version: 2.7.2-p137
rss_kb: 296836
pss_kb: 287310


2.6.6


---
categories:
  50: 30
  75: 30
  90: 31
  99: 51
home:
  50: 40
  75: 42
  90: 60
  99: 79
topic:
  50: 47
  75: 48
  90: 53
  99: 71
categories_admin:
  50: 30
  75: 31
  90: 35
  99: 50
home_admin:
  50: 41
  75: 43
  90: 62
  99: 100
topic_admin:
  50: 48
  75: 49
  90: 65
  99: 93
timings:
  load_rails: 1464
ruby-version: 2.6.6-p146
rss_kb: 332268
pss_kb: 322865
```

Concretely this means a typical "homepage" view is taking 47 milliseconds now, when in the past it was taking 42 or even 40 ms.

There may be some issues with the bench, you can see it here: 

https://github.com/discourse/discourse/tree/ruby-3

The script to run the bench is `ruby script/bench.rb`

I re-tested against a cleanly built Ruby 2.7 direct from the Git branch to ensure compilation was not at fault.


This is pretty concerning, Discourse bench usually tracks Rails performance pretty accurately. 

---Files--------------------------------
aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 175��54 2020-12-07 12-41-12.png (842 KB)
aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 175��54 2020-12-07 12-41-58.png (874 KB)
aaron ��� perf _home_aaron_git_railsbench ��� ssh whiteclaw.local ��� 299��65 2020-12-07 13-19-20.png (142 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>