[ruby-core:106957] [Ruby master Bug#18454] YJIT slowing down key Discourse benchmarks

Issue #18454 has been updated by sam.saffron (Sam Saffron).


 @k0kubun already doing 20 iterations as a warmup, so it is not likely :cry: https://github.com/discourse/discourse/blob/bbca25e875c36d3541b22c520b6d56ae6322dfbe/script/bench.rb#L192-L192


@byroot @noahgibbs this is a tough one Ruby trunk is not in a happy state, between the File.exists? removal and cb_data changes and other stuff that was added to trunk getting Discourse to work is not easy. I backported 5414de4b6e4372af832e338f8eb7a9fe8de17c84 by @jhawthorn, this is what I got.

Note I had to disable bootsnap cause for some reason I am getting: `fiber.so: undefined symbol: ruby_Init_Fiber_as_Coroutine`


no yjit -> yjit
categories: 48 -> 48
home: 84 -> 84
topic: 42 -> 39 
categories_admin: 48 -> 50 
home_admin: 86  -> 82
topic_admin: 42 -> 38
RSS: 293 -> 599 


- Looks improved, however still seeing a regression in categories admin bench so there may be another spot that needs fixing. Additionally perf is flat on a few of the benchmarks which is a concern
- Oddly bootsnap may be causing an RSS increase  338 -> 370 (on 2.7.5  290 -> 315) 
- Huge memory impact for the fixes in the 3.1.0 tag! 
- 3.2 is going to be a rough upgrade it seems, the File.exists deprecation is going to be very rough, as are a bunch of other changes there.


----------------------------------------
Bug #18454: YJIT slowing down key Discourse benchmarks
https://bugs.ruby-lang.org/issues/18454#change-95780

* Author: sam.saffron (Sam Saffron)
* Status: Open
* Priority: Normal
* ruby -v: 3.1.0p0
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------

4 out of 6 key Discourse benchmarks used in: https://github.com/discourse/discourse/blob/main/script/bench.rb are slower under YJIT. 

Vanilla (median) -> YJIT (median)

Categories: 47ms -> 51
Home page: 85 -> 86
Topic: 42 -> 38
Categories Admin: 48 -> 50
Home Admin: 83 -> 84
Topic Admin: 43 -> 39
Boot: 1740ms -> 2484ms 
RSS: 358756 -> 640540

This was run with a simple `RUBYOPT='--enable-yjit' ruby script/bench.rb -i 100 -s`


To get Discourse working with Ruby 3.1 you will need: 



```
diff --git a/Gemfile b/Gemfile
index 9cdfbf21a9..403c428718 100644
--- a/Gemfile
+++ b/Gemfile
@@ -263,3 +263,11 @@ gem 'colored2', require: false
 gem 'maxminddb'
 
 gem 'rails_failover', require: false
+
+if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("3.0.0")
+  gem 'net-pop', require: false
+  gem 'net-smtp', require: false
+  gem 'net-imap', require: false
+  # waiting on new version of i18n gem
+  gem 'psych', '3.3.2'
+end
```


It is clear that memory usage and boot time have to take a hit here, but slowing down home page seems concerning. Any ideas on what we should test here and how to narrow this down? 



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>