From: "mame (Yusuke Endoh) via ruby-core" <ruby-core@...>
Date: 2024-07-31T02:37:14+00:00
Subject: [ruby-core:118754] [Ruby master Bug#20659] Speed regression of `parse.y` parser after numeric nodes were introduced

Issue #20659 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to yui-knk (Kaneko Yuichiro)

----------------------------------------
Bug #20659: Speed regression of `parse.y` parser after numeric nodes were introduced
https://bugs.ruby-lang.org/issues/20659#change-109300

* Author: alanwu (Alan Wu)
* Status: Assigned
* Assignee: yui-knk (Kaneko Yuichiro)
* ruby -v: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The [`mail` benchmark from yjit-bench][1] is about 20% slower on `master` compared to commit:98eeadc9 ("Development of 3.4.0 started.") as the baseline, comparing running time of the Ruby process running the benchmark for a single iteration. Much of this workload is Ruby parsing.

```
$ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby' '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb'
Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):     889.0 ms ��   2.0 ms    [User: 776.9 ms, System: 111.7 ms]
  Range (min ��� max):   885.7 ms ��� 892.1 ms    10 runs
 
Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):     891.7 ms ��   1.6 ms    [User: 794.5 ms, System: 96.9 ms]
  Range (min ��� max):   889.0 ms ��� 894.7 ms    10 runs
 
Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):      1.086 s ��  0.003 s    [User: 0.951 s, System: 0.134 s]
  Range (min ��� max):    1.080 s ���  1.091 s    10 runs
 
Summary
  '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' ran
    1.00 �� 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb'
    1.22 �� 0.00 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb'

$ for tag in 3.3-equiv master before-numeric-nodes; do echo $tag: $(~/.rubies/$tag/bin/ruby -v); done
3.3-equiv: ruby 3.4.0dev (2023-12-25T09:13:40Z master 98eeadc932) [x86_64-linux]
master: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux]
before-numeric-nodes: ruby 3.4.0dev (2024-01-06T18:26:38Z master 76afbda5b5) [x86_64-linux]
```

Using Valgrind's DHAT reveals that commit:1b8d01136c3ff6c60325c7609d61e19ac42acd9f ("Introduce Numeric Node's") issues roughly 3 times more malloc(3) calls compared to the baseline, most of them coming from strdup() calls in set_number_literal().

Comparison with `--parser=prism`, for interest:

<details>

```
$ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby,master/bin/ruby --parser=prism' \
       '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb'
Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):     889.7 ms ��   2.6 ms    [User: 771.4 ms, System: 118.0 ms]
  Range (min ��� max):   885.8 ms ��� 894.6 ms    10 runs
 
Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):     890.4 ms ��   1.6 ms    [User: 776.8 ms, System: 113.3 ms]
  Range (min ��� max):   888.4 ms ��� 892.9 ms    10 runs
 
Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):      1.087 s ��  0.004 s    [User: 0.968 s, System: 0.119 s]
  Range (min ��� max):    1.083 s ���  1.097 s    10 runs
 
Benchmark 4: ~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb
  Time (mean �� ��):     826.9 ms ��   2.1 ms    [User: 725.8 ms, System: 100.7 ms]
  Range (min ��� max):   823.6 ms ��� 830.9 ms    10 runs
 
Summary
  '~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb' ran
    1.08 �� 0.00 times faster than '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb'
    1.08 �� 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb'
    1.31 �� 0.01 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb'
```

</details>

[1]: https://github.com/Shopify/yjit-bench/blob/b8f1bd8931d4757afdae933ef6f11e0767ba90b8/benchmarks/mail/benchmark.rb



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/