[ruby-dev:50451] [Ruby trunk Bug#14387] Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
From:
daniel.ltw@...
Date:
2018-01-29 19:31:49 UTC
List:
ruby-dev #50451
Issue #14387 has been updated by scardon (Daniel Leong).
~~~ text
/app # ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
1048576
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
131072
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
503808
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
165: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
164: from /usr/local/bundle/bin/rubycritic:23:in `load'
163: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 153 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
165: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
164: from /usr/local/bundle/bin/rubycritic:23:in `load'
163: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 153 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each': stack level too deep (SystemStackError)
/app # rubycritic -f html --no-browser app lib
RubyCritic can provide more feedback if you use a Git, Mercurial or Perforce repository. Churn will not be calculated.
Traceback (most recent call last):
164: from /usr/local/bundle/bin/rubycritic:23:in `<main>'
163: from /usr/local/bundle/bin/rubycritic:23:in `load'
162: from /usr/local/bundle/gems/rubycritic-3.3.0/bin/rubycritic:10:in `<top (required)>'
161: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/cli/application.rb:20:in `execute'
160: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:19:in `execute'
159: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/commands/default.rb:24:in `critique'
158: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `run'
157: from /usr/local/bundle/gems/rubycritic-3.3.0/lib/rubycritic/analysers_runner.rb:27:in `each'
... 152 levels...
4: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject'
3: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `each'
2: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `block in mass'
1: from /usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `mass'
/usr/local/bundle/gems/sexp_processor-4.10.0/lib/sexp.rb:223:in `inject': stack level too deep (SystemStackError)
~~~
This is some output with the rubycritic gem trying to parse a deeply nested AST.
Below is some output with the brakeman gem trying to parse the same deeply nested AST.
~~~ text
/app # ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
1048576
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
131072
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 ruby -ve 'p RubyVM::DEFAULT_PARAMS[:thread_machine_stack_size]'
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
503808
/app # brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
254: from /usr/local/bundle/bin/brakeman:23:in `<main>'
253: from /usr/local/bundle/bin/brakeman:23:in `load'
252: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
248: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
247: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 242 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=100000 brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
256: from /usr/local/bundle/bin/brakeman:23:in `<main>'
255: from /usr/local/bundle/bin/brakeman:23:in `load'
254: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
253: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
252: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 244 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
/app # RUBY_THREAD_MACHINE_STACK_SIZE=500000 brakeman -o brakeman-output.json --no-progress --separate-models --no-branching
Loading scanner...
Processing application in /app
Processing gems...
[Notice] Detected Rails 5 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Processing initializers...
Processing libs...
Traceback (most recent call last):
256: from /usr/local/bundle/bin/brakeman:23:in `<main>'
255: from /usr/local/bundle/bin/brakeman:23:in `load'
254: from /usr/local/bundle/gems/brakeman-4.1.1/bin/brakeman:8:in `<top (required)>'
253: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:20:in `start'
252: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:35:in `run'
251: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:142:in `run_report'
250: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:118:in `regular_report'
249: from /usr/local/bundle/gems/brakeman-4.1.1/lib/brakeman/commandline.rb:133:in `run_brakeman'
... 244 levels...
4: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
3: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
2: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
1: from /usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash'
/usr/local/bundle/gems/brakeman-4.1.1/lib/ruby_parser/bm_sexp.rb:106:in `hash': stack level too deep (SystemStackError)
~~~
Both brakeman and rubycritic never would have such error when we were using ruby 2.4
----------------------------------------
Bug #14387: Ruby 2.5 を Alpine Linux で実行すると比較的浅めで SystemStackError 例外になる
https://bugs.ruby-lang.org/issues/14387#change-69985
* Author: koshigoe (Masataka SUZUKI)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
CircleCI で Alpine Linux を使って Ruby 2.5.0 で Rubocop を実行した時に遭遇した例外です(Ruby 2.4.3 では発生しませんでした)。
Ruby のバージョンによって、再帰が止められるまでの回数に大きな違いがあるのはなぜでしょうか？
これは、意図された挙動なのか、Ruby の変更によるものでは無く Alpine Linux 固有の問題なのか、教えていただく事は可能でしょうか？
Alpine Linux の Tread stack size が比較的小さい事で、Ruby 2.5.0 からこのような挙動になったのでしょうか？
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
## 再現
問題の再現のため、以下の様な再帰するコードを実行します。
~~~ ruby
# test.rb
n = 100000
res = {}
1.upto(n).to_a.inject(res) do |r, i|
r[i] = {}
end
def f(x)
x.each_value { |v| f(v) }
end
f(res)
~~~
Ruby 2.4.3 で実行した場合、 10061 levels で例外があがりました。
~~~
% docker container run \
-v (pwd):/mnt/my --rm \
ruby:2.4.3-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 10061 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
一方で Ruby 2.5.0 で実行した場合、 134 level で例外があがりました。
```
% docker container run \
-v (pwd):/mnt/my --rm \
test/ruby:trunk-alpine3.7 \
ruby -v /mnt/my/test.rb
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
また、Ruby trunk で実行した場合は 2.5.0 同等の結果になりました。
```
ruby 2.6.0dev (2018-01-24 trunk 62017) [x86_64-linux-musl]
/mnt/my/test.rb:9:in `each_value': stack level too deep (SystemStackError)
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:9:in `block in f'
... 134 levels...
from /mnt/my/test.rb:9:in `block in f'
from /mnt/my/test.rb:9:in `each_value'
from /mnt/my/test.rb:9:in `f'
from /mnt/my/test.rb:12:in `<main>'
```
※ trunk の Docker イメージを作った際の Dockerfile は以下。
https://gist.github.com/koshigoe/509be02a3580cdfc7a2cc45a4e6e44c5
--
https://bugs.ruby-lang.org/