From: ko1@... Date: 2020-02-26T17:23:26+00:00 Subject: [ruby-core:97272] [Ruby master Feature#16655] Each test on test-all should run `srand(seed)` at setup Issue #16655 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #16655: Each test on test-all should run `srand(seed)` at setup https://bugs.ruby-lang.org/issues/16655 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Target version: 2.8 ---------------------------------------- # Points * call `srand($seed)` before each test. `$seed` is given seed number by `--seed N` # Background Some tests use `rand()` to produce random number to generate random test examples. Example: ``` def test_local_variable_set_wb assert_ruby_status([], <<-'end;', '[Bug #13605]', timeout: 30) b = binding n = 20_000 n.times do |i| v = rand(2_000) name = "n#{v}" value = Object.new b.local_variable_set name, value end end; end ``` This test generates 2,000 randomly named ivar which point random numbers. However, several tests calls `srand(0)` and this random sequence is affected by test methods order. Two problems: * Test authors of this kind of tests should assume that every test processes uses very different random sequence to increase the test coverage. * We can not get reproducible results on such tests which use random sequence with `--seed` because of `srand(0)` using tests. # Proposal To solve the above two problems, I`ll insert `srand($seed)` at the beginning of each test (`$seed` is given by `--seed` option). This proposal respects `--seed` option more, so we can expect more deterministic results. I measured performance overhead and I can't get meaningful slow down. # Additional note Surprisingly, `--seed` option is ignored just now on test-all, so I'll fix this problem too. -- https://bugs.ruby-lang.org/ Unsubscribe: