[ruby-core:118814] [Ruby master Misc#20661] Stop retrying tests in `make test-all` command by default
From:
"kjtsanaktsidis (KJ Tsanaktsidis) via ruby-core" <ruby-core@...>
Date:
2024-08-09 03:52:54 UTC
List:
ruby-core #118814
Issue #20661 has been updated by kjtsanaktsidis (KJ Tsanaktsidis). I have no objection to disabling automatic re-runs, but one thing: > Just click the "Re-run jobs" button via the GitHub UI. Is this available to people who aren't a member of the Ruby organisation on Github? I think it might not be... I certainly remember pushing empty commits to poke CI because of this. ---------------------------------------- Misc #20661: Stop retrying tests in `make test-all` command by default https://bugs.ruby-lang.org/issues/20661#change-109372 * Author: ono-max (Naoto Ono) * Status: Open * Assignee: ono-max (Naoto Ono) ---------------------------------------- ## Summary Currently, tests are retried once if they fail when tests are executed as parallel tests in the `make test-all` command by default. This is to prevent test failures caused by parallel testing. That's why failed tests are executed serially, not in parallel. Source code: https://github.com/ruby/ruby/blob/master/tool/lib/test/unit.rb#L728-L751 However, there is a problem, which might hide a real issue like https://bugs.ruby-lang.org/issues/20314. To prevent missing this "real" problem, stopping the retry of tests in the `make test-all` command by default would be better. ## Concerns when stopping the retry of tests in the `make test-all` command Here are some concerns about stopping the retry mechanism in `make test-all` command. #### 1. There are some flaky tests in the test suite. Are we okay with that? Yes, [all flaky tests are tracked by Launchable](https://app.launchableinc.com/organizations/ruby/workspaces/ruby/insights/unhealthy-tests), and they're mainly monitored by @mame. Thus, they'll be fixed in the future. #### 2. When a test that is not related to my change fails, what should we do? Just click the "Re-run jobs" button via the GitHub UI. -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/