From: Greg.mpls@... Date: 2018-06-19T22:26:17+00:00 Subject: [ruby-core:87523] [Ruby trunk Feature#14187] `make test` and `make check` to run all test suites Issue #14187 has been updated by MSP-Greg (Greg L). Messy issue, especially taking into account whether the user downloads a pre-built install, or builds their own. Without a build/compile system, tests can't be run with make... Having paid attention to various repos for popular gems, a casual user can very easily install gems that expect a fully working Ruby install. Hence, I agree that casual users may have a need to run test-all & spec. Re ruby-loco, I don't recall ever having problems in bootstraptest or basictest. Spec rarely fails (thanks @eregon), but one could say it is somewhat more 'platform independent' than test-all. > Is there any other idea? Wish I had a good answer. I don't. I run the tests separately so I can split the logs, but for casual users, I think the suggestion should be to run all the tests. I keep thinking of taking one's car in for repair, and the service manager says 'it passed all the quick/basic tests'... ---------------------------------------- Feature #14187: `make test` and `make check` to run all test suites https://bugs.ruby-lang.org/issues/14187#change-72526 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal * Assignee: * Target version: next minor ---------------------------------------- Currently, MRI has many test suites: "`bootstraptest/`", "`KNOWNBUGS.rb`", "`basictest/`", "`test/`", and "`spec/`". And, the target name design of `make` is really complicated: * `make test` runs the first three suites * `make test-all` runs only `test/` * `make check` runs all the suites except `spec/` * `make exam` runs all the suites I think when a casual user build and install ruby, s/he will expect `make test` or `make check` to run all recommended test suites because many other projects uses them as the semantics. Thus, how about changing `make test` and `make check` to run all test suites? I think this change will affect some ruby development tools including `rubyci` (`chkbuild`), `mswin-build`, and `ruby-loco`. @hsbt, @usa, @MSP-Greg, what do you think? ```diff diff --git a/common.mk b/common.mk index 80001b481c..ba5fffdf4b 100644 --- a/common.mk +++ b/common.mk @@ -634,10 +634,6 @@ clean-spec: PHONY -$(Q) $(RM) $(RUBYSPEC_CAPIEXT)/*.$(OBJEXT) $(RUBYSPEC_CAPIEXT)/*.$(DLEXT) -$(Q) $(RMDIRS) $(RUBYSPEC_CAPIEXT) 2> $(NULL) || exit 0 -check: main test test-testframework test-almost - $(ECHO) check succeeded -check-ruby: test test-ruby - fake: $(CROSS_COMPILING)-fake yes-fake: $(arch)-fake.rb $(RBCONFIG) PHONY no-fake -fake: PHONY @@ -660,6 +656,8 @@ no-btest-ruby: PHONY yes-btest-ruby: prog PHONY $(Q)$(exec) $(RUNRUBY) "$(srcdir)/bootstraptest/runner.rb" --ruby="$(PROGRAM) -I$(srcdir)/lib $(RUN_OPTS)" -q $(OPTS) $(TESTOPTS) +test-bootstrap: btest-ruby + test-basic: $(TEST_RUNNABLE)-test-basic no-test-basic: PHONY yes-test-basic: prog PHONY @@ -677,7 +675,6 @@ yes-test-testframework: prog PHONY no-test-testframework: PHONY test-sample: test-basic # backward compatibility for mswin-build -test: btest-ruby test-knownbug test-basic # $ make test-all TESTOPTS="--help" displays more detail # for example, make test-all TESTOPTS="-j2 -v -n test-name -- test-file-name" @@ -755,7 +752,7 @@ $(ENC_MK): $(srcdir)/enc/make_encmake.rb $(srcdir)/enc/Makefile.in $(srcdir)/enc .PHONY: clean clean-ext clean-local clean-enc clean-golf clean-rdoc clean-html clean-extout .PHONY: distclean distclean-ext distclean-local distclean-enc distclean-golf distclean-extout .PHONY: realclean realclean-ext realclean-local realclean-enc realclean-golf realclean-extout -.PHONY: check test test-all btest btest-ruby test-basic test-knownbug +.PHONY: exam check test test-all test-bootstrap btest btest-ruby test-basic test-knownbug .PHONY: run runruby parse benchmark benchmark-each tbench gdb gdb-ruby .PHONY: update-mspec update-rubyspec test-rubyspec test-spec .PHONY: touch-unicode-files @@ -1322,9 +1319,12 @@ info-arch: PHONY change: PHONY $(BASERUBY) -C "$(srcdir)" ./tool/change_maker.rb $(CHANGES) > change.log -exam: check test-spec +test: main test-bootstrap test-knownbug test-basic test-testframework test-almost test-spec + $(ECHO) check succeeded +check: test +exam: test -love: sudo-precheck up all test install exam +love: sudo-precheck up all test install @echo love is all you need great: exam @@ -1356,11 +1356,11 @@ help: PHONY " runruby: runs test.rb by ruby you just built" \ " gdb: runs test.rb by miniruby under gdb" \ " gdb-ruby: runs test.rb by ruby under gdb" \ - " check: equals make test test-all" \ - " exam: equals make check test-spec" \ - " test: ruby core tests" \ - " test-all: all ruby tests [TESTOPTS=-j4 TESTS=]" \ - " test-spec: run the Ruby spec suite" \ + " test: ruby all test suites" \ + " check: equals make test" \ + " exam: equals make test" \ + " test-all: run `test/` suite [TESTOPTS=-j4 TESTS=]" \ + " test-spec: run the Ruby spec suite (in `spec/`)" \ " test-rubyspec: same as test-spec" \ " test-bundler: run the Bundler spec" \ " test-bundled-gems: run the test suite of bundled gems" \ ``` -- https://bugs.ruby-lang.org/ Unsubscribe: