From: Benoit Daloze Date: 2017-01-17T15:05:06+01:00 Subject: [ruby-core:79106] Re: ruby/spec needs help from CRuby committers --===============1924321787== Content-Type: multipart/alternative; boundary=001a1143ddaaf6603705464acb96 --001a1143ddaaf6603705464acb96 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thank you for the great summary Naruse-san! On Mon, Jan 16, 2017 at 4:27 PM, NARUSE, Yui wrote: > But > mspec doesn=E2=80=99t have enough utility in envutil, like assert_separat= ely. That can be added of course. There is ruby_exe("some code"), but it's more limited as it does not require the test framework. > Like above mspec is not as robust as miniunit. > Even though ruby/spec=E2=80=99s example is smaller than test-all, > mspec has more abstraction layers than minitest+test-unit wrapper. Maybe we can make mspec simpler, I would be glad to see that. I do not think there is necessarily a much larger abstraction needed for spec-style. There are also counter examples, such as test-all using refinements and complex code in setup or utilities, which make them really hard to run for other implementations. > If it hits mysterious SEGV, it makes hard to debug. > (I experienced such hard debugging some times through the experience) Could you expand on what is hard to debug? Is it the assertion code, going from one test to another or something else? Debugging at the C level when going through quite a bit of Ruby code is hard, I usually make a minimal reproducible example when the bug is not obvious. > Just as a note > * test-all also has tag feature; JRuby uses that for these days Yes, but it does not work well. The most common case is an alternative implementation passes everything except 1 of the 10 assertions in that test method, and then everything must be excluded for the CI to stay green. --001a1143ddaaf6603705464acb96 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Thank you for the great summary Naruse-san!

On Mon,= Jan 16, 2017 at 4:27 PM, NARUSE, Yui <naruse@airemix.jp> wrote:
> But
> mspec doesn=E2=80= =99t have enough utility in envutil, like assert_separately.

That ca= n be added of course.
There is ruby_exe("some code"), but it&#= 39;s more limited as it does not require the test framework.

> Li= ke above mspec is not as robust as miniunit.
> Even though ruby/spec= =E2=80=99s example is smaller than test-all,
> mspec has more abstrac= tion layers than minitest+test-unit wrapper.

Maybe we can make mspec= simpler, I would be glad to see that.
I do not think there is necessari= ly a much larger abstraction needed for spec-style.

There= are also counter examples, such as test-all using refinements

an= d complex code in setup or utilities, which make them really hard to run fo= r other implementations.

> If it hits mysterious SEGV, it = makes hard to debug.
> (I experienced such hard debugging some times = through the experience)

Could you expand on what is hard to debug?Is it the assertion code, going from one test to another or something els= e?

Debugging at the C level when going through quite a bit of Ruby code= is hard,

I usually make a minimal reproducible example when the = bug is not obvious.

> Just as a note
> * test-all also has = tag feature; JRuby uses that for these days

Yes, b= ut it does not work well.

The most common case is an alternative = implementation

passes everything except 1 of the 10 assertions in= that test method,

and then everything must be excluded for the C= I to stay green.

--001a1143ddaaf6603705464acb96-- --===============1924321787== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Unsubscribe: --===============1924321787==--