ruby-core

On Thu, Sep 8, 2011 at 5:40 PM, SASADA Koichi <ko1@atdot.net> wrote:
> Thank you for summary of needs. =C2=A0Following is problems I feel (and
> discussed on IRC).
>
> Problems:
>
> (1) Most of MRI developers (is it true?) can't change RubySpec. =C2=A0Als=
o
> most of MRI developers don't know RubySpec rules.

I don't think there would be a problem getting MRI developers access
to RubySpec for direct commits. I understand the issues with "rules"
though. Personally I'd be happy to see *any* specs getting in, even if
they require some cleanup after the fact. It would be better than
nothing.

> (2) We can add code modification and test cases in one commit. =C2=A0If w=
e
> can't do it, we need commit commit modifications and RubySpec isolation.

This is an issue, but a small one. You point out later that you could
just keep a copy of RubySpec in MRI's repository and migrate changes
over. That's not a difficult task, if it's done periodically. This is
how Rubinius uses RubySpec.

For JRuby, we chose not to keep our own copy of RubySpec. Instead, our
build will fetch a "known stable" revision of RubySpec as part of the
build and run against that. We make all our RubySpec commits directly
to RubySpec.

> (3) It is difficult to judge test is an independent or not.
> Additionally, we don't need to insert "guard" about version or
> implementation on our "test/*". =C2=A0It seems tough work to insert corre=
ct
> "guard".

That's certainly true. I wouldn't expect that all tests would
immediately start going to the "right place". I believe the
frustration I and others feel mostly stems from the fact that there's
almost no specs coming from Ruby core now, which means those of us
*not* in ruby-core have to reverse-engineer behavior changes and write
specs ourselves. It's a dismal situation, and any improvement would be
welcome.

> (4) Current RubySpec is not portable (especially Windows)
> [ruby-core:39379].

This is just from lack of maintenance. As early as last year the whole
suite ran green on Windows. We can do it again.

> (5) As you know, trunk is not unstable. =C2=A0For example, some modificat=
ions
> are revertd soon. =C2=A0Should we add such features as "Specifications"? =
=C2=A0(I
> know that there is an opinion that the experimental code should have a
> spec).

If MRI maintained a copy of RubySpec in the MRI repository, then it
would be easy to revert spec changes as well. Maybe this would be a
good guide (edits welcome):

* If it is a new experimental feature, tests/specs (if they exist)
could certainly live only in MRI's repository. If that feature becomes
"official" those tests should be migrated to RubySpec.
* If an existing feature changes, the associated specs should change too.
* Anything that's MRI-specific or which Matz deems "implementation
specific" should only be tested by MRI's own suite.

> (6) test/* contains many many corner cases depend on MRI implementation
> to increase code coverage (Thanks Endo-san). =C2=A0It is independent as R=
uby
> language, but dependent on MRI. =C2=A0Where should we write such tests?

Those all stay in MRI's suite.

There will never be a clear line between what's
implementation/MRI-specific and what is true specified behavior since
there's no specification (or rather, the only specification that
exists encompasses only a small subset of Ruby). Nobody expects
ruby-core to be perfect here. But when a commit includes visible
behavioral changes that other implementations will eventually want to
duplicate, we *must* have a test/spec somewhere independent of MRI's
tests.

> I think this problem is "who pay efforts on it?".
>
> One solution is continuing current style (using test/*) and someone
> migrate tests to RubySpec if needed. =C2=A0But we need "someone".

In the past, there have been many contributors migrating tests (and
reverse-engineering MRI to write new ones) by hand. That works, if we
can keep up such efforts and keep finding new people as old ones
leave. Migrating tests is a thankless job, so perhaps we need to thank
people more often :)

One way to start this process would be to start at "a" in the existing
test/* tests and just start moving bits to RubySpec. I've done this
for some MRI tests in the past and for many JRuby tests too. It's a
slow process, but it feels great to delete the old test in favor of a
new spec.

> Other solution is MRI committer join to maintain RubySpec. =C2=A0MRI
> developer (including me) should change development style and several
> overhead to do (I think inserting correct "guard" is difficult).

Guarding isn't too bad. The main guards that ruby-core would need to know a=
re:

ruby_version_is/is_not - for specifying a range of Ruby versions, as
Strings, where the behavior is expected to exist
platform_is/is_not - for specifying platform-specific behaviors (e.g. :wind=
ows).

There are more complex forms in mspec, but we can help teach them to
ruby-core as we get going.

I think ruby-core would only be responsible for making sure specs pass
on the Ruby versions that RubySpec covers: 1.8.7 and 1.9.2. 1.9.3
changes are probably not covered in specs currently.

> Brian Ford told me that we can have our own RubySpec repository in MRI
> repository and migrating updating specs from MRI's RubySpec repository
> to the RubySpec repository is easy. =C2=A0It is also one solution. =C2=A0=
However,
> we need "someone" who can migrate. =C2=A0It solves (1), (2) and maybe (3)=
.

For 1.9.2, I believe yugui took the lead on making sure specs were
written for 1.9.2 behaviors (as much as possible...time was limited)
and also ensuring that 1.9.2 passed all specs (with appropriate
version guards, etc). Could it be release manager's responsibility to
ensure specs get written or migrated, perhaps by asking the developers
making changes to write/migrate those specs?

Those of us outside ruby-core really, really want to help make this
happen. Let me know what I personally can do to help.

- Charlie

Thread

Prev Next

In This Thread

Prev Next