[ruby-core:17593] Re: 1.8 release management

Hi,

A bit of clarifcation:

On Fri, Jul 4, 2008 at 7:39 PM, Igal Koshevoy <[e-mail]> wrote:
> For example, I run the checker against my patched version of p111 and then
> again with your latest SVN release. When the checks finish, I use gvimdiff
> (or meld, kdiff3, etc) to visually compare the reports (e.g., output of the
> RSpec test suite) to determine if the new version if failing tests that
> worked with the previous version, and then review these tests to see why
> they're failing.

RubySpec suite uses "ruby_bug" guards to mark some tests that fail
with MRI, and those guards can have the version info (including
patchlevel) to determine whether the test should be filtered out or
not.

So, if MRI 1.8.6 pl 114 produces clean results, but the very latest
from 1_8_6 branch shows some errors, that does not necessary means
that there is a new regression, it also might mean that ruby_bug
guards are not updated to the latest patchlevel version and allow to
run the spec that still fails.

E.g.:

ruby_bug "#bug_id", "1.8.6.114"

will filter out the test for pl 114, but will allow to run it in pl 256.

It is good to be aware of that.

Also, if you'd like to run ALL specs, even those filtered out by
ruby_bug guards, provide the additional command line parameters: -n
blah  (that will fake the ruby *name* to be 'blah', and ruby_guard
will not recognize MRI as MRI thus running all the specs).

If you run like that, then you could for sure use the "diff" comarison
method for failing specs to find out the new regressions. But be
aware, that the spec run might not even finish if you have -n blah
option (for example, due to MRI bugs like hangups and crashes that are
typically guarded by the ruby_bug guards).

Thanks,
  --Vladimir