[ruby-core:41712] Re: My bug evaluation criteria

Hi,

On Sun, Dec 11, 2011 at 8:59 PM, NARUSE, Yui <[e-mail]> wrote:

>...
>> To pass the "strict superiority test" (SST), the proposed behavior is ju=
dged on:
>> a) usefulness,
>> b) consistency,
>> c) intuitiveness and
>> d) big O level of performance
>> It must be superior to the existing behavior on at least one of these
>> criteria, and not inferior in any.
>
> I add one more, the value of behavior's brekage is prior than compatility=
.

Sorry, I'm not sure I understand your sentence.

>> A bugfix is "straightforward" if it is SST and when the proposed
>> behavior passes the "no intent test" (NIT):
>> 1) doesn't contradict the existing documentation,
>> 2) doesn't break existing tests and
>> 3) the source code doesn't clearly indicate that the current behavior
>> is intended
>
> Add to 3), "the source code nor the commit message".

Agreed.

>> A bugfix is "clear" if it is straightforward (SST+NIT) and if the
>> existing behavior passes the "clear defect test" (ODT):
>> 1) it crashes Ruby,
>> 2) it contradicts the documentation or
>
> On Ruby, a document and code sometimes conflict.
> On such case, they theirselves are equal, but as I wrote above,
> code change introduces incompatibility.
> So usually the code is prior to the documentation.

If a behavior is not intentional (as per test above) and contradicts
the documentation (i.e. the documentation explicitly states some
result which differs from the current result), I have to disagree with
you: it's a clear bug, the code is wrong and the documentation should
be followed. Of course, it's possible to decide to make a feature
change at this point, change the documentation, etc..., but it is not
a bug fix.

>> 3) it is platform dependent (when it can reasonably be platform independ=
ent)
>
> What is "reasonably be platform independent" is not clear yet,

You are right, it's not clear for all cases, but for many cases it is
clear. Having to rewrite a huge library is not reasonable, changing a
couple of lines of code is. Impacting the performance by a factor of
10 isn't reasonable, but a slowdown of a couple of percent is.

> And I add another one:
> 4) regression (it works well before, but is broken by some side effects)

No, this is covered by the Intention test, documentation and
consistency. If it works "well" as you say, then the new behavior can
not be strictly superior and not contradicting intention.

--
Marc-Andr=E9