From: "NARUSE, Yui" <naruse@...>
Date: 2011-12-12T10:59:18+09:00
Subject: [ruby-core:41599] Re: My bug evaluation criteria

Thank you for documentation,

> I don't think there is an official way to judge a bug, but for me it
> comes to a comparison between the existing behavior and the proposed
> behavior that would fix the bug.

As matz said on [ruby-core:41595], there are some more arguments.
But the way is correct, I consider the same things on judging.

> Here's my "bug evaluation criteria" (v 1.0):
>
> To pass the "strict superiority test" (SST), the proposed behavior is judged on:
> a) usefulness,
> b) consistency,
> c) intuitiveness and
> d) big O level of performance
> It must be superior to the existing behavior on at least one of these
> criteria, and not inferior in any.

I add one more, the value of behavior's brekage is prior than compatility.

> A bugfix is "straightforward" if it is SST and when the proposed
> behavior passes the "no intent test" (NIT):
> 1) doesn't contradict the existing documentation,
> 2) doesn't break existing tests and
> 3) the source code doesn't clearly indicate that the current behavior
> is intended

Add to 3), "the source code nor the commit message".

> A bugfix is "clear" if it is straightforward (SST+NIT) and if the
> existing behavior passes the "clear defect test" (ODT):
> 1) it crashes Ruby,
> 2) it contradicts the documentation or

On Ruby, a document and code sometimes conflict.
On such case, they theirselves are equal, but as I wrote above,
code change introduces incompatibility.
So usually the code is prior to the documentation.

> 3) it is platform dependent (when it can reasonably be platform independent)

What is "reasonably be platform independent" is not clear yet,
for example libc/libm behavior and floating point case.
If there is no such external reasonable difference, it should be a bug.

And I add another one:
4) regression (it works well before, but is broken by some side effects)

> What I've typically have been doing so far:
> For non-straightforward cases (failing SST or NIT), I expose the issue
> as clearly as possible along with the best solution I can come up with
> and hope we can all come to an agreement after discussion.
>
> For straightforward bugfixes (SST+NIT), I post my proposed patch and
> say I'll commit it in a couple days unless someone points out
> something I missed.
>
> For clear bugfixes (SST+NIT+ODT), I commit the fix to trunk at the
> same time as I report it on redmine. It appears I should also wait a
> couple of days before I commit in these cases.

I finally add absolute principle, matz decides what is a bug.


Your way is almost correct.
This document will help other new commers.

-- 
NARUSE, Yui �<naruse@airemix.jp>