From: Marc-Andre Lafortune <ruby-core-mailing-list@...>
Date: 2011-12-12T01:54:02+09:00
Subject: [ruby-core:41592] My bug evaluation criteria

I don't think there is an official way to judge a bug, but for me it
comes to a comparison between the existing behavior and the proposed
behavior that would fix the bug.

Here's my "bug evaluation criteria" (v 1.0):

To pass the "strict superiority test" (SST), the proposed behavior is judged on:
a) usefulness,
b) consistency,
c) intuitiveness and
d) big O level of performance
It must be superior to the existing behavior on at least one of these
criteria, and not inferior in any.

A bugfix is "straightforward" if it is SST and when the proposed
behavior passes the "no intent test" (NIT):
1) doesn't contradict the existing documentation,
2) doesn't break existing tests and
3) the source code doesn't clearly indicate that the current behavior
is intended

A bugfix is "clear" if it is straightforward (SST+NIT) and if the
existing behavior passes the "clear defect test" (ODT):
1) it crashes Ruby,
2) it contradicts the documentation or
3) it is platform dependent (when it can reasonably be platform independent)


What I've typically have been doing so far:
For non-straightforward cases (failing SST or NIT), I expose the issue
as clearly as possible along with the best solution I can come up with
and hope we can all come to an agreement after discussion.

For straightforward bugfixes (SST+NIT), I post my proposed patch and
say I'll commit it in a couple days unless someone points out
something I missed.

For clear bugfixes (SST+NIT+ODT), I commit the fix to trunk at the
same time as I report it on redmine. It appears I should also wait a
couple of days before I commit in these cases.

I will also have to give more justification for future bug reports.

Comments my evaluation criteria most welcome.