From: shevegen@... Date: 2018-10-04T21:34:55+00:00 Subject: [ruby-core:89281] [Ruby trunk Misc#15202] Adding Coverity Scan to CI to see the result casually Issue #15202 has been updated by shevegen (Robert A. Heiler). > However there are around 1000 "not important" issues. I think the ruby core team always likes to remove bugs from ruby, if they are bugs that is. On mruby one can see that some sort of systematically try to find ways to break ruby, if you look at some of the issues there. :P Many examples are really quite contrived. I think that makes a huge difference towards bugs that are caused by a "regular" use of ruby, by a real person - that is, you write ruby code in some application and discover that things are failing or breaking. I remember being able to do so with the ruby-gtk bindings - not on purpose, but simply because some of the code is quite long, and very old (written by me too), and it is hard to make changes (I got a bit lazy from gtk2 to gtk3 so I don't write anywhere near as much gtk-code in ruby these days); and a bit cumbersome to find and report bugs too. (Even if Kouhei Sutou does a great job improving, fixing bugs and maintaining the ruby-gnome bindings.) But I think developer manpower is still limited, in numbers alone. Perhaps not only in numbers but in knowledge too. I am sure there are many people who know ruby quite well, but significantly fewer who know both ruby and C very well. I also don't think there is any ... realistic way that the ruby bug tracker could handle reports like +1000 bugs started. :) The total amount of bugs reported on the issue tracker so far here is 8307. Even though nobu is the patch monster from Mount Fuji with 20 arms, even he may have a hard time tackling 1000 different bugs (even if they are well documented and reproducible), What may be useful, perhaps, is if there could be some way to not only find "real" bugs, but those that seem to be more "promising" and worthwhile to fix. Like to somehow semi-automatically "promote" the more outstanding bugs from Coverity Scan or any other method that may have a real, larger impact. And then say that you may distribute a few of these onto the bug tracker here somehow, distributed somewhat slowly so that they would not take too much time away (I think people also prefer to add features since it is something new, whereas fixing bugs is not quite as fun as playing around with new features :D ). Developer time is limited and I think different members of the ruby core team said it before, that often priority will be given to "real" problems, as opposed to theoretical problems or ... not sure about the word ... very unlikely bugs? Anyway, perhaps some people like to look at CI to discover "worthwhile" bugs to fix. A few bug reports in the last ~3 months really amazed me, how they were found. Some other bugs (or "surprising behaviour") are interesting too, like the one where << can change the encoding of Strings (https://bugs.ruby-lang.org/issues/14975), even though that one was found by regular use of ruby rather than any automated or semi-automated CI (may not be a bug and perhaps just a specific behaviour but it still surprised me when I read it). ---------------------------------------- Misc #15202: Adding Coverity Scan to CI to see the result casually https://bugs.ruby-lang.org/issues/15202#change-74310 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal * Assignee: ---------------------------------------- Recently I reported issues detected by code analysis tool mainly using Coverity Scan. The 9 issues categorized as "important" was fixed by #15116. (Thank you!) > https://bugs.ruby-lang.org/issues/15116 > > However as a "not important" issues, around 1000 issues were detected by the tool for the ruby 2.5.1. > I am considering how to deal with this or report those. > I might open an another ticket for that. However there are around 1000 "not important" issues. Right now I do not share the report file (840KByte) for that, because it makes people tired. If someone want to see it, I am happy that to share it here as an attachment. Instead of that, It looks good to me that someone could see the result of coverity scan casually anytime to fix those in long term. What I want to propose it to add coverity scan test on rubyci or Travis CI. I do not know how coverity scan is used on current Ruby project as a regular workflow. But I could see it is actually used from the setting [2] and some tickets. [3] I found how to use Coverity Scan on Travis CI [4], and the used cases [5][6]. How do you think? * [1] rubyci: https://www.rubyci.org/ * [2] coverity scan ruby project: https://scan.coverity.com/projects/ruby * [3] coverity scan used tickets: * https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61862 * https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/55763 * https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/50734 * [4] How to use Coverity Scan on Travis CI: https://scan.coverity.com/travis_ci * [5] The cases for coverity scan on Travis CI: * https://github.com/nanoporetech/scrappie/blob/master/.travis.yml * https://github.com/JanusGraph/janusgraph/blob/master/.travis.yml -- https://bugs.ruby-lang.org/ Unsubscribe: