From: ruby-core@... Date: 2017-11-18T05:26:40+00:00 Subject: [ruby-core:83811] [Ruby trunk Feature#13901] Add branch coverage Issue #13901 has been updated by marcandre (Marc-Andre Lafortune). We're thinking about an output format (for simplecov in particular) and I wanted to suggest for branch, method and "callsite" coverage to separate the "map" data from the "runs" data. What I call the "map" data is all the information that specifies what the branches/methods/callsites are (type, line, column, ...), while the "runs" data is the number of time it is run (or eventually `nil` if it is not executable). ``` { "path" => { :callsite_map => [range, other_range, ���], :callsite_runs => [runs, other_runs, ���], :branch_map => [branch, other_branch, ���], :branch_runs => [[runs, ���], [other_runs, ���], ���], ... }, "other_path" ��� } ``` The reason to separate the run data from the map is to allow for easier merging, and more efficient output in different files more efficiently too. The map can be written once while the runs can be written as many times as needed and merged at the end. Imagine a big app that runs a test suite in 42 parallel processes, and that callsite coverage is generated. One could generate the callsite map once, write it down, and each of the 42 process could write the runs at the end. The map for each callsite would probably be at least 4 integers (start_line, start_column, end_line, end_column), with maybe a type (:send, :def, etc.), so would be quite a bit larger than each runs file. The 43 files would be easier to merge later on too. This splitting is what Istanbul (the coverage tool of Google's Babel) is doing. It is the direction that DeepCover is likely to take too. Note that this is "compatible" with the existing line coverage in the sense that for line coverage, the "map" is trivial and thus not needed; only the runs are output. It's only for the other coverage types that the map has to be specified. ---------------------------------------- Feature #13901: Add branch coverage https://bugs.ruby-lang.org/issues/13901#change-67844 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- I plan to add "branch coverage" (and "method coverage") as new target types of coverage.so, the coverage measurement library. I'd like to introduce this feature for Ruby 2.5.0. Let me to hear your opinions. ## Basic Usage of the Coverage API The sequence is the same as the current: (1) require "coverage.so", (2) start coverage measurement by `Coverage.start`, (3) load a program being measured (typically, a test runner program), and (4) get the result by `Coverage.result`. When you pass to `Coverage.start` with keyword argument "`branches: true`", branch coverage measurement is enabled. test.rb ~~~ require "coverage" Coverage.start(lines: true, branches: true) load "target.rb" p Coverage.result ~~~ target.rb ~~~ 1: if 1 == 0 2: p :match 3: else 4: p :not_match 5: end ~~~ By measuring coverage of target.rb, the result will be output (manually formatted): ~~~ $ ruby test.rb :not_match {".../target.rb" => { :lines => [1, 0, nil, 1, nil], :branches => { [:if, 0, 1] => { [:then, 1, 2] => 0, [:else, 2, 4] => 1 } } } ~~~ `[:if, 0, 1]` reads "if branch at Line 1", and `[:then, 1, 2]` reads "then clause at Line 2". So, `[:if,0,1] => { [:then,1,2]=>0, [:else,2,4]=>0 }` reads "the branch from Line 1 to Line 2 has never executed, and the branch from Line 1 to Line 4 has executed once." The second number (`0` of `[:if, 0, 1]`) is a unique ID to avoid conflict, just in case where multiple branches are written in one line. This format of a key is discussed in "Key format" section. ## Why needed Traditional coverage (line coverage) misses a branch in one line. Branch coverage is useful to find such untested code. See the following example. target.rb ~~~ p(:foo) unless 1 == 0 p(1 == 0 ? :foo : :bar) ~~~ The result is: ~~~ {".../target.rb" => { :lines => [1, 1], :branches => { [:unless, 0, 1] => { [:else, 1, 1] => 0, [:then, 2, 1] => 1 }, [:if, 3, 2] => { [:then, 4, 2] => 0, [:else, 5, 2] => 1 } } }} ~~~ Line coverage tells coverage 100%, but branch coverage shows that the `unless` statement of the first line has never taken true and that the ternary operator has never taken true. ## Current status I've already committed the feature in trunk as an experimental feature. To enable the feature, you need to set the environment variable `COVERAGE_EXPERIMENTAL_MODE` = `true`. I plan to activate this feature by default by Ruby 2.5 release, if there is no big problem. ## Key format The current implementation uses `[