From: Florian Gross <flgr@...>
Date: 2004-12-22T03:17:02+09:00
Subject: Re: [ANNOUNCE] Ruby Coverage Validator

Stephen Kellett wrote:

> In message <32qvlvF3p991aU1@individual.net>, Florian Gross 
> <flgr@ccan.de> writes
> 
>>> Ruby Coverage Validator graphical, stats updated in real time as the 
>>> app  executes - enabling you to direct your testing sequence to 
>>> ensure maximum coverage if you are running interactively rather than 
>>> as a regression test. You can run merge results from one run into 
>>> another, or a central session - ideal for regression testing. You can 
>>> export the results in HTML or XML. Multiple views onto the same data, 
>>> etc.
>>
>> That sounds like it could be useful, though I'm not 100% sure how it 
>> would be used in practice.
> 
> Well, our C++/Java/Python customers seem to find it useful. If you were 
> performing a unit test, you'd probably find it useful to see which lines 
> a particular test executed. If you had all your tests running off a GUI 
> and could run each one on demand, you could view the code coverage 
> results as you executed each command. If the commands were separate Ruby 
> scripts you could view the results in the central session merged into 
> the current session (automatically if you choose) at the end of each 
> script run.
> 
> The HTML/XML export allow you to run your regression test suite 
> overnight and come in to work the next day with a session ready to use 
> (in the GUI) and also HTML or XML results for management. We supply XML 
> as well as HTML in case people want to build a custom report for 
> management rather than use the HTML report.
> 
> One of our customers (for the C++ product) went from spending 33 hours 
> to perform their regression test with a competing product to 8.5 hours 
> our C++ version of Coverage Validator. Ruby Coverage Validator uses a 
> similar GUI and data collection framework with custom bits for Ruby.
> 
> We've added various data export and data visualisation options as people 
> ask for them.

Ah, that's very interesting points. I can see how those features are 
useful in practice now.

>>>> I especially wonder if you are using trace_funcs (which can be quite 
>>>> slow)
>>>
>>>  Yes. Although the slowness would be compounded by using a trace func 
>>> written in Ruby. Our trace func is written in C++. Matz and a few 
>>> helpful people in this newsgroup provided enough information for us 
>>> to put things together after examining the source code and writing 
>>> quite a few test applications.
>>
>> Interesting -- I've not done this before, but if writing a trace_func 
>> in a lower level language speeds it up severely that might be a very 
>> important option. Thanks for mentioning this.
> 
> I thought it would be obvious :-) - I can't speak for Ruby users, but 
> the Python people routinely move more expensive operations over to 
> C/C++. I'd expect the same to be true for Ruby.

Sure, we do that, too. But I always felt that trace_funcs were too slow 
to have them implemented permanently. That might not be true when 
writing them in C, however. I'll have a look at that.