Re: testy.rb - ruby testing that's mad at the world

On Mar 29, 2009, at 1:51 PM, Brian Candler wrote:

> Ara Howard wrote:
>>       result.check :name, :expect => 42, :actual => ultimate.answer
>
> I'm afraid I'm missing something. Why is this better than
>
>  assert_equal 42, ultimate.answer, "name"


you can basically do that too, but i continually forget which is  
expected and which is actual and, as you know, that's a slippery error  
to track down at times.


>>   . testing should improve your code and help your users, not make
>> you want to
>>   kill yourself
>
> Hear hear to that!
>

;-)

>> requiring programmers learn exactly 2 new method calls
>
> Well, it would be nice to know what those 2 methods calls were, and
> their semantics, without reverse-engineering the code. Are these
> Testy.testing and Result#check ?
>

yes.  that is it.

> I did find the gem, and installed it, but the generated rdoc is  
> entirely
> free of comments or explanation.
>


what are these comments you speak of?

seriously, this is an experiment at this point but they will be  
forthcoming if it sprouts wings



> The two samples don't really help me understand why testy is good or  
> how
> to use it effectively, since they are both more verbose that what I
> would have written in test/unit to get the same result.
>
> How would I do something equivalent to these?
>
>   assert_raises(RuntimeError) { somecode }
>
>   assert_match /error/i, response.body
>

at this point my feeling is that breaks ths concept of BDD

"Using examples to describe the behavior of the application, or of  
units of code"

because littering the example code with esoteric testing framework  
voodoo turns it into code in the testing language that does not  
resemble how people might actually use the code - at least not without  
mentally unraveling considerable indirection.  i always end up writing  
both samples and tests - one of the goals of testy is that, by having  
a really simple interface and really simple human friendly output we  
can just write examples that double as tests.  in the latest testy you  
can do this



cfp:~/src/git/testy > cat a.rb
require 'testy'

Testy.testing 'my lib' do

   test 'foo' do |result|
     list = [42]
     result.check :fooness, :expect => 42, :actual => list.last
   end

   test 'bar' do |result|
     list = [42.0]
     result.check :barness, :expect => 42.0, :actual => list.last
   end

end


get a listing of which tests/examples i can run

cfp:~/src/git/testy > ruby -I lib a.rb --list
---
- foo
- bar


run one of them (actually regex matching so you can select more that  
one)


cfp:~/src/git/testy > ruby -I lib a.rb bar
---
my lib:
   bar:
     success:
       barness: 42.0


you can also do something like this (experimental) to just make a  
simple example

cfp:~/src/git/testy > cat a.rb
require 'testy'

Testy.testing 'my lib' do

   test 'just an example of summing an array using inject' do
     a = 1,2
     a.push 3
     sum = a.inject(0){|n,i| n += i}
   end

end



cfp:~/src/git/testy > ruby -I lib a.rb 'just an example'
---
my lib:
   just an example of summing an array using inject:
     success: 6


testy will just display the return value if no results are explicitly  
checked but, of course, exceptions are still reported and cause a  
failed test in the normal way.


so the goal is making it even easier to have a user play with your  
tests/examples to see how they work, and even to allow simple examples  
to be integrated with your test suite so you make sure you samples  
still run without error too.  of course you can do this with test/unit  
or rspec but the output isnt' friendly in the least - not from the  
perspective of a user trying to learn a library, nor is it useful to  
computers because it cannot be parsed - basically it's just vomiting  
stats and backtraces to the console that are hard for people to read  
and hard for computers to read.  surely i am not the only one that  
sometimes resorts to  factoring out a failing test in a separate  
program because test/unit and rspec output is too messy to play nice  
with instrumenting code?  and that's not even getting to what they do  
with at_exit exception raising...



> I think I'd also miss the ability to have setup and teardown before  
> each
> test (something which 'shoulda' makes very simple and effective).


yeah that's on deck for sure.  i *do* really like contexts with  
shoulda.  but still


cfp:/opt/local/lib/ruby/gems/1.8/gems/thoughtbot-shoulda-2.9.1 > find  
lib/ -type f|xargs -n1 cat|wc -l
     3910


if we accept the research and assume that bugs scale linerarly with  
the # of lines of code this is not good for robustness.  this is one  
of my main gripes with current ruby testing - my current rails app has  
about 1000 lines of code and 25,000 lines of testing framework!


>
>
> Please don't get me wrong - I'm absolutely interested in something  
> which
> will make testing simpler and easier, if I can understand how to use  
> it
> effectively.


feedback is awesome - this is an new idea i'm just fleshing out so i  
really appreciate it.

cheers.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama