Re: testy.rb - ruby testing that's mad at the world

On Mar 30, 2009, at 2:12 AM, Brian Candler wrote:


( a LOT of good stuff to which i'll reply to selectively here)



> Perhaps - but it's one rule that only needs to be learned once.
>
> I notice that testy supports check <name>, <expected>, <actual> too.
>
> Testy does (intentially) force you to name your tests, whereas
> Test::Unit will happily let you write
>
>  check <expected>, <actual>
>
> I really don't like having to name each assertion, maybe because I'm
> lazy or maybe because it feels like DRY violation. I've already said
> what I want to compare, why say it again?

hmmm.  yeah i see that, but disagree that the effort isn't worth it  
for the *next* programer.


>>
>
> Hmm, this is probably an argument *for* having a DSL for assertions  
> - to
> make the assertions read as much like example code ("after running  
> this
> example, you should see that A == B and C < D")
>
> Neither
>
>  result.check "bar attribute", :expected => 123, :actual => res.bar
>
> nor
>
>  assert_equal 123, res.bar, "bar attribute"
>
> reads particularly well here, I think.

yeah i agree.  i'm open to suggestion, just has to be very very simple.


> Going too far this way down this path ends up with rspec, I think.
>
> In fact, I don't really have a problem with writing
>
>  res.foo.should == 456
>
> The trouble is the hundreds of arcane variations on this.


bingo!  i really think the key is having *one* assertion method.

> You solve this problem by only having a single test (Result#check),  
> and
> indeed if rspec only had a single method (should_equal) that would be
> fairly clean too. However this is going to lead to awkwardness when  
> you
> want to test for something other than equality: e.g.

i dunno - ruby is pretty good at this

   value = begin; object.call; rescue => e; e.class; end

   result.check :error, SomeError, value

that seems perfectly fine to me.  gives me an idea though - maybe  
check should take a block

   result.check(:error, SomeError){ something that raises an error }

and use the block to get the actual value as in

   value = begin; block.call; rescue Object => e; e; end


>
>   result.check "foo should contain 'error'", foo, :=~, /error/
>
> But again this is getting away from real ruby for the assertions, in
> which case it isn't much better than
>
>   assert_match /error/, foo, "foo should contain 'error'"
>
>   assert_match /error/, foo  # lazy/DRY version
>

check actually uses === for the comparison so you can do

   result.check :instance_of, SomeClass, object

   result.check :matches, /pattern/, string


i need to nail that down though.



>> get a listing of which tests/examples i can run
>
> Yes, parseable results and test management are extremely beneficial.
> Those could be retro-fitted to Test::Unit though (or whatever its
> replacement in ruby 1.9 is called)
>
> Getting rid of the at_exit magic is also worth doing.
>

i actually thought of simply patching test/unit... but then there are  
good test names, contexts, etc.


>> you can also do something like this (experimental) to just make a
>> simple example
>>
>> cfp:~/src/git/testy > cat a.rb
>> require 'testy'
>>
>> Testy.testing 'my lib' do
>>
>>   test 'just an example of summing an array using inject' do
>>     a = 1,2
>>     a.push 3
>>     sum = a.inject(0){|n,i| n += i}
>>   end
>>
>> end
>
> Nice, could perhaps show the (expected) result inline too?


well - here actual always === expected with my current impl.  it's  
essentially example code wrapped in assert_nothing_raised ;-)

>>
> I agree. Part of the problem is that when one thing is wrong making 20
> tests fail, all with their respective backtraces, it can be very  
> hard to
> see the wood for the trees. What would be nice would be a folding-type
> display with perhaps one line for each failed assertion, and a [+] you
> can click on to get the detail for that particular one.
>

funny you mention that as i also hate that.  my first version of testy  
actually just failed fast - if one test failed the code reported and  
aborted.  maybe i should consider going back to that?  i am finding  
that good output makes a ton of failures much easer - even using less  
and searching is easier with testy  that anything else.

>> if we accept the research and assume that bugs scale linerarly with
>> the # of lines of code this is not good for robustness.
>
> I disagree there - not with the research, but the implied conclusion
> that you should never use a large codebase. Shoulda works well, and  
> I've
> not once found a bizarre behaviour in the testing framework itself  
> that
> I've had to debug, so I trust it.

shoulda does work well - i stole it's context concept just yesterday ;-)


>> about 1000 lines of code and 25,000 lines of testing framework!
>
> Yeah, but how many lines of Rails framework? :-)
>


i used to be on the ramaze list alot too you know ;-)


a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama