From: Caleb Clausen <vikkous@...>
Date: 2010-05-12T01:16:54+09:00
Subject: Re: Unit Test of method calling system() - how?

On 5/11/10, Martin Hansen <mail@maasha.dk> wrote:
> @Caleb
>
>> I would extract a helper from both methods which does the bulk of the
>> work (the first 2 lines) but does not exit. Then test the helper and
>> don't bother testing the methods that call it.
>
> Hm, I thought all this unit testing was about testing the interface -
> these are private methods. But perhaps this is one of those rare cases
> where you have to do all sorts of tricks? I was wondering if a special
> class for printing usage would be in order?

"Testing the interface" means different things to different people, I
guess. I think it is fairly obvious that the method below does not
need any tests, it's clear to see that it's correct just by looking at
it:

def foo_and_exit
  foo
  exit
end

I'm assuming that foo itself is tested, of course.

Yes, foo may not be the direct interface as seen by callers, but it is
very very close to it. Good enough for government work, as a former
colleague used to say.

I encourage you not to use mocks as others in this thread are
suggesting. Mocks divorce your tests from reality and force you to
test the implementation rather than the interface. The fact that you
program's usage feature invokes the print_wiki command is surely an
internal detail, not an essential part of its interface, yes? Binding
your test to that internal detail is a greater sin than skipping the
test of exit in my opinion.

Mocks are necessary when testing externalities like databases and
network servers which are hard to set up or may not be available when
tests are run. So, ask yourself this question: is it reasonable to
expect that print_wiki will always be available when your tests are
run? Is it something that will already be installed or be fairly easy
to install and set up?

>> You then have the problem that this print_wiki command is printing
>> something to stdout, which you want to test. Either capture stdout in
>> the test and verify it contains what you want, or use backticks
>> (`...`) instead of system in your helper method.
>
> Actually, 'print_wiki' outputs to stderr - which makes sense if you are
> executing a command and piping the output to a file. You want to see the
> usage info as an indicator that your did something wrong (if you did
> mess up), and not have the usage information in the output file!

All right, stderr then. Same problem really. Except backticks won't
work for you.

There are some existing utilities out there for capturing stderr and
stdout, but I'm not sure what to suggest. It's also pretty easy to
write your own.

>> Since print_wiki might change its behavior, you might think about
>> making the assertion that tests print_wiki's output string somewhat
>> loose.... just verify that some expected substring occurs in there
>> somewhere, rather than verifying that it is exactly equal to some
>> expected string.
>
> Yup, that makes sense.
>
>> $0 is also going to present you some problems in a test... it will not
>> be the same value in the test as it is when run from your main
>> program. Not sure what to suggest about that without more context.
>
> Better to supply the base name of the script as an argument?

Yeah, probably.

>> 3 other points:
>>
>>   looks like both methods have an unintended extra trailing + on their
>> first line
>
> No??? On my screen they end with + ".wiki".

Ah, pastie hid their horizonal scroll bar way down at the bottom where
I couldn't see it. My mistake. ( Your love of excessive indentation
isn't helping here. ;)

>>   standard style is to use 2 spaces for indentation; you appear to be
>> using 8
>
> I do use a tab space of 2, however, this is the result of pasting from
> vi and then tabs are expanded wildly (I have no solution for this).

I suggest that you not use tabs. Use 2 spaces.