From: Ryan Davis Date: 2009-07-17T13:55:06+09:00 Subject: Re: Test::Unit run order On Jul 16, 2009, at 15:30 , Dylan wrote: > On Jul 16, 12:16 pm, Ryan Davis wrote: >> On Jul 15, 2009, at 10:15 , Dylan wrote: >> >>> Hey there! I had a question about the order tests are run in >>> Test::Unit. I have a class with about 10 tests and I'd like to be >>> able >>> to better control the order they run in. Right now they just run >>> alphabetically [...] >> >> order dependent tests are a bug. every single test should be able to >> run and pass in isolation. eg: >> >> % ruby -Ilib test/test_something.rb -n test_one_test >> >> should always pass no matter what the file or test name is. > > Which is nice in theory, but not always the best option. In my case, > I'm testing a web app, and I need to test different things in > different logins, so I'd like to order it so I log in to a normal user > account, run some tests, then log in to an admin account, run some > other tests, etc... without having to re-login for every test, which > wastes a lot of time and doesn't give me anything in return other than > being able to say "Ooh, my tests are independent" (I already have a > separate test for the login system). If the login system doesn't work, > none of the tests are going to pass even if I had separate login > scripts for each one. > > So I either need a way to order the tests so I can use certain logins > for certain tests, or I need a way to create only 1 XML file from > CI_reporter from multiple suites [...] ...or you could deal with the real problem of having slow tests and not the symptoms of it. I've talked about this a lot in the past[*], so I'm not going to repeat myself here. I'll simply reassert: _all_ tests should _always_ pass when run by themselves. and I'll expand on that: _all_ tests should run fast enough that they're a pleasure to run _all_ the time. [*] See my talks on Ruby Sadism, Electric Kool-ade Acid Testing, and many others for related infos.