From: Eric Mahurin Date: 2005-09-27T02:10:10+09:00 Subject: random testing with Test::Unit Does anybody else do random testing on their ruby code besides me? I picked it up because of my background (IC design) where random testing is one of the verification techniques of hardware. So far, I've used 3 random testing strategies for my ruby code. Here are the different ways that the expected response is determined using those strategies: - compare multiple implementations of the same functionality (i.e. StringIO vs. IO, ordered hash vs. Hash). - AOP. Make a wrapper class or module that does pre and post checks of the methods. Use that wrapper class/module when testing. When the pre-check fails, the current test would be aborted/skipped (random generation of inputs yielded something invalid). When the post-check fails, the test would fail. - determine the expected response (from some random input) within the test suite. For all of these strategies, I've had to hack up Test::Unit in the same way to add various features. Here are the things I've added: + instead of running the test_* methods in a fixed order once, run them in a random order for N passes. + abort testing when the first failure is reached. + easy way to skip the rest of the current test (when the random input is invalid). Don't include this in the number of tests or mark it as skipped. + way to pass number of passes of the test_* methods on the command line. + display random seed and way of passing it in on the command line. + test suite should have access to the --verbose level or another switch to control the debugging verbosity. + way to pass command-line options down to the test suite to control various things - what methods to test, what classes to test, various other flags. I would like to see most of these (optional) features added to test/unit. Anybody else have opinions on this subject? ______________________________________________________ Yahoo! for Good Donate to the Hurricane Katrina relief effort. http://store.yahoo.com/redcross-donate3/