From: Florian Gross Date: 2007-07-31T20:14:59+09:00 Subject: Re: Ruby test suites? On Jul 31, 10:35 am, Alex Young wrote: > Has anyone got an overview of the current status of the various Ruby > spec suites? I know of at least 3 (rubicon, jRuby's tests, and > rubinius' specs), but I've got no idea how complete or active they are, > or which Ruby version they're attempting to track. A wild guess says > that jRuby's has probably got the best coverage, but is there anyone > actually knee-deep in them that can give me a better idea? You need to differentiate between JRuby's own tests and the ones they are importing. They also include Rubinius and Rubicon ones in their repository. I'm working together with Rubinius to make their spec better. I'd say that it is the most complete spec for the classes we have already completed. Here's some numbers that are easy to extract: (I'd really like someone to do a more complete comparison on this, though.) Lines of code: freya:~/svn/rubinius/trunk flgr$ wc -l spec/core/*.rb | sort: [...] 1000 spec/core/file_spec.rb 1331 spec/core/hash_spec.rb 3473 spec/core/array_spec.rb 5540 spec/core/string_spec.rb 17831 total Note that I'd consider hash and array quite complete. You can have a look for yourself at http://code.fallingsnow.net/svn/rubinius/trunk/spec/core/ Number of assertions: (this could be off a bit) freya:~/svn/rubinius/trunk flgr$ grep -R should spec/core/ | grep -v '#' | egrep -v specify\|it | wc -l 11058 Number of tests: freya:~/svn/rubinius/trunk flgr$ bin/mspec -t r spec/core 2> /dev/null | grep examples 2243 examples, 38 failures According to the out-dated comparison from Christian Neukirchen (http://chneukirchen.org/blog/archive/2007/05/analyzing-the-jruby-test- suite.html) this would make us the biggest test suite. We've more than doubled from 8.4 kLOC to 17.8 kLOC in two months. Oh, and I totally ignored the non-core specs. There also is language and library specs, but those might be less complete than the core ones. And we're always looking for new contributors. Specifying Ruby completely is a huge task, but one that contributes a lot to the community by making it possible to find bugs in implementations that will cause compatibility issues if they go overlooked. If we want to have one single Ruby language in the future instead of five different and incompatible flavours we will need this.