From: Luis Lavena Date: 2012-06-25T23:55:09+09:00 Subject: The Ruby greener test challenge (a Windows challenge) Hello, Since I started RubyInstaller 4 years ago, ensuring that Ruby worked on every single release took time. Back then, my hardware was limited, so running Ruby tests took a considerable amount of time (more than one cup of tea using tea cups scale). Not only that, even after 4 years and different versions of Ruby and Windows, Ruby still have some failures:    Finished tests in 526.522392s, 21.2128 tests/s, 3665.2629 assertions/s.    11169 tests, 1929843 assertions, 9 failures, 2 errors, 80 skips Wouldn't be awesome that Ruby own tests were all green on Windows? The truth is that having Ruby achieve `0F0E` (0 failures, 0 errors) will help detect real errors, not to mention automate the testing process, perhaps allowing teams have *nightly builds* of Ruby to play with. Following a similar idea from Ruby 1.9.3 Documentation Challenge [1], I would like to propose some test-fixing hunt! ## Objective Reach 0 failures and 0 errors for Windows (MinGW) in the upcoming weeks, is not that hard: 9 failures and 2 errors. The target is Ruby **trunk** (upcoming 2.0), but I expect these fixes be backported to Ruby 1.9.3 too. ## I have no idea how to compile things... I heard *I would like to help but have no idea how to compile Ruby*, and I know is not easy, so that is why I present to you the **Ruby Challenge Kit** Those of you who are well versed in the art of compiling can skip to the next section (Compiling). ### Ruby Challenge Pack The Ruby Challenge Pack is a single-file package that provides all the tools require to compile, test and fix Ruby. Once extracted, it provides: - Ruby 1.9.3-p194 (required as base to compile Ruby). - 32bits DevKit GCC 4.6.1 (based on TDM builds). - 32bits support libraries (OpenSSL, zlib, libffi, libyaml, etc). - Easy 1-click batch to start a tailored Command Prompt - Easy 1-step batch to compile Ruby and run basic tests - A document (README) describing how compile and send your fixes. What is not included and you need is: - A working Git installation on your computer (recommended is msysGit). - A clone of Ruby source from GitHub [2] Download here: http://cdn.rubyinstaller.org/archives/experimental/RubyChallengePack-v1.exe MD5 for verification: 0e6259c256beb5fba7e7ec53bd9e5c21 After download of the package, extract into a directory *without spaces* (e.g. C:\RubyChallenge) and check the included README for instructions (or keep reading) ## Compiling For those using the Ruby Challenge Pack, simply run `quick-compile.bat` as indicated in the documentation. For others, do what you do and always verify Ruby core tests pass:    make test Compiling will take long, depending on your system. You can go and grab a cup of tea while it completes. ## Running and fixing tests Once you've compiled Ruby, you can run the tests:    make test-all TESTS="-q" Above command will run **all** Ruby tests, which could take around 10 minutes. Once it completes, you should get a list that only contains the errors. For example, let's look at the following one:    [ 3348/11169] TestEnv#test_win32_blocksize = 0.00 s      4) Failure:    test_win32_blocksize(TestEnv) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_env.rb:404]:    Errno::EINVAL expected but nothing was raised. Notice the file where the test failed: `test/ruby/test_env.rb` and the line too. Now open your editor/IDE, find that file, open it, and locate the failing test. Figure out if the test is valid and Ruby is failing or is test that is not meant to be run on Windows. At any time, you can email RubyInstaller group [3] and ask questions, everybody will be pleased to help you out. Once you find a solution, modify the file and run the tests again, but this time just for a single file:    make test-all TESTS="-q ruby/test_env.rb" ## Sending your patches You fixed a test, great! Now is time to send this to Ruby developers so they can apply it. Open a ticket on Ruby-trunk [4] tracker and assign it to me (luislavena). I'll review and merge accordingly. To reduce changes people stepping into each other, please email RubyInstaller group [3] before to announce or see if others are working on that particular failure. ## A big thank you There is no such thing as small contributions, all contributions are valuable. Help us make a better Ruby. Thank you. -- [1] http://blog.segment7.net/2011/05/09/ruby-1-9-3-documentation-challenge [2] http://github.com/ruby/ruby [3] https://groups.google.com/forum/?fromgroups#!forum/rubyinstaller [4] http://bugs.ruby-lang.org/projects/ruby-trunk/issues/new -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry