[ruby-core:28012] Re: [Bug:trunk] some behavior changes of lib/csv.rb between 1.8 and 1.9

Hi,

On Tue, Feb 2, 2010 at 5:18 AM, Vladimir Sizikov <[e-mail]> wrote:
> Hi,
>
> On Tue, Feb 2, 2010 at 1:17 PM, Yusuke ENDOH <[e-mail]> wrote:
>> 2010/2/1 James Edward Gray II <[e-mail]>:
>>> I say that meaning that CSV has a lot of tests in Ruby itself. oes RubySpec not make an effort to port existing test suites?
>
> I'd say that a lot of tests in Ruby are very good candidates for
> porting to RubySpec. But the devil is always in details: which tests
> are good RubySpec ones and which are not.

Yes, we make an attempt to port good tests from other test suites
including MRI's tests. However, someone has to volunteer to do the
work.

>
>> Here is just my personal opinion: ubySpec is (at least, aim to be)
>> "spec", not test suite.
>>
>> The library "spec" can be referred to know the standard, strict and
>> guaranteed behavior of library. n the sense, it is similar to
>> document, rather than test suites.
>> "The code is the spec" philosophy is too cumbersome for the purpose.
>> It is difficult to know what behavior is guaranteed. nd, some
>> optimization and refactoring may lead to spec change easily.
>
> I have a bit different view here.
>
> Personally, I'd consider anything (reasonable) that any
> external/public Ruby library or any other 3rd party Ruby code expects
> from the Ruby implementation to be a RubySpec worthy material.
> Otherwise, those libs/application could be working differently on
> different implementations, which is always not good. :)
>
> At a minimum, Rubyspec should have tests for public API behavior, including:
> * Regular use cases and boundaries
> ideally, with Equivalence Class Partitioning
> http://www.testing-world.com/58828/Equivalence-Class-Partitioning)
> * Invalid/exceptional use cases (how API/impl react to those
> invalid/exceptional situations)
> * All the assertions explicitly stated in the public/official
> documentation (all *testable* assertions, I mean).
>
> What are the not-so-good candidates for RubySpec:
> * white-box tests
> * tests for private API, and for internal implementation-details
> * most of regression tests (they are implementation specific, typically)
> * heavily platfrom-dependent or platfrom-specific behaviors (very hard
> to maintain such tests and keep the sanity)
> * performance/stress tests

Vladimir has explained this very well here. (Thanks Vladimir.)

There is an explanation of rubyspec on the wiki
http://rubyspec.org/wiki/rubyspec, especially under the Style Guide.
Please also see this thread on the rubyspec ML
http://groups.google.com/group/rubyspec/browse_frm/thread/36a082f4db5f155b#.
We can improve these explanations if you have specific questions.

Cheers,
Brian

>
> For tests that are in Ruby repository, while there are lots and lots
> of good tests that are very suitable for RubySpecs, they often mixed
> among other ones which are not ideal candidates, so it could take
> quite an effort to detect which ones belong to which category and to
> port only them (and to check that RubySpec doesn't have similar tests
> already). Big task.
>
> In JRuby, we tend to encourage to write RubySpec tests first, and only
> if there is a good reason why such test is not good for RubySpec, only
> then we end up with jruby-specific tests. We also use various
> 3rd-party test suites, which we also trying to move to RubySpec, where
> appropriate, to reduce the number of such 3rd-party suites and to
> simplify test maintenance.
>
> Thanks,
> -Vladimir
>
>> In addition, test suites may include "white-box test," which is
>> involved with the specific implementation.
>> For example, when yet another CSV library appears (like FasterCSV
>> for old CSV) and aims to be strictly compatibile to lib/csv.rb,
>> the test suites may be too implementation-specific to be used as
>> comformance test.
>>
>>
>> Well, in [ruby-core:27930], to be exact, I had to ask you "are these
>> new behaviors guaranteed (at least in 1.9 series)?"
>>
>> --
>> Yusuke ENDOH <[e-mail]>
>>
>>
>
>