[squeak-dev] Re: ENH: SUnit - shuffling the order tests are executed in

Zulq Alam me at zulq.net
Fri Aug 15 00:04:48 UTC 2008


Is your gut thinking something like this?

   - A test fails
   - You do something you believe fixes it
   - The test passes when rerun in different sequence
   - You move on without actually fixing the bug

I think that simply rerunning the failed test after you believe you've 
fixed the problem may not be sufficient because other tests may have 
modified the TestResource since the failure.

For the shuffle to be useful you need to be lucky. Lucky enough that the 
sequence that was causing a test to pass incorrectly is broken. Then you 
need to be just as lucky to repeat the failing sequence.

Z.

David Pennell wrote:
> I like the ability to shuffle the test order, but I think this should be 
> a preference and that you should be able to re-run the same exact test 
> sequence.
> I can't give you a crisp reason, but my gut tells me that you will want 
> a way to re-run the same test sequence when things go strange. 
> 
> -david
> 
> On Thu, Aug 14, 2008 at 11:12 AM, Brent Pinkney <brent at zamail.co.za 
> <mailto:brent at zamail.co.za>> wrote:
> 
>     Hi,
> 
>     I would like to propose an enhancement to SUnit:
> 
>     Currently the order in which TestCase #test... methods are executed
>     by TestSuite is fixed.
>     This can cause dependencies to creep in if
>            a) the TestCase class uses a TestResource and
>            b) one #test.. method alters an aspect of the TestResource
>     that another #test... assumes is invariant.
> 
>     By adding #shuffle to TestSuite >> run:, it is more likely that this
>     corruption is exposed.
> 
>     viz.
>            TestSuite >> run: aResult
>                    self tests shuffled do:
>                            [ :each |
>                            self changed: each.
>                            each run: aResult ]
>            !
> 
>     Please comment and I will submit a fix.
>     PS. Who maintains SUnit ?
> 
>     Brent
> 
> 
>     --
>     Brent
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 




More information about the Squeak-dev mailing list