[squeak-dev] Re: ENH: SUnit - shuffling the order tests are executed in

David Pennell pennell.david at gmail.com
Fri Aug 15 00:14:52 UTC 2008


Yep, I think that's what it was telling me.  Either that or I'm hungry :)

On Thu, Aug 14, 2008 at 7:04 PM, Zulq Alam <me at zulq.net> wrote:

> Is your gut thinking something like this?
>
>  - A test fails
>  - You do something you believe fixes it
>  - The test passes when rerun in different sequence
>  - You move on without actually fixing the bug
>
> I think that simply rerunning the failed test after you believe you've
> fixed the problem may not be sufficient because other tests may have
> modified the TestResource since the failure.
>
> For the shuffle to be useful you need to be lucky. Lucky enough that the
> sequence that was causing a test to pass incorrectly is broken. Then you
> need to be just as lucky to repeat the failing sequence.
>
> Z.
>
> David Pennell wrote:
>
>> I like the ability to shuffle the test order, but I think this should be a
>> preference and that you should be able to re-run the same exact test
>> sequence.
>> I can't give you a crisp reason, but my gut tells me that you will want a
>> way to re-run the same test sequence when things go strange.
>> -david
>>
>> On Thu, Aug 14, 2008 at 11:12 AM, Brent Pinkney <brent at zamail.co.za<mailto:
>> brent at zamail.co.za>> wrote:
>>
>>    Hi,
>>
>>    I would like to propose an enhancement to SUnit:
>>
>>    Currently the order in which TestCase #test... methods are executed
>>    by TestSuite is fixed.
>>    This can cause dependencies to creep in if
>>           a) the TestCase class uses a TestResource and
>>           b) one #test.. method alters an aspect of the TestResource
>>    that another #test... assumes is invariant.
>>
>>    By adding #shuffle to TestSuite >> run:, it is more likely that this
>>    corruption is exposed.
>>
>>    viz.
>>           TestSuite >> run: aResult
>>                   self tests shuffled do:
>>                           [ :each |
>>                           self changed: each.
>>                           each run: aResult ]
>>           !
>>
>>    Please comment and I will submit a fix.
>>    PS. Who maintains SUnit ?
>>
>>    Brent
>>
>>
>>    --
>>    Brent
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20080814/3083c93f/attachment.htm


More information about the Squeak-dev mailing list