On 14-Aug-08, at 5:50 PM, Andreas Raab wrote:

> Colin Putney wrote:
>> It doesn't need to be anything that exotic. If some dependency  
>> between tests *does* creep in, the shuffling will cause the failure  
>> to be intermittent. Test B fails only when it comes after Test A,  
>> but before TestC or something like that. If the suite is shuffled  
>> on every run it'll be hard to reproduce the failure. A better  
>> strategy would be to shuffle when the suite is created, then always  
>> run in the same order. That way you can reproduce the sequence that  
>> failed, and figure out how to fix it.
>
> Alternatively, only shuffle if the last run succeeded. If it fails,  
> re-run the tests in the same order until it's been fixed.

Even better.

Colin