On 10-Sep-09, at 1:52 PM, Eliot Miranda wrote:

> Hi All,
>
>     if I want to run through a complex test that tests a number of  
> combinations, some of which have failures, its convenient to wrap  
> the test in an exception handler for TestFailure and run the tests  
> resuming on each failure.  But I can't because TestFaiure isn't  
> resumable.  Any good reason why?
>
> e.g. I'm currently testing the generation of a divide/remainder  
> routine for the x86 where I'm enumerating over many combinations of  
> register quads, dividend, divisor, quotient, remainder, and I'd like  
> to do the following to count how many failures I'm getting with each  
> tweak of the register juggling code:

Well, SUnit wasn't designed to be used that way. Each test is supposed  
to have a binary result. Either it passes, or it doesn't. Sometimes  
the UI distinguishes between an assertion failing and an error, but  
that's sort of incidental. So TestFailure isn't resumable because we  
already have a result for the test: it failed. There's no need to  
continue executing the test, because further failures don't give any  
new information. In fact, further assertions are meaningless - we  
already know that *something* isn't as it should be. If further  
assertions fail, does that indicate additional problems, or the same  
one manifesting its self in different ways?

The "proper" way to do this in SUnit would be to have a whole bunch of  
different tests that run the same code with different inputs. That's  
probably not practical.  A better way would be to write your own test  
harness. SUnit is great, but it's not the last word on testing, right?

Colin