Am 27.03.2006 um 12:54 schrieb Markus Gaelli:

>
> On Mar 27, 2006, at 12:28 PM, Bert Freudenberg wrote:
>
>>>> And you are suggesting to indicate clearly, which tests depend  
>>>> on some external resource?
>>>
>>> Well, really, what I'm looking for is something that instead of  
>>> saying "all tests are green, everything is fine" says "all the  
>>> tests we ran were green, but there were various that were *not*  
>>> run so YMMV". I think what I'm really looking for is something  
>>> that instead of saying "x tests, y passed" either says "x tests,  
>>> y passed, z skipped" or simply doesn't include the "skipped" ones  
>>> in the number of tests being run. In either case, looking at  
>>> something that says "19 tests, 0 passed, 19 skipped" or simply "0  
>>> tests, 0 passed" is vastly more explicit than "19 tests, 19  
>>> passed" where in reality 0 were run.
>>>
>>> Like, what if a test which doesn't have any assertion is simply  
>>> not counted? Doesn't make sense to begin with, and then all the  
>>> preconditions need to do is to bail out and the test doesn't  
>>> count...
>>>
>>> In any case, my complaint here is more about the *perception* of  
>>> "these tests are all green, everything must be fine" when in  
>>> fact, none of them have tested anything.
>>
>> Other Unit Test frameworks support skipping tests. One pattern is  
>> to raise a SkipTest exception, in which case the test it added to  
>> the "skipped" list.
>>
>> The good thing about implementing this with exceptions is that it  
>> would work nicely even if the particular test runner does not yet  
>> know about skipping.
>>
>> Another nice XPish thing is to mark tests as ToDo - it's an  
>> expected failure, but you communicate that you intend to fix it soon.
>>
>> See, e.g., http://twistedmatrix.com/projects/core/documentation/ 
>> howto/policy/test-standard.html#auto6
>
> So the circle is closing... exceptions and preconditions again! ;-)
> So Andreas, want to introduce some ResourceNotAvailable and ToDo  
> exceptions ;-) , or do we get away without them and just throw a  
> PreconditionError that I was suggesting in an earlier thread?

It's all about communicating the test writer's intent to the test  
runner. And I think I'd prefer "x tests, y passed, z skipped" as  
Andreas suggested.

> As said in the previous mail ToDo's could be easily figured by just  
> sticking to the convention not to even start the method under test,  
> which is a good idea in that case anyhow.
> As a nice effect one would not even have to touch the tests later  
> when the method under test gets implemented.

However, you wouldn't get the "unexpected success" mentioned in the  
link above.

- Bert -