On Mar 27, 2006, at 11:09 AM, Andreas Raab wrote:

> Markus Gaelli wrote:
>> If it's not possible to put the data zipped into a method because  
>> it would be too big somehow, I'd consider your two examples  
>> logically equivalent to "If the moon is made out of green cheese  
>> anything is allowed". So it is kind of ok that these tests are green.
>
> It's 8MB a pop so no, I think it's not really feasible to stick  
> that test data into a method ;-)
>
>> And you are suggesting to indicate clearly, which tests depend on  
>> some external resource?
>
> Well, really, what I'm looking for is something that instead of  
> saying "all tests are green, everything is fine" says "all the  
> tests we ran were green, but there were various that were *not* run  
> so YMMV". I think what I'm really looking for is something that  
> instead of saying "x tests, y passed" either says "x tests, y  
> passed, z skipped" or simply doesn't include the "skipped" ones in  
> the number of tests being run. In either case, looking at something  
> that says "19 tests, 0 passed, 19 skipped" or simply "0 tests, 0  
> passed" is vastly more explicit than "19 tests, 19 passed" where in  
> reality 0 were run.
Yeah, and I think my precondition mechanism could just do that. I  
mean you want to annotate your test somehow of being such kind of  
beast, so that the TestRunner can know about them, and indicate them  
as you suggest, no?
I was banging on the "external resource" a bit, cause until now this  
is the only reason I can see for writing such kinds of tests, and I  
wanted to make that very explicit... ;-)
>
> Like, what if a test which doesn't have any assertion is simply not  
> counted? Doesn't make sense to begin with, and then all the  
> preconditions need to do is to bail out and the test doesn't count...
I don't understand this remark within that context.

I know a guy who is using that
	shouldnt: aBlock raise: anExceptionalEvent : []
idiom a lot ;-) , which is good for knowing what is really tested ;-)  
but otherwise does not provide any real assertion in the test. (See  
most of the BitBltClipBugs tests, which should be platform independent)

Also, tests without any assertions still could execute lots of  
methods, which have nice post conditions with them.
So besides being good smoke tests, they also could be seen as tests  
of that very methods.
>
> In any case, my complaint here is more about the *perception* of  
> "these tests are all green, everything must be fine" when in fact,  
> none of them have tested anything.

Fine for me, all I proposed was a mechanism to denote them. :-)

Cheers,

Markus