TestRunner in 3.8 -- ARGH!!!
Colin Putney
cputney at wiresong.ca
Fri Nov 5 13:52:20 UTC 2004
Andreas Raab wrote:
> [... and later we have Colin argueing about essentially the same issue...]
>
>> The problem with the old behavior is that it *also* causes you to
>> have to select each and every test manually. You can't just use the
>> 'Filter' button to select some tests and run them all, because the
>> abstract classes will cause tests to get run twice - or more than
>> twice if you have several levels of abstract classes in your test
>> hierarchy. This makes moderately-long-running test suites take much
>> longer.
>
> I might not have been clear on this issue - it is the whole idea (!)
> that you run these tests "multiple times". Consider the following
> situation: Let us assume we have a set of invariants for a class which
> must hold true for all subclasses. Such as "when the mouse moves over
> [any subclass of] button its state changes from normal to over". Yes, we
> can rely on copying the test for each and every subclass of Button. Or,
> alternatively, we put it in the superclass (abstract or not) with the
> specific intent that this test is an invariant for the (abstract or not)
> class and all of its subclasses. So that if you make a new button
> subclass these invariants can be automatically validated without you
> much doing about it.
>
> Also, it is likely that situations in which this is relevant (lots of
> tests which should be grouped together) will expose some common behavior
> which is best put into a common superclass.
No, you were clear. Apparently it is I that was not clear. I completely
agree about the utility of abstract test cases. I use them a lot.
Consider the situation you mentioned in your first mail, something like
this:
FooAbstractTest
FooTestA
FooTestB
FooTestC
FooAbstractTest has a single test method #testBar, which gets run in
slightly different contexts in FooTestA, FooTestB and FooTestC.
Now, suppose I use the filtering in the test runnner to select all the
Foo-related tests with the pattern 'Foo*', and click Run All. This is
what gets run:
FooTestA(FooAbstractTest)>>testBar
FooTestB(FooAbstractTest)>>testBar
FooTestC(FooAbstractTest)>>testBar
FooTestA(FooAbstractTest)>>testBar
FooTestB(FooAbstractTest)>>testBar
FooTestC(FooAbstractTest)>>testBar
I have no problem with #testBar being run 3 times. As you say, that's
the whole point. But I don't want it to run 6 times, as having it fail
twice doesn't tell me any more than if it had failed once.
Ultimately this boils down to different mechanisms used for building
suites. I use names and filters to group tests, while you use the
inheritance hierarchy.
Colin
More information about the Squeak-dev
mailing list
|