Hi Andreas,
I am in the interesting situation that I'm writing a few tests that require large data sets for input and where I don't want people to require to download the data sets. My problem is while it's easy to determine that the data is missing and skip the test there isn't a good way of relaying this to the user. From the user's point of view "all tests are green" even though that statement is completely meaningless and I'd rather communicate that in a way that says "X tests skipped" so that one can look at and decide whether it's useful to re-run the tests with the data sets or not.
Another place where I've seen this to happen is when platform specific tests are involved. A test which cannot be run on some platform should be skipped meaningfully (e.g., by telling the user it was skipped) rather than to appear green and working.
Any ideas?
Cheers,
- Andreas
If it's not possible to put the data zipped into a method because it would be too big somehow, I'd consider your two examples logically equivalent to "If the moon is made out of green cheese anything is allowed". So it is kind of ok that these tests are green. But you are right, one usually does not think about tests to have prerequisites, one does think about them as commands, which "always" bring their necessary context.
And you are suggesting to indicate clearly, which tests depend on some external resource? I'd suggest to use (and introduce in general into Squeak) preconditions using blocks (*1) like:
testCroquetOnXBox self precondition: [SmalltalkImage current platformName = 'XBox']. (...)
Having that in place one could easily collect and indicate all the tests, which have a failed precondition. They should be rare and all depend on an external resource, which is too cumbersome to recreate as a scenario.
Cheers,
Markus
(*1) Except backwards compatibility I wouldn't have problems to use method properties/pragmas for the introduction of pre- and postconditions either.