Milan Zimmermann wrote:
On 2007 January 29 16:58, Colin Putney wrote:
On Jan 29, 2007, at 9:23 AM, Bert Freudenberg wrote:
I actually side with Ralph on this one. It is very satisfying to see the test runner turn green. With tests in the image that you cannot fix, you will never get this satisfaction.
True, the green bar is very satisfying. But Ralph's reasons go a bit deeper. The purpose of a test suite is to provide immediate feedback on during development. Running the suite is like asking "how am I doing?" A green bar means "fine," and a red bar means, "STOP! You
Yes except that the current suite of tests take ages to run, and so does not fulfil this goal.
The solution which already exists! Is to categorise the tests, so that a suite of short tests can be run more frequently in less than 2 minutes.
If you have this categorisation then it is trivial to have a category of 'known issues', which you simply do not run if you want to see your green light.
The improved TestRunner, times each test, and can automatically sort the long from the short, the network using tests from the non-network using tests etc.
Not forgetting that there are other categories of tests needed in order to get that all hallowed green bar, such as: tests that I would not expect to work on my platform, or tests that will not work on the vm that I am using, and tests that will not work in this version of the image.
Pulling your known issues tests into a separate class, is not a good solution because then you loose the ability to subclass from TestCase sensibly.
Pulling your known issues into a separate package is not a good solution, because then you loose the context for those tests, since they belong in the same context as those tests which pass. You need that context if you are ever going to fix them.
Finally, breaking things up physically, rather than tagging things 'mentally' so to speak ruins any kind of smooth workflow. Write test fix test, becomes write test in one place, when it works move it to another place, debug it again in the new context.
To try the improved TestRunner, try
Installer fixBug: 5639.
best regards
Keith
___________________________________________________________ All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html