(Copying squeak-dev too).

I'm not sold on the whole test timeout thing.  When I run tests, I
want to know the answer to the question, "is the software working?"

Putting a timeout on tests trades a slower, but definitive, "yes" or
"no" for a supposedly-faster "maybe".  But is getting a "maybe" back
really faster?  I've just incurred the cost of running a test suite,
but left without my answer.  I get a "maybe", what am I supposed to do
next?  Find a faster machine?  Hack into the code to fiddle with a
timeout pragma?  That's not faster..

But, the reason given for the change was not for running tests
interactively (the 99% case), rather, all tests form the beginning of
time are now saddled with a timeout for the 1% case:

  "The purpose of the timeout is to catch issues like infinite loops,
unexpected user input etc. in automated test environments."

If tests are supposed to be quick (and deterministic) anyway, wouldn't
an infinite loop or user-input be caught the first time the test was
run (interactively)?  Seriously, when you make software changes, we
run the tests interactively first, and then the purpose of night-time
automated test environment is to catch regressions on the merged
code..

In that case, the high-level test-controller which spits out the
results could and should be responsible for handling "unexpected user
input" and/or putting in a timeout, not each and every last test
method..

IMO, we want short tests, so let's just write them to be short.  If
they're too long, then the encouragement to shorten them comes from
our own impatience of running them interactively.  Running them in
batch at night requires no patience, because we're sleeping, and
besides, the batch processor should take responsibility for handling
those rare scenarios at a higher-level..

Regards,
  Chris


On Sat, May 29, 2010 at 2:53 AM, stephane ducasse
<stephane.ducasse at free.fr> wrote:
> Hi guys
>
> in Squeak andreas introduced the idea of test time out
> Do you think that this is interesting?
>
> Stef
>
> SUnit
> -----
> All test cases now have an associated timeout after which the test is considered failed. The purpose of the timeout is to catch issues like infinite loops, unexpected user input etc. in automated test environments. Timeouts can be set on an individual test basis using the <timeout: seconds> tag or for an entire test case by implementing the #defaultTimeout method.
> _______________________________________________
> Pharo-project mailing list
> Pharo-project at lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>