[squeak-dev] Re: SUnit infinite loop detection

nicolas cellier ncellier at ifrance.com
Mon Jul 7 21:31:11 UTC 2008


Colin, Igor,
thank you, you have some good ideas.
They do not match my requirement though.
I don't want to test a specific implementation,
I want to test a feature.

     self should: [33 printStringBase: 1] raise: Error.

The idea is to have some generic tests automated to check base images 
and/or packages.

Unfortunately, current implementation will not fail, it will loop!
Knowing this, I can protect the test with something like

     self
         should: [self should: [10 printStringBase: 1] raise: Error]
         notTakeLongerThanMilliseconds: 1000.

In little better English:
     self
         should: [10 printStringBase: 1]
         notTakeLongerThanMilliseconds: 1000
	toRaise: Error.

The 1000ms figure here is quite arbitrary.
If delay is too short, would the test pass on a 486 µ-proc running a 
Simulator?
If a long delay is ran on a fast machine, some recursive loop might 
exhaust memory and crash the VM (the VM should be robust, but...).
This can easily happen, you load incompatible changes and 
printStringBase: send printOn:base: sends printStringBase: etc...

I thought the idea of a generic SUnit support message for counting loops 
would be smarter:

     self
         should: [self should: [33 printStringBase: 1] raise: Error]
         notLoopMoreThan: 100.

Up to test maker to provide a reasonnable hint.
Here, even a loop on bits should hardly loop more than (33 highBit + 1).

As said before, support is easy for recursive loop, not for iterative...

Now, when i think of it, does it really add some value?
The point is why would i protect only this specific TestCase? Because I 
know it fails, but maybe there are plenty others ready to fail and they 
will block the testing image...

That should rather be a general rule: a single test case consuming too 
much resource (memory and/or cpu) should be halted and declared failing. 
That would be a a nice application: an image monitoring another...
A job for a HydraVM?

...or maybe monitoring a bunch of others: you want to test N packages? 
Well that is 2^N different partitions, hope the multi-core won't stop at 
1000, that makes only 10 packages to test together... As an exercize, 
compute number of message sends required to test all the SqueakMap 
packages combinations (Who is the smart guy that will fight the 
combinatorial hydra?).

Cheers

Nicolas

Igor Stasenko a écrit :
> Temporarily, replace a method in method dictionary with 'test' object
> or compiled method
> (Squeak can use any object as 'method', not just to a compiled method).
> There you can inspect arguments and see, if your loop is still infinite.
> Of course its implies , that you have sends within a loop, which can
> be intercepted in that way.
> 
> 2008/7/6 Colin Putney <cputney at wiresong.ca>:
>> On 6-Jul-08, at 1:33 PM, nicolas cellier wrote:
>>
>>> Some infinite loops have been removed (presumably).
>>> And I want to be able to write non regression SUnit tests that's all.
>>>
>>> How would you do it?
>> It's try to find some way to measure the progress of the algorithm. If
>> you're testing #printOn:, can you monitor the stream to see if any  digits
>> have been printed? Are there some intermediate results that you can check?
>>
>> Another approach to isolate the bugfix and test that specifically. When you
>> fixed the infinite loop, what did you change? Is there a way to test that in
>> a narrow sense? For example, say the bug was that an index into an array
>> wasn't getting incremented at the end of the loop. Now you've added code to
>> do the increment. Is there a way to test that the index gets incremented
>> rather than testing the overall behavior of the loop?
>>
>> Maybe this "untestability" is a code smell - perhaps there's some
>> refactoring that could be done to expose more parts of the algorithm to
>> testing in isolation.
>>
>> Colin
>>
>>
> 
> 
> 




More information about the Squeak-dev mailing list