[squeak-dev] Re: SUnit infinite loop detection
nicolas cellier
ncellier at ifrance.com
Mon Jul 7 21:31:11 UTC 2008
Colin, Igor,
thank you, you have some good ideas.
They do not match my requirement though.
I don't want to test a specific implementation,
I want to test a feature.
self should: [33 printStringBase: 1] raise: Error.
The idea is to have some generic tests automated to check base images
and/or packages.
Unfortunately, current implementation will not fail, it will loop!
Knowing this, I can protect the test with something like
self
should: [self should: [10 printStringBase: 1] raise: Error]
notTakeLongerThanMilliseconds: 1000.
In little better English:
self
should: [10 printStringBase: 1]
notTakeLongerThanMilliseconds: 1000
toRaise: Error.
The 1000ms figure here is quite arbitrary.
If delay is too short, would the test pass on a 486 µ-proc running a
Simulator?
If a long delay is ran on a fast machine, some recursive loop might
exhaust memory and crash the VM (the VM should be robust, but...).
This can easily happen, you load incompatible changes and
printStringBase: send printOn:base: sends printStringBase: etc...
I thought the idea of a generic SUnit support message for counting loops
would be smarter:
self
should: [self should: [33 printStringBase: 1] raise: Error]
notLoopMoreThan: 100.
Up to test maker to provide a reasonnable hint.
Here, even a loop on bits should hardly loop more than (33 highBit + 1).
As said before, support is easy for recursive loop, not for iterative...
Now, when i think of it, does it really add some value?
The point is why would i protect only this specific TestCase? Because I
know it fails, but maybe there are plenty others ready to fail and they
will block the testing image...
That should rather be a general rule: a single test case consuming too
much resource (memory and/or cpu) should be halted and declared failing.
That would be a a nice application: an image monitoring another...
A job for a HydraVM?
...or maybe monitoring a bunch of others: you want to test N packages?
Well that is 2^N different partitions, hope the multi-core won't stop at
1000, that makes only 10 packages to test together... As an exercize,
compute number of message sends required to test all the SqueakMap
packages combinations (Who is the smart guy that will fight the
combinatorial hydra?).
Cheers
Nicolas
Igor Stasenko a écrit :
> Temporarily, replace a method in method dictionary with 'test' object
> or compiled method
> (Squeak can use any object as 'method', not just to a compiled method).
> There you can inspect arguments and see, if your loop is still infinite.
> Of course its implies , that you have sends within a loop, which can
> be intercepted in that way.
>
> 2008/7/6 Colin Putney <cputney at wiresong.ca>:
>> On 6-Jul-08, at 1:33 PM, nicolas cellier wrote:
>>
>>> Some infinite loops have been removed (presumably).
>>> And I want to be able to write non regression SUnit tests that's all.
>>>
>>> How would you do it?
>> It's try to find some way to measure the progress of the algorithm. If
>> you're testing #printOn:, can you monitor the stream to see if any digits
>> have been printed? Are there some intermediate results that you can check?
>>
>> Another approach to isolate the bugfix and test that specifically. When you
>> fixed the infinite loop, what did you change? Is there a way to test that in
>> a narrow sense? For example, say the bug was that an index into an array
>> wasn't getting incremented at the end of the loop. Now you've added code to
>> do the increment. Is there a way to test that the index gets incremented
>> rather than testing the overall behavior of the loop?
>>
>> Maybe this "untestability" is a code smell - perhaps there's some
>> refactoring that could be done to expose more parts of the algorithm to
>> testing in isolation.
>>
>> Colin
>>
>>
>
>
>
More information about the Squeak-dev
mailing list
|