On Wed, Nov 05, 2003 at 09:15:01AM +0100, Andreas Raab wrote:
> Thanks for the info. One thing I am wondering about is the following. When
> we write tests, we typically model our "internal assumptions" quite
> precisely which helps us to track may of the problems that occur when those
> assumptions (even slightly) change. Is there anything in research that
> explains why we don't do this on the environment? Is it just considered too
> big a task? Or is it that we merely assume that we have an unchanging target
> (which requires people/vendors to fix the API once and forever)? Or is it
> that - because in many cases we simply don't know enough about the
> environment (black box reuse) - we don't even know what assumptions we (and
> the environemnt itself) is making? What I am interested in here is to
> understand what the causal effect(s) for our effectively blindly relying on
> the environment is.
> 

Two random thoughts:

1) Joseph did a very nice "Pre-Testing" chnageset for SUnit. The idea
of that was, that the code tests the environment (if this class there? does
it has method y? Does method y work?) And if the environment is not corrent,
it tells you what to implement (that is: It dumps out code for you to
accept). Very nice. 

This was done by hand, of course. The Question is now: Could we do this automatically? 
If we had complete testsuite (hah! impossible...), than you could generate a 
"environmental" test by collecting all the tests for all the methods you call. 
But we don't know about the quality of the tests, and if they are actually testing 
what you need to have tested. So this doen't work.

What about this: While your programm is running on Squeak 3.5, it generates
instances of Objects and calls methods on them, resulting in a differnt
instance/changed instance. Couldn't we record these instances? with 
state1/methodcall/state2 we have everything we'd need for building tests 
automatically. This test-suite would be an exact testsuite in the sense
of "Program X thinks the environment should behave this way".

Taking this test-suite to the new system will show those methods
that behave in a different way, thus showing all the changes in the
environment that might keep the programm from running.

2) Lots of the non-backward compatible changes are introduced by
refactorings. The nice thing with a refactoring is that they do not
change semantics, they are a pure syntactic translation that can be done
automatically and that could be saved for later re-use.

So we could provide a "refactoring tool" that can help the programmer to do the 
same refactorings we did in the image, but on his code.  

  Marcus