[Seaside-dev] Current load-order test failures

[James Foster]

On Apr 8, 2009, at 12:28 PM, Lukas Renggli wrote:

>>> Test failures are **perfectly fine** during the development cycles.
>>
>> As I'm sure you are aware, this is not a universally held opinion.
>
> This is not a corporate environment where a ticket is created and
> eventually somebody is forced to fix the bug.
>
> This is an open source environment and there is nothing that happens
> without the necessary motivation. If the test is disabled or removed,
> the issue is simply not going to be resolved anytime soon. I have seen
> that many times, just look at the issues that are older than a year.
>
> If it bothers you that much I suggest that you commit a fix to the
> PharoInbox, write an e-mail explaining the importance of your change,
> convince the Pharo maintainers to include the changes with the next
> version, and tomorrow the test is green.
>
> A reason for the existence of this test is to bother people and to
> provoke a solution.

I understand that you are trying to provoke Seaside developers to work  
on Squeak/Pharo to improve ANSI compliance. As a contributor to the  
GemStone port of Seaside this leaves me feeling a bit manipulated. In  
order to validate a port (by comparing test results in two  
environments), I need to investigate test failures that have nothing  
to do with Seaside functionality.

Also, this risks the unintended consequence that Seaside developers  
will become accustomed seeing test failures and dismiss it with the  
explanation that the core developers are comfortable with broken  
tests--whatever the reason. The fact that you are willing to write a  
failing test and not fix it sends that message to me. Why should I fix  
Pharo when it bothers you but you aren't fixing it? I am concerned  
that your approach might dilute the meaning of tests so that they are  
simply a parallel task tracking system.

If you are interested in developing an ANSI Compliance Test Suite,  
could you start another package for this purpose? Alternatively, could  
you mark this test as an expected failure in Squeak/Pharo? I suspect  
that you would not accept these approaches because they won't "bother  
people" enough to get them to take on your suggested task.

Keep in mind that on an open source environment, your actions might be  
pushing people in a different direction than you intend. You present  
me with the choice of fixing Pharo or accepting failing tests. Fixing  
Pharo is outside my comfort zone and apparently too much trouble for  
you (which makes me even more scared to try it since I respect your  
abilities). If accepting failing tests is good enough for you then I  
guess it will be good enough for me. In at least one case, your impact  
is to reduce the motivation to maintain quality (as measured by tests).

James Foster