[Seaside] onAnswer: problems

[Colin Putney]

On Jun 3, 2004, at 8:41 PM, C. David Shaffer wrote:

> Avi Bryant wrote:
>
>> Yup, that'll fix your immediate problem, but I'm not convinced it's a 
>> good long term solution either.  For example, if the child you hung 
>> the onAnswer: block on was two deep (say there was a navigation 
>> component like a tab widget in between the parent and the child it 
>> was really interested in), then things would still break.
>
>
> OK, here's one that works several levels deep.  Before I do any more 
> with this kind of thing I need to write some test cases...to do that I 
> need a framework for testing seaside apps.  Anyone got one (or part of 
> one) that they're willing to share?  Off to Maine for a week tomorrow 
> morning...thanks for all your help Avi.

I too would like to see such a framework.

My Nori templating framework provided a way of testing Seaside 
components, but since templating is so passé, I've stopped developing 
it. I'm not sure how applicable it is to the current situation, but 
here's how it worked.

1. Instead of using HtmlRenderer, components generated a DOM tree which 
was then serialized to serve the page.

2. The tests would set up a component, render it, and then make 
assertions about the DOM tree. This would be stuff like "has a text 
node containing 'Copyright Evil Corporation 1976, all rights reserved'" 
or "has an anchor with the label 'Launch Nukes'" or "background color 
for a div with id #main is #ffffff".

3. Callbacks were implemented with MessageSends instead of blocks. This 
allowed them to be easily examined as well. So tests would also make 
assertions like "some element of the page is bound do #doSomething." 
That assertion would pass if there was an anchor or form button that 
triggered #doSomething when clicked. You could do similar assertions 
for value callbacks.

4. Testing of the callbacks themselves was done separately. A component 
would be set up, the callback invoked, the resulting component rendered 
and assertions made about the resulting DOM tree. So you could make 
assertions like #doSomething calls SomeOtherComponent, which then 
displays a page containing a link that calls #answer.

The intent of all this was to make is simple to write tests that 
capture the spirit of the requirements that usually come up in web 
applications. Requests from users often take the form of "There should 
be a link on this page that will take me to a page that does X." So to 
test this, we can write two tests: one that asserts the link is present 
and calls a certain callback (ie, test the rendering), and another that 
asserts the callback does what it's supposed to (ie, test the 
behaviour).

I wrote a fairly complex real-world application using this kind of 
testing (not in Seaside - Nori was a port of that system to Seaside), 
and I can't stress enough what a benefit it is to be able to test 
application workflow and response to specific user actions.

So where do we go from here? I have some thoughts about how to 
implement a framework for testing Seaside apps that doesn't rely on a 
templating system the way Nori did.

1. For testing, we use a mock renderer instead of a genuine 
HtmlRenderer. The test would set up the MockRenderer with a set of 
expectations, which would capture the things we want to be true about 
the way the component is rendered. During the render, the MockRenderer 
would check each call by the component against its expectations and 
fail the test if they aren't met. In some cases, this would have to 
wait until the render is finished (such as certain things being 
present), but sometimes we could fail the test at the exact point the 
render goes wrong.

2. Analysing block callbacks is much trickier than MessageSends, but I 
think it could be done using the same sort of techniques that are in 
the RefactoringBrowser. We'd parse the source code to the block or 
decompile it's bytecodes into an AST which we could then examine. Even 
with an AST, it's pretty difficult to predict what code will do without 
running it, but I suspect that most blocks passed to the renderer are 
pretty simple, so it ought to be possible to use the same sort of 
convention that Nori did. We could assert that #doSomething gets called 
by the callback, or the value gets stored in an instance variable etc.

3. Alternatively we could modify HtmlRenderer to allow the use of 
messages or selectors in place of blocks. This makes rendering a bit 
more awkward, but it makes testing easier.

4. I'd like to tie into the halo system as well. There ought to be a 
halo that allows the user to view and edit the rendering assertions 
that get used in the component's tests, and run the tests. I'm 
imagining something a bit like FIT here; this would be aimed at 
allowing non-programmers to specify the rendering tests. Ideally this 
should be implemented in a way that allows for test-driven development 
- ie, we can create the rendering assertions before actually writing 
the rendering code.

Thoughts?

Colin