Web app Testing frameworks was: [Seaside] onAnswer: problems

[Roger Whitney]

Some of us in San Diego have been talking about the lack of testing of 
Smalltalk Web apps. Have you thought about something like HttpUnit 
where the tests are run on the results of http calls to the server? 
Such a framework would not be restricted to testing Seaside apps. Since 
one would be testing the http output the tests should give a bit more 
coverage of the web app but also make it harder for the tests to access 
the state of objects.

On Jun 4, 2004, at 10:45 AM, Colin Putney wrote:

>
> I too would like to see such a framework.
>
> My Nori templating framework provided a way of testing Seaside 
> components, but since templating is so passé, I've stopped developing 
> it. I'm not sure how applicable it is to the current situation, but 
> here's how it worked.
>
> 1. Instead of using HtmlRenderer, components generated a DOM tree 
> which was then serialized to serve the page.
>
> 2. The tests would set up a component, render it, and then make 
> assertions about the DOM tree. This would be stuff like "has a text 
> node containing 'Copyright Evil Corporation 1976, all rights 
> reserved'" or "has an anchor with the label 'Launch Nukes'" or 
> "background color for a div with id #main is #ffffff".
>
> 3. Callbacks were implemented with MessageSends instead of blocks. 
> This allowed them to be easily examined as well. So tests would also 
> make assertions like "some element of the page is bound do 
> #doSomething." That assertion would pass if there was an anchor or 
> form button that triggered #doSomething when clicked. You could do 
> similar assertions for value callbacks.
>
> 4. Testing of the callbacks themselves was done separately. A 
> component would be set up, the callback invoked, the resulting 
> component rendered and assertions made about the resulting DOM tree. 
> So you could make assertions like #doSomething calls 
> SomeOtherComponent, which then displays a page containing a link that 
> calls #answer.
>
> The intent of all this was to make is simple to write tests that 
> capture the spirit of the requirements that usually come up in web 
> applications. Requests from users often take the form of "There should 
> be a link on this page that will take me to a page that does X." So to 
> test this, we can write two tests: one that asserts the link is 
> present and calls a certain callback (ie, test the rendering), and 
> another that asserts the callback does what it's supposed to (ie, test 
> the behaviour).
>
> I wrote a fairly complex real-world application using this kind of 
> testing (not in Seaside - Nori was a port of that system to Seaside), 
> and I can't stress enough what a benefit it is to be able to test 
> application workflow and response to specific user actions.
>
> So where do we go from here? I have some thoughts about how to 
> implement a framework for testing Seaside apps that doesn't rely on a 
> templating system the way Nori did.
>
> 1. For testing, we use a mock renderer instead of a genuine 
> HtmlRenderer. The test would set up the MockRenderer with a set of 
> expectations, which would capture the things we want to be true about 
> the way the component is rendered. During the render, the MockRenderer 
> would check each call by the component against its expectations and 
> fail the test if they aren't met. In some cases, this would have to 
> wait until the render is finished (such as certain things being 
> present), but sometimes we could fail the test at the exact point the 
> render goes wrong.
>
> 2. Analysing block callbacks is much trickier than MessageSends, but I 
> think it could be done using the same sort of techniques that are in 
> the RefactoringBrowser. We'd parse the source code to the block or 
> decompile it's bytecodes into an AST which we could then examine. Even 
> with an AST, it's pretty difficult to predict what code will do 
> without running it, but I suspect that most blocks passed to the 
> renderer are pretty simple, so it ought to be possible to use the same 
> sort of convention that Nori did. We could assert that #doSomething 
> gets called by the callback, or the value gets stored in an instance 
> variable etc.
>
> 3. Alternatively we could modify HtmlRenderer to allow the use of 
> messages or selectors in place of blocks. This makes rendering a bit 
> more awkward, but it makes testing easier.
>
> 4. I'd like to tie into the halo system as well. There ought to be a 
> halo that allows the user to view and edit the rendering assertions 
> that get used in the component's tests, and run the tests. I'm 
> imagining something a bit like FIT here; this would be aimed at 
> allowing non-programmers to specify the rendering tests. Ideally this 
> should be implemented in a way that allows for test-driven development 
> - ie, we can create the rendering assertions before actually writing 
> the rendering code.
>
> Thoughts?
>
> Colin
>
>
>
> _______________________________________________
> Seaside mailing list
> Seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/listinfo/seaside
>
>

----
Roger Whitney              Department of Computer Science
whitney at cs.sdsu.edu        San Diego State University
http://www.eli.sdsu.edu/   San Diego, CA 92182-7720
(619) 583-1978
(619) 594-3535 (office)
(619) 594-6746 (fax)