On Tuesday, August 5, 2003, at 12:59  AM, Julian Fitzell wrote:

> Richard A. O'Keefe wrote:
>> I've just been asked this question by a student,
>> and I didn't really know the answer.
>> "How do you test the (visual) output of a GUI program?"
>> I suggested all sorts of things, from grabbing a Form and comparing it
>> with a stored Form to saving the Morph to a file and comparing the 
>> files,
>> but it all seems a bit vulnerable (user changes round/square corners
>> preference => all comparisons fail).
>> Is there a way of testing the visual output of a GUI program
>> that doesn't involve handcuffing a user to the desk to look?
>
> I'm sure Colin will reply as soon as he gets up tomorrow, so I won't 
> go into the details, but he did a presentation at Smalltalk Solutions 
> about this sort of thing.

Yup, this is exactly the sort of situation I addressed in my talk.

The gist of it was that to write robust tests one needs a clear notion 
of what he considers correct behaviour. In terms of user interfaces, 
particularly on the web, I found that what I wanted to test for had 
little to do with how the thing actually looked. Given the number of 
possible page designs that could be considered "correct," there's no 
good way to textually test the html output of say, a Seaside app. But, 
if you parse it into a DOM first, it's pretty easy to make assertions 
of the kind Julian mentioned - text "XYZ" appears somewhere on the 
page, there's a form input hooked up to callback #foo:, there's a link 
that's hooked up to #bar, etc.

I recently did some work to see if I could do the same kind of thing 
with Morphic, and (somewhat to my surprise) it's even easier than with 
html. The tree of morphs is like a DOM, only simpler, and you can 
easily walk the tree looking for a morph that satisfies your assertion. 
With most morphs there are convenient methods that you can call to 
simulate user actions. You can run the test without actually putting 
the morphs in the world, so they don't show up on screen during the 
test.

My first stab at doing this is in the current release of Monticello - 
have a look at MCSnapshotBrowserTest. A snapshot browser is just like a 
regular browser, except that it operates on a MCSnapshot, not the 
classes and methods actually in use in the image. Let's say I want to 
test that every thing looks right after you select a class in the 
browser. So I want to assert the following:
	- all the class categories in the package are visible
	- all the classes in the selected category are visible
	- all the protocols in the selected class are visible
	- no methods are visible
	- the class definition is displayed in the text pane.

The code for that would be something like this:

testClassSelected
	self clickOnListItem: 'Monticello-Mocks'.
	self clickOnListItem: 'MCMockClassA'.
	
	self assertAListMatches: self allCategories.
	self assertAListMatches: self mockClasses.
	self assertAListMatches: self classAProtocols.
	self denyAListIncludesAnyOf: self allMethods.
	self assertTextIs: self classADefinitionString.


Note that I'm not testing the layout here at all, just the contents of 
the various panes. It is possible to make assertions about the layout, 
but I haven't come up with a way to test that it "looks right," 
whatever that means. I'm not sure that esthetics should be tested 
anyway, so I've just punted that particular issue.

Hope this helps.

Colin