On Sat, 25 May 2002, Avi Bryant wrote:

> Trivial benchmarks, observed but not timed.  How long it takes a
> browser window to come up, how smoothly scrolling through a browser list
> updates, delay between clicking on a category and seeing the list of
> methods, etc.  The qualitative difference is striking.  I will see if I
> can come up with some numbers, although I don't have a good idea of how to
> automate such tests.

Here's the most telling thing I've come up with, trying to automate the
act of switching classes in the browser:

b := Browser new.
Browser openBrowserView: (b openEditString: nil) label: 'System Browser'.
(Collection allSubclasses collect:
	[:c |
	c methodDictionary size ->
	(Time millisecondsToRun: [b setClass: c selector: nil])])
		asSortedCollection

On 2.8:
a SortedCollection(0->10 1->12 2->10 3->11 4->9 5->13 6->14 7->10 8->9
9->12 10->12 11->13 12->12 13->16 14->9 15->13 16->14 18->13 19->17 20->10
21->13 23->14 24->12 26->15 27->11 29->17 31->13 34->15 43->13 44->15
46->12 49->12 55->9 68->12 86->16 107->10 114->12 147->17 221->15)

On 2.9a:
a SortedCollection(0->15 1->12 2->11 3->12 4->17 5->16 6->19 7->16 8->15
9->18 10->17 11->20 12->16 13->21 14->20 15->16 16->18 18->18 19->22
20->23 21->22 23->19 24->17 26->22 27->20 29->28 31->22 32->21 34->23
46->27 49->30 51->29 54->28 68->32 87->40 107->38 119->42 162->58 245->76)

In 2.8, the amount of time it takes to switch classes is fairly low and
fairly constant.  In 2.9, however, although it starts off the same, it
increases directly with the number of methods in the class.  This is
consistent with my observations just clicking around.  I don't know what
the root cause is, but this symptom alone has a major effect on how
responsive the system feels.

Really scary is to see the same numbers for 3.2:
a SortedCollection(0->20 1->21 2->21 3->26 4->20 5->24 6->23 7->26 8->33
9->29 10->27 11->37 12->33 13->31 15->26 16->32 17->30 18->29 19->31
20->35 21->29 23->40 24->44 25->45 26->40 27->41 29->39 30->42 31->55
33->47 34->47 46->51 48->52 55->58 58->73 60->65 69->68 87->105 107->85
124->100 176->143 291->272)

For a large class, it can take 1/3rd of a second to bring it up in the
browser.  That's 20x as long as it took on 2.8!  Still usable on a fast
machine, but far from snappy.

Anyone have any guesses?

Avi