Benchmarks for Comparing Squeak Builds
Greg & Cindy Gritton
gritton at ibm.net
Sun Jan 24 03:23:27 UTC 1999
At 01:14 PM 1/23/99 -0500, you wrote:
>I'm messing arounds with MPW builds of Squeak, using various types and
>levels of optimization. Got the thing running (at last) consistently with
>all capabilities (some significant reorganization of interp.c is necessary
>to compile under -opt speed; some magic is necessary to get named
>primitives internal to the VM to work right), but I'm concerned that the
>supposedly "highly optimizing" MrC may not have produced the best results
>possible.
>
>Which benchmarks, whether or not found in the standard image, have people
>found to be most meaningful/relevant to evaluation of a squeak build?
>Unsurprisingly, I have found the several benchmarks yielding quite
>different results?
>
>Is there a benchmark suite people rely upon to evaluate whether a change is
>an improvement? If not in the standard image, where would I found them.
>
>
>
I have a set of benchmarks that I have used to compare the speed of
verious Smalltalk implementations with C. These are based on the
Stanford Integer benchmarks and were originally put online
as part of the Self distribution. They are derived from C benchmarks
so they are heavy array accessing. I also have the old
slopstone and smopstone benchmarks. (Squeak won't run smopstones.)
I could post or mail the benchmarks if anyone is interested.
In addition to Squeak I have versions for Smalltalk-V, Visual Works,
Smalltalk MT, and C. I have ported a couple of the benchmarks to
Python. The grand total is under 100K of source files.
Sincerely,
Greg Gritton
Some interesting results of various Smalltalk's
Benchmark V V-try2 Dolphin MT Squeak Sq-JIT Sq2.3 VW3.0 C
Python
(Time in ms)
BubbleSort 880 880 902 259 1349 1048 1086 245
21 3479
BubbleSort2 880 820 805 560 2083 2083 1107 206
21 3479
IntMM 1310 770 428 135 745 564 934 124 17
IntMM2 1160 1210 650 197 1655 1634 1458 153 17
MM 990 1260 741 1590 1497 2341 934 357 22
MM2 1260 940 660 2090 1721 2765 1289 367 22
Perm 930 990 701 136* 961 728 815 133
21 2630
Perm2 720 770 675 239 1273 1207 801 120 21
Queens 380 380 375 80 632 458 522 71 11
Queens2 280 280 285 59 468 327 555 67 11
Quicksort 550 490 384 105 599 485 481 95
11 1555
Quicksort2 490 500 365 199* 815 853 502 99
11 1555
Towers 550 550 930 215 1266 996 1010 171
22 3157
Towers2 330 330 538 118 756 623 557 79
22 3157
Puzzle 4720 2780 7695 1490 15591 13527 12789 1679 77
Total 15340 14950 16134 7472 31411 29639 25393 3966 305
Fixed 1242 1194 1183 2176 2000 1727 277 28
Float 1571 1515 1388 2646 2745 2138 383 54
* Did not run correctly
Versions
Smalltalk-V (from Smalltalk Express - version ?)
Squeak 2.0, Squeak 2.3
Dolphin 2.1, patch level 2
Smalltalk MT 1.5
VisualWorks NonCommercial 3.0
All runs are on a 100MHz Cyrix 6x86 PR120 with 512K of L2 cache, 24M of RAM.
More information about the Squeak-dev
mailing list
|